There are basically two places you can start with the books:
From the beginning, with The Sins of the Fathers. The first four or so novels or so are with Scudder as an alcoholic, and his condition gets steadily worse until he enters AA. The AA novels roughly start around Out on the Cutting Edge and continue through to the most recent full novel, All The Flowers Are Dying. The most recent, A Drop of the Hard Stuff, is a flashback novel.
There are advantages to both periods, and you can certainly jump around, as the backstory is basically recounted for about a page in each novel, but I will say one of the pleasures of the series is that Scudder and the characters around him age and change approximate to the arc of the series. Additionally - and this is Block's great contribution to the detective canon (and I would argue American lit as a whole) - the series vividly captures New York throughout the last 40 plus years, from the bad old days of the seventies, to the Giuliani 90s, to the hollowed out corporate canyons of today. In fact, one of the reasons Block stopped writing the series in 2005 is even he admitted, post 9/11, that the city had become something he didn't really recognize, and all the places he'd been writing about for decades had either closed or been forced out.
Cool. I shall peruse the local library catalogue to see if they carry them. I like starting at the beginning with series, so I'll probably take that route. I'll be interested to read some of the background stuff too, particularly to contrast it with the way London has evolved over the last 40 years I've been around the place.