In an ill-lit New York basement crammed with people, Deborah Gordon is trying to explain why the subterranean club we are sitting in — the legendary Village Vanguard — has been the go-to place for jazz aficionados from all over the world for the past 80 years. The problem is, every time she puts a thought into words, she is interrupted by a blast of tenor sax or a wail of trumpet or the crack of a snare drum. The 16-piece Vanguard Jazz Orchestra is warming up for its Monday night set, and this is no place to be engaging in philosophical exchanges about the lure of a particularly American musical art form. Better to sit back and let this sweet thunder envelop us for a couple of hours then talk later.

When we finally do chat, Gordon explains that although she is nominally in charge of the Vanguard, “This place has a life of its own and is in charge of itself”. This is, she says, something she learnt from her father Max, founder of the club and one of the key custodians of New York jazz.

Max Gordon died in 1989 and you only have to reel off the names of the artists he hired to play here — Thelonious Monk, Charlie Parker, John Coltrane, Billie Holiday, Sonny Rollins, Bill Evans, Miles Davis — to gauge how important he was. Dizzy Gillespie even wrote “Blues for Max” specially for him. I’ve been visiting the Vanguard since the late 1970s, and Max once introduced me to Miles Davis here. Although the glowering genius was at the time vocalising contempt for white people, his admiration for Max was clear (though Max later told me that Miles was “the most difficult of them all”).

The Vanguard is a long, narrow, low-ceilinged room that takes just 123 customers. At one end is a dimly lit, lived-in bar; at the other is a small bandstand, and along the walls are portraits of the great musicians who once lit up the room. The audience is the slightly older local crowd, with a smattering of foreigners, here to absorb themselves in complex, thoughtful American music.

No food is served at the Vanguard and there are only minimal interruptions from the staff serving the drinks. Here, the music is the thing, and in this tiny space, every Monday for more than 40 years, a 16-piece orchestra has exploded into life and filled the room with blissful sounds. In Europe and east Asia, audiences pay high prices to hear musicians such as these playing in large auditoriums. In New York you pay a $30 cover charge and it’s like having a jazz band in your living room.

The Vanguard’s 80th anniversary has conveniently coincided with a jazz club revival in New York. Over the past few years, established clubs such as Minton’s, the Jazz Standard, Dizzy’s and the Vanguard have all thrived, while new clubs such as Subrosa and Mezzrow have taken off. Even venues hosting esoteric, avant-garde jazz such as the Stone in the East Village are doing well. As a life-long fan, it is reassuring to see that in this age of social networks and cellphones there is an appetite for something authentic and cerebral.

On this trip to New York, I stopped in at five of these clubs and listened to hard bop, swing, blues, Dixieland, avant- garde, a little New Orleans jazz-funk, and even found time to dash out to Brooklyn to talk to provocative jazz writer Stanley Crouch.

The first night I took in the early set at the Jazz Standard on East 27th, a venue far removed from the traditional cramped West Village clubs. It’s located in the basement of Danny Meyer’s highly regarded restaurant Blue Smoke, and Meyer’s barbecue cuisine plays a big part of the experience. The band that night was the SFJazz Collective, an impressive group of musicians from all over the world who played stirring, avant-garde original compositions mixed in with the work of tenor saxophonist Joe Henderson. There was just enough time to draw breath and dash uptown to the Time Warner building on Columbus Circle to catch the second set at Dizzy’s Club Coca-Cola. Dizzy’s is about as far as you can get from the smoke-stained basement clubs of jazz mythology. It is a huge, high-ceilinged space with a panoramic view of Central Park, and is one of three performance spaces under the aegis of Jazz at Lincoln Center (JALC), the programme of jazz education, broadcasts and performances guided by Wynton Marsalis.

At 53, the former wunderkind of modern jazz is, alarmingly quickly, morphing into the elder statesman of the genre. Marsalis’s admiration for traditional forms and disdain for post-1970s avant-garde jazz — something he shares with Stanley Crouch, his intellectual mentor — has attracted criticism from some modernists but nobody can deny his impact and, as Crouch says “In the music of Marsalis, as composer and player, one hears the whole of jazz remade into his image”.

That night at Dizzy’s, the spirit of Marsalis’s love of the traditional shone through as a big band of young, up-and-coming musicians performed a stirring tribute to Billie Holiday with four vocalists. Its frontman was Michael Mwenso who — until Marsalis lured him to JALC in 2012 — had run the late-show gigs at Ronnie Scott’s and been a vivacious presence on London’s jazz scene. He told me Marsalis had been an influence since childhood, and that being invited to join JALC “was a great privilege — believe me, at the moment the New York jazz scene is thriving.”

As if to prove his point, Mwenso led the band through the 1930s Holiday repertoire but with flourishes that effortlessly transported the music into the 21st century. Great jazz has a timeless quality, and for the 140 customers at Dizzy’s that night, this was as good as anything they are likely to hear all year. At the end they rose as one and gave the young band a rousing ovation.

The following night I headed up to Minton’s on 118th Street in Harlem, a neighbourhood where white New Yorkers once feared to tread. These days, Harlem has become gentrified, multicultural and relatively crime-free, like most of Manhattan. Minton’s was once the fabled Minton’s Playhouse, opened before the second world war and a hothouse for the development of bebop in the early 1940s. It was here that Parker, Gillespie, Monk, Kenny Clarke and the other founders of bebop held their famous jam sessions, and it was here that Gillespie and Roy Eldridge, that other titan of the jazz trumpet, conducted their trumpet duels.

In the late 1970s, on my first trip to New York, I rushed to Jimmy Ryan’s on West 54th Street because I’d heard that Eldridge was playing there. And indeed he was — I was one of seven or eight customers in a tomblike club that night and the great man played his heart out. It was an early reminder that jazz comes in and out of fashion — and when it’s out, even the greatest practitioners struggle to make a living.

But Minton’s, reopened in 2013, was full and thriving in the current boom, and the house band that night was JC Hopkins’ Biggish Band. Like the Vanguard, there is a sense of history in this room; unlike the Vanguard, it serves food and is a rather shiny reincarnation of the original Playhouse. It is a small club, taking just 25 customers in the bar area at the back and 50 dining guests at tables in front of the minuscule bandstand. Just before taking his band on stage, Hopkins told me that the success of the new Minton’s was down to the “interaction between older cats and younger cats”. That applies not just to the audience, but to the musicians who play here — from 93-year-old Jon Hendricks to 19-year-old Solomon Hicks, the Harlem kid who has been lead guitarist of the Cotton Club band since he was 13.

It turned out that the band was also doing a tribute to Billie Holiday, 2015 being the centenary of her birth and, as Hopkins explained, because she had a special connection to Minton’s: “She was a felon in New York, so they took away her performer’s licence, but she could play here because she was jamming and didn’t get paid for it.” So with Hopkins at the piano, the band swung and bopped behind three wonderful vocalists through the Billie Holiday songbook, including superb readings of “God Bless the Child” and “What a Little Moonlight Can Do”. Music created in another age that remains as affecting today must be regarded as serious art, and that was what I heard at Minton’s that night.

The last two clubs on my jazz pilgrimage offered up opposite ends of the spectrum. The Café Carlyle is located in the Carlyle Hotel (favourite of Mick Jagger, David Bowie and the Duke and Duchess of Cambridge) and there’s a real uptown feel about the place: the clink of champagne flutes as opposed to the clunk of downtown bourbon glasses. Every Monday night Woody Allen plays clarinet here in the Eddie Davis New Orleans Jazz Band. In this room he’s more a workaday musician than a celebrity director — so much so that when I took my daughter to hear him last year, he walked past us at the end of the set and said humbly: “Thanks very much for coming to listen to us.”

Then there was the Blue Note, the famous Greenwich Village club that is a big commercial operation (a souvenir shop takes up much of the first floor) and offers much more than jazz. On my night there the Donald Harrison Band played a combination of New Orleans jump-up music and funk jazz, with the Meters’ guitar ace Leo Nocentelli and James Brown band trombonist Fred Wesley as featured soloists. Hell, they even finished off with “Iko Iko”, a signature tune of Mardi Gras music. It was a long, long way from the intellectual and spiritual rigour of the Village Vanguard.

And then it was over. A week of booming clubs and hot jazz. There is no other city that offers such range and such easy access to American jazz, and even if you’ve never felt a great affinity for the music, an evening in a New York jazz club might just convert you.

Listen to a playlist of jazz club classics by the artists mentioned at ft.com/nyjazz