Szabó was born in Budapest and began playing guitar at the age of 14, inspired by jazz music on the Voice of America broadcasts. He escaped Hungary and moved to the United States in 1956, a year of attempted revolt against Soviet-dominated Communist rule, and attended the Berklee School of Music in Boston. In 1958, he was invited to perform at the Newport Jazz Festival. Szabó performed with the Chico Hamilton quintet from 1961 to 1965, and beginning in 1966 he recorded a well-received span of albums under his own name on the Impulse! label.