Sanderling was born in Arys, Kingdom of Prussia, German Empire (now Orzysz, Poland), to Jewish parents. His early work at the Deutsche Oper Berlin, where he served as repetiteur (rehearsal director) for Wilhelm Furtwängler and Erich Kleiber, was cut short when the Nazi regime removed him from his post because he was Jewish. He then left for the Soviet Union in 1936, where he worked with the Moscow Radio Symphony Orchestra. In 1939, he became conductor of the Kharkiv (now Kharkov) Philharmonic Orchestra. During the siege of Leningrad, he worked in Novosibirsk, Siberia. From 1942 to 1960, he was joint principal conductor with Yevgeny Mravinsky of the Leningrad Philharmonic. Around 1942-1943, Sanderling first met Dmitri Shostakovich, which marked the start of their professional working relationship and personal friendship.