Joost was born into a musical family in Wissembourg, Alsace on 9 June 1898. His father, Albert Joost, was a cellist in the military. Oskar attended high school in Leipzig and participated in the First World War as a volunteer.
In 1924, Oskar and his brother Ali formed the Oscar Joost Dance Orchestra, with Oskar spelling his name with a 'c' instead of a 'k'. Oscar played first violin, and his brother percussion. From 1930, the Orchestra began playing in the Berlin Eden hotel, which led to a recording contract with Electrola, followed by subsequent contracts with other record labels: Pallas (1931), Crystal (1931–1934), Ultraphon/Telefunken (1932), Grammophon/Polydor (1934–1941). In this time, he was also involved in film work.