Karim was born on February 15, 1916, in Ujan Dhol village, Derai Thana, in Sunamganj District in Sylhet Division. Growing up in poverty, he started expressing his thoughts and emotions musically from a very early age. The ektara, a traditional single-stringed instrument, has been his constant companion. However, poverty forced him to seek work in agricultural labor. As he was growing up, he received training in spiritual and Baul music from Shah Ibrahim Mastan Baksh. Mastan was from Sreepur Peer Mohol.