Cole was born to Edward and Paulina Cole, and grew up in Chicago with siblings Eddie, Ike and Nat King Cole. He began playing piano at the age of six, and continued his musical education at the Roosevelt Institute in Chicago. He moved to New York in 1951, where he studied at the Juilliard School of Music, before completing a master’s degree at the New England Conservatory of Music.
Following the moderate success of Whispering Grass in 1953 Cole spent several months on the road with Johnny Coles and Benny Golson as theEarl Bostic band. He went on to work with Grover Washington, Jr. and to record jingles for various companies, including Turner Classic Movies.
During the 1970s, Cole recorded several albums for European and English based labels. He was the subject of the 2006 documentary The Cole Nobody Knows. In June of that year, Cole was added to the Steinway Artist roster.
Cole was inducted into the Georgia Music Hall of Fame in 2007. In July 2009, he released a recording featuring his own quartet (guitarist Randy Napoleon, drummer Curtis Boyd, and bassist Elias Bailey), along with alto saxophonist Jerry Weldon and pianist John DiMartino, playing live at Dizzy's jazz club in Lincoln Center. His 2010 album, Freddy Cole Sings Mr. B, was nominated for the Grammy in the category Best Vocal Jazz Album. The album features tenor Houston Person, pianist John DiMartino, guitarist/arranger Randy Napoleon, drummer Curtis Boyd, and bassist Elias Bailey.