Williams began pursuing an acting career after spending two years as a paratrooper in C Company, 506th Infantry, of the 101st Airborne Division.
He first appeared on Broadway in The Long Dream (1960). Continuing his work on stage, he appeared in Walk in Darkness (1963), Sarah and the Sax (1964), Doubletalk (1964), and King John. His breakout theatrical role was in William Hanley's Slow Dance on the Killing Ground, for which he received a Tony Award nomination. The New York Times drama critic Howard Taubman wrote of his performance, "Mr. Williams glides like a dancer, giving his long, fraudulently airy speeches the inner rhythms of fear and showing the nakedness of terror when he ceases to pretend." He also served as artist-in-residence at Brandeis University in 1966.
Williams' breakout role was as undercover cop Linc Hayes on the highly popular counterculture TV cop series The Mod Squad (1968), along with fellow relative unknowns Michael Cole and Peggy Lipton. Since the series ended in 1973, he has worked in a variety of genres on stage and screen, from comedy (I'm Gonna Git You Sucka; Half-Baked) to sci-fi (Star Trek: Deep Space Nine) and drama (Purple Rain).
Spanning over forty years, his career includes the role of Prince's tormented father, who was also a musician, in Purple Rain (1984), a recurring role in the surreal TV series Twin Peaks (1990), a good cop in Deep Cover (1992), a rioter in the mini-series Against the Wall (1994), and Wesley Snipes' chemically dependent Dad in Sugar Hill (1993). Other TV roles include Hill Street Blues, the Canadian cult classic The Littlest Hobo, Miami Vice, The Highwayman, Burn Notice, Everybody Hates Chris, Justified, Law & Order. He can be seen in movies such as 52 Pick-Up, Life, The Cool World, Deep Cover, Tales from the Hood, Half-Baked, King: A Filmed Record... Montgomery to Memphis, Hoodlum, Frogs for Snakes, Starstruck, The General's Daughter, Reindeer Games, Impostor, and The Legend of 1900. He also played a supporting role as George Wallace's fictional African-American butler and caretaker in the 1997 TNT movie George Wallace.
From 2003 to 2007, Williams had a recurring role as Philby Cross in the Mystery Woman movie series on the Hallmark Channel. He appeared in all but the first of the eleven movies alongside Kellie Martin. (J.E. Freeman played Philby in the Mystery Woman first movie.) In the seventh (Mystery Woman: At First Sight) movie, he reunited with his Mod Squad co-star Michael Cole. He played Bumpy Johnson in the film American Gangster.From 2005 to 2007 Williams had another recurring role as the voice of Councilor Andam on the Disney series American Dragon Jake Long