Smith grew up in a wrestling family in Del City, Oklahoma, a suburb of Oklahoma City. His high school career record was 105–5, where he earned state medal each year – placing 2nd as a freshman, 3rd as a sophomore, and a 2× state champion winning it his junior and senior year. His older brother Leroy and younger brother Pat Smith also wrestled for Del City High School and Oklahoma State University. All three became national champions in college. The youngest Smith brother, Mark, wrestled at Oklahoma State and was also an All-American. Smith's nephews Mark and Chris Perry are also national collegiate champions. Mark won two titles while wrestling for the University of Iowa. Chris won two titles under John's tutelage at Oklahoma State University.
Smith became the head wrestling coach at Oklahoma State University beginning with the 1991–1992 season. He succeeded Joe Seay, who resigned during an investigation by the NCAA. Smith has coached at OSU for 18 seasons, longer than any other Cowboy mentor except the legendary Edward Gallagher. Smith has a dual meet record of 281-42-4 and the 281 wins are an Oklahoma State record.
His teams have won five NCAA wrestling team titles, in 1994 and 2003 through 2006. His wrestlers have won 26 individual NCAA titles and 102 All-American honors. Smith has also captured 10 conference (Big Eight and Big 12) team titles and his wrestlers have won 82 conference crowns.
Under Smith's watch, Oklahoma State compiled a 21-0 dual meet record in 2005 and capped the season with one of the most dominant showings in the history of the NCAA Championships when an NCAA-record five separate Cowboys (half of the lineup) were crowned as NCAA champions. Zack Esposito won at 149, Johny Hendricks taking the 165 championship, Chris Pendleton repeating as an NCAA champion at 174, Jake Rosholt claiming the title at 197 and Steve Mocco taking the heavyweight championship.
John coached his nephew Chris to All American honors with a 3rd-place finish at NCAA's in 2012 and a national championship in 2013 and 2014.
John coached Jordan Oliver, a 4x All American, 3x NCAA finalist, 2x NCAA Champion.