The son of a coach himself, Smart started his football career at Bainbridge High School. He then moved on to play defensive back for the Georgia Bulldogs, where he was teammates with defensive linemen Antonio Cochran, Emanos Leroy, linebacker Brandon Tolbert and All-Pro cornerback Champ Bailey. As a senior, he was a first-team All-SEC pick. He finished his career with 13 interceptions, which is currently fourth on the all-time Georgia ranks. He was also a four-time member of the SEC Academic Honor Roll.