Dooley was born in Athens, Georgia in 1968, the son of University of Georgia coach Vince Dooley and his wife, radio talk show host Barbara Meshad Dooley. Dooley played high school football at Clarke Central High School in Athens under legendary coach Billy Henderson. He was a star tight end on the school's 1985 AAAA State Championship team. Dooley played alongside other notable Clarke Central (and later NFL) players, including kicker John Kasay (Seattle Seahawks, Carolina Panthers, New Orleans Saints), defensive end and former U.T. defensive line coach Chuck Smith (Atlanta Falcons, Carolina) and wide receiver Willie Green (four teams).