Mallory is the son of former Indiana Hoosiers football coach Bill Mallory. He played high school football at Bloomington South High School. He was a starter at linebacker starting with the last game of his freshman year at Bloomington South. Mallory's older brothers, Mike and Doug Mallory played college football at the University of Michigan. In February 1988, and despite having an offer to play for his father at Indiana, Mallory announced that, like his older brothers, he had committed to Michigan.