From 1950 to 1954, he coached at Oklahoma A&M, and compiled a 22–27–1 record. From 1955 to 1957, he coached at Alabama, where he posted a 4–24–2 record that included a 14-game losing streak from 1955 to 1956. In his first year at Alabama, Whitworth was only allowed to hire only two of his own coaches and forced to retain the rest of former coach Harold Drew's assistants. This included athletic director Hank Crisp, Whitworth's boss, who was in charge of the defense. Whitworth brought assistant coach Moose Johnson with him from Oklahoma A&M. Following successive 2–7–1 seasons in 1956 and 1957, Whitworth was fired and replaced by Bear Bryant. In 1951, while Whitworth was coaching Oklahoma A&M, he instigated the infamous Johnny Bright Incident, which occurred in the football game in Stillwater