Born in Mobile, AL, John Mitchell played football at Eastern Arizona Junior College from 1969-1970 and was twice named an NJCAA All-American. Mitchell transferred to the University of Alabama, where he was the first African American to play for the Crimson Tide, along with teammate Wilbur Jackson, and was the first African American to be named captain of the team. Mitchell started in all games he appeared in, leading the team to a compiled 23-3 record, winning two Southeastern Conference (SEC) Championships. In 2009, Mitchell was inducted into the Alabama Sports Hall of Fame. 

Mitchell was taken in the seventh round of the 1973 NFL Draft by the San Francisco 49ers, but his playing career was cut short. Instead, he took on a full-time coaching position under legendary coach Paul "Bear" Bryant from 1973-1976, becoming the first African American football coach at Alabama and the youngest assistant coach in the country at the time. Mitchell then was the defensive line coach at Arkansas from 1977-1982 under Lou Holtz. Among an extensive list of coaching accolades, Mitchell notably became the first African American defensive coordinator in SEC history with Louisiana State University, serving four years there from 1987-1990. 

Today, the Steelers coach is the longest-tenured member of the coaching staff with 28 years of experience. Mitchell served as the team's defensive line coach from 1994-2006, before being named assistant head coach/defensive line (2007-17).