Athletic director Greg Byrne announced Sunday that Alabama and Johnson "made the decision to mutually part ways" after his fourth season coaching the Crimson Tide.
Johnson's buyout under a contract extension reached in August 2017 would be $8 million.
"It was an honor and privilege to work with these young men and their families," Johnson said in a statement. "This was an opportunity of a lifetime, and we truly enjoyed our experience at Alabama."
The former NBA point guard and head coach led the Tide to a 75-62 record in four seasons, including a trip to the NCAA Tournament in 2018 that broke a six-year drought. Alabama lost five of its last six games this past season, including a first-round upset by Norfolk State in the NIT on Tuesday night.
Johnson had success on the recruiting trail, landing five-star prospects in Collin Sexton and current freshman Kira Lewis Jr. Sexton left after one season and was an NBA lottery pick.
Alabama also set single-season total and per-game attendance records at Coleman Coliseum during his tenure. But a lackluster performance in the 80-79 overtime loss to Norfolk State was the final blow after late-season stumbles kept the Tide from making another trip to the NCAA Tournament.
Johnson's hiring made a splash for a program often overshadowed by Nick Saban and football. As a player, he helped the San Antonio Spurs win an NBA championship in 1999. He also played for Golden State, Seattle, Denver, Dallas and Houston during a 16-year professional career.
He was named NBA coach of the year after leading the Dallas Mavericks to their first appearance in the finals in 2005-06. Johnson coached the Mavericks for four seasons and the Brooklyn Nets for two-plus seasons.
Alabama was his first college coaching job.
Tide associate head coach John Pelphrey will serve as interim head coach.
Byrne said this "was not an easy decision," but expressed optimism that it would be an attractive job.
"There are so many desirables here at the University of Alabama, and the brand itself gives all of our teams the ability to recruit nationally," he said. "This is such a great place, and people want to be part of it."
Enjoy all things sports?
Don't miss a story! Sign up for daily stories delivered to your inbox.