When ABC made the decision to pull Jimmy Kimmel Live off the air in the wake of comments Kimmel made on the show following the death of Charlie Kirk, Americans across the nation apparently were not the only ones who feared Kimmel may never return to ABC.

In his first proper interview since his suspension and return to ABC, Kimmel dropped by The Late Show with Stephen Colbert where he candidly broke down the events that unfolded the day of his suspension. Kimmel revealed to Colbert that the news came down in the afternoon as he and his team were working on the night’s planned show.

“I’m in my office typing as I usually do and I get a phone call. It’s ABC. They say they want to talk to me, this is unusual. As far as I knew, they didn’t even know I was doing a show previous to this. I have a lot of people who work in my office with me, so the only private place to go is the bathroom,” Kimmel recalled of the day.

“So I go into the bathroom and I’m on the phone with the ABC executives, and they say, ‘Listen, we want to take the temperature down. We’re concerned about what you’re going to say tonight and we decided that the best route is to take the show off the air.’ I said, ‘I don’t think that’s a good idea’ and they said, ‘Well, we think it’s a good idea.’ And then there was a vote, and I lost the vote. I walked out to my office and I called in some of the executive producers, and I said, ‘They’re pulling the show off the air,’” Kimmel continued.

After getting the news from ABC, Kimmel had to break the news to his writers and crew and recalled his wife telling him he was white, or whiter than Jim Gaffigan as Kimmel joked.

When Colbert inquired about the first thing that occurred to Kimmel after learning the news, Kimmel candidly admitted he was certain he’d never return to the air following his suspension.

“I thought, that’s it. It’s over. I was like I’m never coming back on the air. That’s really what I thought”

Fortunately, that did not end up being the case; however, it was looking like a possibility in the immediate wake of his suspension. Reports surfaced that suggested there was no way back for Kimmel, and others hinting he didn’t even want to come back to ABC following their decision. Luckily, ABC reversed its decision less than a week after pulling Jimmy Kimmel Live from the airwaves.

Kimmel’s return brought about the show’s biggest ratings in years, with his monologue from the episode being viewed by millions across YouTube and social media. While he has returned to strong ratings, it’s certainly an uncertain time for late-night and there is no telling what the future could hold given the way in which President Donald Trump continues to single out Kimmel and his fellow late-night hosts.