Jon Stewart and Stephen Colbert nearly left Comedy Central in 2012

Former "Daily Show" host Jon Stewart and former "Colbert Report" host Stephen Colbert almost left Comedy Central in 2012 amid heated contract negotiations, according to Chris Smith's new behind-the-scenes book, "The Daily Show (The Book): An Oral History."

Variety reported that the book goes into great detail about how both hosts were engaged in tough contract negotiations with Comedy Central owner Viacom's then-CEO, Philippe Dauman.

Dauman took a particularly hard stance on Colbert. Luckily, Colbert and Stewart shared the same agent, James Dixon. Dixon decided to tie Colbert and Stewart's negotiations together to gain leverage.

For Stewart, he and the company couldn't agree on the amount of time he can take off from the show to direct the film, "Rosewater," in 2013.

"I don't think Philippe in any way saw what we do as special," Stewart said in the book. "As far as he was concerned the star is the real estate, and whether or not we are the ones who carved out that real estate and made it valuable is not important to them."

When it was clear the parties couldn't come to an agreement, Stewart told their agent he was done and Colbert followed his lead.

"How close can you get to it all falling apart? I mean, Jon quit and we were a package," Colbert said.

That was enough to shift the negotiations. Dauman called their agent and said the company would give the men the deals they wanted.

Stewart would stay on with "The Daily Show" until 2015. He currently has a production deal at HBO, with an upcoming digital program with HBO planned.

Colbert extended his contract for "The Colbert Report" to 2014. In 2015, he replaced David Letterman as the host of CBS's "The Late Show."

Earlier this year, Dauman was ousted from his chief executive role at Viacom amid a bitter dispute with owner Sumner Redstone.