Here's why Warren Buffett is giving his billions away

Warren Buffett was dialing for dollars at the inception of The Giving Pledge, a philanthropic initiative he created with and Bill and Melinda Gates.

The pledge asks the world's wealthiest individuals to dedicate the majority of their wealth to philanthropy.

One fellow he spoke to was on the fence, and his wife wasn't keen on it, Buffett recalled at the 2016 Concordia Summit in New York on Tuesday. But their children asked them to do it, and they agreed to give their fortunes away.

That memory stuck with Buffett, who asks those who agree to the pledge to write letters explaining why they've chosen to spread their wealth, rather than keeping it in the family.

He doesn't want his grandchildren to sit on their laurels, knowing that grandpa's wealth has them set.

"The world delivers very unequal results. I am a white man, born in the US, and I was wired to do well in capitalistic world ... but it shouldn't enable my Buffett grandchildren to do nothing but to fan themselves," Buffett said.

Here's his pitch to billionaires:

"I'm sitting with a bunch of certificates of Berkshire Hathaway that I bought 55 years ago. They're sitting on a safe deposit box, they have no utility to me. They can't do anything for me nor my family. Here's a lot of money that has no real utility to me but has enormous utility to hundreds and millions of people around the world- for education, research, healthcare. They can change the lives of other people but not for my life- anything I can buy, I have. So there's a million use for that money that can really change people's lives."

There are now 156 individuals signed up to the pledge, including prominent figures in the finance and tech world like Carl Icahn and Sheryl Sandberg.

When asked if Buffett's tried to recruit presidential candidates Donald Trump and Hillary Clinton, he quipped:

"I don't think the Clintons are on the Forbes list. And Trump was on the Forbes list, but he seems to have an active argument about exactly where he should be placed. He's not one I've called. I'm not sure whether Bill's called him or not. We've probably called a majority of the names, but maybe we haven't gotten to the Ts yet… Maybe he'll call us!"