A $9 billion hedge fund manager gives his best career advice to Wall Street hopefuls. Here's the 1 thing he wishes he'd done differently.

Spencer Platt/Getty

• David Abrams, who manages nearly $9 billion with his fund Abrams Capital, rarely makes public appearances.
• He gave career advice to attendees at a New York conference on Friday that included students looking to land jobs on Wall Street.

David Abrams, who has worked on Wall Street for over 20 years, manages nearly $9 billion at his Boston-based hedge fund Abrams Capital.

He also rarely makes public appearances, with hedge funders describing him as a "unicorn," because he's often talked about, but never seen. He gave a rare address on Friday at a conference in New York for Project Punch Card, a charity aspiring to improve access to investing and finance jobs for underrepresented people

For students attending his talk, Abrams told them to pick their future jobs by focusing on the "half-dozen people you'll be working with everyday," and "not prestige of the firm."

Read more: $9 billion hedge fund manager David Abrams, who rarely makes public appearances, lays out his investing strategy - and cautions against being too patient

"It's not about if your parents or mother-in-law have heard of the place you'll be working," he said.

He also advised students to do the opposite of what he did from an education standpoint. Abrams graduated from the University of Pennsylvania with a degree in history and then worked for Seth Klarman at Baupost Group in 1988. He started his firm in 1999.

"I would have taken some business courses, that's for sure," he said about things he would have done differently.

"I would not advise people doing what I did, which is entering the business knowing absolutely nothing."