But you might also spy him at The Whales, an Asian-fusion night spot on New York's bustling Lower East Side, donning a T-shirt and a white pair of headphones.
Or, Beautique in Midtown - dubbed a "millionaire playpen" by the New York Post.
But Solomon isn't so much playing with fellow millionaires as he is playing for them - and countless others as well.
The investment banking stalwart has of late been moonlighting as a DJ on the Electronic Dance Music scene, sharing the stage with stars like Paul Oakenfold and, last month, even spinning vinyl at Willow Studios in Los Angeles for the Electronic Music Awards.
It's an unorthodox hobby for a high-powered finance exec, a breed that more commonly spends its free time hitting the golf links, curating art, cavorting at charity galas, or jetting off to tony weekend homes in the Hamptons or Martha's Vineyard.
But Solomon has been an audiophile his whole life, as he recently explained in an episode of the podcast "Exchanges at Goldman Sachs."
"I always loved music," Solomon said. "I was a very analog guy, when I was in college, I must have had over 1,000 LPs, vinyl albums."
A lot has changed since Solomon's college days in the early 1980s. The digitization of music over the past 15 years has facilitated easy access to artists and genres that might previously have remained muted or cordoned off to outsiders.
"I think this year I just saw a statistic that 63 percent of all music is being consumed in a streaming form now," Solomon said. "The ability to kind of curate, study, listen to, try different things, expand your horizons, it just went up exponentially. "
That's how you get a guy like Solomon, age 55 and ensconced in the world of high finance, manning the turntables at beach parties in the Bahamas overrun with bikini-clad millennials.
He appears to have been reeled in as much by the business angle and the platforms commanded by top DJs as the sound itself.
Here's Solomon:
"And so as that happened I started getting more and more interested in different kinds of music. Music I really didn't have a lot of experience with. And five, seven years ago, I started really kind of taking note of club and EDM music and what was happening with all the electronic music, and I said: This is really interesting. Big business. And started looking at it, and I said: You know, I like some of this music. And started playing around with it, started reading about these DJs that really had these incredible platforms, and I said: You know what? This looks interesting. And kind of stumbled into it as a hobby, and now I just do it for fun."
He indulges his side hobby about once a month.
The Wall Street veteran says finding and exploring a passion is crucial to longevity in a career known for grinding people down and burning them out.
"If you can't find a way to have passions and pursue those passions and mix them into your professional life and your personal life in some way, shape or form, it's just harder to have the energy to keep on doing this, and to keep moving forward professionally," Solomon said.
Get the latest Goldman Sachs stock price here.