The Cleveland Fed breaks down just how punishing the retail business is right now

As we've noted many times before, retail is a hard business to be in.

And while CEOs and analysts have noted this, this view has even made it into the Federal Reserve's latest roundup of notes about the economy.

In Wednesday's release of the Fed's Beige Book, a collection of economic anecdotes from each of the Fed's 12 districts, commentary from the Cleveland branch laid out just how tough the business has it right now.

First the report talked about the structural changes that are dragging on the sector:

The retail environment is changing rapidly. Chains are allocating larger shares of their capital budgets to technology and repositioning existing stores to be more attractive to Millennials. Underperforming assets are being shuttered at a faster pace.

So there you've got the concerns over constantly changing consumer tastes and the threat of internet retailers like Amazon dragging down profits.

The report also highlighted some cyclical factors that are hurting the businesses:

Vendor and shelf prices were fairly stable, though supply-side prices for selected food products and cotton apparel declined. Two contacts noted they are working on simplifying supply chains to become more cost effective. Retailers are facing stiff labor competition. Higher turnover combined with a smaller pool of qualified workers is driving up wages.

Retailers spend a significant among of their revenues on labor and so increases hit their already-thin margins in a more forceful way.

And finally there were issues that dinged sales just in the last few months.

"Consumers seem reluctant to take on additional debt, especially for large-ticket items. One contact suggested this may be attributable to financial market volatility. Apparel sales, especially cold-weather items, were lower," said the report.

And so in sum, the Cleveland Fed is saying that short-run factors such as the weather, medium-run factors such as higher labor costs, and long-term factors such as the structural shift to online shopping are all keeping retailers up at night.

A doozy of an outlook.