The pope said he had been particularly struck by a headline saying workers at the factory near Dhaka were being paid just 38 euros ($50) a month.
"This is called slave labour!" the pope was quoted by Vatican radio as saying in his homily at a private mass in his residence to mark May Day.
More than 400 workers have been confirmed dead and scores are missing in the collapse, which occurred in a suburb of the capital Dhaka last week in the country's worst-ever industrial disaster.
"Today in the world this slavery is being committed against something beautiful that God has given us -- the capacity to create, to work, to have dignity," the pope said at the mass.
"How many brothers and sisters find themselves in this situation!" he said, as protesters in May Day demonstrations around the world rallied against unfair work conditions and unemployment.
"Not paying fairly, not giving a job because you are only looking at balance sheets, only looking at how to make a profit. That goes against God!" the pope said in his strongly-worded address.
The Argentine pope, formerly the archbishop of Buenos Aires Jorge Bergoglio, became a powerful voice on the side of the poor during his homeland's devastating economic crisis.
Since being elected pontiff in March, he has repeatedly called for the Roman Catholic Church to be closer to the needy and has said he wants "a poor Church for the poor".
The 76-year-old later spoke to thousands of followers in St Peter's Square, urging politicians to fight unemployment and calling for greater "social justice" against "selfish profit".
"I call on politicians to make every effort to relaunch the labour market," he said in his traditional weekly address.
"Work is fundamental for dignity," he said.
He spoke of "labour market difficulties in various countries" -- an apparent reference to the unemployment crisis afflicting Europe.
Unemployment is often caused by "an economic conception of society based on selfish profit outside the bounds of social justice," he said.
The Vatican has often been sharply critical of unregulated capitalism, particularly in recent years during the global financial crisis.
"We do not get dignity from power or money or culture, no! We get dignity from work," he said, adding that many political and economic systems "have made choices that mean exploiting people".