"Initial inspections at every factory will be completed at the latest within nine months, and plans for renovations and repairs put in place where necessary," a statement from the pact's steering committee said.
Western retailers, including Carrefour, Primark and Tesco, started signing up to the Accord on Fire and Building Safety in May to improve shocking factory conditions in Bangladesh, the world's second biggest apparel maker, with clothing accounting for 80 percent of its exports.
A headquarters to oversee implementation of the pact will be set up in the Netherlands and inspectors will aim to "identify grave hazards and the need for urgent repairs," according to the statement, giving details of the pact.
The deal requires top retailers to underwrite renovations and make a two-year commitment to the factories where renovations will be undertaken.
Labour umbrella groups, including Swiss-based IndustriALL, stepped up pressure on retailers to sign the agreement after the nine-storey building crumbled on April 24, causing one of the world's worst industrial disasters.
"Our mission is clear: to ensure the safety of all workers in the Bangladesh garment industry," said Jyrki Raina, general secretary of IndustriALL.
The task of inspecting and improving factories could prove hugely daunting. A survey by a prestigious Dhaka-based engineering university last week found nine out of ten Bangladeshi garment plants are risky structures, and many were built without qualified engineers.
The Bangladesh Garment Manufacturers and Exporters Association (BGMEA), which represents 4,500 garment factories, initially welcomed the accord, saying it reflected the retailers' long term commitment to the country.But in recent weeks manufacturers have criticised the organisers, saying the BGMEA should have been brought on board.
"They should have definitely included the BGMEA and the knitwear manufacturers in the accord and its decision-making bodies. After all, it's our factories they are going to inspect," BGMEA vice-president Reaz-Bin-Mahmood told AFP.
Scott Nova, head of the US-based Worker Rights Consortium, told AFP the BGMEA was not included because "this agreement is focused on the responsibility of the brands to ensure that factories are made safe".
While leading European retailers have joined the agreement, American brands such as Walmart and Gap have snubbed the accord and opted for self-regulation.
Walmart, the world's largest retailer and one of Dhaka's top buyers, has promised to inspect its Bangladeshi suppliers and publish the results, while Gap says it launched its own drive last October.