Sri Lanka's The Sunday Times newspaper said the project has been asked to shift by Urban Development Ministry Secretary Gotabhaya Rajapaksa, brother of President Mahinda Rajapaksa following the advice of a consultant brought by the World Bank.
"He showed me this land and said if you are building anything on it, it must be in the form of two towers so that you can see the Beira Lake through them," the newspaper quoted Rajapaksa as saying.
"He said you must also especially be able to see the Lotus Tower which is now coming up."
The Lotus Tower is a state-run project built with Chinese involvement.
The investors have been asked to shift to another location on D R Wijewardene Mawatha, where state land was available, the report said. These included land owned by Sathosa, a state trading agency, fisheries ministry and the port.
Rajapaksa had said the investors had a choice of two towers or shifting. The designs had not yet come to the urban agency for approval.The land was recently transferred to the project on a long-term lease but the report said had been first given to the Rank Holdings for a car park before Rajapaksa's time.