It is not known what caused the accident, but images from the scene showed part of a huge red crane had crashed through the mosque's roof.
Mecca is currently preparing for the annual Muslim Hajj pilgrimage.
Hundreds of thousands of people are expected to arrive in the Saudi city from all over the world later this month.
Saudi authorities began a major expansion of the site last year to cope with ever-growing numbers of worshippers.
The crane that collapsed on Friday was part of the construction work taking place at the site.
Reports suggested the crane fell during high winds. The Arabian peninsula has been has been hit by strong sand storms over the past week.
A video posted on YouTube, which could not be independently verified, appeared to record the moment the crane fell, with a loud crash heard in the background followed by panic and shouting.
Images circulating on Twitter appeared to show numerous bodies and blood on the floor of the mosque.
for more details check cheap umrah packages