A number of roads around Binfield Street in Tipton have been cordoned off after residents reported hearing a loud bang before finding nails and debris.
There are no reports of any injuries, West Midlands Police said, but streets were evacuated as a precautionary measure.
Tipton councillor Ian Jones said he was “very shocked”.
A force spokesman said: “Police have been called to Binfield Street in Tipton this afternoon following reports of a loud bang.
“Some residents have reported finding debris in the area and finding nails.
“This is being investigated by forensic officers at the scene.”
BBC WM reporter Steve Hermon, who is at the scene, said he had been told by passers-by that prayers at the mosque were usually held there at 13:00 BST but had been moved back an hour on this occasion.
Among the road closures are parts of Sedgley Road East, Dudley Port, Jays Avenue, Tudor Court, Park Lane East, Crompton Road and Victoria Road.
Tipton councillor Mr Jones said he was “very shocked” at what had happened. said: “We have always had a stable relationship between different people in this community.
“This has always been a settled community where everyone lives together peacefully.”
Mr Jones told BBC WM it could not “go unnoticed” that the incident happened on the same day of the funeral of Fusilier Lee Rigby, who was killed in Woolwich in May.
West Midlands Police are still investigating an explosion which occurred near a mosque in Caldmore in Walsall last month.
About 150 people were evacuated from homes near Aisha Mosque and Islamic Centre after a device was found in an nearby alleyway.