Communities Secretary Eric Pickles said the number of signs was damaging the character of towns and villages.
Mr Pickles said: "Our streets are losing their English character. We are being overrun by scruffy signs, bossy bollards, patchwork paving and railed off roads, wasting taxpayers' money that could be better spent on fixing potholes or keeping council tax down. We need to 'cut the clutter'."
But Richard Kemp, the Liberal Democrat vice-chairman of the Local Government Association, told the BBC that decisions about street furniture were for councils, not ministers, to make.
"One man's clutter is very much another man's simple signing," he said.
"In our shopping streets in particular, we have to get a balance between pedestrians, residents, business needs and motorists, and that's a local decision, not something that a secretary of state should be involving himself in."