Sarsen is exceptionally hard, the Pollisher on the Marlborough Downs still has a shiny surface and could be thousands of years older than Stonehenge.
The weathered appearance in mostly due to solution weathering that occured before the sarsen was lithified, ie before it was silcreted to become as hard as it is now.
The stones of Stonehenge are not much more ragged and weathered looking now than they were when they were erected. Fanciful reconstructions a la Inigo Jones, where they look all Roman and straight and shiny are just that, fanciful.
If I had the time I would scan a copy of my university dissertation 'The Sarsen Stones of the Marlborough Downs' in which I talk about the formation and subsequent history of the Sarsens.
Anyway, any wear and tear due to Stonehenge being used as a big granary would be very evident today.