I'd have to agree, and add -
1. There was no mention of all the local graves that we'd heard had bluestones in.
2. The only one that had a few flakes in was in the ditch - hardly a surprise since there were thousands in the central area,

On the hand, if you ignore the chippings, it's quite plausible the place attracted long distance pilgrims who believed it would heal them (inter alia). Lourdes attracts more people from far away all over Europe than other purely "religious" sites.

That's just conjecture as well, but at least it fits very comfortably with how people act even today. Taking chips away yet leaving thousands doesn't. I give you the Dutch and Croatian metal detectorists that have been hard at work in English fields this very weekend as an illustration. They'll snaffle everything they find, not leave some.

See how I did that?! ;)