"People have been choosing to live in Alphamstone (Essex) for a very long time... it was already special in the Bronze Age; the church is built on a barrow or tumulus which excavations in the early twentieth century dated to about 1500 BC. A sacred well was discovered then, and three fine cinerary urns which are now in Colchester Museum. Even more significant are the boulders around the churchyard, perhaps the remains of a stone circle and evidence of ritual importance. These sarsen... stones were probably deposited by glaciers at the end of the last ice-age, although popular folklore holds that they were dug out of Wiltshire or Dorset downland, brought perhaps hundreds of miles by unreliable transport on rough tracks, and literally manhandled into place..."*

There's also the more recent accounts of small stones and turf being brought considerable distances to Silbury and deposited on the primary mound there (and perhaps also in the later phases of the monument's construction). In more recent times there's the removal of the Stone of Scone** from Scotland to Westminster Abbey (and then back again). You may have touched on an important point here - ie that stones from various locations were seen as having special properties or powers and their removal and relocation elsewhere would somehow transport those properties and powers to the new location (and its inhabitants).

* http://www.ansellsfarm.co.uk/alphamstone/alphamstone.html

** http://en.wikipedia.org/wiki/Stone_of_Scone