I think it would depend on the size of the largest stones being used.

If you think of long cairns, they could have been accumulated over a period of years, as mobile groups passed through the land. Perhaps using the cairn as a focal point in the landscape for the purpose of meeting other groups at agreed times of the year.

Then there's always the blurry line between harvesting 'encouraged' wild crops that have been partially selected by human activity (as described in detail by Mr Mithen), and full-blown sessile farming. That could make organised shifting of larger stones more feasible for mobile populations.