You know us country types, anything to play up to the townies' idea of us as stupid, so we can laugh at their gullibility when they go home.
But apparently, it was a common belief in East Anglia (and probably elsewhere - the Growing Stone in South Wales) that pebbles grow in the soil into large stones.
The Blaxhall stone is on the land of Stone Farm, Blaxhall. Apparently it was only the size of a small loaf a century or two ago. But now it's about 5 tons. It's good soil round here.