It may not be man-made but it's certainly 'megalithic', and (like in Cornwall??) may well be related to the siting of tombs like the nearby Coetan Arthur?:
The Rocking Stone (we give it a title of courtesy which has survived its peculiar property) stands on the western slope of Carn Llidi, at a short distance from the cliff, and from the road leading to St. David's Head. Its height is five feet, its extreme length and breadth each six feet six inches; and it narrows somewhat towards the bottom. A little to the east there is a point of rock which may have acted formerly as the fulcrum.
Its displacement is ascribed to the Puritans by E. Lhuyd; but we fortunately possess an account of its original state in the unknown author quoted by Browne Willis:-
About a quarter of a mile from hence is the famous Stone which they call here the Shaking-Stone, Y-maen-sigl. It is so large, that 20 Yoke of Oxen will not remove it from its Place, and yet it lieth upon a Bank; and notwithstanding its Hugeness, (they say) a Child of eight Years of Age will shake it. I never saw any Child shake it; but I can move it, tho' six Men should stand upon it, with less than the Strength of one of my Hands, so much that a Man that stands upon it, would be afraid of falling.
From p24 of 'The History and Antiquities of Saint David's' by WB Jones and EA Freeman (1856). Browne Willis was an antiquary who wrote many books in the first half of the 18th century.
You can read Jones and Freeman's book online at Google Books. It's got lots of information about the location of destroyed sites in the area, which I'm sure will be of interest to those that know the area well.