In a Welsh manuscript from 1600* there is a list of giants and their abodes. It includes
“Clidda Gawr in the parish of Bettws Newydd, and his abode in the place called Cloddeu Caer Clidda, and that land today is called Tir Clidda in the parish of Llanarth.”
Coed y Bwnydd is near/on Clytha Hill, so it seems Coed y Bwnydd may well be Clidda’s castle. It’s now a wooded nature reserve looked after by Monmouthshire council. (There is a Clytha Castle to the north, but that’s far too new for a giant).
*Owen, H., ‘Peniarth Ms. 118, fos. 829-837’ Y Cymmrodor, XXVII, (1917) pp.115-52. Text and translation.
cited here on the Arthurian Gwent page.