John Garland says (in his 2007 'Haunted Somerset'):
Hill climbers and ramblers often report strange and 'distant voices' and on occasions feel as if 'being watched' by unseen eyes in and around Porlock's Berry Castle, situated on a lower ride of one of Exmoor's combes, acclaimed for its scenic beauty.
The Somerset HER says that this site is likely to be an Iron Age / Romano-British era hillslope enclosure. It might never have been finished (you know how these DIY projects drag on). Another small enclosure exists up the slope to the south west. The record suggests the land is open access here.