Phil Quinn (Third Stone 26) mentions that fairies used to live here at Cadbury. But they couldn’t stand the noise from the new church bells – they buried their gold and left (perhaps too heavy to take with them?).
The folklore mentioned by Purejoy below is included by Ruth Tongue in her 1965 ‘Somerset Folklore’ – she also says that the local people (unsurprisingly) called the camp ‘Camelot’.