According to W B Dawkins' 1880 "Early Man in Britain" (an excerpt online at Showcaves.com),
the 'Pin Hole Cave' is so called because people used to drop a pin in (for luck? for a wish? as an offering?) - part of the ritual being that they took away a pin left by another visitor previously.