The church is dedicated to All Saints.
The hole in the stone had water in it so I didn't look to closely. It seemed to go off to the right into the stone but not that far in. However I didn't look that closely so...

For any church fans, have a look at the saxon St Marys just across the stream. The christianisation of this site is older than Alton Priors church.