The Cornwall HER says that traditionally, people used to stroll out here after their Sunday lunch (= after church?). And why not. Surely they might like to yet (despite the doubting of the HER).
Tom Thumb (so the 1621 pamphlet by Richard Johnson goes) was a very tiny person "of the bignes of my thumb." He was born to a ploughman and his wife - they'd specifically asked Merlin for a bit of assistance, and the birth was attended by the queen of the fairies. "After some boyhood adventures appropriate to his size and character, he is eaten in succession by a cow, a giant, and a fish, by which means he ends up at King Arthur's court (the fish having been caught for Arthur's table)." He sits at the round table and is the king's companion, and goes out riding with him. "For more than a century, the tale of Tom Thumb was the most widely known and popular version of the Arthurian legend in circulation in England". Which seems quite strange now. The Tom Thumb stories have international parallels too.
Later, Henry Fielding wrote a satirical book in which Tom was a giant-killer, and there was lots of murder and love-intrigue. Later on he became a more child-friendly figure.
(quotes and info. from 'Tom Thumb and Jack the Giant-Killer: Two Arthurian Fairytales?' by Thomas Green, in
Folklore , Vol. 118, No. 2 (Aug., 2007), pp. 123-140.)
But what does this have to do with the rock? Is it just supposed to be amusing because Tom Thumb is tiny and the rock is massive? Or is it related to his giant-slaying days? The rock gets a slight blob on the map in the 1870s, but only gets its name marked in the 1970s. The HER suggests that this means the name is quite recent (it being English too). But who knows.
More juicily, the HER says "There is a piece of oral folklore associated with the rock that suggests it was used as a place for sacrifice at the time of the St Just feast." Those druids eh. Though I don't know what they were doing following the Christian calendar. I notice that the St Just festival "was always held on the nearest Sunday to All Saints-day"* so maybe the druids were celebrating more of a Samhain thing instead. All a bit muddled up but never mind.
*Cornish Feasts and 'Feasten' Customs - M A Courtney, The Folk-Lore Journal v4 (1886).