Yes, of course you are right, we will never be able to prove it. However, I think the idea didn't occur to any modern person (except Gordon) because we just don't have a need to move big stones anymore, or if we do then we hire a crane/lorry/forklift, whatever. Hollywood has left everyone with the image of hundreds of slaves being lashed while they struggle to drag huge stones by rope and subconsciously we cling to that notion, even though it's pure fiction.

On the other hand, imagine you are around in 2500BC and there's a big lump of stone lying on (perhaps half buried in) the ground and you think to yourself, that would look rather nice standing up near my village. How would you go about even getting it out of the ground and onto a sled/track/rollers, whatever, in order to drag it? You'd use levers. For a big stone you'd need lots of people with levers and you'd have to co-ordinate the effort. A shared fulcrum log would make a lot more sense than individual bits of stone or timber. You'd soon realise that not only can you lift the stone, but you can move it sideways quite easily. We know that the lever method is more efficient than dragging, so why would you then bother to even consider doing it in any other way? Or if you did, you'd soon come back to using levers when you realised how much effort they would save.