It's rather an oblique issue on which to judge what Neolithic people did with stones. Maybe boats are more easily dragged, being the shape they are. And what about strength - maybe the skin of the hull wouldn't be strong enough to be undamaged by levers.
If stone rowing existed perhaps it was only suited to stone shifting, and soon forgotten after the stone age ended. The whole hypothesis is that it's forgotten technology, so we've no more reason to doubt it's existence if the Vikings didn't know about it than if the R*mans didn't, which they didn't.