Its the idea that the distance makes the difference that I don't really get.

It seems to be acceptable to some to imagine they picked them up say, 15 miles away, where a glacier might have dropped them, but a much longer distance seems to be totally out of the question! Why should that be so?

Surely, if you accept they may have brought them 15 miles, then you can accept it may have been a vastly bigger project than you imagine, and that they brought them from much further?