A tangent on this from listening to Radio 4 yesterday - one commentator on a programme about legends of giants in Cornwall suggested that the 'giants' were Scandinavian invaders, while the 'pixies/piskies' were Picts.
The idea was that, if you're a Pict (average height c. 5 feet), the average Viking is fairly tall, and as with all legends, is likely to grow in the telling. The fact that invaders would have been likely to have occupied easily-defensible positions such as hilltops would also fit with so many of the legends of giants depicting them as living at the tops of hills.
The Picts, meanwhile, are small from a Viking viewpoint, and also likely to get smaller as the legend is told, and their familiarity and affinity with the landscape could appear as 'magic'. There may also be an etymological connection between 'Pict' and 'Pixie'.
Needs some work, but it's an interesting theory.