Citing from an earlier reference, that grand folklorist Katherine Briggs ("A Dictionary of Fairies," 1976) describes Duergars as "the worst and most malicious order of fairies"; being "black dwarves of the north of England, always full of malice and the enemies of mankind." (a bit like Yanks in Iraq then!) A tale is told of them in F. Grice's "Folk Tales of the North Country" (1944), pp.130-33...
Dr Margaret Murray in her "God of the Witches" published in the 1930s suggested that the little people, or fairies or dwarfs etc originated in an earlier tribe/race being driven back to inhospitable regions of Britain by later invaders, and forced to live in exposed moorland places or uninviting dark forests. She suggested they would have been more slight of build than the invaders/settlers and would have worn clothing that allowed them to blend into their surroundings as a defensive measure.