I've been told that it's remaindering spoil from the digging of the 1776 shaft on the basis that if you dig a hole you can't get all the spoil back in the same hole.
Also that -
When he arrived at Silbury in 1849 Merewether describes a bank of this spoil atop Silbury, two decades on there was just a mound left which is what you see in the drawing - the 1849 tunnel saw material from the shaft drop into the tunnel (sounds familiar) and with more people visiting due to the attention of the dig the hole up top was dangerous so was topped up by the spoil.
The material in the hole continued to settle and due to a collapse in 1916 the 1849 tunnel re-opened and the hole appeared uptop again. The tunnel entrance was sealed in 1923 and the remaining mound that you see in this drawing was used to top up the hole in 1936.