Check out Edwin Giddings Off Licence at 15 St. John Street in Devizes. Not sure how good their cider is now but they've always got lots of other goodies to eat and drink (they do a great Wiltshire ham and if they still sell Welsh ice cream get a bucket of that as well).

The Wiltshire Heritage Museum is a bit further down the road.