It's a lovely walk in. I generally like to end walks at Avebury from elsewhere on the Downs. It's always such a contrast arriving there, as you can walk for hours without seeing anyone, then Avebury comes as a bit of a shock! The route along the Ridgeway is one of the less quiet options, but still peaceful.
Here's an old blog of a walk from Barbury - Avebury: https://www.themodernantiquarian.com/post/127242/ridgeway-to-heaven-barbury-castle-to-avebury-21-may-2011