Depending on what the composition of the paint was on a medium timescale you'll find lichen feeding on the surface materials. When the lichen breaks off or is grazed - and sheep'll eat it if it's snowy and they're desperate - then the paint is gone too. I wish somebody could get into the Upper South Tyne Valley and get to look at these ochre-washed walls. Fifty years later and there's not much left (all covered in rusty lichen). There's a fine-grained fallen menhir also that has been painted along the narrow faces - I've two pictures of it that show a clear soil line, but little else.