The correct term for that flaking is 'spalling' and it's generally attributed to freeze-thaw conditions. It can also be found in old or low grade brickwork and reinforced concrete. It's perhaps where the term 'flaky' originated.