Well I have a book called "the Ruined Abbeys of Britain by an American Frederick Ross it must be late 19th C - usual over the top and florid style writing but lots of historic facts....
"in the year 177, William Malmesbury informs us, ............... Saints Phaganus and Duravianus - (who baptised the king) and had a grant of the isle of Avallon. The old wattled church having fallen to decay, they rebuilt it of stone, and dedicated it to Peter and Paul" the king placed 12 anchorets on the island in connection with the new church" and they lived on water and bread for five days, allowed a little wine on Sunday! and he got rid of the druids as well...
skipping through to anglo saxon times and the newly arrive Benedictine order - 630 Paulinus, Bishop of Rochester as abbot, he found the church, the "vetusta Ecclesia", in great state of disrepair, and at his own expense caused it to be cased from foundation to roof, all around, with planking, and removing the thatch, roofed it with lead".
So therefore it seemed to have been both stone and timbered...