Aye, was actually thinking about structural above ground remains (though thanks to tiompan and HD for their links).

Seahenge (before it was ‘relocated’) could be seen as a wooden, in situ structure if not an actual building. Having said that what constitutes a building anyway. Seahenge has a worked wooden wall, an entrance that could be opened and closed and a central monumental feature. That would surely place it into the building category and, as such, a fair contender for the oldest wooden building in the world?