We may need to have small guy ropes either side to prevent the pole from pivoting sideways, but that's a minor issue. Although the pole seems like it's acting as a big lever, it's actuallyh the rope that pulls the stone up. The pole merely acts like an A-frame keeping the rope in the correct place. It's the angle between the rope and the load that gives the mechanical advantage. All the load on the pole is compressive, so it doesn't need to be especially stout.

It's also possible to reset the angle of the pole as follows:

Suppose the pole is set initially quite low and we only manage to pull the stone from 70 to 80 degrees before the pole reaches the ground. If the back of the stone is then rammed with rubble to retain it, the end of the pole can be raise back up and the rope retensioned ready for a second go. This is not a big deal because we are only talking of a 2-3 ton load at the end of the pole.