Sitting on the fence

What is the origin of the phrase "sitting on the fence"?

I think the allusion is fairly straightforward. If I sit astride my garden fence, one leg either side (I imagine it would uncomfortable, so I don't do it often, or indeed, at all) I am neither on one side nor the other, but 'distributed' evenly between the two, and can, with equal ease, drop to either side.

DFG

There may also be an implication that the politican or whoever is on the fence, is trying to decide which side to jump off.