No skin off my back

What is the origin of the phrase "no skin off my back"

Guessing here: skin off one's back as in being whipped?

The OED cites several instances of "no skin off my nose" and various other places, such as "bugle" for nose, "off me" and "off my ass." It did not mention "off my back." It means, of course, "It's all the same to me," or "I don't care." SS