Object
Then how do you explain "He was the subject to their affection" Henry?
I don't recognise that as a normal construction. I would say either
He was the subject of their affection, or
He was subject to their affection.
Both carry a similar meaning.