"with all due respect"

This is a phrase I have heard used in many circumstances and with many different shadings. Used to demean a questioner or to recognize the legitimacy of a discussion partner --- I wonder -- where did it originate?