Can't live with 'em, can't live without 'em

Does anyone know the origin of this saying? I assume it probably originated by probably a man referring to a woman - or possibly vice versa.

The character Flounder said that in Animal House . It was already a well-known saying by then. So all I know is it's pre-1978.