Origin of "lose the rag"

anyone got any information on the background to the phrase "lose the rag",any info would go toward solving a familly dispute.
Regards
Darrell Mccloskey