A bunch of fives
A fist. The fives are the five fingers.
The phrase appears in print in 1825, in Charles Westmacott's The English Spy:
"With their bunch of fives."
It is also reported as appearing, slightly earlier, in Boxiana by Pierce Egan, 1821. Boxiana is a classic work on boxing history and folklore, and just the place to look for such a reference. It isn't easy to obtain copies of it these days, so that hasn't been confirmed here as yet.