The common masses.
This term is of Greek origin and a literal translation from the Greek οἱ πολλοί is 'the many'. There are many examples of it in print in its original Greek form, dating back to the 17th century. The earliest known is a 1668 essay by John Dryden - Of dramatick poesie:
"If by the people you understand the multitude, the οἱ πολλοί"
Many believe that this term was adopted into English by the American writer James Fenimore Cooper. He did use 'hoi polloi' in his Gleanings from Europe in 1837, but before then it was in common use by those whom we might expect to have been familiar with classical Greek - scholars of Oxford and Cambridge universities. For instance, the various classes of degree of Cambridge's Mathematical Tripos were Wranglers and Senior and Junior Optimes (what we would now call First, Second and Third Class), followed by Hoi Polloi - also called Poll Men or Polloi Men. The first record I can find of this in print is from a listing of Cambridge degree awards in The Times, 22nd January 1833. Under a heading of Cambridge List of Honours and Degrees - Mathematical Tripos are listed the names of those who had been awarded various classes, including a rather long list under Hoi Polloi.
There are two linguistic points of interest concerning hoi polloi. The first is whether or not to precede it with 'the'. Some argue that, since 'hoi' means 'the' in Greek, then 'the hoi polloi' translates as 'the the many', so we should omit the article and just say 'hoi polloi'. Others argue that this is merely pedantic, not to say inconsistent with other uses of articles inherited from others languages; for example, alchemist, which comes from the Arabic, where al means the - and yet no one complains that the alchemist is incorrect. Whatever your views on that, it's a fact that 'the hoi polloi' is so widely used (not least by Dryden, as we see above) that whatever grammarians say about it won't alter its general usage.
The second point is that some believe hoi polloi to mean 'the upper classes'; for example, this from the Chicago Daily Herald, October 1984:
"Brent Musburger, whose talks with WGN are continuing, was among the hoi polloi in the rich seats."
This usage is possibly influenced by a mistaken association with 'hoity-toity'.