The original word for the garment covering the legs of men was pantaloon. The word pant was often used as a synonym for pantleg, that is, the covering of only one leg. In this case the complete garment would have two pants, or better, a pair. Pantaloon as a garment could be used in the singular, but most often the plural was preferred.
In the U.S. in particular, the single word "pant" came be used in the retail clothing industry (late 19th c. onward) to represent what every one else would call either "pants" or "a pair of pants," e.g., " A practical and well made pant for general sportswear" (L.L. Bean Catalogue, 1962).
Sometimes clerks would use pant alone as a part-for-the-whole expression, meaning a garment with pantlegs. But "pants" as we know them really got their start as a colloquialism in the U.S. in the 19th century. There is still a widespread feeling that "pants" are not quite as dignified as "trousers."
I mentioned pantaloon. The word has an interesting history, which I'll only touch on. It was orginally the name of a character in the Italian commedia dell'arte. He represented a Venetian type and spoke in Venetian dialect (as did the Zanni character), and wore close-fitting trousers down to the foot. (Most commedia characters were known not only by their name, their calling, their dialect, and their manner, but also by their costume.)
The French picked up on the name Pantalon (their version of Pantaleone), and passed it to the English as Pantaloon, both of which were used both for pants and for the character. Pantaloon is an old fool, whether in Italian, French or English.
This does not answer the question about using the plural for a singular garment. This is also the case with shorts (that is, short pants). After the term "pants" came to be accepted for female attire, we had also pantaloons and panties, the latter being used both in the singular and in the plural, mostly in the plural. In a panty-raid, one steals panties. If a girl on spring break accidentally drops her bikini bottom you may say, "She dropped her panties." I don't know if anyone would ever say, "She lost her panty," but I don't think so. (Panty in the singular is used mostly in trade language, like pant.)
In any case, it's a garment with holes for the legs. This is also true of short shorts and tighty whities and briefs. But the actual covering of the legs is omitted altogether, unless you dignify the origin of the legs, or stump, as a pair of legs. When you don't even get the stump, as in a thong, no one uses the plural.
So, one shirt, a pair of shorts, but NOT two shorts. Originally a pant was a pant-leg.