It could be the Ethernet port specs. According to the Mactracker database, all iMac G4s have 10/100 ports, typical of Apple "home" computers of the time. Their "pro" models had the much faster 10/100/1000 port, sometimes called "gigabit ethernet" or "gigE."
My late-model G4 tower has the 10/100/1000 port and my wife's eMac, like G4 iMacs, has only 10/100 ports. Both computers are sharing our 50mbps cable internet service. The G4, as well as a PowerBook G4 and a MacBook Pro that also have the gigE port, can find servers that deliver full 50mbps speeds. The eMac can do no better than about 5mbps, commensurate with it port's max capacity being 1/10th that of the gigE ports on the "pro" models.
Ports cannot be upgraded as they are built into the logic board.