What is the purpose of dual gigabit ethernet ports?

I use one Ethernet port to connect to my ADSL modem/router and the other to network directly to my G5.

You can combine them to get an aggregated 2 Gb bandwidth. They're very useful for multi-homing your computer on two different networks that must be isolated. For example, I use one for my work VPN and one for my Internet access. Everything works transparently on my computer, whether I'm accessing it over the business network or Internet. If you run virtual machines you might want one on a separate network interface.

The two network ports on the Mac Pro as 1Gbps ( giga bit per second ).

A typical broadband connection is in the range of 1-20 Mbps, and at most 50Mbps ( mega bit per second ).

This means that the internet connection is going to use, at most, 5% of the port bandwidth of a single port on the mac, so using both ports isn't going to make any difference to performance of internet activities.

Two ports become useful if the machine has to connect to two different networks, eg. LAN and Storage, or if you need to transfer large files within a LAN between multiple machines/file servers where you can aggregate the ports to give 2Gbps of bandwidth.

Slightly off-topic, but related and the answer may be useful:

Would a NAS drive via Gb Ethernet be faster than FW800?

GB ethernet does have more bandwidth than FW800, but I don't know how things would work with protocol overhead. The quality of the ethernet controller used in the NAS would also be an issue.