Useful Information about Drivers for Interconnects

During benchmarking/testing of VMI 2.0 Beta, we came across some interesting things about different interconnect hardware and drivers installed on our cluster of Dell Poweredge 2650 machines running Linux kernel 2.4.18-27. Some of experience we think users of VMI might find helpful are mentioned here on this page.

1. Ethernet Driver

We benchmarked VMI for the TCP device using 3 different Gigabit Ethernet NICs (Broadcom Gigabit Ethernet, Netgear Gigabit Ethernet and Intel epro1000 Ethernet) with their corresponding drivers (tg3, ga622, e1000). Broadcom gave markedly higher bandwidth (~117 MB/sec) compared to Netgear ( ~96MB/sec) and epro1000 (~87MB/sec) for large messages. The latency for small messages for Broadcom and Netgear was very close; ~50 us for Braodcom and ~44 us for Netgear. For epro1000. however, latency went up to 120 us for small messages. The overall performance, therfore, was better for Broadcom than any other NIC. However, the tg3 driver than comes with Linux 2.4.18-27 kernel has a bug that causes the machine to either halt or crash. Until this problem with tg3 is resolved, using Broadcom GigE with tg3 for the best performance might not be too tempting.