Communication benchmarks with the Ancor Fibre Channel Fabric

Conclusion

All the results presented in this note represent maximum reachable values when implementing a parallel application in which some processing tasks will interleave the communication processes. The best measurements we obtained, are summarized below, according to the hardware configuration used:

- communications between HP workstations: the overhead of the system is around 25 microseconds, and the speed can reach 7.7 MBytes/s for transmitting 64 KByte messages (for a 64 Kbyte socket buffer size).
- communications between RS/6000 590 and HP/9000 700 workstations:
* the overhead of the system is around 25 microseconds, and the speed can reach 7.2 MBytes/s for transmitting 105 byte messages (for a 64 KByte socket buffer size), when the IBM machine is receiving data;
* the overhead of the system is around 25 microseconds, and the speed can reach 8 MBytes/s for transmitting 106 byte messages (for a 64 KByte socket buffer size), when the IBM machine is sending data.

For a IBM-HP communication direction, the shape of the measured performance chart (either speed or time) is alarming: it is not continuous. For instance, a drop of several hundreds of percent in the performance has been reported between sending a message of 128 bytes and a message of 1024 bytes (for 32KByte and 64KByte socket buffer sizes). Moreover this result is completely deterministic: it can be reproduced at any time. Similar behaviours when measuring communications involving an IBM workstation, in a similar experiment using the same TCP implementation have been previously reported (see [1] for more details). In that case, for a given parameterization of the socket buffer size and the message size, a deadlock can occur between the two machines, and it is then deblocked by time expiry on the receiver side. We believe this is a likely explanation, which unfortunately can not be verified here, as no network analyser for Fibre Channel is yet available.

The overhead of the system (interface plus fabric plus all the software) is around 25 microseconds. The overhead is defined as the limit for the communication time when the message size to be transferred tends towards zero. For our application, this definition gives an over optimistic expectation of overhead for protocols which wait to merge messages before sending. In that case, for instance instead of sending 10 times a 100 byte message, the system will send 5 times a 200 byte message. This problem has also been already mentioned in [1].

Fabrice Chantemargue - 30 AUG 94

[Next] [Prev] [Top]

Generated with WebMaker