Communication benchmarks with the Ancor Fibre Channel Fabric

Results using TCP/IP

The program used to test out the performance is `ttcp'. This program is in the public domain and was originally created by the US Army Ballistics Research Lab (BRL).

There are three input parameters to this program, which can be chosen independently:

• - the size of the message to be sent,

• - the number of times the message is sent (to perform an average measurement),

• - the socket buffer size.

The number of times the message is sent is equal to 105 if the message size is lower than 512 bytes. In the case of a message size greater than 512 bytes, the number of times the message is sent is equal to 104. These values have been chosen like this in order to provide relevant time measurements.

Seven values (in bytes) for the socket buffer size have been tested: 1K, 2K, 4K, 8K, 16K, 32K and 64K (1K is equal to 1024 bytes)

Graphs (see figures 2a and 2b) show how the communication time and therefore the communication speed scale with the size of the message to be transferred and the socket buffer size. Figures 2a and 2b display results when the 590 workstation acts as a sender and the 250 workstation acts as a receiver, and vice versa respectively.

Figure 2a: Results obtained for transmission using TCP (Speed, Time).

Figure 2b: Results obtained for transmission using TCP (Speed, Time).

From these graphs, we can conclude:

• - the best performances have been obtained when the RS/6000 590 is the sender and the RS/6000 250 is the receiver (the 590 workstation is more powerful than the 250).

• - for both directions of communications, the shape of the measured performance chart (either speed or time) is alarming: it is not continuous. For instance, a drop of several hundreds of percent in the performance has been reported between sending a message of 900 bytes and a message of 1000 bytes (for socket buffer sizes greater than 8192 bytes). Moreover this result is completely deterministic: it can be reproduced at any time. We note that similar behaviour has been previously reported in a similar experiment using the same TCP implementation, with another type of medium (see appendix). In that case, for a given parameterization of the socket buffer size and the message size, a deadlock can occur between the two machines, and it is then deblocked by time expiry on the receiver side. We believe this is a likely explanation, which unfortunately can not be verified here, as no network analyser for Fibre Channel is yet available.

• - the overhead of the system (interface plus fabric plus all the software), under TCP/IP is around 70 microseconds. The overhead is defined as the limit for the communication time when the message size to be transferred tends towards zero. For our application, this definition gives an over optimistic expectation of overhead for protocols which wait to merge messages before sending (see appendix). In that case, for instance instead of sending 10 times a 100 byte message, the system will send 5 times a 200 byte message.

Generated with WebMaker