FIBRE CHANNEL PERFORMANCES WITH IBM EQUIPMENT
For application in 100 KHz trigger, the conclusion of these measurements is that today, Fibre Channel does not satisfy our application's requirements. We measured a minimum overhead for a parallel application in which processing tasks will interleave with communication processes. Assuming zero processing time, we can state for the combination of hard- and software under scrutiny (based on general purpose workstations like IBM RS/6000-560s interconnected to a Fibre Channel Fabric) a maximum aggregate working frequency of around 1.5 Khz (it corresponds to an overhead of around 600 ms. measured with TCP/IP). It is also important to stress the fact that the communication performances are correlated with the power of the host workstation inside which the Fibre Channel adapter is located. Working with IBM RS/6000-590s, we already measured an overhead reduced to 375 ms [5]. In that case, it would imply a working frequency of 2.7 KHz.
For Low Energy Physics applications, the conclusions are more optimistic, since the bandwidth that was measured for a point to point communication satisfies the requirements of the experiment. This experiment will create indeed very short events (about 500 bytes with fragments of about 50 bytes), which can be grouped into bigger events (called super events) allowing to fully exploit Class 1 service of the Fibre Channel Standard (dedicated connection). Connectionless services (Class 2 and Class 3) could help the design of the software and probably make more efficient the use of the switch, but unfortunately it was not possible to test it with the equipments and the software that was available.
The overhead is a critical issue for applications in which small messages have to be transmitted, like for instance in our High Energy Physics application (remember that our largest message is only 1 Kbyte). We will not use TCP/IP for our eventual implementation, but some light-weight protocol, expected to improve considerably the overhead (mostly for High Energy Physics) and the bandwidth (for both applications). Using Raw Socket as protocol, we were a bit disappointed, because we did not observe any improvement over TCP/IP, and moreover, this protocol did not seem to be very robust. Of course, the Raw Socket application we used was a preliminary version and we can expect a performance improvement in the next release, since Raw Socket is by-passing TCP and IP layers. Improvements are mandatory for both physics applications. The only advantage in favour of the Raw Socket protocol (versus TCP/IP), at that time, is that the CPU utilization is half when transmitting large messages: for instance for sending around 64 Kbytes, the CPU utilization was around 90% with TCP/IP and less than 50% with Raw Socket.
When several independent communications were performed in parallel (loading of the switch of approximately 30 percent), we observed a maximum degradation of 5-10 percent on the performance of each communication when it was relative to the transmission of small messages. In case of large messages, no degradation was measured. Such a test should be extended to a fully loaded switch.
We would like to have confidence in the future evolution of the technology for improving considerably performances. This seems easier for high throughput, more speculative for hypothetical very low overhead. For short term future, commercial products like Fibre Channel may more typically be needed in general-purpose applications, for which high throughput is of greater interest than low overhead.
Generated with CERN WebMaker