We have demonstrated a large packet switching system, based on the DS link technology. The network is performing reliably, and has provided quantitative measurements of the performance of 2-dimensional grid, torus and Clos topologies. The results show that, although grid and torus networks are easier to physically implement, the Clos networks clearly give better performance. Given the type of traffic, the required throughput and the maximum latency, it is possible to use the results presented to evaluate the suitability of a given topology to meet those requirements. The network designer needs to consider not only the average latency, but also the effect of the long latency tail occurring in packet switching networks due to contention. The measurements presented give an upper limit of the network performance obtainable but the performance will be reduced further if the network interfaces are unable to handle the low packet overheads required.
In practise, the system is extremely stable and measuring the upper limit of the error rate was governed principally by unstable Ethernet interfaces and mains power failures. The work presented here will be extended to cover other topologies and the application of IEEE 1355 as a switching technology for other interconnects (for example Ethernet) will be investigated.
We have shown that the event rate achieved with a 512 node Clos applied to the SCT subdetector of the ATLAS level two trigger is about 120 kHz. Work on other subdetectors and trigger architectures is on-going.
Initial experience with GBaud HS links and switches is encouraging.