Between bursts, each SDPC sends a fraction of its event fragments across a Fast Ethernet switch to each of several Event-Building PCs (EBPC). The EBPCs combine all the event fragments belonging to one event and write the completed events to a local disk buffer. In that way each EBPC generates a burst fragment ("burstlet") file per SPS burst.
The burst fragment files are then picked up by the Central Data Recording (CDR) mechanism and sent to the CS-2 in the CERN computer centre where they are assembled to complete burst files, processed by the level-3 trigger, reconstructed and written to permanent tape storage.
In the current configuration of the readout systems each SPS burst generates
ca. 270-280 MByte of data, this corresponds to a continuous data rate of
18-20 MByte/s ( 1.1-1.2 Gbyte/min)
A sketch of the NA48 data acquisition system is shown below.
The PCs connected to the subdetectors are standard PentiumII-266 systems equipped with 128MByte RAM, a DT16 interface and a 100MBit/s Ethernet card.
Each Event-Building PC is equipped with 2 PII-266 processors, it has 192MByte RAM, 4 x 4.5GByte SCSI harddisks and a 100Mbit/s Ethernet card.
To fulfill the requirement of compatibility with the old data acquisision system based on 4 DEC Alpha TurboChannel Workstations with FDDI interfaces, we are using the 4 CDR Routing PCs as protocol converters between Ethernet and FDDI. Each CDRPC has 64MByte RAM, an FDDI interface and a 100MBit/s Ethernet card and acts a an IP router. The CDRPCs will become obsolete as soon as the direct Gigabit Ethernet link from the switch to the computer centre has been installed.
Internally the ports are linked accross a 1.2 Gbit/s backplane. The switch is of a store-and-forward type which means that incoming packets are stored in a buffer at the input and then forwarded to the required output port when the backplane is available. This type of switch can also be described as connectionless.
Each input port has a 32kByte buffer which allows incoming packets to
be held if they cannot be immediately forwarded to the output port. Each
output port has a 192kByte buffer which allows some pile-up to occur if
the output port receives packets from the backplane faster than it can
send them on to the external destination (the EBPC). This can happen if
several consecutive packets on the backplane happen to be directed to the
same output port (recall that the backplane delivers data at 1.2 Gbit/sec
whereas the port can only get rid of data at 100 Mbit/sec).
05-08-1998: It seems, that 2.1.114 with recent AIC7xxx, EEpro100 and RAID patches does run with SMP ( now since 10h) but we still need more running time to test it thoroughly. The second processor gives us the computing power to run the fast event filter in the PC farm.
21-08-1998: We are still using 2.1.114 which turns out to crash about once per week per EBPC ...
During installation of the farm we found one harddisk that was not working and we had to replace 64MBytes of RAM which did pass the BIOS test but did then not work with Linux. In the last 4 months another harddisk and another 64MByte of RAM failed, but the environment in the experimental hall is quite rough.
The stability of Linux is excellent, when running "stable" 2.0.x versions
on the EBPCs, we see much less than one OS crash per week (in total,
not per PC).