S-LINK News 1997

Abusing S-LINK as a Fibre Channel data sink

12 December 1997. The Fibre Channel implementation of the S-LINK interface uses Fibre Channel components, but only implements a very small part of the signalling protocols of Fibre Channel. At CERN we also have real Fibre Channel equipment, including switches and interfaces. To test out the switches, we would like to maximise the traffic as much as possible. For this we would have to load all 16 ports, which is a problem as we have only 8 interfaces in total at CERN and they are relatively expensive. As the end of the year is a good time to do some fun projects, we had the idea of modifying the FCS-LINK interface so that it could be used as a FCS data drain. Zoltan Meggyesi had to modify the Altera on the board so that the S-LINK card would send a Fabric Login frame (so that the card is recognised by the switch) and after that it will acknowledge the reception of each frame by sending a Receiver Ready (R_RDY). The design of this Class 3 data sink took only two working days and some tuning of timing parameters. With this fun design we will be able to fully load the FC switch completely for a low cost. In fact we probably have enough S-LINK cards at CERN to load the switch completely. We might try to expand the design so that it can also sink Class 2 traffic. This is more difficult as complete Acknoledge frames have to be sent back.

NA48 goes well over the 100 MByte/sec limit

12 December 1997. NA48 has modified the Linux driver for the S-LINK to PCI interface to be as lightweight as possible. With the SLIDAS generating data, they now have measured a speed from S-LINK into the main memory of the PC of 117 MByte/sec. With this result, S-LINK can place itself in the range of the fastest interfaces ever built. The DT-16 to S-LINK interface (that will plug onto the S-LINK to PCI card) that NA48 designed is working well and did not need any modifications. Plans are to take data in the beginning of next year over S-LINK. Although the PCI interface can receive at 117 MByte/sec, they will use maximum 30 MByte/sec, leaving enough room for future upgrades.

First electrical S-LINKs delivered

12 December 1997. The first commercial electrical S-LINK cards have been delivered to Wolfgang Suttrop from the ASDEX Upgrade experiment. The cards, which are based on LVDS drivers and a SCSI cable, were completely designed and produced in Poland. Prototypes of the electrical S-LINK cards have already been in use for the ATLAS L2 trigger demonstrators in Saclay and Mannheim and also by NIKHEF in Amsterdam.

NA48 moves data at 97 MByte/sec into PC memory

17 October 1997. NA48 is currently designing a data acquisition system upgrade. The present system it is based on Fibre-optic links, a datamerger, a HIPPI switch, and HIPPI interfaces. Most of those devices were specially designed for NA48. To be able to use standard PCs (instead of expensive workstations used now), NA48 is planning to replace this home-built DAQ system by simple DT-16 (parallel electrical ECL link) Link Destination Cards connected to commercially available S-LINK to PCI interfaces in PCs.The datamerging will then be done in parallel over 100 Mbps switched Ethernet.

This upgrade using S-LINK will make the NA48 DAQ system cheaper and will increase the reliability. The design of the DT-16 LDC is underway and will contain only ECL/TTL conversion chips and one small Altera chip. The software, which will be based on LINUX, is being developed as well. First tests with an S-LINK to PCI interface that receives data from a SLIDAS, showed the following throughput measurement on a P5/200 MHz PC: 

33 seconds, 3200.000000 MBytes, 96.969696 MBytes/s
This is a measurement of the speed of datamoving from the S-LINK interface into the host memory. This is without any S-LINK control words (NA48 probably will not use them), so it is basically a measurement of the DMA speed into host memory. Under LINUX it will not be necessary to make an extra copy of the data from kernel memory to user memory, so this will be the actual throughput seen in the system and is more than sufficient, This used 0.5% processor time only. Even with a copy, the read() command still gave a througput of around 40 MBsec, using 50% of the processor's time. The software written is based on a LINUX AMCC library that Steffen Luitz found on the web and is not using the LINUX libraries from ORL or KFKI/RMKI. The addition of the S-LINK specifics to this generic AMCC library costed Steffen Luitz and Andre Lacourt only one night of hard work.

ASDEX Upgrade will use S-LINK

13 October 1997. The ASDEX Upgrade tokamak (Axially Symmetric Divertor EXperiment) went into operation at the Max-Planck-Institut für Plasmaphysik (IPP), Garching in 1990. This fusion device, Germany's largest at present, is for investigating crucial problems in fusion research under reactor-like conditions. For this purpose essential plasma properties, particularly the plasma density and the  wall load, have been adapted to the conditions that will be present in a future fusion reactor.

Demand to diagnose fast plasma events (and the computer power to process the data) steadily increases and, particularly at the ASDEX Upgrade, they are now going to upgrade several data acquisition systems for faster sample rate and larger memory. To handle those higher rates, a new, fast multichannel Data Aquisition (DAQ) system, that uses S-LINK, is being developed for the Electron Cyclotron Emission (ECE) diagnostics.

MDT sends physics data over S-LINK!

9 October 1997. The MDT (Monitored Drift Tube) detector made by Rome/Pavia is currently taking  physics data over a Fibre Channel S-LINK. The S-LINK data is received by an S-LINK to PMC interface that is mounted on a RIO2 from CES. The test beam, which is located at H8 in Prevessin at CERN, will run for another week. This is a very special occasion as this is the first time real physics data has been sent over an S-LINK.

The S-LINK specification is only two years old (the first release was on 12 September 1995). Since then all S-LINK interfaces, links and test tools have been built and tested. In fact all S-LINK devices that the MDT group is using are now all commercially available.

S-LINK will be presented at IEEE Real Time conference

19 September 1997. The Xth IEEE Real Time Conference (IEEE-NPSS Technical Committee on Computer Applications in Nuclear and Plasma Sciences) will take place in France from 22 to 26 September 1997 under the auspices of the Commissariat à l'Energie Atomique (CEA) and Institut National de Physique Nucléaire et de Physique des Particules (IN2P3/CNRS).
The Conference will be devoted to the "Impact of modern technologies in Real Time systems" on small as well as large present and future scientific apparatus. Various areas will be covered from Nuclear Physics, High Energy Physics, Astroparticle Physics, Astrophysics and Plasma Physics. Links to similar applications of such techniques to R&D in medical, biology, aerospace and other industrial disciplines are presented.
An exhibition of new products by companies active in the various fields will provide a valuable meeting ground for vendors and users.

The S-LINK will be presented at several occasions at this conference:

The paper describing the last presentation can be downloaded in Postscript (310 KB).

Electrical S-LINK commercially available

NE logo9 September 1997. The electrical version of the S-LINK, which drives with LVDS drivers a standard SCSI cable up to 15 m long, is now available via the company Nowoczesna Elektronika. The commercial offer, including pricing is available on the web. You will see that the price is low for cards that have specifically been designed for High Energy Physics applications. The cards, that have a throughput of 80 MByte/s, have an 8-week delivery time and come with a one-year warrantee.

"S-LINK speed is unbelievable"
CompactPCI version will be built by Olivetti & Oracle Research Laboratory?

8 September 1997. At the Olivetti & Oracle Research Laboratory (ORL) they are looking to solve the problem of noisy workplace by interposing a dedicated fibre-optic link between a workstation and its terminal (display, keyboard and mouse). The use of such links allows workstations to be withdrawn from offices and relocated to a central equipment room. In June this year they received some S-LINK cards and PCI interfaces, in July they had written already a Linux driver for it and in August they had made some real nice demo's with this setup as can be read in the mail from Steve Hodges:

Hi Erik,

Well the S-LINK demo is wowwing everyone here! The speed of it is unbelievable.Thanks also for your reply to the return line problem.
We would  like to build Compact-PCI versions of the SSPCI and SPCIS boards incorporating the modifications you suggest. Is it possible to get schematics for these boards?

With some kind of friendly licencing scheme, of course ORL can have access to the schematics.They also have made public the Linux driver they wrote for the PCI/S-LINK interfaces.
 
 

PCI and PMC boards can do Memory Read Multiple cycles

4 September 1997. The S-LINK to PCI and PMC  interfaces use the S5933 PCI interface chip from AMCC. Until now it was supposed that the S5933 could not do any of the special PCI cycles that will speed up reading and writing when the cards are used in a system with multiple PCI bridge chips. A mail from an application engineer from AMCC however told that there is the functionality of doing Memory Read Multiple cycles: Markus Joos from CERN has verified if this works on a RIO2 8062: It is recommended to use those special cycles anyway when you write software as other systems may give a much higher throughput if the MRM cycles are used.
The article Efficient Use of PCI from Frank Hady* (Intel Corp. Platform Architecture Labs) recommends: Use the MR (normal Memory Read) command for small data reads of a single PCI word (4 bytes). (Use MRL for reading data that resides within one cache line). Use MRM for reading data that is held on more than 1 cache line. While many chipsets fetch an entire cache line to service a MR and alias MRL and MRM it is best to use the commands as recommended, since not all chipsets make these simplifications.

Note that the MRM cycles are only useful on the PMC and PCI to S-LINK boards as they do memory reads. The S-LINK to PMC and PCI boards don't have any use for this cycle.

*Efficient Use of PCI, Frank Hady, Intel, published in RTC Europe, Volume 1, Number 1, May 1997
see also the more recent article "Efficient Use of PCI" at the Intel web site.

Read-out Buffers

ATLAS DAQ system21 August 1997. In the ATLAS experiment, the Read-out Buffer, also called ROB, will receive detector data from the Read-out Drivers over S-LINK.The ROB will store data and send it away only when a positive trigger decision has been made or when specific parts of the data are asked for. Otherwise the data simply will be removed from the memory. In the final experiment about 2000 of those ROB modules with each a single S-LINK input will be needed.

In Royal Holloway (UK) they are building a ROB module using a i960 processor. The first version will be in the PCI form-factor, while the final version will be using PMC.

At NIKHEF (The Netherlands) work is started on the Multi-ROB. This project integrates up to 16 ROB modules onto one 9U VME card. This higher integration will reduce the amount of crates needed, increase the functionality, reliability and reduce the final cost.

One picture...

29 July 1997. Is S-LINK real? Often one picture tells more than a 1000 words. In this case 23 photos of 12 different S-LINK cards will show you that S-LINK is more than just a specification of a connector...

GOLD : getting full speed into PCI and PMC

22 July 1997. A simple change of two chips on the S-LINK to PMC or S-LINK to PCI card will make it possible to move data at full speed from S-LINK into the PCI bus. Before the boards could only run at half the maximum PCI bus speed. In practive the speeds will be limited to around 90 MByte/sec, dependent on the design of the host machine. This high-speed version of the boards is nicknamed 'Gold'. No changes in software are needed.
All existing prototype boards will be upgraded on demand. Users having boards from INCAA Computers should contact INCAA directly about an upgrade.

Don't you like your noisy workstation? Use S-LINK!

2 June 1997. A common problem in offices with a large number of computers is the heat and noise generated by the workstations. At the Olivetti & Oracle Research Laboratory (ORL) they are looking to solve this problem by interposing a dedicated fibre-optic link between a workstation and its terminal (display, keyboard and mouse). The use of such links allows workstations to be withdrawn from offices and relocated to a central equipment room. ORL will receive in the first week of June a set of PCI/S-LINK cards with FCS-LINK cards to investigate the use of S-LINK over Fibre Channel to implement the remoting of a workstation from its terminal. S-LINK is well suited to this application because it provides a high-bandwidth forward channel (for the display) and a limited bandwidth return channel (for the keyboard & mouse).

Moving a few cards to Italy

20 May 1997. Stefano Veneziano requested a few different S-LINK cards for INFN. First of all he needed a Fibre Channel based S-LINK for the readout of the MDT detector in the testbeam. He will get one of the 'pool' of S-LINKs that we have available at CERN. Secondly, INFN is designing it's own S-LINK based on the MATCH chip. The transmitter design is finished and in two weeks time a PCB is expected back. To test this card, Stefano will need a SLIDAS and a SLITEST so he can inject known data into the link. He has ordered those cards from INCAA who will deliver them soon. The design of the MATCH based receiver is not yet finished.

S-LINK with SCSI cable will be available from Poland

9 May 1997. The S-LINK implementation that uses SCSI cable to transfer data has passed all tests in CERN's S-LINK lab.This Scsi-LINK has been designed at the Institute of Nuclear Physics in Cracow. The link, which is able to transfer data at more than 50 MByte/sec, is perfect for laboratory tests and matches the PMC and PCI/S-LINK cards in bandwidth. The advantage of the link is that it is cheap and uses a readily available standard 68-pin SCSI cable. A Polish company will start producing the cards soon and will be able to deliver the link cards for less than CHF 900.

Another link that is commercially available, is the FCS-LINK. It uses fibre optic cable with SC connectors and is able to transfer data at up to 103 MByte/sec over a distance up to 500 meter.

SLIDAS, a cheap S-LINK data generator

9 May 1997. The SLIDAS is a stand-alone device that connects to the S-LINK connector of a ROMB. Its purpose is to test the read-out motherboard at its S-LINK interface without having to set up a real S-LINK just for this task. With the SLIDAS, the user can select among a wide variety of data patterns to transfer to the ROMB. The data patterns are choosable in terms of contents, length, bandwidth and error contents. Not only normal patterns such as walking ones, alternating ones and zeros can be generated, but also TRT data as it will come from the ATLAS Transition Radiation Tracker Read Out Driver can be emulated. This format is defined in the ATLAS Read-out Link Data Format document.

The SLIDAS is available from INCAA Computers and is costing less than CHF 1000. It is the cheapest possible way to emulate many channels coming into your data acquisition system, as you don't need the S-LINK link cards and detectors to generate data.

SLITEST User's Guide

7 April 1997. The SLITEST is an easy to use device to test links. In fact it is so easy to use that no user has asked to get a manual for it. Anyway it is written now, and you can find in there some technical details and pictures on how you can use it to test links or to make an S-LINK data sink. The last weeks the SLITEST has been used extensively to test the SLIDAS, the S-LINK data generator.

Testing links made easy

11 March 1997. One of the strong points of the S-LINK project is that apart from defining a specification for users, it also provides test tools. It is those test tools that make that implementations will be inter-operable and that many cross checks for compliance are made. Among the test tools are the SLIDAD , SLIDAS , SLIBOX and the SLITEST. The latter is a new, software-less, device in the range that can be used to test links, but that also can be used to make a data sink for S-LINK Link Destination Cards.

Real-Time Computer Show features S-LINK

10 March 1997. At the Real-Time Computer Show, the companies INCAA Computers and Dr. Bernd Struck will demonstrate S-LINK. This show, which is seen as the largest European exhibition for real-time products and developers of embedded applications, will be held on March 13 in Geneva.

There will be a live demonstration of a SLIDAS generating data that will be sent over a Fibre Channel S-LINK, after which the data is received by a SLIDAD. All those cards are available commercially. Apart from that INCAA will show the PMC and PCI interfaces to S-LINK. Also a prototype of a G-LINK implementation of S-LINK will be shown. This latter, in combination with a PCI to S-LINK interface will show the compactness and simplicity of S-LINK.

SLITEST and PMC interfaces to Sweden

10 March 1997. Christian Bohm, team leader of the design team of the S-LINK using the Motorola Optobus component, has taken a SLITEST, SLIDAD and PMC to S-LINK and S-LINK to PMC interfaces to Sweden. The System and Instrumentation Physics Group of the University of Stockholm has now made prototypes of the S-LINK that uses parallel ribbon fibre with MT-connectors, and will use the above mentioned devices to test those links. For more information on the Optobus, take a look at Motorola's Optobus homepage.

INCAA delivers first cards

4 March 1997. Last week INCAA Computers delivered the first PMC to S-LINK, S-LINK to PMC, SLIDADs and SLIBOX cards to CERN and other customers. All ten of the PMC cards are all sold already, so INCAA will start immediately another production of those cards. Struck will be able to deliver the first FCS-LINK cards by the beginning of April.

Other S-LINKs come to CERN

4 March 1997. Osamu Sasaki from KEK, Japan, brought yesterday to CERN a set of S-LINK boards that use the Hewlett Packard G-LINK as physical layer components. In this case the signals are sent over three coax cables. Of course we directly tried those cards out on the VME/PMC setup here and they worked. There only appear some problems when datablocks of a size larger than 100.000 words are sent. We are looking into this problem right now. Those cards are only prototypes, and it is not known yet when the G-LINK S-LINK cards will be produced in quantity. It is great to see descriptions of the cards in Japanese writing!

The parallel electrical S-LINK, using a standard SCSI cable, also has arrived at CERN. Jolanta Olszowska and Wieslaw Iwanski of the Institute of Nuclear Physics in Krakow/Poland are currently at CERN to test the cards. They use the first prototype of the SLIDAS, a SLIDAD and the SLITEST baseboard to do the first tests with full speed data patterns. When those tests give satisfactory results, a larger scale test will be done with the PMC/S-LINK cards.

Both the G-LINK and the parallel electrical S-LINK fully comply to the mechanical form factor of the S-LINK specification. That means that all components are mounted on the bottom of the cards and that the connectors are mounted in the front-panel, just like PMC cards. Those cards, if mounted on the PCI to S-LINK and S-LINK to PCI cards, will make a one-slot PCI interface. We are waiting for people who would like to write a Windows/NT driver for those cards.

Level 2 trigger demonstrator Architecture B uses S-LINK

27 February 1997. In the ATLAS Level 2 trigger demonstrator Architecture B, the S-Link will be used to connect the Trigger Supervisor output with the ROI Distributer Input. First tests, using a RIO-2 and PMC/S-LINK cards will be started mid March 1997. Those tests will use the LynxOS S-LINK library written by the ATLAS "DAQ prototype-1" team. In January the Trigger Supervisor has been tested at Sacley with the demonstrator Architecture C.

How many links needed in ATLAS?

20 February 1997. Robert McLaren presented at the second International Data Acquisition Workshop (DAQ96) on Networked Data Acquisition systemsin November 1996 in Osaka Japan the paper An Application of S-LINK, a Data Link Interface Specification, in the ATLAS Readout System". This paper (Postscript 220 KByte) describes the S-LINK interface specification, the use of the S-LINK in ATLAS, the physical links being designed, and the test equipment that has been developed. One can read that almost 2000 Readout Links from the detector and trigger sub-systems to the Readout Buffer will be needed for the final ATLAS DAQ system.

S-LINK overview paper

18 February 1997. The S-LINK was presented at the 1996 IEEE Nuclear Science Symposium in Los Angeles. The article written for this symposium gives a concise overview of the the S-LINK specification, the products and the applications. You can get a copy of the paper in a 310 KByte Postscipt file: "S-LINK, a Data Link Interface Specification for the LHC Era".

News split

18 February 1997. The S-LINK web pages exist now for over one year. The project itself is slightly older, about one and a half year ago the first specification was released. The S-LINK News pages from 1996 have now been split out of the 1997 S-LINK news.

S-LINK tests at SACLAY

31 January 1997. Argonne National Laboratory (ANL) shipped the prototype version of the ATLAS Trigger Supervisor/ROI Builder to Saclay. Jim Schlereth is going Saturday for 10 days to do the testing. He will integrate with the Saclay hardware and they will try to get some results from Level 2 trigger Architecture C to present at the demo meetings at end of February and ATLAS week at the beginning of March.

At Saclay they will use the S-LINK to PMC card with a RIO2 to generate ROI fragments which will be carried via the S-LINK to the ANL input router. The input router emulates the S-LINK PMC card so it needs only the physical layer card. As physical layer card, ANL uses a buffered version of the "just a cable S-LINK" described in the S-LINK implementation guide.

What can you buy?

30 January 1997. The S-LINK project is more than a project. It's about specially designed links for high energy physics that you can use to connect your readout or trigger modules to other modules in the DAQ system. To make sure that the you can get enough S-LINK cards, many of the cards are now commercially available. Those companies will provide support in the form of guarantee and such. If you need cards soon, place your orders now as only a limited amount of cards is made in the first production run. If you need cards for only a limited amount of time, you may contact Erik van der Bij to get some cards on loan.

The following cards can be ordered from the company INCAA Computers, with delivery possible in the month of March.

The only link card commercially available come from the company STRUCK, again with delivery around the month of March:

Fibre Channel implementation in USA

30 January 1997. One set of Fibre Channel S-LINK cards has been sent to the USA, to the ATLAS Level 2 Trigger Supervisor (Argonne/Michigan State University). This set, which is from the prototype run of the Fibre Channel S-LINK cards, completes all S-LINK components that are needed. This is an important milestone as the Trigger Supervisor has to integrate to many other parts in the ATLAS DAQ system that will use S-LINK.

Six projects known to use S-LINK

30 January 1997. There are currently six projects known to the S-LINK team that will use S-LINK. Those projects are all in the ATLAS experiment, although also other experiments and even companies outside high energy physics have shown interest in it.

SLIDAS PCB is being manufactured

9 January 1997. The design of the long awaited SLIDAS, a dummy Link Destination Card that can be used as a data generator, is finished. This board has several switches that can be used to select the data patterns, inject errors, change speeds, etc. The card will also be able to send data in the same format as required by the ATLAS ROD/ROB links. Five PCBs of the SLIDAS are being manufactured, they will arrive at CERN in two weeks time.

S-LINK goes PCI

9 January 1997. Two S-LINK to PCI and two PCI to S-LINK boards will be mounted soon. The two designs are basically a copy of the interfaces that link S-LINK to PMC and PMC to S-LINK. With the right software those boards can be used in any computer with a PCI bus. When an S-LINK card is plugged onto the PCI boards, the full module will still use only one PCI slot. 

Old S-LINK News

CERN - High Speed Interconnect - S-LINK
Erik van der Bij - 3 March 1998