S-LINK News 1997
Abusing S-LINK as a Fibre Channel data sink
12 December 1997. The Fibre
Channel implementation of the S-LINK interface uses Fibre Channel components,
but only implements a very small part of the signalling protocols of Fibre
Channel. At CERN we also have real Fibre Channel equipment, including switches
and interfaces. To test out the switches, we would like to maximise the
traffic as much as possible. For this we would have to load all 16 ports,
which is a problem as we have only 8 interfaces in total at CERN and they
are relatively expensive. As the end of the year is a good time to do some
fun projects, we had the idea of modifying the FCS-LINK interface so that
it could be used as a FCS data drain. Zoltan Meggyesi had to modify the
Altera on the board so that the S-LINK card would send a Fabric Login frame
(so that the card is recognised by the switch) and after that it will acknowledge
the reception of each frame by sending a Receiver Ready (R_RDY). The design
of this Class 3 data sink took only two working days and some tuning of
timing parameters. With this fun design we will be able to fully load the
FC switch completely for a low cost. In fact we probably have enough S-LINK
cards at CERN to load the switch completely. We might try to expand the
design so that it can also sink Class 2 traffic. This is more difficult
as complete Acknoledge frames have to be sent back.
NA48 goes well over the 100 MByte/sec limit
12 December 1997. NA48
has modified the Linux driver for the S-LINK to PCI interface to be as
lightweight as possible. With the SLIDAS generating data, they now have
measured a speed from S-LINK into the main memory of the PC of 117
MByte/sec. With this result, S-LINK can place itself in the range
of the fastest interfaces ever built. The DT-16 to S-LINK interface (that
will plug onto the S-LINK to PCI card) that NA48 designed is working well
and did not need any modifications. Plans are to take data in the beginning
of next year over S-LINK. Although the PCI interface can receive at 117
MByte/sec, they will use maximum 30 MByte/sec, leaving enough room for
First electrical S-LINKs delivered
12 December 1997. The first commercial electrical
S-LINK cards have been delivered to Wolfgang Suttrop from the ASDEX
Upgrade experiment. The cards, which are based on LVDS drivers and
a SCSI cable, were completely designed and produced in Poland. Prototypes
of the electrical S-LINK cards have already been in use for the ATLAS L2
trigger demonstrators in Saclay and Mannheim and also by NIKHEF in Amsterdam.
NA48 moves data at 97 MByte/sec into PC memory
17 October 1997. NA48
is currently designing a data acquisition system upgrade. The present system
it is based on Fibre-optic links, a datamerger, a
HIPPI switch, and HIPPI interfaces. Most of those devices were specially
designed for NA48. To be able to use standard PCs (instead of expensive
workstations used now), NA48 is planning to replace this home-built DAQ
system by simple DT-16 (parallel electrical ECL link) Link Destination
Cards connected to commercially available S-LINK to PCI interfaces in PCs.The
datamerging will then be done in parallel over 100 Mbps switched Ethernet.
This upgrade using S-LINK
will make the NA48 DAQ system cheaper and will increase the reliability.
The design of the DT-16 LDC is underway and will contain only ECL/TTL conversion
chips and one small Altera chip. The software, which will be based on LINUX,
is being developed as well. First tests with an S-LINK
to PCI interface that receives data from a SLIDAS,
showed the following throughput measurement on a P5/200 MHz PC:
33 seconds, 3200.000000 MBytes, 96.969696 MBytes/s
This is a measurement of the speed of datamoving from the S-LINK interface
into the host memory. This is without any S-LINK control words (NA48 probably
will not use them), so it is basically a measurement of the DMA speed into
host memory. Under LINUX it will not be necessary to make an extra copy
of the data from kernel memory to user memory, so this will be the actual
throughput seen in the system and is more than sufficient, This used 0.5%
processor time only. Even with a copy, the read() command still gave a
througput of around 40 MBsec, using 50% of the processor's time. The software
written is based on a LINUX AMCC library that Steffen Luitz found on the
web and is not using the LINUX libraries from ORL or KFKI/RMKI. The addition
of the S-LINK specifics to this generic AMCC library costed Steffen Luitz
and Andre Lacourt only one night of hard work.
ASDEX Upgrade will use S-LINK
October 1997. The ASDEX
Upgrade tokamak (Axially Symmetric Divertor EXperiment) went into operation
at the Max-Planck-Institut
für Plasmaphysik (IPP), Garching in 1990. This fusion device,
Germany's largest at present, is for investigating crucial problems in
fusion research under reactor-like conditions. For this purpose essential
plasma properties, particularly the plasma density and the wall load,
have been adapted to the conditions that will be present in a future fusion
Demand to diagnose fast plasma events (and the computer power to process
the data) steadily increases and, particularly at the ASDEX Upgrade, they
are now going to upgrade several data acquisition systems for faster sample
rate and larger memory. To handle those higher rates, a
new, fast multichannel Data Aquisition (DAQ) system, that uses S-LINK,
is being developed for the Electron Cyclotron Emission (ECE) diagnostics.
MDT sends physics data over S-LINK!
9 October 1997. The MDT
(Monitored Drift Tube) detector made by Rome/Pavia is currently taking
physics data over a Fibre Channel S-LINK. The S-LINK data is received by
an S-LINK to PMC interface that is mounted on a RIO2 from CES. The test
beam, which is located at H8 in Prevessin at CERN, will run for another
week. This is a very special occasion as this is the first time real physics
data has been sent over an S-LINK.
The S-LINK specification is only two years old (the first release was
on 12 September 1995). Since then all S-LINK interfaces, links and test
tools have been built and tested. In fact all S-LINK devices that the MDT
group is using are now all commercially available.
S-LINK will be presented at IEEE Real Time conference
19 September 1997. The
Xth IEEE Real Time Conference (IEEE-NPSS Technical Committee on Computer
Applications in Nuclear and Plasma Sciences) will take place in France
from 22 to 26 September 1997 under the auspices of the Commissariat à
l'Energie Atomique (CEA) and Institut National de Physique Nucléaire
et de Physique des Particules (IN2P3/CNRS).
The Conference will be devoted to the "Impact of modern technologies
in Real Time systems" on small as well as large present and future scientific
apparatus. Various areas will be covered from Nuclear Physics, High Energy
Physics, Astroparticle Physics, Astrophysics and Plasma Physics. Links
to similar applications of such techniques to R&D in medical, biology,
aerospace and other industrial disciplines are presented.
An exhibition of new products by companies active in the various fields
will provide a valuable meeting ground for vendors and users.
The S-LINK will be presented at several occasions at this conference:
The paper describing the last presentation can be downloaded in Postscript
Electrical S-LINK commercially available
September 1997. The electrical
version of the S-LINK, which drives with LVDS drivers a standard SCSI
cable up to 15 m long, is now available via the company Nowoczesna
Elektronika. The commercial
offer, including pricing is available on the web. You will see that
the price is low for cards that have specifically been designed for High
Energy Physics applications. The cards, that have a throughput of 80 MByte/s,
have an 8-week delivery time and come with a one-year warrantee.
"S-LINK speed is unbelievable"
8 September 1997. At the Olivetti
& Oracle Research Laboratory (ORL) they are looking to solve
the problem of noisy workplace by interposing a dedicated fibre-optic
link between a workstation and its terminal (display, keyboard and mouse).
The use of such links allows workstations to be withdrawn from offices
and relocated to a central equipment room. In June this year they received
some S-LINK cards and PCI interfaces, in July they had written already
a Linux driver
for it and in August they had made some real nice demo's with this setup
as can be read in the mail from Steve Hodges:
CompactPCI version will be built by Olivetti & Oracle
Well the S-LINK demo is wowwing everyone here! The speed of it is
unbelievable.Thanks also for your reply to the return line problem.
We would like to build Compact-PCI versions of the SSPCI and
SPCIS boards incorporating the modifications you suggest. Is it possible
to get schematics for these boards?
With some kind of friendly licencing scheme, of course ORL can have
access to the schematics.They also have made public the Linux
driver they wrote for the PCI/S-LINK interfaces.
PCI and PMC boards can do Memory Read Multiple cycles
4 September 1997. The S-LINK to PCI and PMC interfaces use
the S5933 PCI interface chip from AMCC. Until now it was supposed that
the S5933 could not do any of the special PCI cycles that will speed up
reading and writing when the cards are used in a system with multiple PCI
bridge chips. A mail from an application engineer from AMCC however told
that there is the functionality of doing Memory Read Multiple cycles:
"Enable bit 15 in the MCSR enable memory read multiple during bus
mastering. In the new databook. Don't know why it was kept a secret."
Markus Joos from CERN has verified if this works on a RIO2 8062:
"The S5933 does indeed generate MRM cycles with MCSR-bit 15 = 1.
There is, however, no difference in performance. With and without MRM the
bridge inserts 6 wait cycles (180ns) after every transfer of 8 words (=
1 cache line)".
It is recommended to use those special cycles anyway when you write software
as other systems may give a much higher throughput if the MRM cycles are
The article Efficient Use of PCI from Frank Hady* (Intel Corp.
Platform Architecture Labs) recommends: Use the MR (normal Memory Read)
command for small data reads of a single PCI word (4 bytes). (Use MRL for
reading data that resides within one cache line). Use MRM for reading data
that is held on more than 1 cache line. While many chipsets fetch an entire
cache line to service a MR and alias MRL and MRM it is best to use the
commands as recommended, since not all chipsets make these simplifications.
Note that the MRM cycles are only useful on the PMC and PCI to S-LINK
boards as they do memory reads. The S-LINK to PMC and PCI boards don't
have any use for this cycle.
*Efficient Use of PCI, Frank Hady, Intel, published in RTC Europe,
Volume 1, Number 1, May 1997
see also the more recent article "Efficient
Use of PCI" at the Intel web site.
August 1997. In the ATLAS experiment, the Read-out Buffer, also called
ROB, will receive detector data from the Read-out Drivers over S-LINK.The
ROB will store data and send it away only when a positive trigger decision
has been made or when specific parts of the data are asked for. Otherwise
the data simply will be removed from the memory. In the final experiment
about 2000 of those ROB modules with each a single S-LINK input will be
In Royal Holloway (UK) they are building a ROB
module using a i960 processor. The first version will be in the PCI
form-factor, while the final version will be using PMC.
At NIKHEF (The Netherlands) work is started on the Multi-ROB.
This project integrates up to 16 ROB modules onto one 9U VME card. This
higher integration will reduce the amount of crates needed, increase the
functionality, reliability and reduce the final cost.
29 July 1997. Is S-LINK real? Often one picture tells more than
a 1000 words. In this case 23 photos
of 12 different S-LINK cards will show you that S-LINK is more than
just a specification of a connector...
GOLD : getting full speed into PCI
22 July 1997. A simple change of two chips on the S-LINK
to PMC or S-LINK to PCI card will
make it possible to move data at full speed from S-LINK into the PCI bus.
Before the boards could only run at half the maximum PCI bus speed. In
practive the speeds will be limited to around 90 MByte/sec, dependent on
the design of the host machine. This high-speed version of the boards is
nicknamed 'Gold'. No changes in software are needed.
All existing prototype boards will be upgraded on demand. Users having
boards from INCAA
Computers should contact INCAA directly about an upgrade.
Don't you like your noisy workstation? Use S-LINK!
2 June 1997. A common problem in offices with a large number of
computers is the heat and noise generated by the workstations. At the Olivetti
& Oracle Research Laboratory (ORL) they are looking to solve this
problem by interposing a dedicated fibre-optic link between a workstation
and its terminal (display, keyboard and mouse). The use of such links allows
workstations to be withdrawn from offices and relocated to a central equipment
room. ORL will receive in the first week of June a set of PCI/S-LINK cards
with FCS-LINK cards to investigate the
use of S-LINK over Fibre Channel to implement the remoting of a workstation
from its terminal. S-LINK is well suited to this application because
it provides a high-bandwidth forward channel (for the display) and a limited
bandwidth return channel (for the keyboard & mouse).
Moving a few cards to Italy
20 May 1997. Stefano Veneziano requested a few different S-LINK
cards for INFN. First of all he needed a Fibre Channel based S-LINK
for the readout of the MDT detector in the testbeam. He will get one
of the 'pool'
of S-LINKs that we have available at CERN. Secondly, INFN is designing
it's own S-LINK based on the
MATCH chip. The transmitter design is finished and in two weeks time
a PCB is expected back. To test this card, Stefano will need a SLIDAS
and a SLITEST so he can inject known data
into the link. He has ordered those cards from INCAA who will deliver them
soon. The design of the MATCH based receiver is not yet finished.
S-LINK with SCSI cable will be available from Poland
9 May 1997. The S-LINK
implementation that uses SCSI cable to transfer data has passed all
tests in CERN's S-LINK lab.This Scsi-LINK has
been designed at the Institute of Nuclear Physics in Cracow. The link,
which is able to transfer data at more than 50 MByte/sec, is perfect for
laboratory tests and matches the PMC and PCI/S-LINK cards in bandwidth.
The advantage of the link is that it is cheap and uses a readily available
standard 68-pin SCSI cable. A Polish company will start producing the cards
soon and will be able to deliver the link cards for less than CHF 900.
Another link that is commercially available, is the FCS-LINK.
It uses fibre optic cable with SC connectors and is able to transfer data
at up to 103 MByte/sec over a distance up to 500 meter.
SLIDAS, a cheap S-LINK data generator
9 May 1997. The SLIDAS is a stand-alone
device that connects to the S-LINK connector of a ROMB. Its purpose is
to test the read-out motherboard at its S-LINK interface without having
to set up a real S-LINK just for this task. With the SLIDAS, the user can
select among a wide variety of data patterns to transfer to the ROMB. The
data patterns are choosable in terms of contents,
bandwidth and error
contents. Not only normal patterns such as walking ones, alternating
ones and zeros can be generated, but also TRT
data as it will come from the ATLAS
Transition Radiation Tracker Read Out Driver can be emulated. This
format is defined in the ATLAS
Read-out Link Data Format document.
The SLIDAS is available from INCAA
Computers and is costing less than CHF 1000. It is the cheapest possible
way to emulate many channels coming into your data acquisition system,
as you don't need the S-LINK link cards and detectors to generate data.
SLITEST User's Guide
7 April 1997. The SLITEST is an
easy to use device to test links. In fact it is so easy to use that no
user has asked to get a manual
for it. Anyway it is written now, and you can find in there some technical
details and pictures on how you can use it to test links or to make an
S-LINK data sink. The last weeks the SLITEST has been used extensively
to test the SLIDAS, the S-LINK data generator.
Testing links made easy
11 March 1997. One of the strong points of the S-LINK project is
that apart from defining a specification for users, it also provides test
tools. It is those test tools that make that implementations will be inter-operable
and that many cross checks for compliance are made. Among the test tools
are the SLIDAD , SLIDAS
, SLIBOX and the SLITEST.
The latter is a new, software-less, device in the range that can be used
to test links, but that also can be used to make a data sink for S-LINK
Link Destination Cards.
Real-Time Computer Show features S-LINK
10 March 1997. At the Real-Time
Computer Show, the companies INCAA
Computers and Dr. Bernd Struck will
demonstrate S-LINK. This show, which is seen as the largest European exhibition
for real-time products and developers of embedded applications, will be
held on March 13 in Geneva.
There will be a live demonstration of a SLIDAS
generating data that will be sent over a Fibre
Channel S-LINK, after which the data is received by a SLIDAD.
All those cards are available commercially. Apart from that INCAA will
show the PMC and PCI
interfaces to S-LINK. Also a prototype of a G-LINK
implementation of S-LINK will be shown. This latter, in combination
with a PCI to S-LINK interface will show the compactness and simplicity
SLITEST and PMC interfaces to Sweden
10 March 1997. Christian Bohm, team leader of the design team of
the S-LINK using the Motorola
Optobus component, has taken a SLITEST, SLIDAD and PMC to S-LINK and
S-LINK to PMC interfaces to Sweden. The System and Instrumentation Physics
Group of the University of Stockholm has now made prototypes of the S-LINK
that uses parallel ribbon fibre with MT-connectors, and will use the above
mentioned devices to test those links. For more information on the Optobus,
take a look at Motorola's
INCAA delivers first cards
4 March 1997. Last week INCAA
Computers delivered the first PMC to S-LINK, S-LINK to PMC, SLIDADs
and SLIBOX cards to CERN and other customers. All ten of the PMC cards
are all sold already, so INCAA will start immediately another production
of those cards. Struck will be able
to deliver the first FCS-LINK cards by the beginning of April.
Other S-LINKs come to CERN
4 March 1997. Osamu Sasaki from KEK, Japan, brought yesterday to
CERN a set of S-LINK
boards that use the Hewlett Packard G-LINK as physical layer components.
In this case the signals are sent over three coax cables. Of course we
directly tried those cards out on the VME/PMC setup here and they worked.
There only appear some problems when datablocks of a size larger than 100.000
words are sent. We are looking into this problem right now. Those cards
are only prototypes, and it is not known yet when the G-LINK S-LINK cards
will be produced in quantity. It is great to see descriptions of the cards
in Japanese writing!
The parallel electrical
S-LINK, using a standard SCSI cable, also has arrived at CERN. Jolanta
Olszowska and Wieslaw Iwanski of the Institute of Nuclear Physics in Krakow/Poland
are currently at CERN to test the cards. They use the first prototype of
the SLIDAS, a SLIDAD and the SLITEST baseboard to do the first tests with
full speed data patterns. When those tests give satisfactory results, a
larger scale test will be done with the PMC/S-LINK cards.
Both the G-LINK and the parallel electrical S-LINK fully comply to the
mechanical form factor of the S-LINK specification. That means that all
components are mounted on the bottom of the cards and that the connectors
are mounted in the front-panel, just like PMC cards. Those cards, if mounted
on the PCI to
S-LINK and S-LINK
to PCI cards, will make a one-slot PCI interface. We are waiting for
people who would like to write a Windows/NT driver for those cards.
Level 2 trigger demonstrator Architecture B uses S-LINK
27 February 1997. In the ATLAS Level 2 trigger demonstrator Architecture
B, the S-Link will be used to connect the Trigger Supervisor output with
the ROI Distributer Input. First tests, using a RIO-2 and PMC/S-LINK cards
will be started mid March 1997. Those tests will use the LynxOS S-LINK
library written by the ATLAS "DAQ prototype-1" team. In January the Trigger
Supervisor has been tested at Sacley with the demonstrator Architecture
How many links needed in ATLAS?
20 February 1997. Robert McLaren presented at the second International
Data Acquisition Workshop (DAQ96) on Networked Data Acquisition systemsin
November 1996 in Osaka Japan the paper An
Application of S-LINK, a Data Link Interface Specification, in the ATLAS
Readout System". This paper (Postscript 220 KByte) describes the
S-LINK interface specification, the use of the S-LINK in ATLAS, the physical
links being designed, and the test equipment that has been developed. One
can read that almost 2000 Readout Links from the detector and trigger sub-systems
to the Readout Buffer will be needed for the final ATLAS DAQ system.
S-LINK overview paper
18 February 1997. The S-LINK was presented at the 1996
IEEE Nuclear Science Symposium in Los Angeles. The article written
for this symposium gives a concise overview of the the S-LINK specification,
the products and the applications. You can get a copy of the paper in a
310 KByte Postscipt file: "S-LINK, a Data
Link Interface Specification for the LHC Era".
18 February 1997. The S-LINK web pages exist now for over one year.
The project itself is slightly older, about one and a half year ago the
first specification was released. The S-LINK News
pages from 1996 have now been split out of the 1997 S-LINK news.
S-LINK tests at SACLAY
31 January 1997. Argonne National Laboratory (ANL) shipped the prototype
version of the ATLAS
Trigger Supervisor/ROI Builder to Saclay. Jim Schlereth is going Saturday
for 10 days to do the testing. He will integrate with the Saclay hardware
and they will try to get some results from Level 2 trigger Architecture
C to present at the demo meetings at end of February and ATLAS week at
the beginning of March.
At Saclay they will use the S-LINK to PMC card with a RIO2 to generate
ROI fragments which will be carried via the S-LINK to the ANL input router.
The input router emulates the S-LINK PMC card so it needs only the physical
layer card. As physical layer card, ANL uses a buffered version of the
"just a cable S-LINK" described
in the S-LINK implementation guide.
What can you buy?
30 January 1997. The S-LINK project is more than a project. It's
about specially designed links for high energy physics that you can use
to connect your readout or trigger modules to other modules in the DAQ
system. To make sure that the you can get enough S-LINK cards, many of
the cards are now commercially available. Those companies will provide
support in the form of guarantee and such. If you need cards soon, place
your orders now as only a limited amount of cards is made in the first
production run. If you need cards for only a limited amount of time, you
may contact Erik van der Bij
to get some cards on loan.
The following cards can be ordered from the company INCAA
Computers, with delivery possible in the month of March.
The only link card commercially available come from the company STRUCK,
again with delivery around the month of March:
Fibre Channel implementation in USA
30 January 1997. One set of Fibre
Channel S-LINK cards has been sent to the USA, to the ATLAS
Level 2 Trigger Supervisor (Argonne/Michigan State University). This
set, which is from the prototype run of the Fibre Channel S-LINK cards,
completes all S-LINK components that are needed. This is an important milestone
as the Trigger Supervisor has to integrate to many other parts in the ATLAS
DAQ system that will use S-LINK.
Six projects known to use S-LINK
30 January 1997. There are currently six
projects known to the S-LINK team that will use S-LINK. Those projects
are all in the ATLAS experiment, although also other experiments and even
companies outside high energy physics have shown interest in it.
SLIDAS PCB is being manufactured
9 January 1997. The design of the long awaited SLIDAS,
a dummy Link Destination Card that can be used as a data generator, is
finished. This board has several switches that can be used to select the
data patterns, inject errors, change speeds, etc. The card will also be
able to send data in the same format as required by the ATLAS ROD/ROB links.
Five PCBs of the SLIDAS are being manufactured, they will arrive at CERN
in two weeks time.
S-LINK goes PCI
9 January 1997. Two S-LINK to PCI
and two PCI to S-LINK boards will be
mounted soon. The two designs are basically a copy of the interfaces that
S-LINK to PMC and PMC
to S-LINK. With the right software those boards can be used in any
computer with a PCI bus. When an S-LINK card is plugged onto the PCI boards,
the full module will still use only one PCI slot.
Old S-LINK News
CERN - High Speed Interconnect
Erik van der Bij - 3 March 1998