Fibre Channel Examination

I have written this document to help me improving my knowledge on Fibre Channel. I started by writing down questions which seem to be reasonable to ask and found the answers afterwards. I allowed myself to use the specification when making the exam.

As I am not a specialist and don't have much practical experience, the information in this document is not necessarily correct; it merely shows my understanding of the Fibre Channel specifications I had at 7 April 1994.

If you find any errors in it, I would appreciate if you send me an e-mail.

Erik (Erik.van_der_Bij@cern.ch)

Updates

21/09/95 Jean-Francois GILLOT found error in 1.1.2. He earned a coffee with it!

Table of Contents

1. Physical and Signaling Interface (FC-PH)
FC-PH version 4.2

1.1 Hierarchy

1.1.1 Which handshakes are performed before the first frame can be sent?

  1. FC-0: In case of Laser Links: Open Fibre Control state machine.
    [Disconnect state, Stop State, Reconnect State, Active State]
    This is to make sure that a laser link when open can be classified as a Safety Class 1 laser product. The protocol works on the level of switching on and off the laser.
  2. FC-1: Receiver and Transmitter state machines.
    These are to achieve bit and word synchronization on a link. It is not a real handshake between the Receiver and Transmitter.
    Transmitter: [Not Enabled, Working, Failure, Open Fibre]
    Receiver: [Reset, Loss-of-Sync, Sync Acquired]
    The methodology to acquire bit and word synchronization is not specified in FC-PH. It works on detection of three Ordered Sets with a Comma character in their leftmost bit positions (Comma character: 7 bits that in the absence of transmission errors cannot appear in any other location of a Transmission Character).
  3. Primitive Sequence Protocol (16.4)
    [NOS: Not-Operational, OLS: Offline Sequence, LR: Link Reset, LRR: Link Reset Response, Operational link]
    The Primitive Sequence Protocol is uses to indicate specific conditions within a Port or conditions encountered by the Receiver logic of a Port. It is based on sending continuously Primitive Sequences (K28.5 + 3 specific words) until a response (at least three times the correct Primitive Sequence received) from the other port is received. The protocol ends with each port sending Idles.

1.1.2 Which handshakes are performed before the first data can be sent?

Fabric login and N-port login. The logins are started by sending a FLOGI or a NLOGI frame in which the payload contains the service parameters. The responses are coming back in ACC frames (LS_Command code of ACC). The logins are done in seperate Exchanges.

In class 1 connections, the first frame must have a frame starting with SOFc1 to open a dedicated connection to the Destination port. The first frame can contain data as well.

1.1.3 Which parameters are exchanged during those handshakes?

The parameters are split up in: Common Service Parameters and Service Parameters for the different classes (23.6).

  1. N-port Common Service Parameters
    1. FC-PH version: lowest and highest
    2. Buffer to buffer credit
    3. Continuously Increasing and/or Random Relative Offset
    4. N-port / F-port
    5. Buffer to buffer Receive Data Field Size
    6. N-port total Concurrent Sequences
    7. Relative Offset by Info Category
    8. R_A_TOV
    9. E_D_TOV
  2. N-port Class Service Parameters
    1. Class validity
    2. Intermix mode
    3. Stacked connect requests
    4. Sequential delivery
    5. Initiator control
      1. X_ID reassignment
      2. Inititial Responder Process_Associator
      3. ACK_0 capable
      4. ACK_N capable
    6. Recipient control
      1. ACK_0 capable
      2. ACK_N capable
      3. X_ID interlock
      4. Error policy support
      5. Categories per Sequence
    7. Receive Data Field Size
    8. Concurrent Sequences
    9. N-port End-to-end Credit
    10. Open Sequences per Exchange

1.1.4 What is an Exchange, what is a Sequence?

An Exchange is the mechanism for coordinating the interchange of information and data between two N_Ports or nodes.

A Sequence is a set of one or more related Data frames transmitted unidirectionally from one N_Port to another. There may be only one Sequence active in an Exchange. However, there may be many Exchanges open between two N_Ports.

1.1.5 How are the ports called in an Exchange and in a Sequence?

Exchange: Originater and Responder
Sequence: Sequence Initiator and Sequence Recipient

1.1.6 What is the Error recovery hierarchy?

Abort Exchange Abort Exchange Protocol-frames
Abort Sequence Abort Sequence Protocol-frames
This is a frame that is sent with the same SEQ_ID as the last frame sent.

Link Reset Link Reset Protocol-primitives
Link Initialization Link Init Protocol-primitives
Link Failure Link Failure Protocol-primitives
Those recovery primitives are going less or further back in the Primitive Sequence protocol. Link Failure is the worst, Link Reset is the most gentle recovery in the hierarchy (see 4.15).

Back to Table Of Contents

1.2 Sequence Qualifier

1.2.1 What is the Sequence_Qualifier?

S_ID, D_ID, OX_ID, RX_ID, SEQ_ID.

1.2.2 Which fields in the Sequence_Qualifier change and when?

All fields are fixed for a given Sequence. All but RX_ID are known by the Originator at the start of the Sequence. RX_ID will be known after reception of the first ACK.

1.2.3 Which fields in the frame header exist and when do they change?

1.2.4 Where does an N-port get its S_ID from?

S_ID (Source Identifier) gets assigned by the fabric at fabric login if the fabric supports it. Otherwise it is known implicitly.

Back to Table Of Contents

1.3 F_Port login

1.3.1 Give four examples of when a fabric login can fail

  1. F_Login with S_ID=0 does not result in an ACC with a D_ID set by the fabric, but it gives back a F_RJT with reason code Invalid S_ID (23.3.1). In this case the fabric does not support N_Port Identifier assignment.
  2. if a retry is done after a) a F_Login with an S_ID < > 0, and the fabric still responds with a F_RJT with reason code Invalid S_ID.
  3. F_Login responds with F_RJT class not supported.
  4. F_Login responds with a LS_RJT with a reason code

1.3.2 What to do in those cases?

  1. Retry F_Login with S_ID < > 0.
  2. Retry F_Login with another value of S_ID.
  3. Retry F_Login in another class.
  4. The Service Parameters were logically inconsistant or in error. Correct and retry.

Back to Table Of Contents

1.4 N_Port login

1.4.1 Give two examples of when an N-port login can fail

  1. P_Login responds with P_RJT class not supported.
  2. P_Login responds with LS_RJT with reason code. E.g. the Originator doing the P_Login does not support X_ID reassignment while the Responder N_Port requires it.

1.4.2 What to do in those cases?

  1. Retry P_Login with another class.
  2. Originator should support X_ID reassignment or else it cannot communicate to this port.

Back to Table Of Contents

1.5 Flow control

1.5.1 Which flow control is used in the different classes? 

          Buffer-to-buffer   End-to-End
[R_RDY]          [ACK]
Class 1     SOFc1: yes
other frames: no       yes
Class 2         yes             yes
Class 3         yes             no

1.5.2 Do Link Control frames participate in the flow control?

Link Control frames such as ACK_x, P_RJT, P_BSY, do not participate in end-to-end flow control (20.3.4). An N_Port shall not transmit P_BSY response frames for them either (20.3).

How can an N_Port make sure that it will not get an overflow in class 1 because of those Link Control frames? The N_Port should have enough resources to receive one Link Control frame in response to each data frame sent. This means that the N_Port should have at least as many receive buffers as it has sent Data Frames in addition to the End-to-end credit that it has given at P_Login.

1.5.3 How are the fields in the ACK frames filled in?

(See 20.3.2.2)

Back to Table Of Contents

1.6 Exchanges and Sequences

1.6.1 Which frames are not part of an Exchange or a Sequence?

Link Control Command frames such as Link Credit Reset (LCR) are not part of an Exchange or a Sequence (20.3.4).

Back to Table Of Contents

1.7 Sequence recovery

1.7.1 What happens if a frame is missing or in error?

In case of a missing frame, the recipient of the Sequence will find this by either timing out after E_D_TOV, or in case of class 1, by having a missing 'SEQ_CNT' value between two frames. In this case, the Recipient will transmit an ACK corresponding to the missing frame with the Abort Sequence bits set to (01).

In case of errors in a frame, it could be found by the CRC that is located at the end of a frame. The Recipient will send a P_RJT when it has detected a frame in error.

The Sequence will be aborted by the Initiator (24.3.10) after he has detected the error or has received the P_RJT frame or has received an ACK with the Abort Sequence bits set to (01). It will do so by sending the Abort Sequence (ABTS) Basic Link Service command.

1.7.2 Is it possible that FC-4 still receives a good frame in case of errors?
If yes, is it possible in all classes?

There are four different ways of handling errors. The Policy used, is decided by the Exchange Originator, which has to take into account the Error Policy supported by the Responder, which is shown at N_Port login (29.6.1). The method chosen will be shown in the F_CTL field in the data frame. The discard policies will deliver a complete Sequence or, it will not be delivered to FC-4 at all.

  1. Discard multiple Sequences: a Sequence will be delivered only if it is complete and received and all previous Sequences are delivered as well.
  2. Discard a single Sequence: a Sequence will be delivered only if it is complete. So there is no requirement that the previous Sequences were delivered as well.
  3. Discard multiple Sequences with retransmission: in this case the Sequence Recipient may request for a retransmission of a Sequence. This policy is only possible in Class 1. Note that the Retransmission Protocol is prohibited in the FCSI profiles.
  4. Process with infinite buffering: frames will be delivered to FC-4 even if there is a frame in error. This method can be useful for frame buffers and such.

1.7.3 Is there a maximum number of exchanges between N-ports?
If no, why not? If yes, how is this determined?

At N_Login the following parameters are defined:

As there can be maximum one Sequence Open per Exchange, the maximum number of Exchanges is limited to Total Concurrent Sequences.

The Open Sequences per Exchange parameter is used only in case of Streamed Sequences, where a Sequence Initiator can send several Sequences without waiting for the ACK on the last frame of the Sequence. Note that there is a difference between Streamed Sequences (18.6) and Chained Sequences(18.5, see as well question 2.2.4.).

Back to Table Of Contents

2. FCSI Common FC-PH Feature Sets
Used in Multiple Profiles
FCSI - 101 Revision 2.0

2.1 General

2.1.1 What does the "FCSI Common FC-PH Feature Sets" define?

The "Common Feature Sets" document makes certain selections from the FC-PH specification. The Profiles, such as SCSI and IP prescribe which Feature Sets to use and define also other, extra features. A device that is compliant to a certain Profile will interoperate with another device that is defined in the same Profile document, but there still are different options within each profile document, such as the physical layer and usage of Class 1 and Class 2.

The "Common Feature Sets" document defines as well the procedures to follow for error recovery. Here as well it defines a subset from the possibilities that the FC-PH document allows.

Furthermore the document specifies certain Features that a Fabric should have to be able to interoperate with N_Ports that operate according to the FCSI profiles. Also Complementary Fabric Profiles and Name Server Cooperating Profiles are defined.

Back to Table Of Contents

2.2 FC-2 Layer Feature Sets for Normal Operations

2.2.1 Which classes are used in the "Common FC-PH feature sets"?

Only Class 1 and Class 2 are used in the Profiles. The F-FC2-C2 Feature Set defines features and behaviour required when all Exchanges are performed in Class 2 only. F-FC2-CM defines the features required when both Class 1 and Class 2 are used in the same Exchange. In F-FC2-CM intermix mode support (sending Class 2 frames within a Class 1 connection), is required for both Transmission and Reception.

2.2.2 Which ACK types are used in the "Common FC-PH feature sets"?

The usage of ACK_N is prohibited in both Class 1 and Class 2. In Class 1 usage, only ACK_1 may be used. In Class 2 usage, both ACK_1 and ACK_0 may be used.

2.2.3 How is Relative offset used in the "Common FC-PH feature sets"?

Only Continuous Relative Offset is needed to be supported. This means that when the frames are put in order, the data can be put in the memory in the same order. Random Relative Offset is therefore prohibited.

2.2.4 Are Chained Sequences allowed in an Exchange?

Chained Sequences, which is that the Sequence Initiator requires a reply Sequence from the Sequence Recipient within the existing Dedicated Connection, is not allowed in FCSI compliant devices. This to prevent that Dedicated Connections stay open too long.

Back to Table Of Contents

2.3 FC-2 Layer Feature Sets for Error Detection and Recovery

2.3.1 Which Error Recovery mechanisms are used?

The Error Recovery mechanisms used do not resend the data (Process policy is prohibited). The recovery procedures are only to recover from failures such as lost frames or frames in error. The mechanisms as defined in FC-PH are used (sending ABTS, see question 1.7.1.) with additional specifications such as time-out usage, SEQ_CNT, RX_ID assignment and such.

If the ABTS protocol fails, different Second-Level Error Recovery mechanisms are defined for Class 1 and Class 2.

Back to Table Of Contents

2.4 Fabric Profiles

2.4.1 Which Fabric Profiles are defined?

Two fabric profiles are defined: P-FAB-C1, which has Full Class 1 and Limited Class 2 capability, and P-FAB-BC, which has Full Class 1 and Full Class 2 capability. The difference is on the Buffer-to- Buffer Data Field size (128 and 512 byte respectively) and that the -BC version can support Class 2 operations in full duplex at the speed of the N_Port technology. The -BC version is also required to do Class 2 speed matching, which is that if both N_Ports support different link speeds. Class 3 is allowed, but not required and Intermix support is required.

2.4.2 Which Complementary Fabric Profiles are defined?

The following four Complementary Fabric Profiles are defined in the "Common FC-PH Feature Sets" document:

  1. P-CFAB-H1 : Host support for Full Class 1 / Limited Class 2 fabric
  2. P-CFAB-P1 : Peripheral support for Full Class 1 / Limited Class 2 fabric
  3. P-CFAB-HB : Host support for Full Class 1 / Full Class 2 fabric
  4. P-CFAB-PB : Peripheral support for Full Class 1 / Full Class 2 fabric

These Profiles specify extra features needed, such as Local Mapping of N_Port IDs and World-Wide- Names and Stacked Connect Requests. It is not clear what is the difference between -H1 and -HB Complementary Fabric Profiles and -P1 and -PB.

2.4.3 Which Cooperating Profiles are defined?

  1. P-COP-NSR : Name server registration functions
  2. P-COP-NSQ : Name server query plus registration functions

These two Cooperating Profiles are needed in a Name Server is being used in an environment with a fabric. Those profiles are not yet defined. Also the Name Server Logical Profile is not yet defined.

Back to Table Of Contents

3. FCSI SCSI Profile
FCSI - 201 Revision 2.0

3.1 FCSI SCSI Profile

3.1.1 How many different types of FCSI SCSI implementations are possible?

Physical layer: 3 versions: 25/50/100-M5-SL-I (all laser, multi-mode).

SCSI Application Logical Profiles: 4:

  1. PSCSI-HD2 : Multi-drive Mass Storage Host using Class 2
  2. PSCSI-HDM : Multi-drive Mass Storage Host using Mixed mode
  3. PSCSI-PD2 : Multi-drive Mass Storage Peripheral using Class 2
  4. PSCSI-PDM : Multi-drive Mass Storage Peripheral using Mixed mode

So there are two possibilities for each Host and Peripheral, of which each one has three possible physical options. Note that electrical versions will be defined later.

3.1.2 Which implementations interoperate?

The -HD2 and PD-2 will interoperate in point-to-point connections, and with a Full Class1 / Full Class 2 fabric (P-FAB-BC). In the latter case the Class 2 speed matching makes that even the different physical layers will interoperate. They won't work with a Limited Class 2 fabric as the limited Class 2 bandwidth and Class 2 frame Receive Data Field Size make them unsuitable.

The -HDM and -PDM will interoperate when the physical layers are matched.

Back to Table Of Contents

4. FCSI IP Profile
FCSI - 202 Revision 1.0

4.1 FCSI IP Profile

4.1.1 How many different types of FCSI IP implementations are possible?

Physical layer: 3 versions: 25/50/100-M5-SL-I (all laser, multi-mode).

IP Application Logical Profiles: 4:

  1. P-IP-H2 : IP Host using Class 2
  2. P-IP-HM : IP Host using Mixed mode
  3. P-COP-ARPC : Cooperating profile for ARP Client (using Class 2)
  4. P-ARP-SRVR : ARP Server (using Class 2)

So there are two possibilities for the Hosts, of which each one has three possible physical options. Note that electrical versions will be defined later. If needed, the ARP Server or ARP Client profiles have to be added to the host.

4.1.2 Which implementations interoperate?

The -H2 hosts will interoperate in point-to-point connections, and with a Full Class1 / Full Class 2 fabric (P-FAB-BC). In the latter case the Class 2 speed matching makes that even the different physical layers will interoperate. They won't work with a Limited Class 2 fabric as the limited Class 2 bandwidth and Class 2 frame Receive Data Field Size make them unsuitable.

The -HM and -PM will interoperate when the physical layers are matched.

Back to Table Of Contents

This is one of the CERN High Speed Interconnect pages - 21 September 1995 - Erik van der Bij