Figure 9: Number of packets handled per interrupt versus message length for Comms1 for various maximum size packets.
In the DSNIC, an interrupt handler takes care of all the packet handling to communicate messages. Figure 9 shows for Comms1 the number of packets handled per interrupt versus message length. The results in this graph are strongly related to the way in which the interrupt handler operates. The interrupt handler is activated by a number of hardware events, e.g., FIFOs, or mailboxes, becoming full or empty. The interrupt handler could terminate as soon as the events that interrupted the processor are handled. However, in order to improve performance, the interrupt handler, after having handled the events, checks whether any other hardware events need handling. If so, it handles them, and checks again for new events. Therefore, the interrupt handler can handle many packets each time it is called.
Handling many events per interrupt improves performance, since it avoids unneccesary processor interrupts. However, if many events are available, the interrupt handler is active for a long time. During this time no other interrupts can be handled, which has proven to cause problems to the timer interrupt. To avoid interrupt handling that takes too long, we limit the maximum number of events handled per interrupt. The effect of this is shown by the limit of about 70 handled packets per interrupt for the smaller packet sizes.
In Figure 9 two stable states can be identified, many and one: the state many, handling the maximum number of packets per interrupt, and the state one, handling one packet per interrupt. The small packet sizes result in such a large event handling frequency compared to the event handling time, that state many is reached. The large packet sizes result in such a low event handling frequency that state one is reached.
The larger packet sizes show a drop off at message length 270 to about 1 packet handled per interrupt. So, if a message of more than 270 bytes is sent, the interrupt handler only handles one packet, because at the end of the handler no new events are available. Handling one packet on interrupt therefore takes as much time as it takes to communicate a 270 byte packet. This drop in efficiency of the interrupt handler also causes the latency peak and throughput dip at message length 270, see Figure 8 and Figure 6.
Figure 10: Number of packets handled per interrupt versus message length for Comms2.
Figure 10 shows the number of packets handled per interrupt versus message length for Comms2. In Comms2, we are dealing with bidirectional communication. Therefore, three stable states can be identified, the two states from Comms1, i.e., states many and one, and one extra state, state two. In state two, two packets are handled per interrupt, an outgoing and an incoming packet. Medium sized packets, such as packet size 1024, end up in state two.