Concept of Shared IRQs in linux

In this post, i am gonna talk about the shared IRQ and how Linux kernel handle shared IRQs.

As wikipedia states “In a computer, an interrupt request (or IRQ) is a hardware signal sent to the processor that temporarily stops a running program and allows a special program, an interrupt handler, to run instead”.

In any embedded system, When a device needs the CPU it sends a request to the CPU. When the CPU gets this request it stops everything what it is doing (and save in memory where the CPU left for the task it was doing) and then it serve the device that sent the request. After serving the device, it gets the work what it was doing from cache/HDD and carry on what it was doing before that interrupt was sent. This request is known as IRQ (interrupt request). So, interrupts are interruptions of a program caused by hardware, when it needs an attention from CPU.

There are limited Interrupts lines(pins) are available on every SoC. Idealistically, one IRQ line can only serve one device. It menace that number of device that can communicate with the processor is equal to the number of IRQ lines available on processor. Which is not enough as per the modern embedded device complexity. As a solution of this situation, modern hardware,  has been designed to allow the sharing of interrupt line among couple of device. Its a responsibility of a software developer to enable/disable appropriate hardware for interrupt on shared line and maintain the list of IRQs for shared line. On the arrival of interrupt on shared line, appropriate ISR from list should gets called to server the device.

Here as a part of this post we will going to explore how shared IRQ can registered and used with Linux kernel.

request _irq() is the function, which is used to request and register normal IRQ with Linux kernel. The same function is used to registered shared IRQ. The difference is SA_SHIRQ bit must be specified in the flags argument when requesting shared interrupt. On the registration of shared IRQ kernel checks for any other  handler exists for that interrupt and  all of those previously registered also requested interrupt sharing. If it found any other handler for same IRQ number, it then checks dev_id parameter is unique, so that the kernel can differentiate the multiple handlers. So it is very essential that dev_id argument must be unique. The kernel keeps a list of shared handlers associated with the interrupt, and dev_id is used as the signature that differentiates between all handlers. If two drivers were to register same dev_id  as their signature on the same interrupt, things might get mixed up at interrupt occurrence , causing the kernel to oops when an interrupt arrived. When a hardware device raises the interrupts on IRQ line, the kernel invokes every handler registered for that interrupt, passing dev_id, which was used to register the handler via request_irq().

When interrupts occurs on shared IRQ line, kernel invokes each and every interrupt handler registered with it by passing each its own dev_id. Shared IRQ handler should quickly check the dev_id with its own to recognize its interrupts and it should quickly return with return value of IRQ_NONE if own device has not interrupted(dev_id does not match). If dev_id matches ISR should return IRQ_HANDLE so kernel stops calling nest interrupt handler. In addition to this, driver using shared IRQs should not enable or diable IRQ. If it does, things might go wrong for other devices sharing the line; disabling another device’s interrupts for even a short time may create latencies that are problematic for that device.

Be very cautious while playing with shared interrupts !!

cheers 🙂

Concept of ISR in Linux

In this article, i am going to discuss about Interrupt handling mechanism of Linux kernel.

So the story starts from When an interrupts occurs, the processor looks if interrupts are masked. If they are, nothing happens until they are unmasked. When interrupts become unmasked, if there are any pending interrupts, the processor picks one. Then the processor executes the interrupt by branching to a particular address in memory. The code at that address is called the interrupt handler. When the processor branches there, it masks interrupts (so the interrupt handler has exclusive control) and saves the contents of some registers in stack. When the handler finishes executing, it executes a special return-from-interrupt instruction that restores the saved registers and unmasks interrupts.

The problem over here is :

The Corresponding interrupt is disabled during the execution of a interrupt handler, the interrupt handler expected to be finish fast But, what if you have to do a lot of data processing, memory allocation in a interrupt handler.

After kernel release of 2.6 this problem is resolved by designing and developing proper interrupt handling mechanism. Which splits he interrupt handling in to two parts. 

1. Top-Half: The top half is the real interrupt handler: It just tells the kernel to run the bottom half, and exits. The kernel guarantees that the top half is never re-entered: if another interrupt arrives, it is queued until the top half is finished. Because the top half disables interrupts, it has to be very fast.
2. Bottom-Half: The bottom half is run after the interrupts are processed(Top half is executed).The interrupts are not disabled while the bottom half is run, so it can do slower actions.

NOTE: Its totally device driver developers choice to split interrupt processing or not. If the ISR is going to very short and can be managed then there is no need of  bottom-half. Similarly, if the disabled interrupt for a long time is OK for use case, then healthy top half is also possible. So, it is a totally design decision.

Linux provides three mechanism to implement bottom half.

1. softIrq : 

It is a vector of 10 different entries (in kernel 3.16) supporting variety of bottom half processing also called software interrupt. All entries are shown by the image below.

Linux/include/linux/interrupt.h

• Softirqs are statically allocated at compile-time. So there are fixed number of softirq and they run in priority order.
• Softirqs have strong CPU affinity, so they are reserved for most of time critical and important bottom half processing on the system.
• softirq is guaranteed to run on the CPU it was scheduled on in SMP systems.
• It Runs in interrupt context, so Interrupt context cannot perform certain actions that can result in the kernel putting the current context to sleep, such as downing a semaphore, copying to or from user-space memory or non-atomically allocating memory
• it can’t preempted and can’t scheduled
• Atomic execution
• it can run simultaneously on one or more processor, even two of the same type of softirq can run concurrently

Softirqs are most often raised from within interrupt handlers. First the interrupt handler(top half) performs the basic hardware-related work, raises the softirq, and then exits. After the kernel is done processing interrupts, it checks wither any of the softirqs have been raised or not. Code flow in Linux kernel for interrupt handling is explained below.

Interrupt

 | do_IRQ()  (top half which masks all interrupts and invoke softirq)

   | irq_exit() (Release all masked interrupts)

     | invoke_softirq() (Kernel checks for the any pending invoked irq)

       | do_softirq() (Execution of softirq (bottom half) with )

2. Tasklet

Tasklets are build on top of softirq. The central idea of tasklet is to provide rich bottom half mechanisum. Only below points diffres from softirq.

• Tasklets have a weaker CPU affinity than softirqs
• Unlike softirqs, tasklets are dynamically allocated.
• A tasklet can run on only one CPU at a time.
• Runs in interrupt contex, Interrupt context cannot perform certain actions that can result in the kernel putting the current context to sleep, such as downing a semaphore, copying to or from user-space memory or non-atomically allocating memory
• Atomic execution
• Two different tasklets can run concurrently on different processors, but two of the same type of tasklet cannot run simultaneously on same processor.
• Tasklet is strictly serialized wrt itself, but not wrt another tasklets.
• Tasklet runs on same CPU from where is raised

Why softIRQ if tasklet is there ?

I’ll explain the need of softirq. That’s the networking code. Say you get a network packet. But to process that packet, it takes a lot of work. If you do that in the interrupt handler, no other interrupts can happen on that IRQ line. That would cause a large latency to incoming interrupts and perhaps you’ll overflow the buffers and drop packets. So the interrupt handler only moves the data off to a network receive queue, and returns. But this packet still needs to be processed right away. Before anything else. So it goes off to a softirq for processing.But Now you still allow for interrupts to come in. Perhaps the network interrupt comes in again on another CPU. The other CPU can start processing that packet with a softirq on that CPU, even before the first packet was done processing. the same tasklet can’t run on two different CPUs, so it doesn’t have this advantage. In fact if a tasklet is scheduled to run on another CPU but is waiting for other tasklets to finish, and you try to schedule the tasklet on a CPU that’s not currently processing tasklets, it will notice that the tasklet is already scheduled to run and not do anything. So tasklets are not so reliable when it comes to latencies.

3. Workqueue

Workqueues are also like tasklets. They are useful to schedule a task that for future. There is some identical difference between  two,

Runs in kenrel process context. Because work queues run in process context (kernel threads), they are capable of sleeping

• Non atomic execution.
• Workqueue runs on same CPU from where is raised
• Higher latency compared to tasklet.

NOTE: This article contains only conceptual details of Interrupt handling mechanism of Linux.