HPE 3PAR Interrupt Coalescing

I’m currently working on a case with HPE (Hewlett Packard Enterprise) 3PAR support with regard to excessive retransmit errors on our Remote Copy Fibre Channel (RCFC) connection between two 7400 arrays. While troubleshooting we found that a required option was not set appropriately – namely interrupt coalescing on the RCFC ports. In this post I will cover how to change the setting and in brief what it actually means.

 

Interrupt Coalescing – what is it?

First off let us discuss what interrupt coalescing is and how it relates to the 3PAR remote copy ports.

Without delving too deeply into the topic we can simply say that interrupt coalescing is the process by which I/O that would normally trigger an interrupt/context switch to the CPU is gathered together (coalesced) until we either have a certain number of I/O’s or we hit a time value. To give an example we might have a system which is configured to wait for 10 I/O’s to coalesce before interrupting the CPU. Alternatively it might be set to coalesce I/O’s for 2 milliseconds and then send the interrupt.

The idea of interrupt coalescing is to reduce the interrupt/context switching load on processors and in the right scenarios it can be beneficial.

If we consider the HPE 3PAR array specifically and RCFC then the following is accurate to my knowledge –

  • Interrupt coalescence is a way of reducing the number of context switches that the CPU makes when processing events from the fibre channel HBA (Host Bus Adapter)
  • If you enable coalescence you are telling the HBA to delay the generation of interrupts until either:
    • Two interrupt events have arrived
      OR
    • 1ms has elapsed since an interrupt event has arrived
  • If  the I/O profile is multithreaded and generates lots of concurrent I/O then most of the time condition 1 will apply
  • If you have a single threaded application that sends one I/O at a time you will fall into condition 2

 

In this scenario we have two HPE 3PAR 7400 arrays which are 20 miles apart using Remote Copy Fibre Channel (RCFC) to synchronously replicate to one another. Below is a screenshot from the management console which shows the connectivity – we have two 8Gb ports for replication traffic.

 

3PAR Remote Copy Configuration Links

 

Configure Interrupt Coalescing on HPE 3PAR

First let us check which ports have interrupt coalescing (or IntCoal as the 3PAR calls it) configured. To do this on the command line we can make use of the showport command with the –par switch.

 

 

We can also check via the GUI, both through the traditional management console (IMC) and the newer web StoreServ Management Console (SSMC) –

IMC IntCoal Settings

SSMC IntCoal Settings

 

As you can see the RCFC port in question has IntCoal enabled – it should be noted it is enabled by default so it is up to the storage administrator to know to disable it.

The steps involved are as follows –

  • Stop Remote Copy (RC) groups
    • Changing IntCoal settings can result in the port going offline for a short time (seconds) therefore it is safer to stop remote copy synchronisation while making the changes
  • Change IntCoal setting via the command line
  • Start Remote Copy groups

As you can see the process is very simple, it’s just important to make sure you stop your RC groups before and be aware the port going offline will trigger an alert.

We leverage the controlport command as shown below –

 

 

As you can see I ran the command twice to configure both ports – please note you need to configure this on all RCFC ports involved, i.e if you have two arrays the setting must be configured on both.

If we now check the settings on one of the arrays we can see the RCFC ports now have IntCoal disabled. Again we leverage the showport command with the –par switch –

 

 


I hope the information in this post has been useful – if you have any thoughts or questions please drop a comment below.

Leave a Reply