A few weeks ago, I’ve had to look into getting a QLogic QLE7342 InfiniBand adapter working on a Debian Squeeze system, with OFED 1.5.2. This post will probably save quite some time to anyone trying to do the same; it applies to all the QLogic adapters supported by the ib_qib kernel module.
Note: on the ibverbs side of things, the adapters are supported by the ipath plugin, just like older QLogic adapters (that use the ib_ipath kernel module).
First of all: grab a recent version of the ofa-kernel package. The ib_qib module in ofa-kernel from OFED 1.5.2 did not work at all for me. I used the ofa-kernel snapshot from 20110203.
There are two things to know about the QLogic adapters:
- they don’t autodetect the fabric speed by default, a dedicated utility has to be used to set the speed(s)
- the driver exports a sysfs-like pseudo-filesystem, ipathfs, that needs to be mounted
QLogic offers a complete IB stack based on OFED and dubbed “QLogic OFED+”. The complete package weighs in at over 500 MB and contains the QLogic-blessed drivers, OFED stack, libraries and, of utmost importance to us, the QLogic-specific utilities.
Unfortunately, as simple as the QLogic utilities are, they don’t come with source. The package also only exists for RedHat 4 and 5.
So, go to the QLogic support website, select your hardware and download the QLogic OFED+ Host Software for RHEL 5 package. In the tarball, one directory contains the OFED stack and another contains all the utilities (QLogic-Tools.*).
For setting the adapter speed, you’ll want the iba_portconfig utility along with the initscript (shipped as iba_portconfig.sh). The desired adapter speed is set in the initscript by choosing the proper arguments to iba_portconfig (-s 1 for SDR, -s 2 for DDR, -s 4 for QDR or any combination thereof).
The iba_portconfig initscript must run after the drivers have been loaded. Which brings us to loading the driver and mounting the ipathfs.
This is all handled by the openibd initscript provided in ofa-kernel (under ofed_scripts/) and its companion script dedicated to QLogic adapters, truescale.cmds.
This initscript will load the OFED stack and drivers; if a QLogic adapter is present, it’ll mount the ipathfs in the right place.
Voila, once this is done, the adapter should happily report itself ACTIVE/LinkUp.