Free Software. Free Speech. That’s the way it works.
After months working on getting the cluster up and running via proof-of-concept, demos, tests, meetings, negotiations and the like, the press release is now out!
We’ve got that big thingy running and it’s doing pretty well.
Been there, done that, waiting for the t-shirt any moment now (hint, hint) :)
Long time no see! I’ve just packaged Digi’s latest dgap beta release, dgap 1.3-21, for Squeeze.
The packages are available at the usual address, using this APT source:
deb http://debian.technologeek.org squeeze non-free
I have made some changes to the packaging as far as udev and firmware loading is concerned, but this is nothing major. Firmware files moved out of /usr, along with udev-related utilities.
Should you notice any issue with the packages or the APT repository, please let me know.
The previous packages, version 1.3-15, are still available for Lenny, APT source unchanged:
deb http://debian.technologeek.org lenny non-free
A few weeks ago, I’ve had to look into getting a QLogic QLE7342 InfiniBand adapter working on a Debian Squeeze system, with OFED 1.5.2. This post will probably save quite some time to anyone trying to do the same; it applies to all the QLogic adapters supported by the ib_qib kernel module.
Note: on the ibverbs side of things, the adapters are supported by the ipath plugin, just like older QLogic adapters (that use the ib_ipath kernel module).
First of all: grab a recent version of the ofa-kernel package. The ib_qib module in ofa-kernel from OFED 1.5.2 did not work at all for me. I used the ofa-kernel snapshot from 20110203.
There are two things to know about the QLogic adapters:
QLogic offers a complete IB stack based on OFED and dubbed “QLogic OFED+”. The complete package weighs in at over 500 MB and contains the QLogic-blessed drivers, OFED stack, libraries and, of utmost importance to us, the QLogic-specific utilities.
Unfortunately, as simple as the QLogic utilities are, they don’t come with source. The package also only exists for RedHat 4 and 5.
So, go to the QLogic support website, select your hardware and download the QLogic OFED+ Host Software for RHEL 5 package. In the tarball, one directory contains the OFED stack and another contains all the utilities (QLogic-Tools.*).
For setting the adapter speed, you’ll want the iba_portconfig utility along with the initscript (shipped as iba_portconfig.sh). The desired adapter speed is set in the initscript by choosing the proper arguments to iba_portconfig (-s 1 for SDR, -s 2 for DDR, -s 4 for QDR or any combination thereof).
The iba_portconfig initscript must run after the drivers have been loaded. Which brings us to loading the driver and mounting the ipathfs.
This is all handled by the openibd initscript provided in ofa-kernel (under ofed_scripts/) and its companion script dedicated to QLogic adapters, truescale.cmds.
This initscript will load the OFED stack and drivers; if a QLogic adapter is present, it’ll mount the ipathfs in the right place.
Voila, once this is done, the adapter should happily report itself ACTIVE/LinkUp.
As part of my work for EDF, I’ve had to package and integrate a set of tools for performance measurement and profiling of HPC code. This toolbox comprises tools for analysis of both sequential and parallel codes, MPI communications profiling and, of course, visualization frontends.
Without going into too much details, here’s the list:
If you’ve been anywhere near HPC code, the names probably ring a bell; they’re the best tools out there in their category, developed and used by the top laboratories.
Over the past year, all the tools have received some level of testing, meaning we know they do work at the very least to some extent. As you can imagine, testing such tools is no easy task and takes an insane amount of time and resources of all kinds.
Testing is all the more important that I had to produce a number of patches to integrate the tools properly in the distribution and, in some cases, to even get them to build.
Now, we would like to share this work with the HPC community in and around Debian. How exactly we are going to do that isn’t clear just yet; most probably, we’ll end up building a team with other interested parties and offer our packages as a base to build upon.
There are a number of challenges with these tools: they’re not easy to build, they’re not easy to maintain, they’re not easy to use, they’re not easy to understand. Basically, nothing is easy. Some of those tools were never meant to be packaged and integrated in a distribution and no sane amount of patching will fix that, so we have to live with packages that aren’t quite as polished as we like them to be.
And then, there are licenses. Some tools are non-free due to usage restrictions. Others rely on non-free dependencies. Although I’ve been looking at the licenses, a thorough license check will be required and decisions will need to be made.
It’s not for the faint of heart! If you are interested in these tools and in bringing them to Debian, please get in touch.
I’ve also had to package the Apache Derby database (Java); if someone out there cares about Derby, I’d be more than happy to provide my packages as a start base for getting Derby into Debian. The packages need some work by someone who knows a thing or two about Derby and who can test and enhance the packaging of the server part.
A few weeks ago, I’ve been busy at work updating the OFED packaging for OFED 1.5.2. We needed a version of the stack newer than what’s (partially) available in Squeeze. As this has now been tested, we are all happy to contribute this work to Debian, courtesy of my customer, EDF.
This morning, I’ve pushed out packages for OFED 1.5.2 to the pkg-ofed SVN repository on Alioth. The packages should be made available in experimental at some point too, though a number of them will have to go through NEW.
With this update, I’ve cleaned up & updated the packaging where needed and also made two important changes that I need to highlight:
A number of libraries have changed their interfaces and bumped their soversions. libibcommon is gone, it has been folded into libibumad. The API/ABI between libibverbs and the IB driver modules has changed, so the new libibverbs comes with appropriate Breaks in place.
Thanks to Benoît Mortier from pkg-ofed for his 2-day IB & OFED crash course that was instrumental in getting started with the OFED maze.
Thanks to the work of Mark Heily, both upstream and in Debian, this week saw the arrival of libdispatch in Debian.
libdispatch is the userspace component of Apple’s Grand Central Dispatch technology, designed to help write applications that really take advantage of multicore processors without having to fiddle directly with threads and locking (and getting it terribly wrong).
To get that working on Linux, Mark wrote a userspace implementation of the pthread workqueues and a kqueue compatibility library (libkqueue). He also packaged the Blocks runtime (libblocksruntime) for Debian so as to build libdispatch with Blocks support. That means code using libdispatch must be built with CLang (or llvm-gcc-4.2) rather than gcc, as getting Blocks support in gcc in the near future looks highly improbable.
On Mac OS X and FreeBSD, a kernel-based implementation of the pthread workqueues is used; we don’t have that for Linux yet. The all-userspace implementation we have at the moment works really well; there are some caveats, though, but that’s no reason for not starting to play with libdispatch right now!
Try it out, it’s great. Thanks again, Mark!
When I got my Mac Mini, one of the very first things I did was to test the Debian build of rEFIt on the machine, given the EFI firmware is x64 on the newer machines whereas it’s ia32 on my MacBookPro; it was the very first time I could test the x64 build.
And the results weren’t good: some images couldn’t be loaded, and it all seemed to be random.
Disabling optimization in both GNU EFI and rEFIt for the x64 builds helped somewhat, at least it made rEFIt reliable enough to be usable.
Now, with this new release, I started getting error messages, requiring a key press before the menu would be displayed. It’s annoying but it’s also very helpful as I now had an error message to work with, and even if the error message wasn’t reliable, it would still point me to some part of the code. The message was: Invalid argument.
On x64, the use of a call wrapper is mandatory because the EFI x64 calling convention doesn’t match the calling convention of the ELF executables we’re building and later disguising as EFI executables. The call wrapper has always been the prime suspect in this issue, it’d be an obvious candidate for anyone looking at this issue.
Today, digging into the issue with some new data, I realized what the problem was: when using the call wrapper, all arguments need to be of the UINT64 type. That’s the type the call wrapper uses when extracting the arguments from its va_list.
Introduce some more macros on top of the call wrapper, build, test: voila, it’s fixed! That was tricky one.
Let’s sum it up for the search engines: all the arguments to the EFI interface called through uefi_call_wrapper() in GNU EFI need to be cast to UINT64.
rEFIt 0.14-1, available in unstable in a couple hours.
Next, I’ll see if I can reenable optimization in both GNU EFI and rEFIt.
After weeks of trying to get around to doing that, I am finally starting the process of replacing my 10-year old 1024 bits DSA key with a shiny new 4096 bits RSA key.
The old key ID is F5D65169, the new key ID is FA1E5292; it’s available from the keyservers or my website.
I have put up a transition document, signed with both keys; if you’ve signed my old key, grab it, verify it and if all is OK on your end, please sign the new key.
You’ll notice that one of the old UIDs did not make it to the new key; that’s because I’m not using that email address and don’t actually intend to use it in the future.
Thanks!
Ten years ago, pretty much to the day, I was installing my first Debian system, a frozen Potato, using a release-candidate version of the boot-floppies. A few days later, I was reading the Policy, New Maintainer’s Guide and Developer’s Reference while building my first package.
It’s been a fun ten years, with ups and downs, and I’m looking forward to what’s coming next.
I’ve spent the last week packaging VMware Workstation 7.0.0 at work. Looking around on the net, I’ve been unable to find anything helpful about packaging this new version, so it seems nobody’s got around to packaging it yet.
I’ve asked our customer for its approval for releasing our packaging scripts to the community and got it, so here are our packaging scripts for this version, courtesy of EDF. See the instructions in debian/README.source for what has to be done to turn it into a full source package.
The packaging is based on our previous 6.5.2 packaging, which was itself based (partially, at least) upon the Gento 6.5.2 ebuild. It uses a tweaked vmware-installer to install the products to debian/tmp, then makes use of the vmware-installer database to populate the packages.
I think this method should work with any VMware product using vmware-installer 1.1.
Packages have been built and tested on Etch and Lenny.
Hope it helps! Bugs, comments, questions to the email address listed as Maintainer in debian/control, please :-)