Dr. Brian Robert Callahan
academic, developer, with an eye towards a brighter techno-social life
TL;DR: Grab the patch here.
I think pkgsrc from NetBSD is great. The idea that one can have a truly portable packaging system that will work on (nearly) any Unix-based operating system is great. And genuinely the proof is in the pudding—pkgsrc has bulk builds for NetBSD, Solaris/SmartOS/Illumos, Linux, and macOS. That covers a huge assortment of Unix systems you might want to use. Additionally, pkgsrc has varying levels of support for other Unix systems, such as FreeBSD, OpenBSD, Haiku, HP-UX, and UnixWare.
One Unix that pkgsrc formerly supported is AIX, the IBM Unix system. It seems that at some point in time, testing fell by the wayside and pkgsrc no longer bootstraps on AIX. Indeed, there was an email from April 2022 that stated that pkgsrc did not successfully bootstrap on AIX 7.1 using GCC 8.3 as the compiler.
The state of packages on AIX genuinely surprises me. IBM hosts a repository of open source software for AIX. But it seems pretty sparse compared to what you could get with pkgsrc. Another website offering AIX packages seems quite old. I think pkgsrc would be a great way to bring modern packages to AIX.
I am not the first to think this. There are AIX 7.2 pkgsrc packages available at this repository, however all the packages are compiled as 32-bit RISC System/6000 objects. I would greatly prefer to have everything be 64-bit XCOFF objects, as we could do more with 64-bit programs. There also aren't too many packages in that repository, so I think starting fresh is in our best interest.
As we shall see, this was not as straightforward as I would have hoped.
Let's get to work.
I do not have any POWER machines at home, so I resorted to using QEMU, which can run AIX 7.2 and earlier. I was able to find install media for AIX 7.2 TL4 and was able to install it on a QEMU virtual machine. My virtual machine has 3 CPUs and 8 GB of RAM with a 512 GB virtual hard drive. That will be more than enough RAM and disk space for our needs, but the CPU will still be incredibly slow even when building using all 3 CPUs. Extra so as we are using that Intel N100 mini PC that we have seen before.
After installation, I needed to follow this tutorial to set up networking. I also needed to write a simple /etc/resolv.conf file that contained the single line nameserver 192.168.0.1 so that DNS would work.
AIX 7.2 TL4 comes with a version of OpenSSL too old to run their dnf packaging tool. To overcome this, you need to download the AIX package of OpenSSL (3.0.15, as of this blog post). But you need an IBMid to download it, even though it doesn't cost anything and they don't even ask if you have an AIX license. Fortunately, I have an IBMid from $DAYJOB so I downloaded it onto my MacBook Air and served it up to the AIX machine.
I used the /usr/opt/perl5/bin/lwp-download utility on the AIX machine to download it. I could not find curl or wget on the machine. I discovered this utility studying the dnf_aixtoolbox.sh script, which I would need next anyway to get the AIX Toolbox packages installed.
Following the IBM tutorial, I installed OpenSSL 3.0.15 by extracting the tarball and then running installp -qaXFY -d . openssl.base openssl.license openssl.man.en_US. Not so bad.
Now I could download the AIX Toolbox setup script with /usr/opt/perl5/bin/lwp-download https://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/ezinstall/ppc/dnf_aixtoolbox.sh. I had to increase the size of the /opt filesystem to accommodate all the packages which I did with chfs -a size=+128G /opt. Definitely overkill right now, but I plan to eventually put the pkgsrc packages into /opt/pkg.
You install the dnf tool with ksh dnf_aixtoolbox.sh -d. The -d says you don't have yum, the previous package utility, installed on the machine and you will only use dnf. We are only going to use dnf long enough to switch over entirely to pkgsrc, and future AIX installs will not need dnf at all, being able to use pkgsrc completely.
The easy installer grabs a 276 MB file, dnf_bundle_aix_71_72.tar and installs it; you should then update your packages with dnf update as the easy installer recommends.
You then need to grab a compiler. The latest in the repository is GCC 12.3.0, so I used that. You can install them with:
dnf install gcc12 gcc12-c++ gcc12-cpp less make tar xz wget
I also grabbed some additional packages to make working with pkgsrc easier, like GNU tar and the xz utilities.
You cannot use the default GCC 10, as it does not understand the -m64 flag on AIX; GCC 12 does. The alternative would be to change all references to -m64 to -maix64 just for AIX, and that felt like too much work for too little benefit.
I discovered that libarchive silently assumes pthreads, but AIX does not automatically add the -lpthread flag. But this is simple to add in the bootstrap script in the AIX section you need to add:
LDFLAGS+= -lpthread export LDFLAGS
There is also a problem with the AIX ar, strip, and similar tools: by default they only understand 32-bit objects. We can tell these programs to autodetect and do the right thing with both 32-bit and 64-bit objects by adding this to the AIX section of the bootstrap script:
OBJECT_MODE= 32_64 export OBJECT_MODE
Now you can bootstrap pkgsrc with:
./bootstrap --abi 64 --compiler gcc --full --gzip-binary-kit /opt/aix72-pkgsrc-2025Q1.tgz --make-jobs 3 --prefer-pkgsrc yes --prefix /opt/pkg
Whenever you install a package, you are greeted with this warning:
pkg_add: libnbcompat-20250303: warning: was built for a different platform:
pkg_add: AIX/powerpc64 7.2 (pkg) vs. AIX/powerpc64 2 (this host)
I do not know why pkg_add thinks our system is AIX/powerpc64 2. The build platform is correct, and the packages work fine, so this appears to be cosmetic, though it would be great if it were fixed.
Now I had a bootstrapped pkgsrc, but it would prove pkgsrc itself that would be my undoing. For some reason, every time I tried to build a package, pkgsrc would insist on trying to build GCC 6.5, even though I was using GCC 12.3.0. I don't know why this happens, and looking through the documentation did not uncover any answers. This does not happen on my PowerPC Mac OS X pkgsrc system using GCC 14.2.0. So perhaps it is something about AIX.
IBM has a proprietary compiler called IBM Open XL C/C++ for AIX which actually uses LLVM and Clang instead of their previous proprietary frontend and backend. As far as I can tell, it is simply Clang with some additional proprietary LLVM optimization passes to better use the POWER CPUs found in IBM's machines. The latest version of this compiler as of this writing, 17.1.3, is really LLVM 19.1.7. So one would think great, I'll download this compiler and use that for pkgsrc.
Except no, this compiler costs a staggering $7,010 per authorized user, or $21,200 per concurrent user, or you can use it on an AIX cloud instance for $107 per VPC per month.
So much for it being "open" (gotta love marketing speak). However, for pkgsrc I am not sure that we need all the super fancy optimizations that IBM keeps locked away; after all, the fully free version of LLVM works fine for pkgsrc on all the free Unix systems, and even Apple has the good sense to provide their version of LLVM for free.
So one would think you could grab a package from the AIX open source repository or somewhere else and use that.
Nope!
IBM does not provide such a package, and I cannot find anyone else offering modern AIX packages online.
Eventually, after much trial and error, I stumbled upon this CMake invocation, combined with the environment variable OBJECT_MODE set to 64 and an installation of gcc12, gcc12-c++, gcc12-cpp, cmake, cmake-data, and ninja-build packages from the AIX package repository, to build our stage1 LLVM:
cmake -G Ninja -S llvm -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER="gcc-12" -DCMAKE_CXX_COMPILER="g++-12" -DCMAKE_C_FLAGS="-m64 -pthread" -DCMAKE_CXX_FLAGS="-m64 -pthread" -DLINK_OPTIONS="-lpthread" -DLLVM_ENABLE_PROJECTS="clang;compiler-rt" -DLLVM_HOST_TRIPLE="powerpc64-ibm-aix7.2.0.0" -DCMAKE_INSTALL_PREFIX="/usr" -DLLVM_TARGETS_TO_BUILD="Native" -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" -DLLVM_RUNTIME_TARGETS="powerpc64-ibm-aix7.2.0.0"
And then you build it with:
ninja -C build runtimes
And then you wait a very long time. I added the -v flag to my ninja invocation so I could more easily see what was going on. This took more than 24 hours on my virtual machine.
The -pthread is intentional; GCC on AIX has a split threads/no threads ABI, and you cannot use things like C++11 mutexes unless -pthread is passed. LLVM does not suffer from this deficiency.
I learned part way through the stage1 build that I did not need to manually set -m64 or the default target triple. That's fine for this compiler; I hoped to fix this when building a stage2 clang.
Except this all failed. I could not get clang working no matter what I tried.
So I turned to one final trick, and this one worked.
We're just going to lie to pkgsrc. We are going to claim that our compiler is clang when it is in fact gcc. You can do that by installing the gcc12, gcc12-c++, and gcc12-cpp packages from the AIX repository, then make two symlinks:
ln -s /opt/freeware/bin/gcc-12 /opt/freeware/bin/clang
ln -s /opt/freeware/bin/g++-12 /opt/freeware/bin/clang++
Then you can bootstrap as follows:
CC=clang CXX=clang++ ./bootstrap --abi 64 --compiler clang --full --make-jobs 3 --prefer-pkgsrc yes --prefix /opt/pkg
This works, and now you can build packages to your heart's content. Now pkgsrc won't try to build GCC 6.5; it will simply use clang (or what it thinks is clang).
We did it!
That was an adventure. One that could have been significantly shortened with some real POWER hardware, or a virtual machine with 32 or more CPUs. In terms of pkgsrc itself, it did not take that much for pkgsrc on AIX to be revived. But we took too long of a detour with LLVM when lying worked even better.
I packed up all our changes into a single diff and sent it to the pkgsrc team. Hopefully it will get included soon so everyone can benefit.
