Dr. Brian Robert Callahan
academic, developer, with an eye towards a brighter techno-social life
In maintaining oksh, I want to ensure that as many C compilers and operating systems can successfully build and run the shell. It doesn't necessarily need to be perfect, but it does need to be good enough that a user can get an improved experience compared to what came preinstalled. Even if they go on to use oksh to install GNU Bash, that's fine. That's a win for oksh. Hopefully, they will remember the help they got from some OpenBSD code and pay it back some day.
As a result, I've had the pleasure of using some cool C compilers. For a long time now, oksh has supported Oracle Solaris and Illumos, the open source continuation of OpenSolaris. From the beginning of oksh support for the Solaris family, we have supported building with both GCC and Oracle Developer Studio (formerly known by many names including Oracle Solaris Studio, Sun Studio, Sun WorkShop, Forte Developer, and SunPro Compilers).
However, support for Oracle Developer Studio rested on others who are running Oracle Solaris. I want to change that and get Oracle Developer Studio running on one of my own machines. That is going to mean using one of the Illumos distros, since I am not about to pay Oracle to use Solaris.
I chose OpenIndiana because it is the first Illumos distro I think of, and because it is actively well-maintained. If I am going to invest in learning some Solaris, I would like it to be for the long haul; I want to learn one Illumos distro and stick with it. While I am sure other Illumos distros are also well-maintained, I did not spend much time on deciding. I knew OpenIndiana existed, and I chose it.
With that said, I believe these steps will work on all 64-bit x86 Illumos distros.
Join me on my quest to install Oracle Developer Studio 12.6 on Illumos despite Oracle's best attempts to stop me.
This was very easy. I am using VirtualBox on my Windows 11 laptop, as I have heard VirtualBox has the best support for the Solaris family. This makes sense to me, as Oracle owns VirtualBox as well. I downloaded the OpenIndiana Live DVD and walked through the installation process with VirtualBox. It was very easy to install, nothing to report here. If you're going to have a GUI installation, that's the way to do it.
The latest version of Developer Studio is 12.6. You can download it for free on Oracle's website but you have to sign up for an account with them to do so. Fortunately, Oracle's account creation scripts don't check to see if you're using a throwaway email service, so I did that to get an account.
There are three options when downloading: Solaris 11, Solaris 10, and Linux. Linux is out, since we're running Illumos. The question is whether to use the Solaris 11 package or the Solaris 10 package. The Illumos FAQ states that:
illumos should run binaries and probably even kernel drivers from Solaris 10 update 10 and earlier, reflecting our strong commitment to backwards compatibility and interface stability. More recent versions of Solaris, especially Solaris 11, are not considered compatible -- even if they sometimes seem to be! Oracle develops Solaris 11 as a proprietary fork of OpenSolaris, and has been diverging from illumos for nearly a decade at this point.
However, because I can't help myself, I attempted to install the Solaris 11 package first. Following Oracle's steps to get the package repositories ready for installation just fail on Illumos. I'm not sure why and it was not worth looking into. The Illumos people say Illumos will run Solaris 10 binaries, and Oracle offers a Solaris 10 package of Developer Studio 12.6, so let's go with that. It's also far simpler to download the Solaris 10 package, as that's simply a tarball offered via direct download. The Solaris 11 package requires the use of their complicated package manager.
Now equipped with the Solaris 10 tarball of Developer Studio 12.6, I need to figure out how to install it.
After extracting the tarball, there is a helpful developerstudio.sh script right in the top-level directory that looks like it will do all the installation work for us. Let's run it and see what happens.
It does not take long for the script to error out, informing us that we are running Solaris 11 and this package is for Solaris 10. Fair enough I suppose, but let's not let something as small as that stop us. If we run uname -r, we do get SunOS 5.11 as the output. So we are running Solaris 11. Illumos is technically Solaris 11, even if Oracle Solaris 11 and Illumos have over a decade of divergence at this point.
Reading through the developerstudio.sh script, it really is looking at the output of uname to determine what version of Solaris we are running. More specifically, the script uses uname, uname -a, and uname -r to learn about the system.
Since the source code for Illumos uname(1) is written in C, and we don't have a C compiler on our system, we can't just edit the source code. Yes, I know I could just install GCC from the package manager, but that's cheating and we can do this without it.
We need to lie to the installation script and convince it that we are Solaris 10. I did try editing the script, but if you do that it will claim it has been corrupted and refuse to do much of anything.
We need a program that does this:
#include <stdio.h>
int
main(int argc, char *argv[])
{
if (argc == 1) {
puts("SunOS");
return 0;
}
if (argv[1][1] == 'a')
puts("SunOS openindiana 5.10 illumos-676abcb77c i86pc i386 i86pc");
else
puts("5.10");
return 0;
}
The 676abcb77c is some identifying hash. Probably the git hash. In this program, I changed the instances of 5.11 to 5.10.
Now let's go write that program. Since we don't have a C compiler, or any high-level language compiler it seems, we might have to rely on assembly. I guess we have Java in the form of OpenJDK, but I'm not writing any Java for this.
I guess we need to check to see if OpenIndiana ships with an assembler. It does not appear to. OpenIndiana does ship with a linker, and indeed you can see the source code of the linker driver and the linker library in the Illumos-gate repository, which means that Sun at some point got around to open sourcing the Solaris linker. That's good. Linking binaries and libraries is important.
However, the same appears not to be true of the Sun assembler. We might have to resort to installing the GNU binutils in order to get an assembler. But if we do that, we might as well just install GCC.
In the Developer Studio 12.6 tarball, there is a directory called patches. That directory contains headers and libraries that might not be installed on all Solaris 10 machines but are necessary for Developer Studio to function. Inside the first patch subdirectory, descriptively named 119961-16 (I'm sure it's some internal Oracle code), there are more subdirectories. If we follow them down through SUNWsprot/reloc/usr/ccs/bin, we find a binary named as. That looks like an assembler! Indeed, running as -V shows us this output:
as: Studio 12.5 Compiler Common 12.5 SunOS_i386 Patch 03/07/2016
So we have an assembler after all! As we will see later, this strikes me as a little odd, as the actual Developer Studio package ships a different, updated, assembler to be used with the compiler. Why they didn't just provide a copy of that updated assembler here is a mystery to me. But it's fine, this assembler will work for our needs. I copied the assembler to /usr/ccs/bin/as and made a symlink from /usr/bin/as to /usr/ccs/bin/as since that's what OpenIndiana does for ld.
We can now write our lying uname(1) utility in assembly and get an executable.
The Sun assembler can handle either 32-bit x86 or 64-bit x86. It defaults to 32-bit but if you use the -m64 flag when assembling, that tells it you're working with 64-bit assembly. I chose 64-bit assembly because my 64-bit x86 assembly is far better than my 32-bit x86 assembly.
Here is the program I came up with. You can download it here but I am reproducing it below so that you don't have to download it if you don't want to.
.text .globl _start .type _start,@function _start: movq (%rsp), %rdi leaq 8(%rsp), %rsi movl $4, %eax cmpl $1, %edi jne .Lflags movabsq $.Lstr, %rsi movl $6, %edx syscall jmp .Ldone .Lflags: movl $1, %edi movl $8, %ecx movq (%rsi, %rcx, 1), %rdx movl $1, %ecx movsbl (%rdx, %rcx, 1), %ebx cmpl $97, %ebx jne .Lrflag movabsq $.Lastr, %rsi movl $59, %edx syscall jmp .Ldone .Lrflag: movabsq $.Lrstr, %rsi movl $5, %edx syscall .Ldone: xorl %edi, %edi movl $1, %eax syscall .size _start,.-_start .section .rodata .Lstr: .ascii "SunOS\012\0" .Lastr: .ascii "SunOS openindiana 5.10 illumos-676abcb77c i86pc i386 i86pc" .ascii "\012\0" .Lrstr: .ascii "5.10\012\0"
We are just straight-up lying. Since we know the installation script only uses uname, uname -a, and uname -r, we can just check the number of arguments given to the program. If it's one, then the script issued uname. If it's two, then we have one of the two flags; check if it's a and if not go down the r route without doing any additional checking because we don't care. We're just here to lie and get out as quickly as possible. I needed to find an Illumos syscall table and it turns out the two syscalls we need, _exit(2) and write(2), have the same syscall numbers as OpenBSD. That makes things very simple.
I was able to run as -m64 -o uname.o uname.s to assemble the code and then ld -o uname uname.o to link it. The result was a working lying uname(1) utility. On OpenIndiana, uname(1) lives in /sbin, so I ran sudo mv /sbin/uname /sbin/uname2 followed by sudo mv uname /sbin/ to install our lying utility.
Now that we can convince the installation script that we're Solaris 10, let's try to install again.
While we do get further, we don't get too much further. It turns out the installation wizard is written in Java and only Oracle Java will do. Since we only have OpenJDK on the machine, the installation script errors out.
Fortunately, you can download a package of Java 8 for Solaris here. You don't need to install this version of Java; it's enough to extract is somewhere in your HOME directory and use the --javahome flag with the installation script to point to it. Easy enough.
Unfortunately, this version of Java won't run as-is. Our system is missing some support libraries that Java needs to run.
The Developer Studio tarball includes an install_patches.sh script to install all the patches you need to install Developer Studio. However, the script calls utilities that our system doesn't have. I am not going to bother trying to find them; I'm going to write a shell script to install the patches. You can download the script here but as with our lying uname(1), I'll reproduce it here so that you don't have to download it if you don't want to. The script assumes you extracted the tarball into your HOME directory.
#!/bin/bash cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119961-16/SUNWsprot/reloc/usr/ccs/bin sudo install -c /usr/bin -m 555 -u root -g root as cd /usr/ccs/bin sudo ln -s ../../bin/as as cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119961-16/SUNWsprot/reloc/usr/lib/amd64 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libtdf.so.1 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libxprof_audit.so.1 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libxprof.so.1 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119961-16/SUNWsprot/reloc/usr/lib sudo install -c /usr/lib -m 755 -u root -g bin cm4defs sudo install -c /usr/lib -m 755 -u root -g bin libtdf.so.1 sudo install -c /usr/lib -m 755 -u root -g bin libxprof_audit.so.1 sudo install -c /usr/lib -m 755 -u root -g bin libxprof.so.1 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119964-35/SUNWlibC/reloc/usr/include sudo install -c /usr/include -m 644 -u root -g bin demangle.h cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119964-35/SUNWlibC/reloc/usr/lib/amd64 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libCrun.so.1 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libCrunG3.so.1 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libCstd.so.1 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libdemangle.so.1 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libiostream.so.1 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119964-35/SUNWlibC/reloc/usr/lib sudo install -c /usr/lib -m 755 -u root -g bin libC.so.5 sudo install -c /usr/lib -m 755 -u root -g bin libCrun.so.1 sudo install -c /usr/lib -m 755 -u root -g bin libCrunG3.so.1 sudo install -c /usr/lib -m 755 -u root -g bin libCstd.so.1 sudo install -c /usr/lib -m 755 -u root -g bin libdemangle.so.1 sudo install -c /usr/lib -m 755 -u root -g bin libiostream.so.1 # Begin potential confusion # This script does not install libm.so.1, libm.so.2, or libmvec.so.1 # These already exist in OpenIndiana, might break things? # Only install the sunmath bits cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119967-02/SUNWlibm/reloc/usr/include sudo install -c /usr/include -m 644 -u root -g bin sunmath.h cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119967-02/SUNWlibmr/reloc/lib/amd64 sudo install -c /lib/amd64 -m 644 -u root -g bin llib-lsunmath.ln cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119967-02/SUNWlibmr/reloc/lib sudo install -c /lib -m 644 -u root -g bin llib-lsunmath sudo install -c /lib -m 644 -u root -g bin llib-lsunmath.ln cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119967-02/SUNWlibmsr/reloc/lib/amd64 sudo install -c /lib/amd64 -m 644 -u root -g bin libsunmath.so.1 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/119967-02/SUNWlibmsr/reloc/lib sudo install -c /lib -m 644 -u root -g bin libsunmath.so.1 # End of potential confusion cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/120754-17/SUNWlibm/reloc/usr/include sudo install -c /usr/include -m 644 -u root -g bin libmtsk_db.h sudo install -c /usr/include -m 644 -u root -g bin omp_collector_api.h sudo install -c /usr/include -m 644 -u root -g bin omp.h cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/120754-17/SUNWlibmsr/reloc/lib/amd64 sudo install -c /lib/amd64 -m 755 -u root -g bin libmmheap.so.1 sudo install -c /lib/amd64 -m 755 -u root -g bin libmtsk_db.so.1 sudo install -c /lib/amd64 -m 755 -u root -g bin libmtsk.so.1 sudo install -c /lib/amd64 -m 755 -u root -g bin libstkovf.so.1 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/120754-17/SUNWlibmsr/reloc/lib sudo install -c /lib -m 755 -u root -g bin libmmheap.so.1 sudo install -c /lib -m 755 -u root -g bin libmtsk_db.so.1 sudo install -c /lib -m 755 -u root -g bin libmtsk.so.1 sudo install -c /lib -m 755 -u root -g bin libstkovf.so.1 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/152227-01/SUNWlibm/reloc/usr/lib/amd64 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libfsu_db.so.1 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libfsu.so.1 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/152227-01/SUNWlibm/reloc/usr/lib sudo install -c /usr/lib -m 755 -u root -g bin libfsu_db.so.1 sudo install -c /usr/lib -m 755 -u root -g bin libfsu.so.1 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/152229-01/SUNWlibm/reloc/usr/include sudo install -c /usr/include -m 644 -u root -g bin sunperf.h cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/152229-01/SUNWlibm/reloc/usr/lib/amd64 sudo install -c /usr/lib/amd64 -m 755 -u root -g bin libsunperf.so.3.2 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/152229-01/SUNWlibm/reloc/usr/lib sudo install -c /usr/lib -m 755 -u root -g bin libsunperf.so.4.2 cd ~/OracleDeveloperStudio12.6-solaris-x86-pkg/patches/system/152229-01/SUNWlibm/reloc/usr/lib/locale/C/LC_MESSAGES sudo install -c /usr/lib/locale/C/LC_MESSAGES -m 644 -u root -g bin SUNW_SPRO_libsunperf.cat
As you can see, this script begins by installing the assembler as I did by hand earlier; if you want to install Developer Studio 12.6 on your own Illumos machine you can start with this script.
There was one set of patches that I hesitated with; it is surrounded by comments in the script. One of the patches wanted to replace libm.so.1 and family with new versions. I'm not sure if it makes sense to stick with Illumos libm or use this Oracle libm. I ultimately chose to keep the Illumos libm. I would be interested in talking with some Illumos developers to figure out if it makes more sense to keep the Illumos libm or replace it with the Oracle libm.
In all other cases, the patches installed files that do not ship in OpenIndiana. So I installed them all.
Now Java 8 should work. I also had to make a new directory named ~/temp for temporary files during installation.
Running sudo ./developerstudio.sh --tempdir ~/temp --javahome ~/jdk1.8.0_361 greeted me with an installation wizard. The wizard is very upset and warned me that I am not running official Oracle Solaris and that I might be missing some packages needed for runtime. Those are both problems I either know and don't care about, or problems I can deal with later. I left all the default options and kept pressing Next until installation began.
Installation completed without any issues.
At this point, I no longer needed our lying uname(1) utility, so I deleted it with sudo rm /sbin/uname and put the original utility back with sudo mv /sbin/uname2 /sbin/uname. You can also delete the temporary directory with rm -rf ~/temp and Java 8 if you no longer want it with sudo rm -rf ~/jdk1.8.0_u361.
Turns out you can't compile all that much, since OpenIndiana does not install system headers by default. I know Linux does this too, but I really am spoiled by the BSD mentality of a default install should include everything you need to build more software.
I guess we're going to need to learn the package manager after all...
The Solaris family uses a complicated package manager called the Image Packaging System, or IPS for short. I am sure that once you get to know it and understand its quirks it is perfectly serviceable. However, after more than a decade with the OpenBSD package manager, it is difficult to get my brain to think the way other package managers want me to think. It took me way too long to figure out how to search for packages and find the packages I needed.
I first tried to install the system headers using sudo pkg install header, since the package browser says that's what the package is called. But it kept on failing, telling me something about being blocked by a consolidation/osnet/osnet-incorporation package. I eventually figured out I needed to run sudo pkg install consolidation/osnet/osnet-incorporation. If I understand it correctly, this is how you update the system, and the system needed to be updated before it would let me install any other packages.
The package manager is a little slow, so you need to wait a little for the updates to complete. It was not terribly long, a little more than a couple of minutes. The package manager recommended I restart the machine before continuing on, so I did.
Now I could install the system headers. This time, it worked without issue.
Hopefully now I could compile code.
I could now successfully compile and assemble code. But I could not link. The linker was complaining that it could not find /usr/lib/values-xpg6.o. I eventually discovered this file is necessary for C99 support. After searching around the package repository, I discovered this file was part of the c-runtime package. I ran sudo pkg install c-runtime and that installed the necessary runtime support.
Finally, I could compile, assemble, and link C programs! I finally had Oracle Developer Studio 12.6 running on OpenIndiana.
Interestingly, the installation includes a /opt/developerstudio12.6/bin/sunas that identifies itself as:
as: Studio 12.6 Compiler Common 12.6 SunOS_i386 2017/05/30
As far as I can tell, this is the assembler that is used in the compiler pipeline. Why Oracle included an older assembler in the patches remains a mystery, but I can't imagine that it matters too much. Maybe when Developer Studio 12.7 comes out they can sync the assemblers. So that I didn't have two potentially incompatible assemblers on my system, I replaced the old assembler in /usr/bin with the new one by running sudo cp /opt/developerstudio12.6/bin/fbe /usr/bin/as.
While there may be hidden gaps in support that require additional work, I can confidently say that I am able to install and run Oracle Developer Studio 12.6 on an Illumos machine despite Oracle's best attempts to prevent me from doing so.
I don't blame the Illumos or OpenIndiana people for the lies we encountered; this is Oracle's fault. In fact, I enjoyed using OpenIndiana so much I think I will install it on real hardware, on one of my little netbooks. That seems like a good machine to run OpenIndiana on.
I'm happy to work with Illumos and/or OpenIndiana (or any other Illumos distro) people on making installation of Developer Studio as seamless as possible.
And yes, I am able to successfully build a working oksh on OpenIndiana using Oracle Developer Studio 12.6, so for my use case, mission accomplished.
