Dr. Brian Robert Callahan

OK, now I was ready to start compiling code with vbcc.

Codegen errors

But it didn't take long to begin running into codegen errors. Even a basic hello world program would error out during compilation. The first error was somewhat easy to track down: machine/endian.h includes GNU-style inline assembly, and vbcc doesn't support that. I wrapped the offending lines in machine/endian.h with #ifdef __GNUC__ so that vbcc would ignore it but the usual compilers continue to use it.

With that out of the way, here is our hello world program:

#include <stdio.h>

int
main(int argc, char *argv[])
{

        puts("Hello");

        return 0;
}

I ran vc hello.c and was greeted with the following error messages:

/home/brian $ vc hello.c
/tmp/tmp.0.asm: Assembler messages:
/tmp/tmp.0.asm:45: Error: invalid character '?' in mnemonic
/tmp/tmp.0.asm:54: Error: invalid character '?' in mnemonic
/tmp/tmp.0.asm:64: Error: invalid character '?' in mnemonic
/tmp/tmp.0.asm:74: Error: invalid character '?' in mnemonic
/tmp/tmp.0.asm:84: Error: invalid character '?' in mnemonic
/tmp/tmp.0.asm:94: Error: invalid character '?' in mnemonic
/tmp/tmp.0.asm:104: Error: invalid character '?' in mnemonic
/tmp/tmp.0.asm:114: Error: invalid character '?' in mnemonic
as "/tmp/tmp.0.asm" -o "/tmp/tmp.0.o" failed

OK, that's strange. But at least we have something to look for in the resulting assembly file: a bunch of question marks. vbcc's compiler driver has many similarities with other C compilers, so I re-ran the compilation with the -S flag to generate assembly. Sure enough, I found the first culprit:

	and?	$18374686479671623680,%eax/%edx

That's definitely not a valid instruction. There is no and? mnemonic and there is no %eax/%edx register. There were a few other similar instructions that began with and? and ended with %eax/%edx. It seems like the code generator cannot handle the situation where you are anding together two 64-bit integers.

Unfortunately, the code generator strips away all contextual information when writing out assembly, such as the names of the functions, so I had to figure out another way of figuring out which function this codegen error was in. Fortunately, vc has two switches, -ic1 and -ic2 which lets you write out the intermediate code before optimizing (-ic1) and after optimizing (-ic2). So I recompiled with -ic1 and learned that this codegen error is in the __swap64md function.

Because we are not using the machine-dependent inline assembly version of __swap64md (as that was one of the inline assembly functions we just wrapped in __GNUC__ so that vbcc won't be able to see it), OpenBSD falls back to a C version which lives in sys/_endian.h. It looks like this:

#define __swap64gen(x)                                                  \
        (__uint64_t)((((__uint64_t)(x) & 0xff) << 56) |                 \
            ((__uint64_t)(x) & 0xff00ULL) << 40 |                       \
            ((__uint64_t)(x) & 0xff0000ULL) << 24 |                     \
            ((__uint64_t)(x) & 0xff000000ULL) << 8 |                    \
            ((__uint64_t)(x) & 0xff00000000ULL) >> 8 |                  \
            ((__uint64_t)(x) & 0xff0000000000ULL) >> 24 |               \
            ((__uint64_t)(x) & 0xff000000000000ULL) >> 40 |             \
            ((__uint64_t)(x) & 0xff00000000000000ULL) >> 56)

vbcc does understand the shifts, just not the ands. This proved to be a difficult bug to track down.

Discovering libcall

Clang and GCC have support libraries that handle low-level arithmetic and other functions. For clang, that library is called libcompiler_rt. For GCC, it is called libgcc. vbcc has a similar library called libvc. It turns out that libvc is more than just a low-level support library, but also has a good chunk of a libc in it. Unfortuntately, libvc is not source-available, or at least I couldn't find it if it is, and the only place I could get a copy was in the binary tarball I had downloaded where I found the config file templates.

In the compiler, this support library is referred to as libcall. I don't know why the names are different. The i386 is set to unconditionally use libcall. Within libcall, there are a number of builtins for 64-bit arithmetic of all kinds. Indeed, anding two 64-bit numbers together is one of the builtins.

The code to use libcall, however, was missing and from its list of arithmetic and logical operations. Once I added it, the problem immediately went away and vbcc was correctly producing code to and together two 64-bit integers.

It turned out the libcall builtins list was also missing the code to logical not a 64-bit integer. I only discovered that much later when trying to link oksh. But the fix was easy: I just added it to the builtins list and that fixed the linker errors.

Fixing libvc

But I wasn't finished yet. Now, the linker would complain that it was missing a number of symbols. These were the builtin symbols I had just learned about from libcall. So I had to implement them.

Or, well, I just had to reorganize them into a way that made sense for OpenBSD. Like I mentioned before, libvc is not just a compiler support library but also has a bunch of libc functions. I would rather use OpenBSD libc instead. Fortunately, libvc is a simple Unix archive file and not a shared library, so there was hope that I could just extract out the functions I needed and rearchive just those object files.

Using objdump I examined the disassembly for all the low-level support functions and discovered that they were all generic i386 code with no system calls used. That means that even though they were originally compiled on and for Linux, I would be able to use them just fine on OpenBSD because both systems use the same calling convention and object file format. Great! All I had to do was extract the support functions with ar xv and then rearchive them. I also had to update the config file to teach the linking step where to find the new libvc.a file.

At this point, I could successfully compile the hello world program and it ran correctly!

Building oksh with vbcc

Now I could turn my attention to building oksh with vbcc. Fingers crossed, let's try to configure it:

$ CC=vc ./configure

It was somewhat of a mess. The configure script errored out. It turns out this was a bug in my configure script, which I fixed. I was using a NULL for function parameters that really needed a 0. All other compilers happily accepted the NULL but vbcc did not. And vbcc was right.

OK, now the configure script was doing the right thing. But I very quickly ran into compiler errors. vbcc did not understand the GNU-style stdarg.h defines. I went back into the binary tarball and found a bunch of include files that went with the original libvc. I took the stdarg.h file from there and placed it with the compiler and taught the config file to search for the replacement stdarg.h file before searching for the one in /usr/include. This worked. I ended up having to do the same with stdbool.h as well.

But after this, I was able to finish the build.

Segfaulting shells

Unfortunately, trying to run the newly built oksh immediately segfaulted. So I rebuilt oksh this time replacing -O2 with -O0. And that didn't work either, but for a different reason. I got an internal compiler error when building with -O0:

vc -g -O0 -DEMACS -DVI -D_ANSI_LIBRARY  -c expr.c
Register 2(%ecx):
>	oval = op == O_PLUSPLUS ? vl->val.i++ : vl->val.i--;
error 158 in line 542 of "expr.c": internal error 0 in line 2361 of file ic.c !!
aborting...
unexpected end of file
1 error found!
vbcci386 -quiet "expr.c" -o= "/tmp/tmp.0.asm" -g -DEMACS -DVI -D_ANSI_LIBRARY -elf -safe-fp  -O=0 -I/home/brian/vbcc/targets/i386-openbsd/include -I/usr/include failed
*** Error 1 in /home/brian/oksh (<sys.mk>:87 'expr.o')

So I tried again with -O1. And yes, this time I was able to run the compiled oksh. However, while I could use any shell builtin, as soon as I tried to execute a non-builtin (e.g., ls), this would cause a hang and the command would never run.

Messing around with the optimizer

Now I needed to figure out if there is a happy medium between -O0 and -O1 that would give me a working oksh. Diving back into the vbcc documentation, I learned that there are two ways to specify optimizations: one is to use the common -On and the other is to use -O=n.

What's the difference? The former is a shorthand for the latter. The former takes -O0 to -O4 where -O= takes any number from 0 to 32767 (aka, a 15-bit number). Each bit in the 15-bit number enables or disables a particular optimization. -O2 is shorthand for -O=1023 for example. -O1 is shorthand for -O=991. So I figured let's start selectively turning off bits in the optimization number. First is bit 0, which toggles the register allocator.

The documentation does not say much about the register allocator. All it really says is that there are a number of different algorithms depending on what other optimizations are turned on. At -O1 (aka -O=991), the register allocator is turned on. So let's turn it off by changing the optimization flag to -O=990. This will leave everything else from -O1 on and selectively turn off the register allocator.

And... that worked! I now have a fully working oksh built with vbcc. Clearly, there is some bug in the register allocator that is causing the hang. I am not smart enough to figure that out. I am happy with a working oksh. I made some tweaks to the oksh configure script to detect if you are using vbcc and if you are, force the CFLAGS to change from -g -O2 to -g -O=990. vbcc also discovered a function whose prototype was marked static but the function itself was not, so I fixed that too in the commit.

Benchmarking oksh size

We once benchmarked the size of oksh built with many different compilers. Let's add vbcc to the list:

text    data    bss     dec     hex
453699  1068    28448   483215  75f8f

Using vbcc with vasm

As a final test, I built and installed vasm and taught the vbcc config file to use vasm as the assembler instead of GNU as. Building vasm was very simple as it was a single make command:

$ gmake CPU=x86 SYNTAX=std

Yeah, it needs GNU make and for some reason the assembler calls the arch x86 whereas the compiler calls the arch i386. My guess is because vasm also supports 16-bit and 64-bit x86.

The build of oksh with vasm felt exactly the same as when using GNU as. Interestingly, the resulting binary was slightly larger:

text    data    bss     dec     hex
457063  1068    28272   486403  76c03

It's not a huge difference. But it is a difference.

I think I'll continue to use vasm with vbcc. But as we saw, you can use GNU as just fine so vasm is optional for our purposes.

At this point, I also made a system-wide install and removed the lines I added to my .profile since I won't need them anymore.

Conclusion

The vbcc compiler system is a neat alternative C compiler. It seems to be more targeted towards 8-bit micros, m68k, and powerpc. But that doesn't mean it isn't useful for our i386 systems. I'd be very happy if someone added a port of at least vbcc to amd64, but alas I don't have the time to do it myself. I do wish the documentation was a little more clear for the configuration file but I was able to figure it out eventually. It would be also be nice to have the support-library-only version of libvc available for OpenBSD as well as the two include files needed but it was easy enough to do on my own so it isn't critical.

One last thing that I hope changes in the future is that vbcc is entirely incompatable with parallel compilation like with make or CMake. Every invocation of vbcc writes its temporary assembly file to /tmp/tmp.0.asm and so parallel invocations will clobber each others' temporary files.

I'll be submitting the codegen fixes and following along with this C compiler so that vbcc can stay a supported oksh compiler.

OpenBSD httpd