Dr. Brian Robert Callahan
academic, developer, with an eye towards a brighter techno-social life
-fret-clean flagI start with apologies to williewillus, calvin, and reezer on Lobste.rs. Prior to July 9, I would have agreed with you all. But it appears I was wrong. There is no need to further engage; it is clear that critique-from-within is undesirable. Which is perfectly OK, because I stepped away from being on the inside. So let's just keep evaluating OpenBSD's mitigations and we'll continue to learn interesting things along the way.
Now, onto the interesting parts.
My research center has gotten another article accepted for publication that focuses on our new research line of independently evaluating *BSD mitigations. It received some very strong reviews, which suggests to me this is work that others want to see continued. Sadly, the article will once again be locked behind the paywall that is IEEE Xplore. Once it does publish, I will update this blog post with a link to it. Technically, I can't post the paper because IEEE owns the copyright (welcome to academia...), though I'm sure if you ask nicely I can find a way to share it with you if you don't have institutional access.
But just because IEEE owns the copyright to the paper doesn't mean I can't explain it on my blog, so I am going to do that.
This latest paper focuses on the -fret-clean flag implemented by OpenBSD back in OpenBSD 7.6. This mitigation, exclusively and selectively enabled for amd64, teaches the LLVM compiler to insert movq $0, -8(%rsp) after every call instruction. This instruction adds nine bytes, 48 c7 44 24 f8 00 00 00 00, every time the mitigation is inserted.
Technically there are rare exceptions to this code insertion, but it is not relevant to our paper or this discussion. In fact, we intentionally don't focus on the exceptions in our paper in order to steel man the mitigation as much as possible. The inserted movq instruction effectively zeros out the callee's return address slot on the stack immediately upon returning to the calling subroutine. The overall effect is that stale return addresses are removed as soon as they go stale; the logic is that by removing trivially identifiable pointers into executable code, attackers will have a more difficult time finding return-oriented programming (ROP) gadgets.
OpenBSD applies the mitigation selectively to the kernel, libc, libcrypto, ld.so, and the ssh binaries. You can enable the mitigation yourself with the -fret-clean flag, but outside of those objects it is not by default enabled. OpenBSD chose these because they are high value targets; at least for the kernel and libc, that perception of high value is corroborated in the academic literature.
The logic here is clear and relatively elegant in its simplicity. Unfortunately, it is exactly that clarity, elegance, and simplicity that hides its reality—it can reasonably be argued that it costs more than it's likely worth.
And no, you don't have to take my word for any of this. You can see the raw data in our open science repository. That's the whole point to what we're doing.
There is similar work in this space, all of which is now understood to be not worth the costs. In fact, for the particular strategy of zeroing out the return address slot on the stack, this is at least the third time it has been independently invented, of which OpenBSD was the third.
The closest analog is in cleaning scratch registers, which moved from a runtime monitor to a static binary rewriter, and eventually the GCC and LLVM -fzero-call-used-regs family of flags. This eventually made its way to the Linux kernel in the form of the CONFIG_ZERO_CALL_USED_REGS option. After being put into production, Phoronix found potentially greater performance penalties than originally claimed. The Linux hardened project now recommends against using it. And, of course, my research center's own previous research found that OpenBSD's alternative register allocation and compile-time instruction rewriting did not deliver on its promises: it both removed far fewer gadgets and had far greater binary size and runtime performance costs than originally reported. Finally, GCC once had a -mmitigate-rop flag, similar in spirit to the compiler-based OpenBSD mitigations we previously examined, first incorporated in 2015 and then removed in 2018 due to its ineffectiveness and GCC's preference for Control-Flow Enforcement Technology (CET) as the right mitigation in this space, leading to lack of maintenance for -mmitigate-rop.
All of these mitigations share the same story: an intuitive-sounding mitigation was put into production without empirical testing, then later empirical testing by others found it to be locally excellent but globally ineffective at its task. This global ineffectiveness can be tied directly to a mismatch between what these mitigations do and what attackers need. All these mitigations do exactly what they claim to do, but those things don't constrain attackers in any meaningful way.
And it is here we find our first lesson: just because a mitigation is excellent at what it does, it does not follow that local success translates meaningfully into global success. All of these mitigations confuse local success for global success. Local success makes a number on a chart go up (or down); global success improves your security by actually constraining your attacker in a meaningful way.
This means that -fret-clean needs to overcome the narrative every other mitigation in this family of "simple and intuitive" has failed to overcome.
To be clear: the question we're asking is "what is this mitigation's contribution?" And that can only be answered in isolation. We want to know if these mitigations genuinely pull their weight. There is a difference between proving zero value (which is unprovable anyway) and demonstrating an expected negative value. Because then we have what I think is the genuinely interesting question: in a defense-in-depth world, which I think we all agree is the valuable world, are we acting rationally if we keep mitigations that don't earn their keep? I would argue we should prune those mitigations. If a mitigation doesn't deliver, I don't think it has a place in the larger defense-in-depth story, regardless of how true defense-in-depth is in the aggregate.
The OpenBSD innovations page states: "-fret-clean is a clang extension that, upon return from a function cleans the return value off the stack (one of many information leaks which can be used to determine where functions in a different DSO reside). The kernel, libc, libcrypto, and ld.so(1) are compiled with this option. amd64 only, for now."
It does beg the question: is this an innovation? The definition of the word ties innovation to novelty. So one way to check if -fret-clean is an innovation would be to check the source of all things novel: the Patent Office!
Unfortunately, -fret-clean is so not novel that the original patent explicating the mitigation expired years before -fret-clean was even proposed. Let's quote the relevant passage from the patent:
scan code for "call" operations, and for each "call" operation, inject code following the "call" operation which re-initializes the stack space used to pass arguments to and receive returned arguments from the called routine, and to re-initialized the stack memory used to store the return address used by the called routine to return to the calling routine
This is exactly what -fret-clean does; it is a perfect match. It is actually better than a perfect match: it's a perfect superset; not only does it zero out the return address it also zeros out the arguments on the stack. IBM in this patent was concerned with data privacy and improved debuggability rather than ROP (ROP did not exist as a named concept in 2000, when this patent was initially filed).
In 2018, there was a second indepedent reinvention of zeroing out the return address on the stack. A paper proposed and implemented a modification to LLVM to add a __zero_on_return function annotation that would zero out the entire 128-byte "red zone" upon returning, which includes the function's return address on the stack. While this 2018 paper also does not reference ROP, instead focused on controlling side effects, it is clear that this function attribute would have the same effect as -fret-clean, at the cost (or benefit, depending on one's outlook) of having to add an annotation to each function you wish to protect instead of a blanket compiler flag that applies to all functions.
Which means OpenBSD is at least the third independent reinvention. This limits the innovation claim to perhaps the specific implementation and the decision to try it out in an operating system. The idea is not an innovation; at best is was independently rediscovered some quarter of a century later and then applied to a different problem than it was originally invented for. All the more reason -fret-clean needs additional scrutiny: it is effectively trying to map a mitigation for one set of problems onto another set of problems. That neither IBM's patent nor the 2018 paper made their way into all major compilers is instructive: if it was not adopted for the problems it was designed for, why should we assume it ought to be adopted for problems it was not designed for?
We ported the mitigation from OpenBSD to FreeBSD and turned it on for the FreeBSD kernel, libc, libcrypto, and ld.so. We did this so as to have a clean comparison base. We made no changes to the LLVM optimization pass or its hookup to the FreeBSD LLVM; they exist exactly as they do on OpenBSD. We then used the size(1) utility to get ELF .text sizes of the objects before and after applying the mitigation, wrote microbenchmarks to aggressively find the worst-case scenario, ran benchmarks from the Phoronix test suite, and then timed a kernel build for a real-world test. Finally, we wrote a stack scanner that scans around the current stack pointer looking for things that look like libc pointers, classifying them as return addresses, in .text but not a return address, or outside .text (namely, in .data, .rodata, and .bss).
Across the kernel, libc, libcrypto, and ld.so, we see a .text size increase of around 8%. Runtime performance is heavily workload-dependent: anywhere from negligible to up to a 27% increase in our pathological synthetic microbenchmark we wrote to exercise the mitigation as much as possible. The pathological case may actually be worse than it sounds: the runtime impacts came despite there being virtually no additional instructions retired. Which may mean that the layout changes due to the size increases are the culprit for the runtime impacts; this could be alignment and icache/iTLB behavior changes coming into play. This means that the -fret-clean flag may be entirely unpredictable as to its runtime impacts: you would need to benchmark your specific microarchitecture, each and every time, to discover the true impacts. That makes this mitigation entirely unbudgetable and not able to be reasoned about ahead of time. We intend to study this in future work to narrow down the exact culprit(s).
To make it clear, however, the 27% is the pathological case. Real-world performance penalties are closer to the negligible to 2% increase, so it is usually not catastrophic but it is statistically significant. And I don't necessarily want to benchmark every workload I have to ensure it isn't one of the edge cases that would result in meaningful performance impacts.
To ensure consistency, we compared instructions retired (using pmcstat(8)) before and after, and found that the number of additional instructions retired matched the number of calls instrumented with the mitigation.
The mitigation itself does exactly what it claims to do: we built another benchmark that intentionally dirties the stack and then scans it for libc pointers. Pre-mitigation, we had 110 libc pointers on the stack: 32 in the .text section. Of those 32, 20 were return addresses and 12 were other libc .text pointers that happened to be on the stack. We found an additional 78 pointers into the .data, .rodata, and .bss sections. Post-mitigation, we saw a reduction of 18 libc pointers, all of which were return addresses. That is just over 16% of all libc pointers removed. Even though it is a 90% reduction in return addresses, we are happy to steel man the mitigation and grant that it is perfect at what it does.
All other libc pointers were left untouched. Additionally, our stack benchmark collected the number of unique libc pages we were able to discover by scanning the stack: we saw an about 40% reduction, from 40 unique pages to 24 unique pages. However, 24 unique pages is still probably enough to map out libc pretty well. I should also note that we scanned at most 64 KB from around the stack pointer; in a real attack, you will likely be able to read a lot more than just the 64 KB around the stack pointer. You should assume that an attacker will get everything they need to be successful even with this mitigation turned on. I'm not even sure this mitigation will slow an attacker down even a little, since the majority of libc pointers are still available.
To be clear, that is exactly what the mitigation sets out to do. It does its job perfectly. It is just that its job may not be all that contributory to the overall problem. And you always pay costs. That signals to me that there can be reasonable disagreement as to whether or not this mitigation is worth keeping. I think it is reasonable to say that the costs outweigh the benefits; I also think it is reasonable to say the costs amortize low and when part of a larger defense-in-depth strategy, continued inclusion may be justified.
I think I fall on the costs outweigh the benefits side for one simple reason. OpenBSD already has what I imagine to be far superior compensating controls for exactly this failure: library order randomization and internal random relinking of those high-value targets. It's on our short-list to examine internal random relinking in the near-term future.
At the beginning of the original email, Theo says, in regards to the defense-in-depth strategy that OpenBSD has been building against ROP, "None of them are a silver bullet; added up they are a big deal, but noone is saying they are a comprehensive solution" (emphasis mine).
I agree that no one defense is a comprehensive solution and it would be remarkably unfair, bordering on asinine, to claim otherwise. However, the more we do actual empirical evaluations of OpenBSD's compiler-based anti-ROP mitigations, the less I can get behind the idea that "added up they are a big deal."
This is now the third such defense our research center has evaluated, after reviewing both alternative register allocation and compile-time instruction rewriting as well as testing if the idea behind compile-time instruction rewriting is even sound. In every case, the answer is the ideas appear to underdeliver and probably do not buy much if any security, and I think it is justified to remove them. I think we can have reasonable disagreement about whether or not they should be removed. But the fact remains that OpenBSD rarely publishes empirical evaluations of their mitigations, which makes it impossible to have that reasonable disagreement.
It is that last part, about rarely publishing empirical evaluations of their mitigations, that worries me most. Because a pattern emerges: OpenBSD asserts "obviousness" of a new compiler-based anti-ROP technique, puts it into production with no or minimal empirical evaluation, and claims increased security. That is fine when rapid experimentation or a promise to actually conduct in short order a public empirical evaluation is the goal, and I think for the most part that matches OpenBSD practices. However, one actually must conduct said evaluations and must be prepared to accept that an experiment has failed and remove the failures. Because my fear is that without real data, we cannot reason about the effectiveness of a mitigation and that may lull people into a (ever-so-partial) false sense of security. That manifests in a few ways, but the most critical way is that it suppresses future experimentation: why bother trying a new way to do something if you think you are already protected?
This critique, however, is not universal when you look outside this narrow band of mitigations that are explicitly anti-ROP and compiler-based. OpenBSD has had plenty of successes. I think pledge(2) is an excellent mitigation and is what other systems should likely adopt in some form. And, of course, privilege separation should be considered fundamental by anyone who writes software.
But those mitigations match attacker capabilities. These anti-ROP mitigations do not: they conceptualize an attacker that does not exist. It is this mismatch that is especially worrying to me when it comes to mitigations that feel intuitive. If you walk away with nothing else, it should be the idea that the intuitive security mitigations are the ones that most require rigorous empirical evaluation; we likely should stop accepting any such intuitive mitigations outright until they prove themselves useful. Not just for OpenBSD, for all systems.
The obvious response will be the defense-in-depth argument, encapsulated well by the quote that opens this conclusion. But that only works when you match attacker capabilities. And there is evidence that suggests that too much defense-in-depth and/or defenses that do not match attacker capabilities just produces a false sense of security. We should do better once we know better.
