-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KPTI backport conflicts #25
Comments
The KPTI Patches are coming with v4.9.75. Resolving the conflicts and making them compatible with PaX will take a lot of time, so please be patient everybody. |
Thank you much. We have manually tested the meltdown poc on a bunch of Intel and a couple of opteron systems, and the latter does appear safe from the vector, while every bloody Intel chip we can test on works like a charm... |
4.9.75 with KPTI is out now: |
@minipli: if the KPTI port is too absurd to carry out, is there an option to implement a memory-localization technique for kernel/uspace like the paper suggested:
I figure this would drastically reduce ASLR space, or cost obnoxious amounts of wasted memory, but it might be a safer approach long term. |
@sempervictus The part you're quoting is talking about a yet-to-be-implemented hardware feature. So no, that's no option. The KPTI patches are needed to prevent the meltdown attack. |
Ah, thanks for the clarification. I had read that to mean that the address location was used in speculative context to determine if reads would be allowed. Didn't realize hardware would have to change for a </> comparison of the target address and some boundary address in speculative execution as a way to fail early. I'll start collecting data for benchmarks this weekend to see the resulting cost of kpti and retpolines once we upgrade and rerun.
|
Just found this project, great work! Do we know anything more about when patches might be available? "Resolving the conflicts and making them compatible with PaX will take a lot of time", is this days/weeks/months? |
It will take the time it will take. Thank you for your work minipli amd take your time |
Expect it to be weeks/months/never. It's a pretty invasive change conflicting with a lot of PaX. :( |
@minipli: What about other mechanisms to achieve the same end? IIUC, KPTI prevents speculative access to "pivot" over kernel pages mapped into the userspace memory range to the phys page tables and access memory available to the kernel in that context (prior to the privilege/exec bits being visible) by moving most of the kernel page maps out of the userspace virtual page tables. Could we use the active exploit mitigation mechanisms or UDEREF to catch the pattern of page faults generated when speculative execution gets verified with the privilege checks and discarded? If we lock out the user doing this once they start, we dont eliminate the problem (but then again, neither does KPTI really), but we prevent repetition of the condition preventing the attacker from reading ever more memory. |
@minipli: you can use in first time the function GRKERNSEC_KERN_LOCKOUT for check the logs with "trap invalid opcode ip:". Else use KPTI only on Intel processors that are impacted (disable PaX on a bugs files and waiting for a better solution). I'm trying to understand "linux-4.9.74/Documentation/x86/exception-tables.txt" to find a simple hack at the expense of instruction and or specific features. Thank you ;) |
Oh minipli's comment is sad news :-(. Apart from that the upstream backport of KAISER seems to be flawed, see https://news.ycombinator.com/item?id=16087736 and https://twitter.com/grsecurity/status/950868975789006848 EDIT: zaolin nice coincidence that we posted the same info at the same time both beginning with the word "Oh" |
minipli will you make changes to the code of your kernel for kvm ? |
I managed to block the MELTDOWN PoC by modifying the CPU cache behavior (Using /proc/mtrr or with module to disable the CPU cache), but the performance is severely degraded :/ |
Spenders/pax' implementation according to twitter seems to incorporate the functionality into UDEREF (where we see a ton of conflicts)... So there's hope, especially since older kernels apparently got kaiser instead. Uderef already uses pcid, hopefully permitting use of the kpti version for the relevant functionality. On a separate note, anyone consider PAGEXEC as a means to preempt the speculative bounds violations in branch predictors (spectre)? |
Maybe it's time to migrate away from grsecurity? These unofficial patches will not be maintained forever even if minipli decides to patch meltdown, and 4.9 kernel will live for maybe one more year. I'm putting my hope in http://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project and SElinux :) It was a good run but maybe it's time to face the facts, grsec does not want anything to do with the community anymore |
@indy-independence: KSPP is still far from offering the same protection, the proof: Not to mention their REFCOUNT implementation : @sempervictus: the crowd funding maybe a solution for help minipli. |
KSPP is not targeting full porting anyway, UDEREF, KERNEXEC, and other critical things which are likely to called out as "breaking an insecure userspace for the sake of security," or "too much of a performance hit for consideration" like the PIE code with single digit hits. |
I would definitely support crowd funding, though I have the bad feeling that there would be too few people actually donating, but I can't really judge how many users the patch has. @minipli How is the current status? Could you give a rough estimation of how you would plan to go on from here? Do you plan to solve meltdown in a different approach than upstream or port PaX? And what is your opinion regarding the problem that 4.9 has an old KAISER implementation backported that is not really strong compard to KPTI? Finally, are the spectre patches easier to merge with PaX? Is there something we could do for you? |
It seems that this is one of those problems money doesn't solve directly, or exclusively anyway - @minipli is (most likely) human, meaning only so much actual time and effort can be applied regardless of his level of desire. We need to find more people out there already familiar with the codebase/with time and skill available to grok it and at least try to get an effort going marking up the code with comments/docs to gain a better understanding for everyone (maybe a separate comment branch that everyone can PR on?). This should permit better functional separation, improvements/debugging by community, etc. The mystique of the codebase is likely intentional, and the first/biggest barrier to entry for the uninitiated. Without good documentation, even with all the effort @minipli is putting in this work will not be forward portable to newer kernels, and long-term, that's probably the goal to shoot for - its just too useful to let it die on the vine. |
@sempervictus: Exactly. Can't agree more. For 20 years, all the changes are included in the 20-megabyte patches, with no version control (spender probably did it intentionally), explanations, and they include everything from typo fixes, minor tweaks, hardening tricks and groundbreaking mitigation techniques, compiler plugins, etc. PaX/grsecurity is spender's (and hackers behind the PaXTeam) personal project, and of course he can do whatever he wanted. He made great contributions to the entire community and history of computer security for totally two decades. The problem is the entire Linux kernel security depends on this personal project (and if spender is correct, he should be, OpenBSD doesn't have a UDEREF-level of defence, so the entire Unix kernel security), and it has been here for 20 years, and nobody did anything to merge the code, improve the security, besides the patches from this personal project! Perhaps it is the time to fork the Linux kernel and transform PaX/grsecurity to a community project of the general public now. A good start is splitting the functionality of the grsec patches to different modules (like creating a separate repo for the gcc plugins first), and the project will be supported by developers who share spender's security mindset, actively track upstream and continue development, merge useful fixes back to upstream, etc, you know what I mean. But I doubt who is capable of doing it... Not a script kiddie like me of course. Given the fact that the KSPP is continue pushing the partial solution originated from PaX, "actively track upstream" is already a task only the real hackers capable of doing so. |
@biergaizi |
Strcat works on a different OS, not general compute, they can rely on selinux isolation a lot more, and has somewhat different technical and organizational concerns that Linux users or systems ops. Pretty sure he's not chomping at the bit on this either. Guy's well deserving of all the praise in the world for CHOS, but this work needs time and a willingness to keep dealing with upstream... |
@sempervictus Linux-hardened was the original idea to start out of tree implementing grsecurity patchset for amd64 but not enough people actually were interested so strncat now only does arm64 work. |
The split patches / fork project etc. could be a long-term goal, for now the focus should be on fixing meltdown / spectre without loosing any grsec/pax functionality. Right now the state is sad as upstream kernel fixes those severe bugs but does not really offer good protection in general while the unofficial grsec/pax kernel is pretty vulnerable with those issues. @minipli I hope you can update us soon on your state and how you would like to proceed. |
Agree that we need meltdown addressed and to move on to the spectre variants in-tree, the thought process is that the page table protections are implemented in UDEREF upstream (in grsec code), which will likely require in-depth analysis of the feature proving an opportunity for commentary or even extraction. |
It's not about the money, it's about the time I'm able to devote for this project. So please be patient and use an upstream kernel in the mean time if you need to. |
@sempervictus I wonder how much progress has HardenedBSD made, they already have a more complete version of ASLR than OpenBSD, and it will be really cool if they managed to implement UDEREF. But, Meltdown... |
@Esokrates Thanks @minipli for devoting time to fix this! |
HardenedBSD is a great addition to BSD, and should serve as part a development model for any future OS - security patches are security failures, we need to be defending against classes of problems, not lines of code. However, their (BSD) userspace is unpleasantly different though, enough to make it non trivial to migrate from Linux even with their ABI compatible interfaces and Linux jail capacity as the config overhead is serious, and troubleshooting can get nasty. |
So much for NoScript in FFox:
Looks like "the wild" is getting wilder. |
@sempervictus Not really, I think it's a new feature related to sandboxing since Firefox 55 and it's new IPC architecture. |
Seems a bit odd - i run Arch, so ffox is rather "fresh" here, and 55 was a few months ago. Haven't seen that message till now. |
Does upstream dropping the syscall fastpath help in any way? Or does it have to stay in for other grsec/pax features to work? Interesting to see if/how this works - would produce a 5th mitigation by my count (KPTI, KAISER, RHEL, Grsec, and the unofficial version). Thanks for keeping at it @minipli, might be the best thing the public can use if KPTI and its ilk are flawed. |
I just discover (not tested) Blacksmith is a tool designed to detect and inform of any Meltdown execution on your CPU. The software seems to be monitoring this: |
@minipli please take a look in uderef-SMAP :
|
Are we getting any closer to 4.9.75? |
@kintarowonders who wrote:
minipli wrote:
You can pray to God, if you believe in Him (I do), or you can do something else, to make it happen sooner, rather than... never. |
HacKurx, sorry, I mispelled your name. Corrected it. |
@miroR 1] If on your side you want to help please test all the exploits and PoC MELTDOWN in order to find a viable method of detection. Thank you all and reserve beers for minipli. A Note In The Air: PaX Team and Spender if you go through this, do not hesitate to provide a commit. Out of respect for your former users ;) |
Based on Spender's public posts, i'd say chances are good that whatever
they did to change UDEREF in order to handle meltdown is going to be more
than a commit. What's more is that the 4.9 branch isn't their
current-stable nor their old-stable (which is 4.4), so they're not likely
to have anything on hand which could be merged into this tree by mere
mortals.
If minipli is able to fight off meltdown with this kernel, i imagine it is
likely to be something rather different from other kernels which provide
that defense.
|
@HacKurx
I'll try and look into it. Need time. I'm slow + still have Call Traces which are actually from attacks, but I have not smoking gun proofs to it.
No, I'm on AMD64, all my machines. |
I wrote:
I mean, I have them not in this AMD64 based machine, which is over 10 ys old (and much safer), but on the newer AMD64 based machine which I had issues previously, but which I don't go online with. I think I gave all the info on both the type of AMD64 proc that my systems are based, but can't go look for links now... Just, I'm talking about issues, still opened #17, #18, #19, #20, #21, but time, I'm gasping for time to present those Call Traces somehow on my pages, and only giving the links to that presentation, i.e. without spamming linux-unofficial_grsec. Clarified just so not to cause more confusion with the claim. |
Last (I hope) clarification:
Which I don't go online with since the time of those Call Traces published in the abovementioned issues. |
@minipli interesting for after |
Since PTI and UDEREF/KERNEXEC wont play nice, and the whole point of having an open source branch out here is so that people who cannot afford to purchase current code can have access to the defensive measures it provides, what about using something like capsule8's meltdown detector strategy to keep a page fault counter which decrements every X cycles or has some intelligent pressure-aware fill/flush mechanism which would enact active defenses (kill pid, lockout user, etc) if thresholds of counted faults exceed a sysctl param? |
Hello everyone, I have been silent for quite a long time, and I probably should have posted here sooner. Sorry. I just wanted to avoid the politics around the patchset for as long as possible. Anyway, here I am. I am dapperlinux / msr50 / Matthew and I make a Linux distro called Dapper Linux. Now, my distro is one of the very few to run a grsecurity/PaX kernel out of the box, and just like you, I am a big fan of the grsecurity patchset. Now, when the patchset went private, I had a problem. I had to choose to stay on 4.9.24 forever, move to an upstream kernel or start forward porting the patchset myself. I chose the latter. I had always been interested in kernel development, and an opportunity had arisen, so I thought I would give it a go. I started by forward porting the patchset to newer kernels, and after about 70+ hours of effort, I managed to get to 4.11.8, and actually got it compiling. Don't get too excited, since it didn't boot. I named the patchset "Dapper Secure Kernel Patchset" and started chasing after 4.12 instead of trying to get it to boot. You can find the github repo here: and the current status is that the port to 4.14.x is about 85% complete, but it hasn't compiled since 4.11.8, so I wouldn't expect this to work anytime soon. Now, while I was waiting I went and created "Dapper Secure Kernel Patchset Stable" which continues the 4.9.24+ kernel series, and I made releases in parallel to Minipli. Whenever I got stuck, I would have a peek at Minipli's code, but usually I could work out the conflicts myself and release a working patchset. Repo here: Now, everything was fine until 4.9.75 came out. But when it did, I had huge problems. I could get KPTI merged and compiling, but it would always panic on boot. I tried disabling the PTI config, but no luck. So I just waited around until Minipli came up with a fix to peek at. But no fix has come. So over the last few weeks, I have decided to start maintaining Dapper Secure Kernel Patchset Stable again. I read every single patch hunk out of every point release, keep code which is unrelated to KPTI and retpoline, and then make a patchset which reverts any KPTI and retpoline code, and forward port Dapper Secure Kernel Patchset Stable to match it. It's pretty slow work. Each point release is getting more and more difficult to do, since it includes more and more code to mitigate Meltdown and Spectre. So far I have made it to 4.9.80, and I have compiled it and tested it. Just make to sure to get my 4.9.80a release, since I missed a few compile errors when I first posted it. You can get it here. The current release works just like any other grsecurity patchset. Get your 4.9 kernel, patch 4.9.80 and then patch dapper-secure-kernel-patchset-stable-4.9.80 and off you go. Now, I'm starting to think more and more that I should be moving my distro to an upstream kernel, so I cannot promise that I can keep maintaining my 4.9.x patchset. I've talked to Kees Cook, and he encourages me to learn how to land patches upstream and work with upstream instead. 4.9.100 is out, which means I am 20 releases behind. I can still catch up, but it would be quite a few hours effort. Things would go along more smoothly if I get retpoline support landed, and trust me, I'm working on it. I've spent time on and off over the past few weeks trying to get it working. I just keep running into problems where I break RAP. The patchset is currently not quite compatible with GCC8 either. Which is a problem, since Fedora 28 ships with GCC8, and its blocking the release of Dapper Linux 28. I'll have another look at this too sometime. Minipli, if you want to help out at all, then GCC8 support would be a good thing to do first. Is anyone interested in a 4.9.80+ patchset? 4.9.100 patchset? or a patchset that keeps going till 4.9 reaches EOL? Would anyone be interested in crowd funding me in making small point releases? Perhaps a small amount, like 0.003BTC or 0.004BTC per release? And finally trust. How can you trust me? Well, you have to decide that for yourself. I promise I'm not doing anything bad in the 9mb or so patchset, but you can never be sure. Maybe a livestream of me working on the patchset could be interesting. Anyway, I'm just looking for interest in the patchset. Let me know what you think. Reply here, or open an issue in one of my github repos and talk there. Or feel free to email me. You can get my address on my website. |
Hi Matthew, After some bad news: There is also good news, thank you for not giving up and for your work. As far as I'm concerned, I haven't given up, but I'm having fun on an old version here. Minipli hasn't given up either, he just has a lot to sort out. So to do well we must join forces properly but anyway, you can count on my help. |
Very interested to test, after some four more days that I'm totally busy. |
Okay, I spent all weekend getting dapper-secure-kernel-patchset-stable from 4.9.80 to 4.9.92. It was quite a lot of effort, as it seems minor patches are growing in size rapidly these days. You can get my latest release from here There's a few compile errors in the 4.9.8x series since I forward ported without compiling to save the 2 or so hours needed to compile the kernel, and I have resolved these in 4.9.91 as I went and compiled 4.9.90 and 4.9.91. I haven't tested 4.9.92, but the patch was small, so I think it will be okay. My current plan is to catch up to the latest release (4.9.103+) and then complete the gcc8 port, since some of the 4.9.10x patches include gcc8 support. From there on I'll have a look at retpoline, but it is hard enough simply staying up to date. We will see what happens. If anyone wants to help out, you can help by compiling my code and testing, fixing those pesky kconfig warnings I introduced today, or having a go at getting retpoline working. |
@msr50
It was already done, but you advance on without taking my patch. See issues 2
|
Okay, I have had a busy weekend going from 4.9.92 -> 4.9.105, and it is available to download from my releases page. It is tested and compiles fine on GCC7. GCC8 support is 95% done now. There is a small very strange bug left where the size_overflow plugin generates some weird GCC gimple code for a GIMPLE_ASSIGN statement that for some strange reason has 4 operands in a statement somewhere in mm/vmscan.c, and I'll keep looking into it. If you are impatient you can disable CONFIG_SIZE_OVERFLOW in your config to compile a kernel with GCC8. I'll make an issue for it in the next few days with some more debug info incase anyone is interested in figuring out how to fix it. Boy oh boy was 4.9.104 one crazy little release. I haven't seen a release as large as that before. 14,000 lines of code in a small point release that I had to go through to make sure there wasn't any references to meltdown or spectre. It probably won't be the last large release either. Up next is keeping up to date with point releases and back to working on retpoline. Will update as I go. |
@msr50
I'm testing 4.9.105 (which did compile fine on:
I hope you and @minipli will find a way to join forces... I've had fewer crashes with 4.9.92-dappersec180601-06 then I have had with minipli's grsecunoff... And I've been testing (and still am)the kernel that I uploaded at https://www.croatiafidelis.hr/gnu/deb/linux-4.9.92-dappersec180601-06/ i.e. for any hardware (much wider attack surface than when I compile for my own hardware only). Will report how 4.9.105 is faring, and then hopefully offer that one for download too. |
Spent a bit of time looking at the KPTI thing and trying to figure how the pieces impacted by the changes in this tree actually work, and what the kpti patches are doing there. It looks like there's a bunch of delta around the way tss_struct is handled, and lower down to things i dont yet understand, seemingly related to how data is laid out on the stack. It appears that some of the memory access is actually occuring from kernel context whereas the 4.9 upstream code being patched seems to expect its in the user context (in truct tss_struct doublefault_tss for instance)
@minipli: if this is actually "portable" into the fork, is there any chance you might be able to detail the changes from upstream in the commit log so we could better understand the clockwork? Thanks as always.
The text was updated successfully, but these errors were encountered: