A Dummy's Guide to Disclosing Linux Kernel Vulnerabilities

Bit of a niche one today, but my hope is that at least one person will be able to avoid some of the hurdles I had to tackle and the rest will at least get an interesting insight into the process behind responsibly disclosing vulnerabilities in the Linux kernel.

For context, I recently started my new position as a Sr. Security Researcher and during some of my research I discovered a fairly serious looking vulnerability in a part of the Linux kernel - pretty exciting right?! After ~~double~~ ~~triple~~ quadruple checking what I thought I found was, in fact, what I thought I found (and a couple of fist pumps later) - I asked my boss: what do now???

And that brings you all up to speed, as the post below is the culmination of my various Googlings, sifting through docs, advice from community members and maintainers and array of gotchas encountered all in the pursuit of answering the question: what do now?!

Some More (Important) Context
Initial Preparation
The Disclosure Process
Wrapping Up

Some More (Important) Context

Before we dive into the process proper, it's worth highlighting some important context and disclaimers around the nature of this post:

First and foremost, this is advice based on my personal experience and that of the community members I interacted with. One thing worth knowing is that the open-source community is not a hive mind, so opinions will differ!^[1]
On the topic of personal experience, this is written from the perspective of a zoomer who was born in 1997 and who's prior experience with email was limited to using Gmail for Steam 2FA
Finally, there are different routes for reporting/disclosing based on the nature of the bug. In this case I'm specifically referring to unpatched vulnerabilities within the Linux kernel; where in this instance by vulnerability, I am referring to a "sensitive bug" that could lead to privilege escalations, facilitate PE (KASLR/canary leaks), remote DoS etc.

That being said, I hope the majority of the content of this post has some objective value (otherwise RIP my Saturday morning spent on this)

Initial Preparation

Okay, now that we've laid the groundwork and set some context - let's get stuck into how we're gonna go about disclosing this crazy kernel vuln we've just found! Well, after we make sure we're ready to disclose this crazy kernel vuln ... it is a crazy kernel vuln right?^[1]

The more information and understanding you go into the disclosure process with and are able to share, the easier the job is for everyone involved.

Understandably, you don't want to spend years sat on an 0day to the point you're able to rewrite the whole subsystem blindfolded, but consider the following:

What is the impact of the vulnerability? Does it lead to privilege escalation? Is it remotely reachable? Is it trivial or difficult to exploit?
What kernel versions are affected? What commit was the vulnerability introduced in?
What part of the kernel is affected? Is it in the core kernel, a networking module?
Is this vulnerability reachable in most configurations, or only in specific environments/use cases? What distributions are affected?
How would you fix the vulnerability? Are there existing mitigations?

If you find yourself in this position and doubting yourself, it's okay, me too :) In fact it wasn't until my CVE was assigned that I thought "Huh, it is legit after all"; best solution is harness this doubt into thorough testing, research and notes to prove yourself right (or wrong about being wrong?)

The Disclosure Process

It's time. We've put in the work, verified our findings and got a solid understanding of the vulnerability... so, uh, who do we tell? Good question!

coordinated vulnerability disclosure, or "CVD" (formerly known as responsible disclosure)^[1] is a vulnerability disclosure model in which a vulnerability or an issue is disclosed to the public only after the responsible parties have been allowed sufficient time to patch or remedy the vulnerability or issue. — Wikipedia

CVD for the Linux kernel is handled, in typical Linux fashion, over mailing lists. There's a few different mailing lists out there, but in particular we're interested in the following:

security@kernel.org^[1]: private list for Linux kernel security team, who will help verify the bug & facilitate a fix as part of the initial embargoed disclosure
linux-distros@vs.openwall.org: private list of security contacts for various Linux distributions, used as a channel to notify distributions of "non-public medium or high severity security issues" as part of the initial embargoed disclosure
oss-security@lists.openwall.com: public list which anyone can subscribe to, for "public discussion of security flaws, concepts, and practices in the Open Source community" and used for public disclosure of security vulns

It's worth noting when Googling terms such as "linux vulnerability disclosure" the top kernel doc result is actually for v4.10 and contains less information than the latest versions; can be fixed by adjusting the URL or going here

0x00 An Overview

Before diving into the specifics, below is a quick summary of what the disclosure process will typically look like:

Notify security@kernel.org, linux-distros@vs.openwall.org and relevant maintainers of the vulnerability; establishing details, embargo period, CVE request and possible fix
Discussion ensues and ideally a fix is settled on and the affected parties/distributions are kept in the loop
At the end of the embargo period the patch is committed upstream, oss-security@lists.openwall.com is notified, CVE descriptions are made public, tweets are made etc.
If proof-of-concept code is to be published, this is ideally done several days after to give people time to patch their kernels / apply updates

0x01 First Contact

Alright, finally! We've got all the details covered, now we just need to send an email to security@kernel.org and linux-distros@vs.openwall.org right? WRONG!

Well, technically right, but there's a few crucial details we need to make sure of first, unless you want to stumble your way onto the scene like yours truly.

I've linked each of the relevant emails to their respective pages, which outline mailing list usage and any requirements. There's a lot of information and sometimes it's conflicting, as "linux kernel vuln" disclosure is only a subset of what these lists are used for.

❗

Going forward I may refer to the lists by shorthand as s@k.o and linux-distros.

So to cut short a long and embarrassing saga of trial and error on my part, here's a distilled collection of the relevant requirements and advice learned:

Initial disclosure should be sent to linux-distros, CC'ing s@k.o along with any relevant subsystem maintainers^[1]
The email should be sent in plaintext; referring here to the email type and not just the fact it's not encrypted (e.g. as opposed to RTF emails with fancy HTML and stuff)
The subject should be prefixed with [vs] to avoid linux-distro's spam filter, and the subject shouldn't contain any sensitive information RE the vulnerability
Propose a tentative public disclosure date & time (e.g. 05/02/22 16:00GMT), while ideally a date should be settled on after a fix is decided, note the maximum embargo period for linux-distros is 14 days regardless of a working fix
Include a request for a CVE to be assigned for the vulnerability
As mentioned, provide as much information and detail as necessary (no War & Peace tho); if you have a fix, see the next section on how to format/include that correctly

With all that in hand, you should be ready to make first contact and begin the coordinated disclosure of your Linux kernel vulnerability, awesome!

I was about to write my own War & Peace on mail clients, but suffice to say the kernel docs provide some helpful information on recommended email clients and any necessary configuration required to get them to play nice.^[2]

0x02 Patching & Follow Up

Did I CC the right maintainers?! Did my tabs come through on my patch?? Is this even the right mailing list???? - don't sweat it, the waiting game can be real.

If you don't hear back within 48 hours then make sure to follow up the initial email to confirm that it was received, as sometimes things can get lost in the mix.

The discussion that ensues will vary based on the amount of information initially supplied, but will likely focus around impact, mitigations and sorting out a fix.

Including a Patch

"Submitting patches: the essential guide to getting your code into the kernel" is going to be your best friend here, covering almost everything you need to know.

The only difference is you'll just be including this inline within your disclosure email thread with s@k.o and linux-distros; as opposed to a separate submission.

This is where the whole email client whitespace mangling comes in to play, as the Linux kernel uses tabs for indentation and these should really be preserved in patch diffs.

Also, I'm sure it goes without saying, but make sure you test any patches you submit and make this clear when you suggest a patch!

0x03 Public Disclosure

The finishing line is in site now, you can almost see the retweets, as the embargo period comes to an end. There's not much to cover here, so I'll keep it brief:

Hopefully a patch has been finalised and the upstream fix can now be disclosed
You can request linux-distros to make the CVE details public
You should send an advisory out to oss-security@lists.openwall.com; see their wiki for more info, but content will be similar to your initial report to linux-distros^[1]
If you plan to publish exploit code, mention this in the initial public advisory, but ideally wait several days for users to update their systems

And that's it! It's over! Now you can go back to the easy part and find some more vulns!

Here are some examples of public advisories from Qualys & grsec

Wrapping Up

Hopefully this post has given some insight into a particularly niche subset of coordinated disclosure practice and can provide some help to people who find themselves into a similar position as I did recently!

Admittedly, the process was rather daunting to go into with the self-imposed pressure of wanting to disclose a new vulnerability, complete inexperience with mailing lists and figuring out the relevant processes and requirements which applied to my situation.

That said, it was also extremely rewarding to be able to contribute back to the Linux community and be able to facilitate a fix for something like this. There may have been more than a little bit of fan-boying while interacting with people I've followed for a long time, even if they were schooling me on mailing list 101s.

Despite fumbling my way through the process, everyone was helpful and patient, though maybe with this post you can avoid some of the same fumbling.

exit(0);