How one volunteer stopped a backdoor from exposing Linux systems worldwide

An off-the-clock Microsoft worker prevented malicious code from spreading into widely-used versions of Linux via a compression format called XZ Utils.

Linux, the most widely used open source operating system in the world, narrowly escaped a massive cyber attack over Easter weekend, all thanks to one volunteer.

The backdoor had been inserted into a recent release of a Linux compression format called XZ Utils, a tool that is little-known outside the Linux world but is used in nearly every Linux distribution to compresses large files, making them easier to transfer. If it had spread more widely, an untold number of systems could have been left compromised for years.

And as Ars Technica noted in its exhaustive recap, the culprit had been working on the project out in the open.

The vulnerability, inserted into Linux’s remote log-in, only exposed itself to a single key, so that it could hide from scans of public computers. As Ben Thompson writes in Stratechery.  “the majority of the world’s computers would be vulnerable and no one would know.”

The story of the XZ backdoor’s discovery starts in the early morning of March 29th, as San Francisco-based Microsoft developer Andres Freund posted on Mastodon and sent an email to OpenWall’s security mailing list with the heading: “backdoor in upstream xz/liblzma leading to ssh server compromise.”

Freund, who volunteers as a “maintainer” for PostgreSQL, a Linux-based database, noticed a few strange things over the past few weeks while running tests. Encrypted log-ins to liblzma, part of the XZ compression library, were using up a ton of CPU. None of the performance tools he used revealed anything, Freund wrote on Mastodon. This immediately made him suspicious, and he remembered an “odd complaint” from a Postgres user a couple of weeks earlier about Valgrind, Linux’s program that checks for memory errors. 

Click on the link below to continue…