CVE-2024-3094: XZ Utils backdoor — anatomy of a supply chain compromise

TL;DR

A state-level supply chain attack compromised XZ Utils through a years-long social engineering campaign, inserting a sophisticated backdoor that intercepted SSH daemon authentication. The malicious code modified liblzma to hook OpenSSH's authentication flow, enabling pre-authentication RCE via specially crafted certificates—nearly impacting millions of servers.

Affected versions

• XZ Utils versions 5.6.0 (released Feb 24, 2024) and 5.6.1 (released Mar 9, 2024)
• Distributions that packaged these versions:
  • Fedora 41 (Rawhide) and Fedora 40 (beta)
  • Debian testing/unstable (sid)
  • Arch Linux (briefly)
  • Kali Linux (rolling)
  • openSUSE Tumbleweed and MicroOS
• Enterprise distributions (RHEL, Ubuntu LTS, Debian stable) were NOT affected due to slower update cycles

The backdoor only activated when:

1. Built with specific toolchains (gcc, GNU linker)
2. Target architecture was x86-64 with IFUNC support
3. Linked into sshd through systemd's libsystemd (which depends on liblzma)

This narrow activation window prevented wider damage but demonstrates sophisticated targeting.

Root cause

The attack spanned 2021-2024, with attacker "Jia Tan" (JiaT75) gaining maintainer trust through hundreds of legitimate contributions before introducing malicious commits.

Phase 1 - Social engineering (2021-2022):

• Multiple sockpuppet accounts pressured original maintainer Lasse Collin about slow updates
• "Jia Tan" appeared as helpful contributor, gradually earning commit access
• Established pattern of legitimate, high-quality contributions

Phase 2 - Infrastructure preparation (2023):

• Added binary test files (bad-3-corrupt_lzma2.xz) containing encrypted payload
• Modified build system (m4 macros, configure scripts) to extract and inject payload
• Changes disguised as performance optimizations and test updates

Phase 3 - Backdoor deployment (Feb-Mar 2024):

• Malicious code in liblzma/Makefile.am extracted encrypted payload during build
• Injected code modified liblzma.so to hook symbol resolution (IFUNC mechanism)
• Backdoor intercepted RSA_public_decrypt in OpenSSH's authentication flow

The technical mechanism:

1. Build-time script extracted encrypted blob from test files
2. Decryption used multi-stage obfuscation (bash script fragments)
3. Injected code hooked into liblzma.so initialization
4. At runtime, backdoor located sshd process via /proc/self/maps
5. Patched OpenSSH's RSA_public_decrypt to intercept authentication
6. Special Ed448 certificates triggered backdoor execution with embedded commands

Exploitation primitive

The backdoor enabled pre-authentication remote code execution:

Trigger mechanism: Attacker connected via SSH presenting a specially crafted certificate containing:

• Ed448 signature (uncommon, helps identification)
• Encrypted payload in certificate fields
• Signature verified against attacker's private key

Execution flow:

1. OpenSSH called RSA_public_decrypt during authentication
2. Backdoored function intercepted the call
3. Extracted and decrypted payload from certificate
4. Validated signature using embedded Ed448 public key
5. Executed payload with sshd privileges (typically root)

Capabilities:

• Full remote code execution as root
• No authentication required (pre-auth attack)
• Minimal logging (occurred before SSH audit logging)
• Network-based exploitation without local access

Detection evasion:

• Only activated in production-like environments (not in sandboxes)
• Checked for debuggers and analysis tools
• Disabled itself if LD_PRELOAD or LD_AUDIT detected
• Required specific environment variables absent in testing

Discovery: Microsoft engineer Andres Freund noticed 500ms SSH login delays on Debian sid, investigated with profiling tools, discovered CPU time in liblzma—an unusual dependency for sshd. Reverse engineering revealed the backdoor mere weeks before planned stable distribution releases.

Mitigation

Immediate actions:

• Downgrade XZ Utils to version 5.4.x series immediately
• Fedora 40/41: dnf downgrade xz xz-libs to 5.4.6-3
• Debian/Ubuntu: apt install xz-utils=5.4.5-0.3
• Arch Linux: downgrade via package cache or use xz-5.4.6-1-x86_64.pkg.tar.zst
• Verify installed version: xz --version (must be ≠5.6.0/5.6.1)

Validation steps:

1. Check for backdoor indicators:

   bash copy
   hexdump -ve '1/1 "%.2x"' /usr/lib/x86_64-linux-gnu/liblzma.so.5 | grep -q f30f1efa554889f54c89ce5389fb81e7000000804883ec28488954241848894c2410

2. Examine sshd dependencies: ldd /usr/sbin/sshd | grep liblzma
3. Review auth logs for anomalous SSH connection patterns

Infrastructure response:

• Inventory systems with affected distributions
• Rotate SSH host keys and credentials on potentially compromised systems
• Review authentication logs for unusual patterns (difficult: pre-auth compromise)
• Consider full reinstallation of affected systems (backdoor sophistication suggests possible persistence)

Supply chain security improvements:

• Audit dependencies of security-critical software (why does sshd link liblzma?)
• Implement reproducible builds to detect binary tampering
• Scrutinize binary test files in repositories
• Monitor maintainer changes in critical infrastructure projects
• Support funding for open-source security audits

Organizational policy:

• Delay deploying bleeding-edge package versions in production
• Implement staged rollout with canary systems
• Deploy binary transparency logs for package verification
• Establish baseline monitoring for unusual library dependencies

Systemic lessons:

• Single maintainer projects are high-value targets
• Social engineering timelines span years for sophisticated actors
• Build-time attacks evade most runtime security measures
• Community vigilance (Freund's discovery) remains critical defense

This incident represents the most sophisticated supply chain attack against open-source infrastructure to date, barely averted before reaching production systems worldwide. The attacker's patient, methodical approach and deep technical sophistication suggest state-level resources and expertise.