Carnival Crashers: From a Forgotten Health-Check to Root

Monitoring flagged unusual outbound traffic from web01, the box serving CarnivalCart's storefront. No clean snapshot, no packaged log bundle, just a live host, mid-incident, with whatever the attacker left running still running. This is the walkthrough of that investigation, start to finish.

The entry point nobody was watching

The box exposes a legacy Apache CGI health-check, carried over from an older server during a past migration and never rewritten or decommissioned. That's the kind of thing that looks harmless for years right up until it isn't.

First stop on any live box: confirm your own footing before touching anything else.

sudo -l

Full, passwordless sudo for the responder account, so nothing below needs a password prompt.

With that confirmed, the logs. The access log is root-owned, so:

sudo tail /var/log/apache2/access.log

Buried among a couple of generic scanner probes (/.env, /wp-login.php, noise every internet-facing box gets) are two requests that stand out. Both hit a CGI endpoint, and both carry a User-Agent that starts with () { :;}; instead of anything resembling a browser string. That's the unmistakable shape of a Shellshock (CVE-2014-6271) payload. The first is a harmless verification probe (/bin/true, no lasting effect). A few seconds later, the second is the real thing: it chains a curl that pulls a file into /tmp, makes it executable, and runs it, all backgrounded at the top level so Apache's CGI handler doesn't choke on a synchronous child process.

Why did this work at all? The CGI script's shebang doesn't point at the system /bin/bash. It points at a separate interpreter installed just for this script, a carryover from whatever system this was migrated from. Comparing --version output between the two tells the story: the system shell is a patched 5.x build; the script's own interpreter is a stale 4.3 that predates the fix.

The dropped file sits in /tmp, named to blend into a process listing next to real kernel worker threads rather than anything that screams "malware."

Finding what's still alive

A compromise that happened once is a forensics exercise. A compromise that's still running is an active incident. ps aux turns up a process that's clearly been looping far longer than anything else on the box. Checking /proc/<pid>/cmdline for the untruncated command line shows it's calling out to a URL on a fixed interval. The query string changes every run (hostname, uid, timestamp baked in for check-in purposes), but the base URL doesn't. That's the indicator of compromise worth writing down, not the noise around it.

That same process's parent, per ps -eo pid,ppid,cmd, is PID 1, not whatever script originally spawned it. A backgrounded, disowned process outlives its launcher and gets reaped by init once the parent exits. That mismatch between "what's running" and "what's supposed to have spawned it" is itself a signal.

From www-data to root

sudo -V gives a version old enough to carry a known runas-restriction bypass: CVE-2019-14287. sudo -l -U www-data confirms a restriction exists, but the full rule, read directly from /etc/sudoers.d/ rather than the summary, is what matters:

www-data ALL=(ALL, !root) NOPASSWD: /opt/carnivalcart/tools/*.sh

It looks like it safely excludes root. The !root exclusion is exactly what CVE-2019-14287 bypasses: requesting a UID of -1 (or 4294967295) satisfies "not root" by the letter of the restriction while sudo itself resolves it straight to UID 0.

The one directory that rule allows scripts to run from has exactly one file that doesn't match the age or ownership of the legitimate helper script sitting next to it. That's the attacker's own tooling, dropped and immediately invoked with the bypass.

Persistence and the exfil that got staged

With root, the attacker did three things. They added a cron entry under /etc/cron.d/ that doesn't match anything else in that directory, re-firing the beacon on a schedule even if the live process ever dies. They archived the web root to a path in /tmp, ready to ship out. And they left the live beacon running, the thing that tipped off monitoring in the first place.

Reconstructing the timeline

Correlating access.log timestamps against process start times (ps -o lstart, or the timestamps embedded in /proc/<pid>/stat) lines the whole chain up: initial access through Shellshock, the dropper fetched and executed, privilege escalation through the sudo bypass, persistence via cron, then the live beacon spawned and the web root staged for exfiltration.

Every artifact above, the dropper, the sudoers rule, the planted script, the cron entry, the staged archive, traces back to that one forgotten CGI health-check. Nothing here required anything exotic: two CVEs from 2014 and 2019, chained against a box nobody had touched since the migration that introduced it.

Takeaway

"It's just a health-check, it's not internet-facing traffic really" is exactly the kind of reasoning that keeps a box like this unpatched for years. Inventory what's actually exposed, not what you remember exposing.

Answers