kavklaw@llm ~ /guides/gobuster-ffuf

kavklaw@llm $ cat content-discovery-guide.md

Gobuster & ffuf — Content Discovery

🟢 Beginner

⏱️ 20 min read · Find hidden directories, virtual hosts, and parameters that change everything

Both tools are pre-installed on Kali Linux and Parrot OS. If you need to install them manually:

# Gobuster — Go-based, single binary
sudo apt install gobuster                        # Debian/Ubuntu/Kali
# Or install from source (requires Go 1.19+):
go install github.com/OJ/gobuster/v3@latest

# ffuf — also Go-based
sudo apt install ffuf                            # Debian/Ubuntu/Kali
# Or install from source:
go install github.com/ffuf/ffuf/v2@latest

# You'll also want wordlists (SecLists)
sudo apt install seclists
# Installs to /usr/share/seclists/

What you'll need: A target URL, a wordlist, and terminal access. Both tools run from the command line — no GUI, no dependencies beyond the binary itself.

⚡ Quick Start

Need results fast? Here are the commands you'll use 90% of the time. Each one tries every entry from a wordlist as a path on the web server, looking for pages and directories that aren't linked anywhere on the site:

# Directory discovery with gobuster
gobuster dir -u http://target.htb -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt -t 50

# Directory discovery with ffuf (faster, more flexible)
ffuf -u http://target.htb/FUZZ -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt -t 50

# Virtual host discovery
ffuf -u http://target.htb -H "Host: FUZZ.target.htb" -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt -fs 301

# File extension fuzzing
gobuster dir -u http://target.htb -w /usr/share/seclists/Discovery/Web-Content/raft-medium-words.txt -x php,html,txt,bak -t 50

That covers directories, virtual hosts, and file extensions. Now let's dig into the details.

🌐 Why Content Discovery Matters

What "Directory Brute Forcing" Actually Means

At its core, directory brute forcing is dead simple: your tool sends an HTTP request for every word in a wordlist, appended to the target URL. If the wordlist contains "admin", the tool requests http://target.htb/admin and checks the response. A 200 OK means it exists; a 404 Not Found means it doesn't. Do this for 30,000 words and you've mapped out the hidden structure of the site. It's not "hacking" — it's automated guessing of file and directory names.

HTTP Response Codes You Need to Know

Every HTTP request returns a status code that tells you what happened. Understanding these is essential for interpreting scan results:

200 OK — The page exists and loaded normally. This is your hit.
301/302 Redirect — The resource exists but redirects you somewhere else (e.g., /admin redirects to /admin/ or /login). Still worth investigating.
403 Forbidden — The resource exists, but you're not allowed to access it. This is interesting — the server is actively blocking you, which means something valuable is there.
404 Not Found — The resource doesn't exist. This is the "miss" you're filtering out.
401 Unauthorized — Authentication required. Try default credentials.
500 Internal Server Error — The server crashed processing your request. Sometimes exploitable (indicates the app is processing your input).

Some applications don't return proper 404s — they return 200 with a custom "not found" page. That's when you need to filter by response size (-fs) instead of status code.

What Are Wordlists?

A wordlist is simply a text file with one entry per line — each entry is a directory or filename to try. Wordlists are built from real-world data: crawled websites, leaked source code, common conventions (/admin, /backup, /config), and accumulated pentesting experience. The quality of your wordlist directly determines what you find. A 4,600-entry wordlist catches the obvious stuff; a 220,000-entry wordlist catches the obscure. Using the wrong wordlist (e.g., a password list instead of a directory list) finds nothing.

When you browse a website, you see what the developers intended you to see: the public-facing pages, the navigation menu, the linked resources. But websites almost always have hidden content:

Admin panels — /admin, /dashboard, /manage, /cpanel
Backup files — config.php.bak, database.sql, .htpasswd
Development artifacts — /.git, /.svn, /debug, /test
API endpoints — /api/v1/users, /graphql, /swagger.json
Forgotten pages — /old, /backup, /staging
Virtual hosts (different websites served from the same IP, distinguished by the domain name in the request) — dev.target.htb, staging.target.htb, internal.target.htb

Content discovery tools systematically try thousands of potential paths and names from a wordlist (a text file containing one candidate per line) to find these hidden resources. A /.git directory can leak the entire source code. A /api/swagger.json can reveal every API endpoint. A backup file might contain database credentials.

In most CTFs, the initial foothold depends on finding something hidden. Content discovery is not optional. It's the bridge between port scanning and exploitation.

📚 Choosing Wordlists

Your content discovery is only as good as your wordlist. Picking the right wordlist for the context makes all the difference.

SecLists — The Gold Standard

# Install SecLists (usually pre-installed on Kali)
sudo apt install seclists
# Location: /usr/share/seclists/

# Key wordlists for content discovery:
/usr/share/seclists/Discovery/Web-Content/
├── raft-medium-directories.txt     # 30,000 dirs — good default
├── raft-large-directories.txt      # 62,000 dirs — thorough
├── raft-medium-words.txt           # 63,000 words — for file fuzzing
├── raft-medium-files.txt           # 11,000 files — common filenames
├── directory-list-2.3-medium.txt   # 220,000 — DirBuster classic
├── common.txt                      # 4,600 — quick and dirty
├── big.txt                         # 20,000 — medium coverage
└── quickhits.txt                   # 2,500 — fast common paths

# For subdomain/vhost discovery:
/usr/share/seclists/Discovery/DNS/
├── subdomains-top1million-5000.txt   # Quick subdomain scan
├── subdomains-top1million-20000.txt  # Medium coverage
├── subdomains-top1million-110000.txt # Thorough

# For parameter fuzzing:
/usr/share/seclists/Discovery/Web-Content/
├── burp-parameter-names.txt        # Common parameter names
├── api/api-endpoints.txt           # API path patterns

Which Wordlist When?

Quick first pass: common.txt (4,600 entries, ~30 seconds)
Standard scan: raft-medium-directories.txt (30,000 entries, ~2 minutes)
Thorough scan: directory-list-2.3-medium.txt (220,000 entries, ~15 minutes)
Everything: directory-list-2.3-big.txt (1.2 million entries, ~1 hour)

Custom Wordlists

# CeWL (Custom Word List generator) — crawls the target website and builds
# a wordlist from words found on its pages (great for company-specific terms)
cewl http://target.htb -d 3 -m 5 -w custom_wordlist.txt
# -d 3 = follow links 3 levels deep
# -m 5 = minimum word length of 5

# Combine wordlists and deduplicate
cat wordlist1.txt wordlist2.txt | sort -u > combined.txt

# Generate technology-specific wordlists
# Found PHP? Focus on PHP-specific paths
# Found WordPress? Use wp-specific wordlists
/usr/share/seclists/Discovery/Web-Content/CMS/wordpress.fuzz.txt

# Build from page content
curl -s http://target.htb | grep -oP 'href="[^"]*"' | sed 's/href="//;s/"//' | sort -u

💡 Pro Tip: If you know the technology stack (PHP, ASP.NET, Node.js, Python), use technology-specific wordlists. A WordPress site won't have /manage.py, and a Django site won't have /wp-admin. Context-aware fuzzing is faster and finds more.

📂 Gobuster — Directory Mode

Gobuster's dir mode brute-forces directories and files on web servers. It's written in Go, so it's fast and has low memory footprint.

# Basic directory scan
gobuster dir -u http://target.htb -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt

# Common flags:
# -u    Target URL
# -w    Wordlist path
# -t    Number of threads (default 10, recommend 50)
# -o    Output file
# -x    File extensions to append
# -s    Status codes to show (default: 200,204,301,302,307,401,403)
# -b    Status codes to exclude (blacklist)
# -r    Follow redirects
# -k    Skip TLS verification (for self-signed certs)
# -a    User-Agent string
# -c    Cookie string
# -H    HTTP header (can be used multiple times)

# Full-featured scan
gobuster dir \
  -u http://target.htb \
  -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt \
  -t 50 \
  -x php,html,txt \
  -o gobuster_results.txt \
  -k

# With authentication cookie
gobuster dir \
  -u http://target.htb \
  -w /usr/share/wordlists/dirb/common.txt \
  -c "session=abc123; token=xyz789" \
  -t 50

# With custom header
gobuster dir \
  -u http://target.htb \
  -w wordlist.txt \
  -H "Authorization: Bearer eyJhbG..."

Reading Gobuster Output

# Typical output:
===============================================================
Gobuster v3.6
[+] Url:            http://target.htb
[+] Threads:        50
[+] Wordlist:       /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt
===============================================================
/admin                (Status: 301) [Size: 310] [--> http://target.htb/admin/]
/images               (Status: 301) [Size: 311] [--> http://target.htb/images/]
/login                (Status: 200) [Size: 2847]
/uploads              (Status: 403) [Size: 277]
/api                  (Status: 301) [Size: 308] [--> http://target.htb/api/]
/server-status        (Status: 403) [Size: 277]
/.git                 (Status: 301) [Size: 309] [--> http://target.htb/.git/]

# What to investigate:
# 301 → Directory exists, check its contents
# 200 → Direct hit, visit it
# 403 → Forbidden but EXISTS — interesting!
# 401 → Authentication required — try default creds
# .git → Source code leak! Quick check: curl /.git/config
# If it returns a valid Git config, use git-dumper to extract the full repo

🏠 Gobuster — VHost Mode

Virtual host enumeration discovers subdomains that resolve to the same IP but serve different content based on the Host header. This is a must for HTB/CTF machines.

# VHost brute forcing
gobuster vhost \
  -u http://target.htb \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  --append-domain

# --append-domain automatically adds .target.htb to each word
# Without it, you'd need a wordlist with full domain names

# Filter out false positives by size
# First, note the size of a "not found" vhost response
# Then filter:
gobuster vhost \
  -u http://target.htb \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  --append-domain \
  --exclude-length 301   # Exclude responses with body length of 301 bytes (default vhost redirect)

# Example output:
Found: dev.target.htb Status: 200 [Size: 5829]
Found: admin.target.htb Status: 302 [Size: 0]
Found: staging.target.htb Status: 200 [Size: 12053]

# Don't forget to add discovered vhosts to /etc/hosts!
echo "10.10.10.100 dev.target.htb admin.target.htb staging.target.htb" | sudo tee -a /etc/hosts

💡 Pro Tip: In HTB, almost every medium+ machine uses virtual hosts. If you find a domain name (from nmap's ssl-cert script, page content, or error messages), ALWAYS try vhost enumeration. The attack surface often lives on a subdomain.

🌍 Gobuster — DNS Mode

DNS mode resolves subdomains via actual DNS queries. Unlike vhost mode (which sends HTTP requests), DNS mode queries the DNS server directly.

# DNS subdomain brute forcing
gobuster dns -d target.htb -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt

# With specific DNS server
gobuster dns -d target.htb -w wordlist.txt -r 10.10.10.100

# Show IP addresses
gobuster dns -d target.htb -w wordlist.txt --show-ips

# Example output:
Found: mail.target.htb [10.10.10.100]
Found: vpn.target.htb [10.10.10.101]
Found: dc.target.htb [10.10.10.102]

When to use DNS mode vs VHost mode:

DNS mode: For real-world targets with actual DNS records. Tests if the subdomain resolves to an IP.
VHost mode: For targets where DNS might not have records (like HTB/CTF machines using /etc/hosts). Tests if the web server responds differently to the Host header.

🚀 ffuf — Directory Fuzzing

ffuf (Fuzz Faster U Fool) is the go-to tool for web fuzzing. It's often faster than gobuster in practice, more flexible, and uses the FUZZ keyword to mark where payloads go.

# Basic directory fuzzing
ffuf -u http://target.htb/FUZZ -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt

# The FUZZ keyword marks where wordlist entries are inserted
# http://target.htb/FUZZ → http://target.htb/admin, http://target.htb/login, ...

# Common flags:
# -u    URL with FUZZ keyword
# -w    Wordlist
# -t    Threads (default 40)
# -o    Output file
# -of   Output format (json, csv, html, md)
# -fc   Filter by status code (exclude)
# -fs   Filter by response size (exclude)
# -fw   Filter by word count (exclude)
# -fl   Filter by line count (exclude)
# -mc   Match by status code (include)
# -ms   Match by response size
# -H    HTTP header
# -X    HTTP method
# -d    POST data
# -b    Cookie header
# -r    Follow redirects
# -recursion  Enable recursive scanning

# Full-featured scan
ffuf -u http://target.htb/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt \
  -t 50 \
  -fc 404 \
  -o results.json \
  -of json

# With extension fuzzing
ffuf -u http://target.htb/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/raft-medium-words.txt \
  -e .php,.html,.txt,.bak,.old \
  -t 50 \
  -fc 404

Multi-Position Fuzzing

ffuf can fuzz multiple positions simultaneously, something gobuster can't do:

# Fuzz directory AND extension simultaneously
ffuf -u http://target.htb/W1.W2 \
  -w /usr/share/seclists/Discovery/Web-Content/raft-medium-words.txt:W1 \
  -w extensions.txt:W2 \
  -t 50 -fc 404

# Where extensions.txt contains:
# php
# html
# txt
# bak
# config

# This tries: admin.php, admin.html, admin.txt, login.php, login.html, ...

🔑 ffuf — Parameter Fuzzing

One of ffuf's strongest features is parameter discovery: finding hidden GET/POST parameters that the application accepts.

# GET parameter discovery
ffuf -u "http://target.htb/page?FUZZ=test" \
  -w /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt \
  -fs 4242  # Filter out the default response size

# POST parameter discovery
ffuf -u http://target.htb/login \
  -X POST \
  -d "FUZZ=test" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -w /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt \
  -fs 4242

# Parameter value fuzzing (you know the parameter, fuzz the value)
ffuf -u "http://target.htb/api/user?id=FUZZ" \
  -w /usr/share/seclists/Fuzzing/4-digits-0000-9999.txt \
  -fc 404

# JSON body parameter fuzzing (FUZZ replaces the key name)
# This tests: {"username":"admin","id":"test"}, {"username":"admin","role":"test"}, etc.
ffuf -u http://target.htb/api/login \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","FUZZ":"test"}' \
  -w /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt \
  -fs 200

💡 Pro Tip: Hidden parameters are everywhere. A page might accept ?debug=true, ?admin=1, or ?source=1 that reveals debug information, bypasses authentication, or leaks source code. Always try parameter fuzzing on interesting endpoints.

🧠 Knowledge Check — Directory & Parameter Fuzzing

Complete this ffuf command to discover hidden directories on a target:

ffuf uses the FUZZ keyword to mark where wordlist entries are inserted in the URL. -u specifies the URL, -w the wordlist, and -t 50 sets threads to 50 for speed. The raft-medium-directories.txt wordlist (30,000 entries) is a good default. ffuf is different from gobuster — it doesn't use separate modes; the FUZZ keyword position determines what's being fuzzed.

Complete this gobuster command to fuzz for PHP, HTML, and TXT files:

The -x flag in gobuster appends file extensions to each wordlist entry. So with -x php,html,txt and a word "config", gobuster tries /config, /config.php, /config.html, and /config.txt. This is critical because a path like /config might return 404, but /config.php returns 200. Extension fuzzing multiplies the number of requests, so use it judiciously.

What is the default wordlist size that's recommended as a "starting point" for quick content discovery?

common.txt with ~4,600 entries completes in about 30 seconds and catches the most common paths. It's your quick first pass. Escalate to raft-medium-directories.txt (30,000 entries, ~2 min) for standard coverage, then directory-list-2.3-medium.txt (220,000 entries, ~15 min) if nothing was found. rockyou.txt is a password list, not a directory wordlist!

🏠 ffuf — Subdomain/VHost Fuzzing

# Virtual host fuzzing (like gobuster vhost but faster)
ffuf -u http://target.htb \
  -H "Host: FUZZ.target.htb" \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  -fs 301

How to find the right filter size:

First run WITHOUT filters: ffuf -u http://target.htb -H "Host: FUZZ.target.htb" -w wordlist.txt
Note the most common response size (e.g., all show "Size: 301")
Re-run WITH -fs to filter that size: ffuf ... -fs 301

Now only non-default responses show up = real vhosts!

🔍 Filtering Output

Good filtering is what separates useful results from noise. Both tools support filtering, but ffuf gives you more options.

ffuf Filters

# Filter by status code (exclude 404s and 403s)
ffuf -u http://target.htb/FUZZ -w wordlist.txt -fc 404,403

# Filter by response size (exclude responses of size 1234)
ffuf -u http://target.htb/FUZZ -w wordlist.txt -fs 1234

# Filter by word count
ffuf -u http://target.htb/FUZZ -w wordlist.txt -fw 42

# Filter by line count
ffuf -u http://target.htb/FUZZ -w wordlist.txt -fl 10

# Filter by regex
ffuf -u http://target.htb/FUZZ -w wordlist.txt -fr "not found"

# Match filters (whitelist instead of blacklist)
# Only show 200 and 302 responses
ffuf -u http://target.htb/FUZZ -w wordlist.txt -mc 200,302

# Only show responses larger than 500 bytes
ffuf -u http://target.htb/FUZZ -w wordlist.txt -ms ">500"

# Combine filters
ffuf -u http://target.htb/FUZZ -w wordlist.txt -fc 404 -fs 0 -fw 1

Gobuster Filters

# Show only specific status codes
gobuster dir -u http://target.htb -w wordlist.txt -s "200,301,302"

# Exclude status codes
gobuster dir -u http://target.htb -w wordlist.txt -b "404,403"

# Exclude by response body length in bytes (useful for filtering default/redirect pages)
gobuster dir -u http://target.htb -w wordlist.txt --exclude-length 301

# Hide responses with no content
gobuster dir -u http://target.htb -w wordlist.txt --no-status

📎 Extension Fuzzing

The right file extension can be the difference between a 403 Forbidden and full source code disclosure.

# Gobuster — append extensions to every word
gobuster dir -u http://target.htb -w wordlist.txt -x php,html,txt,bak,old,config

# ffuf — extension fuzzing with -e flag
ffuf -u http://target.htb/FUZZ -w wordlist.txt -e .php,.html,.txt,.bak,.old

# ffuf — dedicated extension fuzzing for a known file
ffuf -u http://target.htb/config.FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/web-extensions.txt

# Common extensions to try by technology:
# PHP:  .php, .php5, .php7, .phtml, .inc, .phps
# ASP:  .asp, .aspx, .ashx, .asmx, .config
# Java: .jsp, .jspx, .do, .action
# Python: .py
# Backup: .bak, .old, .orig, .save, .swp, .tmp, ~
# Config: .xml, .json, .yaml, .yml, .conf, .cfg, .ini, .env
# Data: .sql, .sqlite, .db, .csv, .log

💡 Pro Tip: If you find a file like config.php that returns an empty 200 (PHP is executing), try config.php.bak, config.php.old, config.php.swp, or config.phps. Backup files serve as plain text, exposing the source code with database credentials.

🔄 Recursive Scanning

When you find a directory, you should scan inside it too. Both tools support recursive scanning.

# ffuf recursive scanning
ffuf -u http://target.htb/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/common.txt \
  -recursion \
  -recursion-depth 3 \
  -fc 404

# This will:
# 1. Find /admin/ → scan http://target.htb/admin/FUZZ
# 2. Find /admin/config/ → scan http://target.htb/admin/config/FUZZ
# 3. Up to 3 levels deep

# Gobuster doesn't have built-in recursion
# Workaround: script it
for dir in $(gobuster dir -u http://target.htb -w wordlist.txt -q | grep "Status: 301" | awk '{print $1}'); do
  echo "[*] Scanning $dir ..."
  gobuster dir -u "http://target.htb${dir}" -w wordlist.txt -t 50 -q
done

⚠️ Warning: Recursive scanning generates exponentially more requests. With a 30,000-word wordlist and 3 levels of recursion, a site with 10 directories per level generates 30,000 × (1 + 10 + 100) = 3.3 million requests. Use a smaller wordlist for recursive scans, or limit depth.

⚡ Speed Tuning

# ffuf — adjust threads (default 40)
ffuf -u http://target.htb/FUZZ -w wordlist.txt -t 100

# ffuf — rate limiting (requests per second)
ffuf -u http://target.htb/FUZZ -w wordlist.txt -rate 100
# Useful when the target has rate limiting

# ffuf — delay between requests
ffuf -u http://target.htb/FUZZ -w wordlist.txt -p 0.1
# 0.1 second delay between requests

# ffuf — timeout per request
ffuf -u http://target.htb/FUZZ -w wordlist.txt -timeout 5

# gobuster — thread count
gobuster dir -u http://target.htb -w wordlist.txt -t 100

# gobuster — timeout
gobuster dir -u http://target.htb -w wordlist.txt --timeout 5s

# gobuster — delay between requests
gobuster dir -u http://target.htb -w wordlist.txt --delay 100ms

Speed Guidelines

HTB/CTF (local VPN): 50-100 threads, no rate limit needed
Remote targets: 20-30 threads, watch for errors
Targets with a WAF (Web Application Firewall — a security layer that blocks suspicious requests) or rate limiting: 5-10 threads, add delays
Bug bounty: Be respectful — 10-20 threads, rate limit yourself

🤔 When to Use Which Tool

Use Gobuster When:

Simple directory brute forcing (it's straightforward)
VHost enumeration (vhost mode is clean)
DNS subdomain enumeration (dns mode)
You want minimal output and clear results
You're new to content discovery (simpler flags)

Use ffuf When:

Parameter fuzzing (ffuf excels here)
Multi-position fuzzing (fuzz two things at once)
You need flexible filtering (-fc, -fs, -fw, -fl, -fr)
Recursive scanning (built-in support)
POST request fuzzing
You need JSON output for scripting
Speed is critical (slightly faster than gobuster)

Quick Decision Tree

What do you need?           → Best tool
═════════════════════════════════════════
Find directories            → Either (ffuf for control, gobuster for simplicity)
Find virtual hosts          → ffuf (easier filtering) or gobuster vhost
Fuzz parameters             → ffuf (gobuster can't)
Fuzz POST data              → ffuf (gobuster can't)
Recursive scanning          → ffuf (built-in)
DNS subdomain brute force   → gobuster dns mode

🎯 Real-World Methodology

Content Discovery Workflow
══════════════════════════════════════════════════════════
  Step 1          Step 2           Step 3          Step 4-5
  Quick Sweep     Main Scan        VHost Enum      Deep Dive
  common.txt      raft-medium      subdomains      Scan found
  (~30 sec)       + extensions     + filter size   dirs, params
                  (~3 min)         (~1 min)        (as needed)
       │               │               │               │
       ▼               ▼               ▼               ▼
  /admin? /api?   /config.php?   dev.target.htb?  /api/FUZZ?
  /.git?          /backup.bak?   admin.target.htb? ?debug=true?

Step 1: Quick Sweep (1-2 minutes)

# Fast scan with small wordlist to get immediate results
ffuf -u http://target.htb/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/common.txt \
  -t 50 -fc 404

Step 2: Main Directory Scan (3-5 minutes)

# Medium wordlist with extension fuzzing
ffuf -u http://target.htb/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt \
  -e .php,.html,.txt \
  -t 50 -fc 404 -o dir_scan.json -of json

Step 3: VHost Enumeration (1-2 minutes)

# Find virtual hosts
ffuf -u http://target.htb \
  -H "Host: FUZZ.target.htb" \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  -fs $(curl -s http://target.htb | wc -c)  # Auto-filter default size

Step 4: Deep Dive on Interesting Paths (as needed)

# Found /api? Scan inside it
ffuf -u http://target.htb/api/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/api/api-endpoints.txt \
  -t 50 -fc 404

# Found a PHP app? Try PHP-specific files
ffuf -u http://target.htb/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/Common-PHP-Filenames.txt \
  -t 50 -fc 404

# Check for backup files of known files
ffuf -u http://target.htb/config.FUZZ \
  -w extensions.txt  # bak, old, orig, swp, txt, etc.

Step 5: Parameter Discovery (for interesting endpoints)

# Found an interesting page? Fuzz for hidden parameters
ffuf -u "http://target.htb/page.php?FUZZ=1" \
  -w /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt \
  -fs $(curl -s "http://target.htb/page.php" | wc -c)

⚠️ Common Mistakes

❌ Mistake #1: Using too small a wordlist.
common.txt (4,600 words) is a starting point, not an endpoint. If the quick scan finds nothing, escalate to raft-medium or directory-list-2.3-medium. The path you need might not be in a small wordlist.

❌ Mistake #2: Not fuzzing file extensions.
Directories are only half the story. A path like /config returns 404, but /config.php returns 200. Always fuzz with relevant extensions (-x php,html,txt,bak).

❌ Mistake #3: Not filtering output properly.
If every request returns a 200 with a custom "not found" page, you need to filter by size (-fs), word count (-fw), or line count (-fl). Status code filtering alone isn't always enough.

❌ Mistake #4: Forgetting VHost enumeration.
Finding target.htb doesn't mean you've found everything. dev.target.htb, admin.target.htb, or api.target.htb might serve completely different applications. Always try vhost fuzzing.

❌ Mistake #5: Not adding discovered hostnames to /etc/hosts.
After discovering vhosts, you MUST add them to /etc/hosts or they won't resolve. This is the #1 beginner mistake in HTB.

❌ Mistake #6: Scanning without knowing the technology stack.
Check headers and error pages first. If it's PHP, fuzz for .php. If it's Python, check for .py and framework-specific paths. Technology awareness makes your fuzzing 10x more effective.

🔄 Tool Comparison: Gobuster vs FFUF vs Feroxbuster vs Dirsearch

📂 Gobuster

Speed

8/10

Recursion

0/10

Fuzzing

4/10

Output Formats

5/10

Filtering

5/10

Best for: Simple, fast directory brute forcing and DNS subdomain enumeration. Go binary — zero dependencies.

🎯 FFUF

Speed

9/10

Recursion

6/10

Fuzzing

10/10

Output Formats

9/10

Filtering

10/10

Best for: Multi-position fuzzing, parameter discovery, POST fuzzing, and advanced filtering. The most versatile fuzzer.

🔥 Feroxbuster

Speed

9/10

Recursion

10/10

Fuzzing

3/10

Output Formats

7/10

Filtering

8/10

Best for: Automatic recursive discovery — "set and forget." Finds deeply nested directories that flat scanners miss.

🔎 Dirsearch

Speed

6/10

Recursion

8/10

Fuzzing

2/10

Output Formats

8/10

Filtering

6/10

Best for: Beginners — Python-based, smart extension handling, built-in wordlists. Great default configs out of the box.

📚 Further Reading

Nmap Guide — Find web services to enumerate
Burp Suite Guide — Deeper web testing after discovery
Web Hacking Checklist — Full testing methodology
SecLists on GitHub — The ultimate wordlist collection
ffuf GitHub — Official documentation and examples
Gobuster GitHub — Official documentation
Learning Path — Where content discovery fits in the bigger picture

🏆 Section Assessment — Content Discovery Mastery

You run ffuf and every single response returns status 200 with varying sizes. How do you filter to find real content?

Many web apps return 200 for all paths with a custom "not found" page instead of a proper 404. The trick is to filter by response size: first run without filters, note the most common size (e.g., all showing 1234 bytes), then re-run with -fs 1234. Only responses with different sizes (real pages) will show up. You can also filter by word count (-fw) or line count (-fl) for more precision.

How do you discover virtual hosts (vhosts) on a target using ffuf?

VHost fuzzing works by sending different Host: headers to the same IP. -H "Host: FUZZ.target.htb" substitutes each wordlist entry into the Host header. The web server responds differently for valid vhosts vs invalid ones. -fs 301 filters the default response size (you determine this from an initial run). After finding vhosts, add them to /etc/hosts!

When should you use ffuf instead of gobuster?

ffuf's key advantage is flexibility. It can fuzz parameters (?FUZZ=value), POST data, JSON bodies, headers, and multiple positions simultaneously. Gobuster can't do parameter fuzzing at all. For simple directory brute forcing, both tools work equally well. Use gobuster's dns mode for actual DNS resolution of subdomains, and ffuf for HTTP-level vhost discovery.

You found /.git (Status: 301) in your gobuster output. What should you do?

An exposed .git directory is a critical finding — it can leak the entire application source code, commit history, and potentially credentials committed in past versions. Quick confirmation: curl http://target/.git/config — if it returns a valid Git config, it's game over. Then use git-dumper (or GitTools) to reconstruct the full repository. git clone won't work because the web server doesn't speak the Git protocol. Check commit history with git log and git diff for hardcoded passwords.

After discovering a vhost like dev.target.htb, what critical step must you take before you can access it in your browser?

CTF/HTB machines don't have public DNS records. Your browser needs to know that dev.target.htb resolves to the target IP. Adding an entry to /etc/hosts handles this locally. This is the #1 beginner mistake in HTB — discovering virtual hosts but forgetting to add them to /etc/hosts, then wondering why the browser shows a different page or can't connect.