kavklaw@llm ~ /learn/phase-2
← Back to Learning Path

Phase 2: Reconnaissance

Information is ammunition. The more you know about a target, the more attack surface you see.

02

Phase 2: Reconnaissance

Information is ammunition. The more you know about a target, the more attack surface you find.

🔍 Port Scanning with Nmap

Every pentest starts with "what's running on this machine?" Nmap is how you find out.

The Scanning Methodology

Don't just run nmap -sC -sV target (where -sC runs default enumeration scripts and -sV detects service versions) and wait 20 minutes. Use a tiered approach:

# Step 1: Fast discovery — what ports are open?
nmap -p- --min-rate 5000 -T4 target -oN allports.txt

# Step 2: Deep scan on open ports only
nmap -p 22,80,443,8080 -sC -sV -oN detailed.txt target

# Step 3: Targeted scripts for interesting services
nmap -p 445 --script=smb-vuln* target
nmap -p 80 --script=http-enum target

# Step 4: UDP scan (often forgotten, often valuable!)
sudo nmap -sU --top-ports 50 --min-rate 3000 target -oN udp.txt

Understanding Scan Output

PORT     STATE    SERVICE  VERSION
22/tcp   open     ssh      OpenSSH 7.9p1    ← Version → search for CVEs (known vulnerabilities)
80/tcp   open     http     nginx/1.14.2     ← Web server → enumerate
111/tcp  open     rpcbind  2-4              ← RPC (Remote Procedure Call) → check for NFS/GlusterFS
443/tcp  closed   https                     ← Closed ≠ filtered
3128/tcp open     http-proxy Squid 4.6      ← Proxy → potential pivot
8080/tcp filtered http                      ← Firewall blocking → interesting

Nmap Scripting Engine (NSE) — Beyond Basic Scanning

Nmap's scripts are categorized and incredibly powerful. Key categories:

# Vulnerability scanning
nmap --script vuln target              # Run all vuln scripts
nmap -p 443 --script ssl-heartbleed target  # Specific CVE check

# Information gathering
nmap --script http-headers target      # HTTP response headers
nmap --script http-robots.txt target   # Read robots.txt
nmap --script http-title target        # Page titles
nmap --script banner target            # Service banners

# Authentication testing
nmap --script ftp-anon target          # Anonymous FTP?
nmap --script smtp-open-relay target   # Open relay?
nmap --script mysql-empty-password target  # Empty MySQL root?

# Brute force (use carefully!)
nmap --script ssh-brute --script-args userdb=users.txt,passdb=pass.txt target
🧠 Knowledge Check — Nmap Scanning
Complete the Nmap command to do a fast scan of ALL 65,535 TCP ports:
Fill in the blank:
$ nmap  --min-rate 5000 -T4 target
-p- tells Nmap to scan all 65,535 ports (not just the default top 1,000). Combined with --min-rate 5000 and -T4, this gives you a fast full port scan. This is always Step 1 of the scanning methodology — find ALL open ports first, then do detailed scans on what you find.
In Nmap output, what does a "filtered" port state indicate?
Filtered means a firewall or network filter is preventing Nmap from reaching the port — packets are being silently dropped with no response. Closed means the port responded with RST (explicitly refusing the connection). Filtered ports are interesting because something might be running behind the firewall — try scanning from a different source or using firewall evasion techniques.

📂 Directory & Content Discovery

The web page you see is rarely all there is. Hidden directories, backup files, and admin panels are everywhere. Content discovery tools work by sending thousands of requests with common directory and file names (from a wordlist) to see which ones exist on the server.

# Gobuster directory brute force — tries every word in the wordlist as a directory/file name
gobuster dir -u http://target -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt -t 50

# ffuf (Fuzz Faster U Fool) — a flexible fuzzing tool. "FUZZ" is replaced with each wordlist entry.
ffuf -u http://target/FUZZ -w /usr/share/wordlists/dirb/common.txt -fc 404

# Subdomain enumeration
gobuster vhost -u http://target.htb -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt --append-domain

# Don't forget extensions!
gobuster dir -u http://target -w wordlist.txt -x php,html,txt,bak

# Recursive directory scanning with feroxbuster
feroxbuster -u http://target -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt --depth 3

# API endpoint discovery
ffuf -u http://target/api/FUZZ -w /usr/share/seclists/Discovery/Web-Content/api/api-endpoints.txt
ffuf -u http://target/api/v1/FUZZ -w /usr/share/seclists/Discovery/Web-Content/common.txt

What to Look For

  • /admin, /dashboard, /panel — Admin interfaces (often poorly protected)
  • /.git, /.svn — Source code repositories (goldmine!)
  • /backup, /db, /config — Configuration files with credentials
  • .bak, .old, .swp files — Editor backups that expose source code
  • robots.txt, sitemap.xml — Things the site doesn't want indexed
  • /api, /v1, /graphql — API endpoints with less protection than the frontend
  • /.env, /config.php.bak, /web.config — Configuration files with secrets
  • /server-status, /server-info — Apache information disclosure

Wordlist Selection Strategy

The right wordlist makes or breaks content discovery. Different situations call for different lists:

  • Quick scan: /usr/share/wordlists/dirb/common.txt (4,614 words) — fast first pass
  • Standard scan: /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt (~30K) — good balance
  • Thorough scan: /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-big.txt (~1.2M) — when you're desperate
  • Technology-specific: Use wordlists tailored to the stack (e.g., IIS, Tomcat, WordPress)
  • Custom: Build from CeWL (Custom Word List generator) (cewl http://target -d 3 -m 5 -w custom.txt) — crawls a website and creates a wordlist from the words it finds, useful for guessing passwords and directories based on the site's own content

🎯 Practice: Enumerating a Typical CTF Box (Step-by-Step)

Here's exactly what enumerating a real CTF box looks like from start to finish. Follow this workflow on every box:

# ══════════════════════════════════════════════
# STEP-BY-STEP: Enumerating a CTF Box
# Target: 10.10.10.X (after connecting to VPN)
# ══════════════════════════════════════════════

# STEP 1: Create workspace and add to /etc/hosts
mkdir -p ~/htb/boxes/target/{nmap,web,loot}
echo "10.10.10.X  target.htb" | sudo tee -a /etc/hosts

# STEP 2: Fast TCP scan — find ALL open ports
nmap -p- --min-rate 5000 -T4 10.10.10.X -oN nmap/fast.txt
# → Found: 22, 80, 139, 445, 8080

# STEP 3: Detailed scan on discovered ports
nmap -p 22,80,139,445,8080 -sC -sV -oN nmap/detailed.txt 10.10.10.X
# → SSH 7.9, Apache 2.4.41, Samba 4.11, Tomcat 9.0

# STEP 4: UDP scan (background — takes time)
sudo nmap -sU --top-ports 20 --min-rate 3000 10.10.10.X -oN nmap/udp.txt &

# STEP 5: Web enumeration (port 80)
whatweb http://target.htb                          # Identifies the tech stack (web server, CMS, frameworks, etc.)
curl -sI http://target.htb                          # Response headers
gobuster dir -u http://target.htb -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt -x php,html,txt,bak -t 50 -o web/gobuster-80.txt

# STEP 6: Web enumeration (port 8080 — Tomcat)
gobuster dir -u http://target.htb:8080 -w /usr/share/seclists/Discovery/Web-Content/tomcat.txt -t 50
curl http://target.htb:8080/manager/html           # Default Tomcat manager?

# STEP 7: SMB enumeration (SMB = Windows file sharing)
smbclient -L //10.10.10.X -N                       # List shares (-N = no password, "null session")
enum4linux -a 10.10.10.X                            # Extracts users, groups, shares, and OS info from SMB
crackmapexec smb 10.10.10.X --shares               # Shows shares with your access level (read/write)

# STEP 8: Check for quick wins
curl http://target.htb/robots.txt                   # Disallowed paths
curl http://target.htb/.git/HEAD                    # Exposed git repo?
nmap --script vuln -p 445 10.10.10.X                # Known SMB vulns?

# STEP 9: Document everything in notes.md
# What you found, what looks interesting, what to try next

🏆 After This Phase, You Should Be Able To:

  • Run a complete Nmap scan methodology (fast → detailed → scripts → UDP)
  • Enumerate web servers with gobuster/ffuf and know which wordlists to use when
  • Identify and enumerate at least 10 common services (SSH, HTTP, SMB, DNS, FTP, SNMP, MySQL, Redis, LDAP, NFS)
  • Discover hidden directories, subdomains, and backup files on web servers
  • Read and interpret Nmap output (open vs closed vs filtered)
  • Use Wireshark to capture and analyze network traffic from your scans
  • Explain the difference between active and passive reconnaissance

🎮 Practice This on HackTheBox / TryHackMe

These rooms and boxes are specifically chosen to practice scanning and enumeration:

  • TryHackMe — Nmap room (free): Guided walkthrough of every Nmap scan type. Start here.
  • TryHackMe — Kenobi: SMB + NFS enumeration → foothold. Perfect for service enum practice.
  • TryHackMe — Network Services 1 & 2: Practice SMB, Telnet, FTP, NFS, MySQL, SMTP enumeration.
  • HackTheBox — Lame (Easy): Classic SMB enumeration → exploit. Great first HTB box.
  • HackTheBox — Shocker (Easy): Web directory enumeration is KEY to solving this one.
  • HackTheBox — Nibbles (Easy): Requires careful web enumeration and source code analysis.
  • HackTheBox Starting Point: Guided tier with scan → enumerate → exploit flow. Do ALL of them.

🔎 Service Enumeration

Each open port is a potential entry point. Different services need different enumeration techniques.

Common Services Checklist

  • SSH (22): Secure Shell — remote command-line access. Try default creds, check for key-based auth, version → CVEs, user enumeration (older versions)
  • HTTP (80/443): Source code, headers, directories, virtual hosts, technology stack, JavaScript files for API keys
  • SMB (445): Server Message Block — Windows file sharing protocol. smbclient -L target lists shares, enum4linux extracts users, groups, and policies. Check for anonymous access and known vulns like EternalBlue
  • DNS (53): Zone transfers (dig axfr @target domain.htb), reverse lookups
  • FTP (21): File Transfer Protocol — file upload/download. Check for anonymous login, version vulns, upload capabilities
  • SNMP (161): Simple Network Management Protocol — used to monitor and manage network devices. Often misconfigured with default "community strings" (like passwords). snmpwalk -v2c -c public target can leak system info, running processes, and network interfaces
  • MySQL (3306): mysql -u root -h target — default/empty passwords
  • Redis (6379): An in-memory database often used for caching. Frequently deployed without authentication! redis-cli -h target → INFO, KEYS *
  • WinRM (5985/5986): Windows Remote Management — lets you run PowerShell commands remotely. evil-winrm -i target -u user -p pass gives you an interactive PowerShell session
  • LDAP (389/636): Lightweight Directory Access Protocol — the protocol Active Directory uses to store and query user/computer information. ldapsearch -x -H ldap://target -b "DC=domain,DC=htb" can enumerate the entire AD structure
  • RDP (3389): Remote Desktop Protocol — gives you a full graphical Windows desktop. xfreerdp /v:target /u:user /p:pass
  • NFS (2049): Network File System — Linux/Unix file sharing. showmount -e target lists shared directories that you may be able to mount and access
🧠 Knowledge Check — Enumeration Tools
What does gobuster dir do?
gobuster dir performs directory/file brute-forcing against a web server. It sends HTTP requests for each entry in your wordlist and reports paths that exist (based on status codes). It's one of the first tools you run after finding an HTTP service — discovering /admin, /.git, /config.php.bak, and other hidden paths.
Which tool is commonly used for SMB enumeration on Linux?
enum4linux (or its modern successor enum4linux-ng) is the go-to SMB enumeration tool. It extracts share listings, user accounts, group memberships, password policies, and OS information from SMB/NetBIOS. Other useful SMB tools: smbclient (connect to shares), crackmapexec (spray credentials, execute commands).

🎯 Passive vs Active Recon — Know the Difference

Before you scan a single port, you need to understand the two fundamental categories of reconnaissance. Getting this wrong can get you fired — or arrested.

╔══════════════════════════════════════════════════════════════════════╗
║               RECONNAISSANCE SPECTRUM                               ║
╠══════════════════════════════════════════════════════════════════════╣
║                                                                      ║
║  PASSIVE RECON                        ACTIVE RECON                   ║
║  (No contact with target)             (Directly touching target)     ║
║                                                                      ║
║  ┌─────────────────────┐              ┌─────────────────────┐        ║
║  │  Google Dorking      │              │  Port Scanning       │        ║
║  │  WHOIS Lookups       │              │  Directory Fuzzing   │        ║
║  │  DNS Records (public)│              │  Banner Grabbing     │        ║
║  │  Cert Transparency   │              │  Vulnerability Scans │        ║
║  │  Wayback Machine     │              │  Spider/Crawling     │        ║
║  │  Shodan/Censys       │              │  Login Brute-Force   │        ║
║  │  Social Media OSINT  │              │  DNS Zone Transfers  │        ║
║  │  Job Postings        │              │  Web App Probing     │        ║
║  │  GitHub Repos        │              │  OS Fingerprinting   │        ║
║  │  Pastebin Leaks      │              │  Fuzzing Parameters  │        ║
║  └─────────────────────┘              └─────────────────────┘        ║
║                                                                      ║
║  Risk: ░░░░░░░░░░ ZERO               Risk: ██████████ HIGH          ║
║  Detection: NONE                      Detection: IDS/IPS/LOGS       ║
║  Legal: Always safe*                  Legal: Need authorization      ║
║                                                                      ║
║  * Unless accessing leaked data       ┌──────────────────────┐       ║
║    that's clearly private             │ WITHOUT AUTHORIZATION │       ║
║                                       │  = ILLEGAL IN MOST   │       ║
║                                       │    JURISDICTIONS      │       ║
║                                       └──────────────────────┘       ║
╚══════════════════════════════════════════════════════════════════════╝

Why Start Passive?

  • Zero detection risk — The target never knows you're looking. No logs, no alerts, no firewall triggers.
  • Legal safety — You're querying public data sources, not the target's infrastructure.
  • Scope refinement — You discover the full attack surface (subdomains, IP ranges, technologies) before you touch anything.
  • Context building — Job postings reveal tech stacks. GitHub repos reveal internal tooling. Social media reveals employee names for phishing.
  • Efficiency — Why scan for a WordPress admin panel when a Google dork finds it in 2 seconds?

💡 Pro Tip: In real-world pentests, you should spend 40-60% of your time on passive recon before you ever touch the target. On CTFs you can skip straight to active — but learn both. Interviews WILL ask about this.

The Passive → Active Pipeline

PASSIVE PHASE                           ACTIVE PHASE
═══════════                             ════════════
                                        
Google Dorks ──→ Find subdomains ──────→ Port scan each subdomain
WHOIS ─────────→ Find IP ranges ───────→ Nmap sweep the range  
Cert Transparency → More subdomains ───→ Virtual host fuzzing
Wayback Machine ──→ Old endpoints ─────→ Test if they still work
Job Postings ─────→ Tech stack guess ──→ Targeted directory fuzzing
GitHub Repos ─────→ Config patterns ───→ Try leaked creds on login
Shodan ───────────→ Open services ─────→ Enumerate those services
LinkedIn ─────────→ Employee names ────→ Email format → phishing

🔍 Google Dorking Deep Dive

Google has already indexed most of the internet for you. Google dorking (also called Google hacking) uses advanced search operators to find things the target probably didn't want publicly accessible. It's passive recon — you're querying Google's index, not the target directly.

Core Search Operators

  • site: — Restrict results to a specific domain. site:example.com shows everything Google indexed on that domain.
  • inurl: — Find pages with specific text in the URL. Great for finding admin panels and login pages.
  • intitle: — Find pages with specific text in the title tag. Catches default install pages and exposed dashboards.
  • filetype: / ext: — Find specific file types. filetype:pdf and ext:pdf do the same thing.
  • intext: — Find pages containing specific text in the body. Useful for finding config files and error messages.
  • cache: — View Google's cached version of a page. Useful if the page has been taken down.
  • link: — Find pages that link to a URL. Helps map relationships between sites. (Deprecated in some cases but still partially works.)
  • - (minus) — Exclude results. site:example.com -www finds subdomains by excluding the main site.
  • " " (quotes) — Exact match. "index of /" finds directory listings.
  • * (wildcard) — Matches any word. site:*.example.com finds subdomains.
  • OR / | — Logical OR. site:example.com (filetype:sql OR filetype:bak)

Practical Google Dork Examples (15+ Real-World Dorks)

# ═══════════════════════════════════════
# FINDING LOGIN PAGES & ADMIN PANELS
# ═══════════════════════════════════════

site:example.com inurl:login
site:example.com inurl:admin
site:example.com intitle:"admin panel"
site:example.com inurl:/wp-admin
site:example.com inurl:"/admin/login" OR inurl:"/admin/signin"

# ═══════════════════════════════════════
# FINDING EXPOSED CONFIGURATION FILES
# ═══════════════════════════════════════

site:example.com filetype:env              # .env files with API keys, DB creds
site:example.com filetype:xml inurl:config  # XML configuration files
site:example.com filetype:yml inurl:docker  # Docker compose files with passwords
site:example.com filetype:ini              # PHP/app configuration files
site:example.com ext:conf                   # Server config files (nginx, apache)

# ═══════════════════════════════════════
# FINDING EXPOSED DATABASES & BACKUPS
# ═══════════════════════════════════════

site:example.com filetype:sql              # SQL database dumps
site:example.com filetype:bak              # Backup files
site:example.com filetype:log              # Log files (may contain creds, errors)
site:example.com ext:sql intext:password   # SQL dumps containing passwords

# ═══════════════════════════════════════
# FINDING DIRECTORY LISTINGS
# ═══════════════════════════════════════

site:example.com intitle:"index of /"      # Open directory listings
site:example.com intitle:"index of" "backup"
site:example.com intitle:"index of" ".git"

# ═══════════════════════════════════════
# FINDING SENSITIVE DOCUMENTS
# ═══════════════════════════════════════

site:example.com filetype:pdf "confidential"
site:example.com filetype:xlsx "password"
site:example.com filetype:doc "internal use only"

# ═══════════════════════════════════════
# SUBDOMAIN DISCOVERY VIA GOOGLE
# ═══════════════════════════════════════

site:*.example.com -www                    # Find subdomains indexed by Google
site:example.com -www -blog -shop          # Exclude known subdomains to find new ones

# ═══════════════════════════════════════
# FINDING VULNERABLE / DEFAULT PAGES
# ═══════════════════════════════════════

site:example.com intitle:"Apache2 Ubuntu Default Page"
site:example.com intitle:"Welcome to nginx"
site:example.com "phpinfo()"               # PHP info pages leak server config
site:example.com intitle:"phpMyAdmin"       # Exposed database admin panel

# ═══════════════════════════════════════
# FINDING EXPOSED GIT/SVN REPOSITORIES
# ═══════════════════════════════════════

site:example.com inurl:".git"
site:example.com inurl:".svn"
site:example.com "/.git/config"

# ═══════════════════════════════════════
# GENERIC VULNERABILITY HUNTING
# ═══════════════════════════════════════

site:example.com inurl:page= OR inurl:id= OR inurl:file=   # Potential injection points
site:example.com inurl:redirect= OR inurl:url= OR inurl:next=  # Open redirect candidates
site:example.com filetype:php inurl:?                         # PHP files with parameters

💡 Pro Tip: The Google Hacking Database (GHDB) on Exploit-DB has thousands of pre-built dorks categorized by what they find. Before a pentest, search GHDB for dorks relevant to the target's technology stack. Also check out dorkSearch.com for a searchable dork interface.

⚠️ Warning: Google will temporarily block you if you send too many dork queries too fast (CAPTCHA or "unusual traffic" page). Use a delay between queries, or use tools like googler or pagodo that handle rate-limiting for you. Also, accessing files you find via dorking may cross ethical/legal lines — just because Google indexed it doesn't mean you're authorized to use it.

📜 Certificate Transparency & Subdomain Discovery

Certificate Transparency (CT) is one of the most underrated passive recon goldmines. Every time an organization gets an SSL/TLS certificate from a Certificate Authority (CA), the certificate is logged in a publicly searchable database. These logs contain the exact domain and subdomain names the certificate covers.

How Certificate Transparency Works

┌───────────────┐     Request cert for       ┌───────────────────┐
│  Organization │ ──────────────────────────→ │ Certificate       │
│  (target.com) │                             │ Authority (CA)    │
└───────────────┘                             └────────┬──────────┘
                                                       │
                                              Issues cert AND logs it
                                                       │
                                                       ▼
                                              ┌───────────────────┐
                                              │ CT Log Servers    │
                                              │ (publicly         │
                                              │  searchable!)     │
                                              └────────┬──────────┘
                                                       │
                                              Anyone can query
                                                       │
                                                       ▼
                                              ┌───────────────────┐
                                              │  YOU (pentester)  │
                                              │  → subdomains!    │
                                              └───────────────────┘

Why CT Logs Are a Goldmine

  • Covers wildcard certs — A cert for *.internal.target.com reveals the existence of that internal subdomain structure.
  • Historical data — Even expired/revoked certs are logged. You can find subdomains that used to exist and may still resolve.
  • No target interaction — You're querying third-party log servers, not the target. Pure passive recon.
  • Catches dev/staging environments — Companies often get certs for staging.target.com, dev.target.com, api-internal.target.com — environments with weaker security.

Querying CT Logs

# ═══════════════════════════════════════
# crt.sh — The most popular CT log search engine
# ═══════════════════════════════════════

# Basic query via browser:
# https://crt.sh/?q=%25.example.com
# The %25 is URL-encoded % — wildcard match

# Via curl (JSON output):
curl -s "https://crt.sh/?q=%25.example.com&output=json" | jq -r '.[].name_value' | sort -u

# Extract unique subdomains, removing wildcards:
curl -s "https://crt.sh/?q=%25.example.com&output=json" \
  | jq -r '.[].name_value' \
  | sed 's/\*\.//g' \
  | sort -u \
  | tee subdomains-ct.txt

# Check which discovered subdomains actually resolve:
cat subdomains-ct.txt | while read sub; do
  ip=$(dig +short "$sub" 2>/dev/null)
  if [ -n "$ip" ]; then
    echo "$sub → $ip"
  fi
done | tee live-subdomains.txt

# ═══════════════════════════════════════
# Other CT Search Engines
# ═══════════════════════════════════════

# CertSpotter API (free tier available):
curl -s "https://api.certspotter.com/v1/issuances?domain=example.com&include_subdomains=true&expand=dns_names" \
  | jq -r '.[].dns_names[]' | sort -u

# censys.io also indexes CT logs (requires API key)
# Google Transparency Report: transparencyreport.google.com/https/certificates

💡 Pro Tip: Combine CT log results with other subdomain sources. Run subfinder -d example.com (which queries CT logs, DNS datasets, search engines, and more in one command), then merge and deduplicate with your manual crt.sh results. The more sources you combine, the more complete your subdomain list.

🌐 DNS Enumeration Deep Dive

DNS is the backbone of the internet — it translates domain names to IP addresses. For pentesters, DNS enumeration reveals subdomains, internal infrastructure, mail servers, and sometimes the entire network topology.

DNS Resolution — How It Works

You type: www.target.com
═══════════════════════════════════════════════════

 Your Browser                DNS Resolver             Root DNS (.com)
 ┌──────────┐               ┌──────────────┐         ┌──────────────┐
 │ Where is │──── Query ───→│ ISP's DNS or │── Q ──→│ "Ask .com    │
 │ www.     │               │ 8.8.8.8      │         │  TLD server" │
 │ target   │               │              │←─ R ───│              │
 │ .com?    │               │              │         └──────────────┘
 └──────────┘               │              │
                            │              │         .com TLD Server
                            │              │         ┌──────────────┐
                            │              │── Q ──→│ "Ask target  │
                            │              │         │  .com's NS"  │
                            │              │←─ R ───│              │
                            │              │         └──────────────┘
                            │              │
                            │              │         target.com NS
                            │              │         ┌──────────────┐
                            │              │── Q ──→│ www = IP     │
                            │              │         │ 93.184.216.34│
      IP: 93.184.216.34     │              │←─ R ───│              │
 ┌──────────┐               │              │         └──────────────┘
 │ Got it!  │←── Answer ───│              │
 └──────────┘               └──────────────┘

DNS Record Types Pentesters Care About

  • A — Maps hostname to IPv4 address. The most common record.
  • AAAA — Maps hostname to IPv6 address. Often forgotten in security configs!
  • CNAME — Alias pointing to another domain. Can reveal cloud infrastructure (e.g., CNAME → s3.amazonaws.com means potential subdomain takeover).
  • MX — Mail servers. Reveals what email infrastructure is used (Google Workspace, Exchange, etc.) and gives you targets for email attacks.
  • NS — Nameservers. Which DNS servers are authoritative for this domain. Target these for zone transfers.
  • TXT — Text records. Often contain SPF (email auth), DKIM, domain verification strings, and sometimes internal information.
  • SOA — Start of Authority. Contains admin email, serial number, and DNS zone configuration details.
  • SRV — Service records. Can reveal internal services like _ldap._tcp.target.com (Active Directory), _sip._tcp (VoIP), etc.
  • PTR — Reverse DNS. Maps IP → hostname. Useful for discovering what domains share an IP or finding internal hostnames.

DNS Enumeration Commands

# ═══════════════════════════════════════
# BASIC DNS QUERIES
# ═══════════════════════════════════════

# Query all record types
dig target.com ANY +noall +answer

# Individual record types
dig target.com A           # IPv4 addresses
dig target.com AAAA        # IPv6 addresses
dig target.com MX          # Mail servers
dig target.com NS          # Name servers
dig target.com TXT         # TXT records (SPF, DKIM, verification tokens)
dig target.com SOA         # Start of Authority
dig target.com CNAME       # Aliases
dig target.com SRV         # Service records

# Reverse DNS lookup (IP → hostname)
dig -x 93.184.216.34

# Short output (just the answer)
dig +short target.com A

# ═══════════════════════════════════════
# ZONE TRANSFERS (AXFR) — The Jackpot
# ═══════════════════════════════════════
# A zone transfer returns EVERY record in the DNS zone.
# Misconfigured DNS servers allow this to anyone.
# Always try it — it's low-risk and high-reward.

# First, find the nameservers
dig target.com NS +short
# → ns1.target.com
# → ns2.target.com

# Attempt zone transfer against each nameserver
dig axfr target.com @ns1.target.com
dig axfr target.com @ns2.target.com

# If it works, you get EVERYTHING:
# ─ Every subdomain
# ─ Every IP mapping
# ─ Internal hostnames
# ─ Mail servers
# ─ Service records

# ═══════════════════════════════════════
# AUTOMATED DNS ENUMERATION TOOLS
# ═══════════════════════════════════════

# subfinder — Fast passive subdomain enumeration
# Uses CT logs, search engines, DNS datasets, APIs
subfinder -d target.com -o subdomains.txt

# amass — The most comprehensive subdomain tool
# Combines passive + active techniques
amass enum -passive -d target.com -o amass-passive.txt    # Passive only
amass enum -active -d target.com -o amass-active.txt      # Active + passive
amass enum -brute -d target.com -o amass-brute.txt        # With DNS brute-forcing

# gobuster dns — DNS subdomain brute-forcing
# Tries every word in wordlist as a subdomain prefix
gobuster dns -d target.com -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt -t 50

# dnsenum — All-in-one DNS enumeration
# Zone transfers, brute force, Google scraping, reverse lookups
dnsenum target.com

# fierce — Lightweight DNS recon
fierce --domain target.com

# dnsrecon — Another comprehensive DNS tool
dnsrecon -d target.com -t std       # Standard enumeration
dnsrecon -d target.com -t axfr      # Zone transfer attempt
dnsrecon -d target.com -t brt       # Brute force subdomains

# ═══════════════════════════════════════
# COMBINING RESULTS
# ═══════════════════════════════════════

# Merge all subdomain sources and deduplicate
cat subdomains-ct.txt amass-passive.txt gobuster-dns.txt | sort -u > all-subdomains.txt

# Check which are alive (resolve to an IP)
cat all-subdomains.txt | httpx -silent -o live-hosts.txt

# Or with a simple loop:
while read sub; do
  host "$sub" 2>/dev/null | grep "has address" && echo "$sub is live"
done < all-subdomains.txt

💡 Pro Tip: On HTB and CTF boxes, DNS is often a goldmine. If the box has port 53 open, ALWAYS try a zone transfer first: dig axfr domain.htb @target-ip. It's shocking how often this works on CTF machines. In real-world pentests, it's less common but still worth trying every time — the payoff is enormous when it works.

💡 Pro Tip: Don't forget about SRV records! Run dig _ldap._tcp.target.com SRV and dig _kerberos._tcp.target.com SRV to detect Active Directory. This is pure passive recon and can reveal the entire AD structure before you even scan a port.

🕵️ Web Technology Fingerprinting

Knowing what technology stack a target runs lets you focus your attacks. A WordPress site gets WordPress exploits. A Django site gets Django-specific tests. A site running an outdated jQuery gets known CVEs. Fingerprinting is how you go from "there's a web server" to "there's a WordPress 5.8 with WooCommerce 5.5 on PHP 7.4 behind Cloudflare on nginx."

Fingerprinting Tools

  • WhatWeb — Command-line tool. Fast, detailed, scriptable. Detects CMS, frameworks, server software, analytics, and more.
  • Wappalyzer — Browser extension that identifies technologies as you browse. Great for passive recon during manual browsing.
  • BuiltWith — Web service (builtwith.com) that shows technology profiles. More business-intelligence focused but useful for recon.
  • httpx — Probes web servers for tech stack, status codes, titles, and more. Great for bulk scanning many hosts.
# ═══════════════════════════════════════
# WHATWEB — Detailed technology fingerprinting
# ═══════════════════════════════════════

whatweb http://target.com
# Output example:
# http://target.com [200 OK] Apache[2.4.41], Bootstrap, Country[US],
# HTML5, HTTPServer[Ubuntu Linux][Apache/2.4.41 (Ubuntu)], 
# JQuery[3.4.1], PHP[7.4.3], Script, Title[My Site], WordPress[5.8]

# Aggressive mode (more requests, more info)
whatweb -a 3 http://target.com

# Scan multiple targets
whatweb --input-file=live-hosts.txt --log-brief=tech-stack.txt

# ═══════════════════════════════════════
# HTTP RESPONSE HEADER ANALYSIS
# ═══════════════════════════════════════

# Headers leak tons of information:
curl -sI http://target.com

# What to look for:
# Server: nginx/1.14.2            → Web server + version
# X-Powered-By: PHP/7.4.3         → Backend language + version  
# X-AspNet-Version: 4.0.30319     → .NET version
# X-Generator: WordPress 5.8      → CMS identification
# Set-Cookie: JSESSIONID=...      → Java application (Tomcat, Spring, etc.)
# Set-Cookie: PHPSESSID=...       → PHP application
# Set-Cookie: csrftoken=...       → Django (Python)
# Set-Cookie: _rails_session=...  → Ruby on Rails
# Set-Cookie: connect.sid=...     → Express.js (Node.js)
# X-Drupal-Cache:                 → Drupal CMS
# X-Varnish:                      → Varnish cache in front
# CF-RAY:                         → Behind Cloudflare CDN

# ═══════════════════════════════════════
# FRAMEWORK IDENTIFICATION PATTERNS
# ═══════════════════════════════════════

# WordPress: /wp-content/, /wp-includes/, /wp-admin/
# Django: /admin/ with Django styling, csrfmiddlewaretoken in forms
# Rails: _rails_session cookie, /assets/ directory structure
# Spring Boot: /actuator/ endpoints, Whitelabel Error Page
# Laravel: laravel_session cookie, /storage/ paths
# ASP.NET: .aspx extensions, __VIEWSTATE in forms, .axd handlers
# Next.js: /_next/ directory, __NEXT_DATA__ in page source
# React: "root" div, .js bundles with React patterns
# Angular: ng-* attributes, angular.min.js references

# Check for framework-specific paths
curl -s http://target.com/wp-login.php      # WordPress
curl -s http://target.com/administrator/    # Joomla
curl -s http://target.com/user/login        # Drupal
curl -s http://target.com/actuator/health   # Spring Boot (often leaks info!)
curl -s http://target.com/elmah.axd         # ASP.NET error log

💡 Pro Tip: The X-Powered-By and Server headers are the first things to check — but smart admins remove them. When headers are stripped, look at default error pages (404, 500), cookie names, URL patterns, and HTML comments. Every framework has a unique fingerprint if you know where to look. The default "not found" page alone can identify nginx vs Apache vs IIS vs Express.

⏪ Wayback Machine & Web Archives

The Wayback Machine (web.archive.org) has been archiving the internet since 1996. For pentesters, this is a time machine that reveals what the target's infrastructure used to look like — including pages that have been removed, endpoints that were "hidden", and configuration files that were briefly exposed.

Why Web Archives Matter for Recon

  • Removed pages still work — Developers remove the link but not the endpoint. The Wayback Machine shows you what used to be there, and often it's still accessible.
  • Old JavaScript files leak API endpoints — Archived JS bundles contain hardcoded API URLs, keys, and internal endpoint paths. These APIs may still be active even if the frontend no longer references them.
  • Configuration exposure — Maybe config.php was briefly exposed in 2019 before they fixed it. The archive captured it.
  • Technology evolution — See how the tech stack changed over time. Old versions may have known vulnerabilities.
  • robots.txt history — See what directories they tried to hide at different points in time. Directories they blocked probably had something interesting.
# ═══════════════════════════════════════
# BROWSING THE WAYBACK MACHINE
# ═══════════════════════════════════════

# View archived snapshots of a site:
# https://web.archive.org/web/*/example.com

# View a specific snapshot:
# https://web.archive.org/web/20200101000000*/example.com

# Check archived robots.txt (gold!):
# https://web.archive.org/web/2020/http://example.com/robots.txt

# ═══════════════════════════════════════
# waybackurls — Extract ALL archived URLs for a domain
# ═══════════════════════════════════════

# Install: go install github.com/tomnomnom/waybackurls@latest

# Get all URLs ever archived for a domain
echo "example.com" | waybackurls | tee wayback-urls.txt

# Filter for interesting files
cat wayback-urls.txt | grep -E "\.(js|json|xml|config|env|bak|sql|zip|tar)" | sort -u

# Find API endpoints in archived URLs
cat wayback-urls.txt | grep -iE "(api|v1|v2|graphql|rest|endpoint)" | sort -u

# Find URLs with parameters (potential injection points)
cat wayback-urls.txt | grep "?" | sort -u > parameterized-urls.txt

# ═══════════════════════════════════════
# EXTRACTING SECRETS FROM ARCHIVED JS FILES
# ═══════════════════════════════════════

# Step 1: Get all archived JS file URLs
cat wayback-urls.txt | grep "\.js$" | sort -u > js-files.txt

# Step 2: Download and search for secrets
while read url; do
  curl -s "$url" | grep -iE "(api_key|apikey|secret|password|token|endpoint|internal)" >> js-secrets.txt
done < js-files.txt

# Or use gau (Get All URLs) which combines Wayback, Common Crawl, and more:
# Install: go install github.com/lc/gau/v2/cmd/gau@latest
gau example.com | tee all-urls.txt

# ═══════════════════════════════════════
# PRACTICAL WORKFLOW
# ═══════════════════════════════════════

# 1. Grab all historical URLs
echo "target.com" | waybackurls > urls.txt

# 2. Find interesting paths
cat urls.txt | grep -v -E "\.(css|png|jpg|gif|svg|woff)" | sort -u > interesting.txt

# 3. Check if old endpoints still respond
cat interesting.txt | httpx -silent -status-code | grep "200\|301\|302\|403"

# 4. Download archived JavaScript for analysis
mkdir -p js-archive
cat urls.txt | grep "\.js$" | sort -u | head -50 | while read url; do
  filename=$(echo "$url" | md5sum | cut -d' ' -f1)
  curl -sL "https://web.archive.org/web/2023/$url" -o "js-archive/$filename.js"
done

# 5. Search JS files for secrets
grep -rn "api\|key\|secret\|password\|token\|auth\|endpoint" js-archive/

💡 Pro Tip: Always check the Wayback Machine's archived robots.txt for your target. Companies frequently add directories to robots.txt when they realize something sensitive is exposed — then later remove the entry thinking it's "fixed." The archive remembers everything. Those directories are worth investigating.

🎯 Port Scanning Strategy — Beyond the Basics

You already know the basic Nmap methodology from above. This section goes deeper — when to use which scan type, how to handle large networks, and how to avoid common mistakes.

Nmap Scan Types — When to Use Each

  • -sS (SYN scan) — The default and best general-purpose scan. Sends SYN, reads response, sends RST — never completes the handshake. Fast, relatively stealthy, requires root. Use this 90% of the time.
  • -sT (Connect scan) — Full TCP handshake. No root needed. Louder (full connection logged by target). Use when you don't have root or SYN scan gives unreliable results.
  • -sU (UDP scan) — Scans UDP ports. VERY slow (no handshake → has to wait for timeouts). Always run in background. Reveals DNS, SNMP, TFTP, NTP, DHCP.
  • -sV (Version detection) — Probes open ports to determine service/version. Essential but adds time. Only run on confirmed open ports.
  • -sC (Default scripts) — Runs Nmap's default NSE scripts. Safe and informative. Equivalent to --script=default.
  • -sN/-sF/-sX (NULL/FIN/Xmas) — Stealth scans that set unusual TCP flags. Can bypass some firewalls that only filter SYN packets. Unreliable on Windows targets.
  • -sA (ACK scan) — Doesn't find open ports — maps firewall rules. Tells you which ports are filtered vs unfiltered.
  • -sW (Window scan) — Like ACK scan but examines TCP window size. Can sometimes differentiate open from closed on certain systems.
  • -sI (Idle/zombie scan) — Ultimate stealth. Uses a third-party "zombie" host to scan without your IP appearing. Complex to set up but incredibly stealthy.

Scanning Strategy by Scenario

# ═══════════════════════════════════════
# CTF / SINGLE TARGET — Go fast and thorough
# ═══════════════════════════════════════

# Quick discovery (30 seconds)
nmap -p- --min-rate 5000 -T4 target -oN fast.txt

# Deep scan on what you found (2-3 minutes)
nmap -p 22,80,443,8080 -sC -sV -oA detailed target

# Targeted vuln scripts (1-2 minutes per service)
nmap -p 80 --script=http-vuln* target
nmap -p 445 --script=smb-vuln* target

# UDP quick check (background)
sudo nmap -sU --top-ports 20 target -oN udp.txt &

# ═══════════════════════════════════════
# LARGE NETWORK (e.g., 10.10.10.0/24) — Be strategic
# ═══════════════════════════════════════

# Step 1: Host discovery — who's alive? (30 seconds)
nmap -sn 10.10.10.0/24 -oG alive-hosts.txt
grep "Up" alive-hosts.txt | cut -d' ' -f2 > live-ips.txt

# Step 2: Quick port scan on live hosts (top 100 ports)
nmap -iL live-ips.txt --top-ports 100 --min-rate 3000 -oG quick-scan.txt

# Step 3: Full port scan on interesting hosts only
nmap -p- --min-rate 5000 -iL interesting-ips.txt -oN full-ports.txt

# Step 4: Deep scan on discovered services
nmap -p PORTS -sC -sV -iL interesting-ips.txt -oA deep-scan

# ═══════════════════════════════════════
# STEALTH / EVASION — When detection matters
# ═══════════════════════════════════════

# Slow and stealthy SYN scan
nmap -sS -T2 --max-rate 100 -f target          # -f fragments packets
nmap -sS -T1 --scan-delay 5s target             # 5 second delay between probes
nmap -sS -D RND:10 target                        # Decoy scan (spoofs 10 random IPs)
nmap -sS --source-port 53 target                  # Spoof source port (53 = DNS, often allowed)
nmap -sS -e eth0 --spoof-mac 0 target            # Random MAC address

Common Nmap Mistakes & False Positives

  • Not scanning all 65535 ports — The default scans only ~1000 common ports. Many CTF challenges hide services on high ports (e.g., 50000, 65535). Always do -p- first.
  • "Filtered" doesn't mean "nothing there" — Filtered means a firewall is blocking. The service may exist behind the firewall. Try different scan types or source ports.
  • UDP false positives — UDP open|filtered means Nmap got no response. The port might be open OR filtered. Verify with service-specific tools.
  • Rate limiting causing misses — Scanning too fast can cause dropped packets → missed open ports. If you suspect this, rescan slower: nmap -p- -T3 target.
  • Forgetting output files — Always use -oA basename (saves in all formats: .nmap, .xml, .gnmap). You WILL want to reference results later.
  • Running -sV -sC on all ports immediately — Version detection is slow. Find open ports first (-p-), THEN run -sV -sC only on those ports.

💡 Pro Tip: Save Nmap output in all formats with -oA. The XML output (-oX) can be imported into Metasploit (db_import), parsed with xmlstarlet, or converted to HTML reports with xsltproc. The grepable format (-oG) is great for quick scripting: grep "80/open" scan.gnmap | cut -d' ' -f2 gives you all hosts with port 80 open.

📂 Directory & File Brute-Forcing — The Right Tool for the Job

You've already seen gobuster, ffuf, and feroxbuster in the basics section. Now let's go deeper into when to use each, advanced techniques, and wordlist strategy.

Tool Comparison — gobuster vs ffuf vs feroxbuster

  • gobuster
    • Written in Go. Fast and simple.
    • Modes: dir (directories), dns (subdomains), vhost (virtual hosts), fuzz (generic fuzzing).
    • No recursive scanning (scans only the depth you specify).
    • Best for: Quick, straightforward directory brute-forcing. Beginners.
  • ffuf (Fuzz Faster U Fool)
    • Written in Go. Extremely flexible fuzzer.
    • Can fuzz anything: URLs, headers, POST data, cookies.
    • Supports multiple wordlists with multiple FUZZ keywords.
    • Powerful filtering: by status code, response size, word count, line count, regex.
    • Best for: Advanced fuzzing, parameter discovery, virtual host enumeration, any non-standard fuzzing.
  • feroxbuster
    • Written in Rust. Fast with built-in recursion.
    • Automatically recurses into discovered directories.
    • Auto-extracts links from responses.
    • Resume-able scans (save state, continue later).
    • Best for: Deep recursive scanning. "Set it and forget it." Finding deeply nested content.
# ═══════════════════════════════════════
# GOBUSTER — Quick and simple
# ═══════════════════════════════════════

# Basic directory scan
gobuster dir -u http://target -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt -t 50

# With extensions (critical for PHP/ASP sites!)
gobuster dir -u http://target -w wordlist.txt -x php,html,txt,bak,old,conf -t 50

# Hide specific status codes
gobuster dir -u http://target -w wordlist.txt -b 404,403

# Follow redirects
gobuster dir -u http://target -w wordlist.txt -r

# Add custom headers (e.g., authentication)
gobuster dir -u http://target -w wordlist.txt -H "Authorization: Bearer TOKEN"

# ═══════════════════════════════════════
# FFUF — The Swiss Army Knife
# ═══════════════════════════════════════

# Basic directory fuzzing
ffuf -u http://target/FUZZ -w /usr/share/seclists/Discovery/Web-Content/common.txt

# Filter by status code (exclude 404)
ffuf -u http://target/FUZZ -w wordlist.txt -fc 404

# Filter by response size (remove noise)
# First, see what the default 404 size is, then filter it out
ffuf -u http://target/FUZZ -w wordlist.txt -fs 1234

# Filter by word count
ffuf -u http://target/FUZZ -w wordlist.txt -fw 42

# Match only specific status codes
ffuf -u http://target/FUZZ -w wordlist.txt -mc 200,301,302,403

# Extension fuzzing
ffuf -u http://target/indexFUZZ -w /usr/share/seclists/Discovery/Web-Content/web-extensions.txt

# POST parameter fuzzing
ffuf -u http://target/login -X POST -d "username=admin&password=FUZZ" \
  -w /usr/share/seclists/Passwords/Common-Credentials/10k-most-common.txt -fc 401

# Header fuzzing (virtual hosts — more on this below)
ffuf -u http://target -H "Host: FUZZ.target.htb" \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt -fs 1234

# Two wordlists simultaneously (e.g., username + password)
ffuf -u http://target/login -X POST -d "user=W1&pass=W2" \
  -w users.txt:W1 -w passwords.txt:W2 -fc 401

# ═══════════════════════════════════════
# FEROXBUSTER — Recursive scanning beast
# ═══════════════════════════════════════

# Basic recursive scan (auto-recurses into found directories)
feroxbuster -u http://target -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt

# Control recursion depth
feroxbuster -u http://target -w wordlist.txt --depth 3

# With extensions
feroxbuster -u http://target -w wordlist.txt -x php,html,txt,bak

# Auto-extract links from pages (spider-like behavior)
feroxbuster -u http://target -w wordlist.txt --extract-links

# Save scan state (resume if interrupted)
feroxbuster -u http://target -w wordlist.txt --resume-from ferox-state.json

# Filter by status code
feroxbuster -u http://target -w wordlist.txt -C 404,403

SecLists Directory Structure — Know Your Wordlists

# SecLists is THE standard wordlist collection for pentesting
# Install: apt install seclists (or clone from GitHub)
# Location: /usr/share/seclists/

# Key directories:
#
# Discovery/Web-Content/
#   ├── common.txt              (~4.6K) — Quick first pass
#   ├── raft-medium-directories.txt  (~30K) — Standard scan
#   ├── raft-large-directories.txt   (~120K) — Thorough scan  
#   ├── directory-list-2.3-big.txt   (~1.2M) — Desperation mode
#   ├── raft-medium-files.txt         — File names (not dirs)
#   ├── web-extensions.txt            — Common file extensions
#   ├── apache.txt                    — Apache-specific paths
#   ├── tomcat.txt                    — Tomcat-specific paths
#   ├── IIS.fuzz.txt                  — IIS-specific paths
#   └── api/
#       └── api-endpoints.txt         — Common API paths
#
# Discovery/DNS/
#   ├── subdomains-top1million-5000.txt   — Quick subdomain scan
#   ├── subdomains-top1million-20000.txt  — Standard subdomain scan
#   └── subdomains-top1million-110000.txt — Full subdomain scan
#
# Passwords/
#   ├── Common-Credentials/
#   │   ├── 10k-most-common.txt
#   │   └── best1050.txt
#   ├── Leaked-Databases/
#   └── Default-Credentials/

💡 Pro Tip: Don't just use one wordlist. Run a quick scan first with common.txt, then a medium scan with raft-medium. If the site is PHP, add -x php,php.bak,php.old. If it's ASP.NET, add -x asp,aspx,ashx,asmx. Always try both file and directory wordlists. And remember — feroxbuster's auto-recursion can find paths that non-recursive tools miss (e.g., /secret/admin/upload.php requires finding /secret/ first, then recursing into it).

🏠 Virtual Host Discovery

Virtual hosting is how one web server hosts multiple websites on a single IP address. The server looks at the Host: header in HTTP requests to decide which site to serve. This means a target might have admin.target.htb, dev.target.htb, and api.target.htb all running on the same IP — but you'd never find them with a regular directory scan.

How Virtual Hosting Works

Single IP: 10.10.10.50

     Request: Host: www.target.htb         Request: Host: admin.target.htb
     ┌──────────────────────────┐          ┌──────────────────────────┐
     │ GET / HTTP/1.1           │          │ GET / HTTP/1.1           │
     │ Host: www.target.htb     │          │ Host: admin.target.htb   │
     └────────────┬─────────────┘          └────────────┬─────────────┘
                  │                                     │
                  ▼                                     ▼
     ┌──────────────────────────────────────────────────────────────┐
     │                    Web Server (10.10.10.50)                   │
     │                                                              │
     │  Host: www.target.htb    → /var/www/main-site/               │
     │  Host: admin.target.htb  → /var/www/admin-panel/             │
     │  Host: dev.target.htb    → /var/www/dev-app/                 │
     │  Host: api.target.htb    → /var/www/api-backend/             │
     │  (default)               → /var/www/html/                    │
     └──────────────────────────────────────────────────────────────┘

     Different content served depending on the Host header!
     Standard port scanning only shows the DEFAULT vhost.

Why This Matters for CTFs and Pentesting

  • Hidden attack surface — The default site might be a static page with nothing interesting. The admin vhost has the real application.
  • HTB machines love vhosts — A huge number of HackTheBox machines require vhost discovery to progress. It's often the "a-ha!" moment that unlocks the box.
  • Different security levels — Dev/staging vhosts often have weaker authentication, debug mode enabled, or exposed admin panels.
  • Separate applications — Each vhost could be an entirely different application with its own tech stack and vulnerabilities.

Vhost Enumeration Techniques

# ═══════════════════════════════════════
# STEP 1: Add the known domain to /etc/hosts
# ═══════════════════════════════════════
echo "10.10.10.50  target.htb" | sudo tee -a /etc/hosts

# ═══════════════════════════════════════
# GOBUSTER VHOST MODE
# ═══════════════════════════════════════

gobuster vhost -u http://target.htb \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  --append-domain \
  -t 50

# --append-domain automatically appends .target.htb to each wordlist entry
# Without it, you'd need a wordlist with full hostnames

# ═══════════════════════════════════════
# FFUF — More flexible, better filtering
# ═══════════════════════════════════════

# Step 1: Make a request to see the default response size
curl -s http://target.htb | wc -c
# → 11234 bytes (this is the default vhost response)

# Step 2: Fuzz with Host header, filtering out the default response
ffuf -u http://target.htb -H "Host: FUZZ.target.htb" \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  -fs 11234

# -fs 11234 filters out responses of 11234 bytes (the default page)
# Anything with a DIFFERENT size is likely a valid vhost!

# You can also filter by word count or line count:
ffuf -u http://target.htb -H "Host: FUZZ.target.htb" \
  -w subdomains.txt -fw 42   # filter by word count

# For HTTPS targets:
ffuf -u https://target.htb -H "Host: FUZZ.target.htb" \
  -w subdomains.txt -fs 11234

# ═══════════════════════════════════════
# WHEN YOU FIND A VHOST
# ═══════════════════════════════════════

# Add it to /etc/hosts
echo "10.10.10.50  admin.target.htb dev.target.htb" | sudo tee -a /etc/hosts

# Then enumerate it like any other web target:
whatweb http://admin.target.htb
gobuster dir -u http://admin.target.htb -w wordlist.txt -x php,html -t 50
curl -sI http://admin.target.htb

💡 Pro Tip: When a gobuster vhost scan finds nothing, the issue is often filtering. Use ffuf instead — make a baseline request first, note the response size/word count of the default page, then use -fs or -fw to filter it out. Any response with a different size is potentially a valid vhost. Also try both HTTP and HTTPS — some vhosts only respond on 443.

💡 Pro Tip: On HTB, if the box description or webpage mentions a domain name (like target.htb), ALWAYS add it to /etc/hosts and immediately try vhost enumeration. Many boxes gate their real content behind vhosts. If you're stuck on a box, vhost discovery is one of the first things to retry with a bigger wordlist.

🛡️ Vulnerability Scanning

After you've discovered services, directories, and vhosts, vulnerability scanners can automatically check for known issues. But they're a tool, not a crutch — understanding what they find (and what they miss) is critical.

When to Use Automated Scanners vs Manual Testing

  • Use scanners for: Known CVEs, default configurations, missing patches, common misconfigurations, SSL/TLS weaknesses, banner-based vuln detection.
  • Use manual testing for: Business logic flaws, complex injection chains, authentication bypass, IDOR, access control issues, anything context-dependent.
  • Rule of thumb: Scanners find the "known knowns." Manual testing finds the "unknown unknowns." The best pentesters use both.

Nmap NSE Scripts — Targeted Vulnerability Scanning

# ═══════════════════════════════════════
# NMAP VULNERABILITY SCRIPTS
# ═══════════════════════════════════════

# Run ALL vuln scripts against a target
nmap --script vuln target

# Target specific services
nmap -p 443 --script ssl-heartbleed target             # Heartbleed (CVE-2014-0160)
nmap -p 445 --script smb-vuln-ms17-010 target          # EternalBlue
nmap -p 445 --script smb-vuln-ms08-067 target          # Conficker
nmap -p 80 --script http-shellshock --script-args uri=/cgi-bin/test.cgi target  # Shellshock
nmap -p 443 --script ssl-poodle target                 # POODLE
nmap -p 443 --script ssl-enum-ciphers target           # Weak cipher detection

# HTTP enumeration scripts
nmap -p 80 --script http-enum target                   # Interesting directories
nmap -p 80 --script http-methods target                # Allowed HTTP methods (PUT, DELETE?)
nmap -p 80 --script http-php-version target            # PHP version detection
nmap -p 80 --script http-wordpress-enum target         # WordPress enumeration
nmap -p 80 --script http-sql-injection target          # Basic SQLi detection

# List all available scripts
ls /usr/share/nmap/scripts/ | grep "http-"
ls /usr/share/nmap/scripts/ | grep "smb-"

# Search for scripts by keyword
nmap --script-help "*vuln*"

Nikto — Web Server Scanner

# ═══════════════════════════════════════
# NIKTO — Classic web vulnerability scanner
# ═══════════════════════════════════════

# Basic scan
nikto -h http://target

# Scan specific port
nikto -h http://target -p 8080

# Scan with authentication
nikto -h http://target -id admin:password

# Scan HTTPS
nikto -h https://target -ssl

# Save output
nikto -h http://target -o nikto-results.txt -Format txt

# What Nikto checks:
# ─ Outdated server software
# ─ Dangerous files/programs (e.g., /phpinfo.php, /test.php)
# ─ Server configuration issues
# ─ Default files and scripts
# ─ Known vulnerable software versions
# ─ Missing security headers

Nuclei — Modern Template-Based Scanner

# ═══════════════════════════════════════
# NUCLEI — Fast, template-based vuln scanner
# ═══════════════════════════════════════

# Update templates (do this first!)
nuclei -update-templates

# Basic scan against a single target
nuclei -u http://target

# Scan a list of targets
nuclei -l live-hosts.txt

# Filter by severity
nuclei -u http://target -severity critical,high

# Filter by template type
nuclei -u http://target -tags cve             # Known CVEs only
nuclei -u http://target -tags tech            # Technology detection
nuclei -u http://target -tags exposure        # Exposed panels/files
nuclei -u http://target -tags misconfig       # Misconfigurations

# Specific template
nuclei -u http://target -t cves/2021/CVE-2021-44228.yaml  # Log4Shell

# Output results
nuclei -u http://target -o nuclei-results.txt -json

# Why nuclei is great:
# ─ 6000+ community templates and growing
# ─ Much faster than Nikto
# ─ Easy to write custom templates (YAML)
# ─ Actively maintained with new CVE templates within days of disclosure
# ─ Can scan for everything: CVEs, misconfigs, exposures, tech detection

Handling False Positives

  • Always verify manually — Scanners report potential issues. Open the URL/service yourself and confirm the finding.
  • Check the version — "Vulnerable to X" often means "running a version that COULD be vulnerable." The actual vuln may be patched without updating the version string.
  • Read the scanner's reasoning — Nuclei shows exactly what it matched and why. Nmap shows the script output. Read it — don't just trust the headline.
  • Cross-validate — If Nikto says a vuln exists, verify with Nmap scripts or nuclei. Multiple confirmations = real finding.
  • Context matters — "PUT method allowed" is a finding on a static site, but normal on a REST API. Understand the target before flagging.

💡 Pro Tip: Run nuclei with -severity critical,high first for quick wins, then expand to medium/low. In CTF scenarios, nuclei's -tags cve filter is particularly useful — many CTF boxes are intentionally running old, vulnerable software. For real-world pentests, always include -tags exposure to find leaked config files, admin panels, and debug pages.

🌍 OSINT Framework Overview

OSINT (Open Source Intelligence) tools automate the collection of publicly available information about a target — email addresses, employee names, domains, IP ranges, exposed services, and more. Here's when to use each major tool.

OSINT Tools — What Each Does and When to Use It

  • theHarvester
    • What: Gathers emails, names, subdomains, IPs, and URLs from public sources (search engines, PGP servers, Shodan, etc.)
    • When: First step in passive recon. Quick overview of the target's public footprint.
    • Best for: Email harvesting for phishing campaigns. Subdomain discovery.
  • Maltego
    • What: Visual link-analysis tool. Maps relationships between people, domains, IPs, companies, and more using "transforms."
    • When: Complex investigations where you need to visualize connections (e.g., "which companies share DNS infrastructure with the target?").
    • Best for: Mapping organizational structure. Finding infrastructure relationships. Visual analysis.
  • Recon-ng
    • What: Modular reconnaissance framework (like Metasploit for OSINT). Has modules for dozens of data sources.
    • When: Structured recon projects where you need to query multiple sources systematically.
    • Best for: Combining multiple OSINT sources in one workflow. Exporting structured data.
  • Shodan
    • What: "Search engine for internet-connected devices." Indexes banners, open ports, services, and vulnerabilities across the entire internet.
    • When: Finding what services a target exposes to the internet. Discovering IoT devices, SCADA systems, databases.
    • Best for: Discovering exposed services without scanning (passive!). Finding specific vulnerable software versions.
  • Censys
    • What: Similar to Shodan but with stronger focus on certificates and TLS data. Better search syntax for cert-based queries.
    • When: Certificate-focused recon. Finding all IPs associated with a target's SSL certs.
    • Best for: Discovering the real IP behind Cloudflare/CDN. Finding all services using a specific certificate.
# ═══════════════════════════════════════
# theHarvester — Quick OSINT sweep
# ═══════════════════════════════════════

# Search multiple sources for a domain
theHarvester -d example.com -b all

# Specific sources
theHarvester -d example.com -b google,bing,linkedin,crtsh

# Save to file
theHarvester -d example.com -b all -f results.html

# ═══════════════════════════════════════
# Recon-ng — Modular OSINT framework
# ═══════════════════════════════════════

recon-ng
# Inside recon-ng:
# [recon-ng][default] > marketplace search
# [recon-ng][default] > marketplace install recon/domains-hosts/hackertarget
# [recon-ng][default] > modules load recon/domains-hosts/hackertarget
# [recon-ng][default][hackertarget] > options set SOURCE example.com
# [recon-ng][default][hackertarget] > run
# [recon-ng][default] > show hosts

# ═══════════════════════════════════════
# SHODAN — Internet-Wide Service Discovery
# ═══════════════════════════════════════

# Search via browser: https://www.shodan.io

# CLI (requires API key):
shodan search "hostname:example.com"
shodan host 93.184.216.34

# ═══════════════════════════════════════
# SHODAN DORK EXAMPLES (use on shodan.io)
# ═══════════════════════════════════════

# Find all services for a specific organization
org:"Example Corp"

# Find specific software versions
product:"Apache" version:"2.4.49"           # Known path traversal CVE
product:"Microsoft IIS" version:"6.0"       # Ancient and vulnerable
product:"OpenSSH" version:"7.2"              # Old SSH version

# Find exposed databases
product:"MongoDB" port:27017 -authentication  # Unauthenticated MongoDB
product:"Elasticsearch" port:9200             # Exposed Elasticsearch
product:"Redis" port:6379                     # Open Redis

# Find exposed webcams
title:"webcam" has_screenshot:true

# Find specific countries/cities
country:"US" city:"San Francisco" product:"nginx"

# Find exposed admin panels
http.title:"Dashboard" http.status:200
http.title:"phpMyAdmin"
http.title:"Kibana"

# Find vulnerable services
vuln:"CVE-2021-44228"                         # Log4Shell
vuln:"CVE-2021-34473"                         # ProxyShell

# Combine dorks
org:"Example Corp" port:22 product:"OpenSSH"
hostname:"example.com" has_screenshot:true

# ═══════════════════════════════════════
# ADDITIONAL OSINT SOURCES
# ═══════════════════════════════════════

# GitHub — Search for leaked secrets
# https://github.com/search?q=example.com+password&type=code
# https://github.com/search?q=org%3Aexamplecorp+api_key&type=code

# LinkedIn — Employee names → email format
# Search: "Example Corp" employees on LinkedIn
# Combine with email format from Hunter.io or email-format.com

# Hunter.io — Find email patterns
# https://hunter.io/domain-search — shows email format (e.g., [email protected])

# SecurityTrails — Historical DNS data
# https://securitytrails.com — shows DNS history, subdomains, associated domains

# BuiltWith — Technology profiling
# https://builtwith.com/example.com — detailed tech stack analysis

# Dehashed / Have I Been Pwned — Leaked credentials
# Check if target's domain appears in data breaches

💡 Pro Tip: Shodan has already scanned the entire internet for you. Before you run a single Nmap command, search Shodan for your target's IP or organization name. You'll see every open port, service version, and even screenshots — all without sending a single packet to the target. This is pure passive recon. Free accounts get limited results; the membership ($49 one-time) is worth it for serious pentesting.

💡 Pro Tip: For bug bounty programs, always check GitHub for the target organization's repositories. Search for org:targetcorp password, org:targetcorp api_key, org:targetcorp secret. Developers accidentally commit secrets to public repos constantly. Tools like truffleHog and gitleaks automate this search across entire repositories and commit histories.

🧠 Knowledge Check — Advanced Recon Techniques
You want to find all subdomains of a target using Certificate Transparency logs. Which tool/site queries CT logs?
crt.sh is the most popular CT log search engine. Query it with curl -s "https://crt.sh/?q=%25.target.com&output=json" | jq -r '.[].name_value' | sort -u to extract all subdomains that have ever had an SSL certificate issued. This is passive recon — you never touch the target.
What is virtual host enumeration trying to discover?
Virtual host enumeration discovers different websites sharing the same IP address. The web server uses the Host: header to route requests to different sites. Tools like gobuster vhost and ffuf with Host header fuzzing systematically try different hostnames to find hidden vhosts like admin.target.htb or dev.target.htb.
Which Google dork would find exposed .env files (which often contain API keys and database credentials) on a target domain?
site:example.com filetype:env restricts results to the target domain and filters for .env files. These files are used by many frameworks (Laravel, Node.js, Python) to store environment variables including database credentials, API keys, and secret tokens. They should never be web-accessible, but frequently are due to misconfigured web servers.

Source Code Analysis — Finding Secrets in Plain Sight

# View page source for comments, hidden fields, API endpoints
curl -s http://target | grep -i "comment\|hidden\|api\|key\|password\|token"

# Check JavaScript files for hardcoded secrets
curl -s http://target/js/app.js | grep -i "api_key\|secret\|password\|token\|endpoint"

# Git repository exposed? Dump it!
# Check: http://target/.git/HEAD
git-dumper http://target/.git/ ./git-dump
cd git-dump && git log --oneline  # Check commit history for secrets
git diff HEAD~5                   # Look at recent changes

☁️ Cloud Asset Reconnaissance

Modern targets don't just run on physical servers — they're in AWS, Azure, and GCP. Cloud-specific recon finds exposed storage buckets, misconfigured services, and cloud metadata that traditional tools miss.

S3 Bucket & Blob Storage Discovery

# AWS S3 bucket enumeration
# Naming patterns to check:
#   {company}.s3.{region}.amazonaws.com
#   {company}-backup.s3.{region}.amazonaws.com
#   {company}-dev.s3.us-east-1.amazonaws.com
#   {company}-staging.s3.us-west-2.amazonaws.com
#   {company}-assets.s3.eu-west-1.amazonaws.com

# Check if a bucket exists and is listable
aws s3 ls s3://company-backup --no-sign-request 2>&1
# "AccessDenied" = exists but private
# "NoSuchBucket" = doesn't exist
# File listing    = JACKPOT (public read)

# Azure Blob Storage
# {account}.blob.core.windows.net/{container}
curl -s "https://companyname.blob.core.windows.net/backup?restype=container&comp=list"

# GCP Storage
# storage.googleapis.com/{bucket}
curl -s "https://storage.googleapis.com/company-data"

Cloud Metadata & Exposed Services

# If you find SSRF on a cloud-hosted app, check metadata endpoints:
# AWS IMDSv1 (if not blocked)
curl http://169.254.169.254/latest/meta-data/
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/

# GCP
curl -H "Metadata-Flavor: Google" http://169.254.169.254/computeMetadata/v1/

# Azure
curl -H "Metadata: true" "http://169.254.169.254/metadata/instance?api-version=2021-02-01"
💡 Tools: S3Scanner automates bucket discovery across AWS/GCP/DigitalOcean. cloud_enum checks all three major cloud providers for exposed assets. These find things traditional web scanners completely miss.

🌐 ASN & Network Range Discovery

Big organizations own entire network blocks. Finding their ASN (Autonomous System Number) reveals IP ranges you might not discover through DNS alone.

# Find an organization's ASN
whois -h whois.radb.net -- '-i origin AS15169'    # Google's ASN
curl -s "https://api.bgpview.io/search?query_term=example+corp" | jq '.data.asns'

# Get all IP prefixes for an ASN
whois -h whois.radb.net -- '-i origin AS15169' | grep route:

# BGP toolkit (online)
# https://bgp.he.net/ — Search company name, find ASN, see all prefixes

# Practical workflow
# 1. Find company's main IP
dig +short example.com

# 2. Find what ASN it belongs to
curl -s "https://api.bgpview.io/ip/93.184.216.34" | jq '.data.prefixes[].asn'

# 3. Get all prefixes for that ASN
curl -s "https://api.bgpview.io/asn/15133/prefixes" | jq '.data.ipv4_prefixes[].prefix'

# 4. Now scan those ranges — you might find dev servers, staging, admin panels
🎯 Real-world value: On a corporate pentest, the client might give you a domain name. Through ASN/CIDR discovery, you might find 50 additional IP ranges hosting forgotten servers, development environments, and legacy systems that nobody monitors.

🥷 Stealth & Evasion in Recon

On real engagements (not CTFs), getting detected during recon can trigger incident response and change the entire engagement. Here's how to stay under the radar:

  • Scan timing: Nmap's -T1 (sneaky) or -T2 (polite) options slow scans to avoid triggering IDS alerts. Default -T3 is fine for CTFs but noisy in production.
  • Packet fragmentation: nmap -f fragments packets into 8-byte chunks, which can bypass some older firewalls and IDS systems that don't reassemble fragments.
  • Source port spoofing: nmap --source-port 53 makes your scan appear to come from DNS, which some firewalls whitelist.
  • Decoy scans: nmap -D RND:5 target generates 5 decoy IPs alongside your real IP, making it harder to identify the actual scanner.
  • Rate limiting your tools: gobuster -t 5 --delay 500ms slows directory brute-forcing to avoid WAF blocks and detection.
  • WAF detection first: Before throwing tools at a target, check for WAFs with wafw00f target.com. Knowing what's protecting the target saves time.
# Stealthy Nmap scan
nmap -sS -T2 -f --source-port 53 -D RND:3 -Pn target

# Check for WAF before attacking
wafw00f https://target.com
# Common WAFs: Cloudflare, AWS WAF, Akamai, Imperva, ModSecurity

# IDS evasion with decoys
nmap -D 10.10.10.1,10.10.10.2,ME,10.10.10.3 target
#                                 ^ your real IP hidden in decoys
⚠️ Context matters: On CTFs and lab machines, go full speed — stealth is irrelevant. On authorized pentests where you're simulating a real attacker, stealth matters. On red team engagements (adversary simulation), stealth is everything — getting caught means you failed the objective.

🏋️ Practical Exercises

  1. Full Nmap workflow: Scan scanme.nmap.org (Nmap's legal target). Do a quick SYN scan, then targeted service scan, then NSE scripts. Document everything.
  2. Directory brute force: Set up DVWA or a TryHackMe room and practice with gobuster, ffuf, and feroxbuster. Compare speed and results.
  3. Subdomain discovery: Pick a public bug bounty target (e.g., from HackerOne) and enumerate subdomains. Use subfinder, amass, and DNS brute force.
  4. SMB enumeration: On a TryHackMe room, practice smbclient, enum4linux, and crackmapexec against SMB shares.
  5. Wordlist creation: Use CeWL to generate a custom wordlist from a website, then use it with ffuf.

✅ Key Concepts Checklist

Can you explain these without looking them up?

  • What's the difference between a SYN scan and a connect scan?
  • Why do UDP scans take so much longer than TCP scans?
  • What does "filtered" mean in Nmap output vs "closed"?
  • Why do you scan all 65535 ports first, then do detailed scans?
  • What's virtual host routing and why does subdomain enumeration matter?
  • Name five things you check when you find an HTTP service.

🎤 Common Interview Questions

  • "Walk me through your reconnaissance methodology for a new target."
  • "What's the difference between active and passive reconnaissance?"
  • "How would you enumerate a Windows machine with only port 445 open?"
  • "You found a .git directory on a web server. What do you do?"
  • "What tools do you use for content discovery and why?"

📚 Resources for This Phase

Our Guides
Our Cheat Sheets
External Practice
  • HTB Starting Point — Guided machines with enumeration focus
  • TryHackMe Nmap room — Hands-on Nmap training (free)
  • TryHackMe "Kenobi" — SMB + NFS enumeration practice
📝

Phase 2 Checkpoint

0 / 6
Scenario: You run nmap -sC -sV against a target and get this output:

PORT    STATE   SERVICE
22/tcp  open   ssh     OpenSSH 7.9p1
80/tcp  open   http    nginx/1.14.2
3306/tcp open   mysql   MySQL 5.7.38

What is the most significant finding from this output?
MySQL exposed externally is a big red flag — databases should almost never be directly accessible from the internet. This means you can try default credentials, enumerate databases, and potentially extract sensitive data. While the outdated Nginx is worth noting, the externally-accessible MySQL is the more immediately exploitable finding.
Put the correct enumeration methodology in order:
Fast port scan: nmap -p- --min-rate 5000 target
Run targeted NSE scripts on interesting services
Directory brute-force and subdomain enumeration
Deep scan open ports: nmap -sC -sV -p PORTS target
UDP scan on top ports in the background
The correct methodology: Fast full-port scan → Deep scan on open ports → Targeted NSE scripts → Directory/subdomain enumeration → UDP scan. This tiered approach finds all ports quickly, then progressively deepens your knowledge of each service. UDP scanning runs last/in the background because it's slow.
Which of these hidden paths discovered during content enumeration would be the MOST valuable to investigate first?
An exposed /.git/ directory is a goldmine — it contains the entire source code repository including commit history. You can dump it with git-dumper and search through commits for hardcoded passwords, API keys, configuration changes, and removed features. The other paths (images, about page, CSS) are normal web resources with minimal attack surface.
You want to enumerate virtual hosts (subdomains) on a target. Complete the gobuster command:
Fill in the blank:
$ gobuster  -u http://target.htb -w subdomains.txt --append-domain
gobuster vhost enumerates virtual hosts by sending requests with different Host: headers. Virtual hosting means multiple websites share the same IP, distinguished by hostname. A standard directory scan (gobuster dir) won't find these — you need vhost mode. The --append-domain flag automatically appends the base domain to each wordlist entry.
What Nmap flag do you use to run a UDP scan?
-sU performs a UDP scan. UDP scans are much slower than TCP because there's no handshake — if a port doesn't respond, Nmap has to wait for a timeout before marking it as open|filtered. Key UDP services: DNS (53), SNMP (161), TFTP (69), NTP (123). Always include UDP scanning in your methodology — it's often forgotten but can reveal critical services.
What is the difference between a SYN scan (-sS) and a connect scan (-sT)?
A SYN scan sends SYN, reads the response (SYN-ACK = open, RST = closed), then sends RST — it never completes the handshake, making it stealthier. It requires raw socket access (root). A connect scan uses the OS's TCP connect() call to complete the full handshake — no root needed, but it's louder (appears in logs) and slower.
← Phase 1: Foundations All Phases Phase 3: Web Exploitation →