kavklaw@llm $ cat malware-analysis-guide.md
โฑ๏ธ 30 min read ยท Safely dissect malware and understand what it does โ from first strings to full behavioral analysis
โ ๏ธ CRITICAL SAFETY WARNING: Malware analysis involves handling live malicious software that can damage your system, steal your data, or spread to other machines on your network. NEVER analyze malware on your personal computer or any machine connected to a production network. Always use an isolated virtual machine with no shared folders, no clipboard sharing, and no network access to real systems. If you're unsure about your setup, use an online sandbox instead.
Malware analysis requires specific tooling that you must set up before touching any samples:
sudo apt install yara or via pip: pip install yara-python.file, strings, xxd, sha256sum โ all pre-installed on most Linux distributions.If this seems like a lot: start with just a Linux VM, strings, file, and a VirusTotal account. You can do meaningful static analysis with that alone. Add FlareVM when you're ready for dynamic analysis.
Got a suspicious file and need to analyze it safely? Here's the fast-track:
# 1. NEVER execute unknown files on your main system
# Use an isolated VM or upload to an online sandbox
# 2. Quick static analysis (without executing)
file suspicious.exe # Identify file type
sha256sum suspicious.exe # Get hash for VirusTotal lookup
strings suspicious.exe | head -50 # Extract readable strings
strings suspicious.exe | grep -iE "(http|ftp|cmd|powershell|exec|shell|password|admin)"
# 3. Upload hash to VirusTotal (don't upload the file if it's sensitive)
# https://www.virustotal.com/gui/search/[SHA256_HASH]
# 4. For deeper analysis, use a sandbox:
# - Any.Run (interactive sandbox)
# - Hybrid Analysis (automated)
# - Joe Sandbox (detailed reports)
# 5. If you need to analyze locally:
# Set up an isolated VM (FlareVM for Windows, REMnux for Linux)
# Snapshot BEFORE running malware
# Disable network or route through fakenet/inetsim
Malware analysis is the process of studying malicious software to understand what it does, where it came from, and how much damage it can cause. It answers the key questions that incident responders, threat intelligence analysts, and defenders need:
There are three main approaches, each providing different levels of insight. Don't be intimidated โ you don't need to master all three at once. Most beginners start with static analysis and basic sandbox usage, which is enough to handle many real-world situations:
The cardinal rule of malware analysis: never analyze malware on a system you care about. You need an isolated environment where the malware can execute safely.
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Host System (your real computer) โ
โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ Analysis VM (isolated, snapshotted) โ โ
โ โ FlareVM (Windows) or REMnux (Linux) โ โ
โ โ Host-only or no networking โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ host-only network โ
โ โโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ INetSim/FakeNet VM (optional) โ โ
โ โ Simulates: DNS, HTTP, HTTPS, SMTP, FTP โ โ
โ โ Captures malware network activity โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Step 1: Install a hypervisor โ VirtualBox (free), VMware Workstation, or Hyper-V.
Step 2: Create a Windows analysis VM. Install Windows 10/11, then install FlareVM, which installs 100+ analysis tools automatically: disassemblers (Ghidra, IDA Free, x64dbg), PE tools (pestudio, PE-bear, CFF Explorer), network tools (Wireshark, FakeNet-NG), utilities (Process Monitor, Process Explorer, Autoruns), and string extractors (FLOSS, BinText).
Step 3: Create a Linux analysis VM. REMnux is purpose-built for malware analysis:
curl -O https://REMnux.org/remnux
chmod +x remnux
sudo mv remnux /usr/local/bin/remnux
sudo remnux install
Includes: INetSim, YARA, radare2, oletools, pdftools, and more.
Step 4: Network isolation โ set VMs to "Host-only" networking (no internet access, but note that host-only still allows VMโhost communication; for full isolation use "Internal" networking). Use INetSim or FakeNet (tools that pretend to be the internet โ they respond to DNS queries, HTTP requests, etc., so malware "thinks" it's online while you capture everything it tries to do).
Step 5: SNAPSHOT everything before analysis. Always revert to snapshot after each analysis session.
โ ๏ธ Critical: Always take a VM snapshot before executing malware. After analysis, revert to the clean snapshot. This ensures your lab stays clean and each analysis starts fresh. If you don't snapshot, your analysis environment becomes contaminated and results become unreliable.
Static analysis examines the malware without executing it. Always start here. It gives you initial indicators and helps you understand what you're dealing with before risking execution.
# Identify file type (don't trust the extension!)
file suspicious.exe
# Output: PE32+ executable (GUI) x86-64, for MS Windows
file document.pdf
# Could be: PDF document, version 1.7 (legit PDF)
# Or: PE32 executable... (EXE renamed as PDF!)
# Check magic bytes manually
xxd suspicious.exe | head -5
# MZ header (4D 5A) = Windows PE
# 7F 45 4C 46 = Linux ELF
# PK (50 4B) = ZIP/DOCX/XLSX/JAR
# %PDF (25 50 44 46) = PDF
# Calculate hashes
md5sum suspicious.exe
sha1sum suspicious.exe
sha256sum suspicious.exe
# Use SHA-256 hash to search VirusTotal without uploading the file
Strings are one of the most powerful quick-analysis tools. Readable text in a binary can reveal C2 URLs, registry keys, file paths, error messages, and more:
# Basic string extraction
strings suspicious.exe | less
# Minimum string length (reduce noise)
strings -n 10 suspicious.exe
# Extract both ASCII and Unicode strings
strings -a suspicious.exe # ASCII
strings -el suspicious.exe # Unicode (little-endian)
# Search for interesting patterns
strings suspicious.exe | grep -iE "(http://|https://|ftp://)" # URLs
strings suspicious.exe | grep -iE "(\.exe|\.dll|\.bat|\.ps1)" # File references
strings suspicious.exe | grep -iE "(HKEY_|HKLM|HKCU|CurrentVersion\\\\Run)" # Registry
strings suspicious.exe | grep -iE "(cmd|powershell|wscript|cscript)" # Execution
strings suspicious.exe | grep -iE "([0-9]{1,3}\.){3}[0-9]{1,3}" # IP addresses
strings suspicious.exe | grep -iE "(password|passwd|credential|login)" # Credentials
strings suspicious.exe | grep -iE "(CreateFile|WriteFile|DeleteFile|RegSetValue)" # API calls
# FLOSS โ FLARE Obfuscated String Solver (finds hidden/obfuscated strings)
floss suspicious.exe
# FLOSS finds strings that regular 'strings' misses:
# - Stack strings (built character by character)
# - Decoded strings (XOR, Base64, custom encoding)
# - Tight strings (short decoded strings)
๐ก Pro Tip: UseFLOSS(FLARE Obfuscated String Solver) instead of plainstringsfor malware. Malware authors routinely obfuscate strings to hide IOCs, but FLOSS uses emulation to decode them automatically. It finds URLs, C2 servers, and registry keys thatstringscompletely misses.
PE (Portable Executable) is the file format Windows uses for programs (.exe, .dll, .sys files). For Windows PE files, pestudio provides detailed static analysis in a friendly GUI. It highlights suspicious elements automatically, making it a great starting point for beginners:
pestudio analyzes file metadata, imports, strings, resources, sections (entropy analysis), and integrates with VirusTotal.
Key things to look for:
1. Suspicious imports:
VirtualAlloc + WriteProcessMemory โ process injectionCreateRemoteThread โ remote thread injectionInternetOpen + InternetReadFile โ network communicationRegSetValueEx โ registry modification (persistence)CreateService โ service creation (persistence)IsDebuggerPresent โ anti-debuggingGetTickCount, QueryPerformanceCounter โ timing-based anti-analysis2. Section entropy (a measure of randomness, 0.0 to 8.0):
.text section: ~6.0-6.8 entropy3. Compilation timestamp: future dates โ likely forged; very old dates โ might be compiled from old source; timestamps matching known threat actor campaigns.
Understanding the PE (Portable Executable) format is fundamental to Windows malware analysis. PE is the file format Windows uses for programs and libraries โ every .exe, .dll, .sys, and .scr file follows this structure. If you've ever double-clicked a .exe, you've run a PE file.
# PE File Structure Overview:
# โโโโโโโโโโโโโโโโโโโโโโโโโโโ
#
# โโโโโโโโโโโโโโโโโโโโโโโ
# โ DOS Header (MZ) โ โ Magic bytes: 4D 5A ("MZ")
# โ DOS Stub โ โ "This program cannot be run in DOS mode"
# โโโโโโโโโโโโโโโโโโโโโโโค
# โ PE Signature โ โ "PE\0\0" (50 45 00 00)
# โโโโโโโโโโโโโโโโโโโโโโโค
# โ COFF File Header โ โ Machine type, number of sections, timestamp
# โโโโโโโโโโโโโโโโโโโโโโโค
# โ Optional Header โ โ Entry point, image base, subsystem
# โโโโโโโโโโโโโโโโโโโโโโโค
# โ Section Headers โ โ .text, .data, .rsrc, .rdata, .reloc
# โโโโโโโโโโโโโโโโโโโโโโโค
# โ .text (Code) โ โ Executable code
# โ .data (Data) โ โ Initialized global variables
# โ .rdata (Read-only) โ โ Import/export tables, strings
# โ .rsrc (Resources) โ โ Icons, dialogs, embedded files
# โ .reloc (Relocations)โ โ Relocation information
# โโโโโโโโโโโโโโโโโโโโโโโ
# Important PE elements for malware analysis:
# - Import Address Table (IAT): What DLLs and functions the malware uses
# - Export Table: Functions the malware exposes (relevant for DLLs)
# - Resources: May contain embedded payloads, configs, or additional malware
# - Entry Point: Where execution begins
# - Section permissions: .text should be executable, .data shouldn't be
# Command-line PE analysis:
# Using pefile (Python):
python3 -c "
import pefile
pe = pefile.PE('suspicious.exe')
print(f'Entry Point: {hex(pe.OPTIONAL_HEADER.AddressOfEntryPoint)}')
print(f'Timestamp: {pe.FILE_HEADER.TimeDateStamp}')
print(f'Sections:')
for section in pe.sections:
print(f' {section.Name.decode().rstrip(chr(0)):8s} '
f'Entropy: {section.get_entropy():.2f} '
f'Size: {section.SizeOfRawData}')
print(f'Imports:')
for entry in pe.DIRECTORY_ENTRY_IMPORT:
print(f' {entry.dll.decode()}')
for imp in entry.imports[:5]:
print(f' {imp.name.decode() if imp.name else hex(imp.ordinal)}')
"
strings on a suspicious executable and find the strings "VirtualAlloc", "WriteProcessMemory", and "CreateRemoteThread". What technique does this suggest?Dynamic analysis means actually running the malware and observing what it does. This reveals behavior that static analysis can't catch, especially when the malware is packed, encrypted, or heavily obfuscated.
# Process Monitor โ captures real-time file, registry, network activity
# Essential tool from Sysinternals
# Setup:
# 1. Open ProcMon BEFORE running malware
# 2. Set filter: Process Name โ contains โ [malware name]
# 3. Execute the malware
# 4. Watch the activity stream
# Key operations to watch:
# โโโโโโโโโโโโโโโโโโโโโโโโโโ
# FILE operations:
# CreateFile โ what files does it create/access?
# WriteFile โ what data is it writing?
# DeleteFile โ is it cleaning up after itself?
# Look for: files in %TEMP%, %APPDATA%, startup folders
# REGISTRY operations:
# RegSetValue โ what registry keys is it creating?
# Look for: Run keys (persistence), service entries
# HKCU\Software\Microsoft\Windows\CurrentVersion\Run
# HKLM\SYSTEM\CurrentControlSet\Services
# PROCESS operations:
# Process Create โ what child processes does it spawn?
# Process Start โ cmd.exe? powershell.exe? another copy of itself?
# NETWORK operations:
# TCP Connect โ what IPs/domains does it contact?
# UDP Send โ DNS queries, data exfiltration?
# Save the capture for later analysis:
# File โ Save โ PML format (native) or CSV for grep/awk
# Capture network traffic during execution
# Wireshark โ full packet capture
# Start capture before executing malware
# Filter: ip.addr == [malware_vm_ip]
# Key things to look for:
# - DNS queries (what domains does it resolve?)
# - HTTP requests (C2 communication, downloading payloads)
# - Raw TCP connections (custom C2 protocols)
# - DNS over unusual ports (tunneling)
# - TLS certificate details (C2 infrastructure fingerprinting)
# FakeNet-NG (on Windows) โ simulates internet services
# Responds to ALL DNS queries, HTTP requests, etc.
# Prevents malware from knowing it's in a sandbox
# Shows you exactly what the malware tries to communicate with
fakenet.exe
# INetSim (on Linux/REMnux) โ internet simulation
# Provides fake DNS, HTTP, HTTPS, FTP, SMTP, IRC servers
sudo inetsim
# Configure in /etc/inetsim/inetsim.conf
# Point your analysis VM's DNS to the INetSim IP
# API Monitor โ intercepts Windows API calls in real-time
# More granular than Process Monitor โ shows exact API parameters
# Important API categories to monitor:
# - File I/O: CreateFile, ReadFile, WriteFile, DeleteFile
# - Registry: RegOpenKey, RegSetValue, RegDeleteValue
# - Process: CreateProcess, OpenProcess, VirtualAllocEx
# - Network: connect, send, recv, InternetOpen, HttpSendRequest
# - Crypto: CryptEncrypt, CryptDecrypt (ransomware!)
# - Anti-debug: IsDebuggerPresent, CheckRemoteDebuggerPresent
# x64dbg โ debugger for stepping through execution
# Set breakpoints on interesting APIs
# Step through code to understand logic
# Dump decrypted payloads from memory
Behavioral analysis looks at the overall effects of the malware on the system. After running the malware, compare the system state before and after to identify all changes.
What to compare (before vs after execution):
Using Regshot for Registry Comparison:
Example Regshot output showing persistence:
Keys added: 1
HKCU\Software\Microsoft\Windows\CurrentVersion\Run\WindowsUpdate
Values added: 1
HKCU\...\Run\WindowsUpdate: "C:\Users\Public\update.exe"
This is persistence โ the malware added itself to auto-start.
YARA is the pattern-matching tool that malware researchers use most. Think of it like grep for malware โ you write rules that describe what a specific malware family looks like (certain strings, byte patterns, file properties), and YARA scans files to find matches. Once you write a rule for a malware sample, you can use it to find that same malware anywhere else in your environment.
// Basic YARA rule structure:
rule ExampleMalware {
meta:
author = "analyst"
description = "Detects Example Malware Family"
date = "2024-03-15"
hash = "a1b2c3d4e5f6..."
strings:
$url = "http://evil-c2.com/beacon" ascii
$mutex = "Global\\ExampleMutex" wide
$api1 = "VirtualAlloc"
$api2 = "WriteProcessMemory"
$api3 = "CreateRemoteThread"
$hex_pattern = { 48 8B 05 ?? ?? ?? ?? 48 89 44 24 }
condition:
uint16(0) == 0x5A4D and // Must be a PE file (MZ header)
filesize < 500KB and // Reasonable file size
$url and // Must contain the C2 URL
(2 of ($api*)) // At least 2 of the API strings
}
// Hunting rule โ broader detection:
rule Suspicious_ProcessInjection {
meta:
description = "Detects potential process injection"
strings:
$s1 = "VirtualAllocEx" ascii
$s2 = "WriteProcessMemory" ascii
$s3 = "CreateRemoteThread" ascii
$s4 = "NtWriteVirtualMemory" ascii
$s5 = "RtlCreateUserThread" ascii
condition:
uint16(0) == 0x5A4D and
3 of them
}
// Using YARA:
// Scan a single file
yara rules.yar suspicious.exe
// Scan a directory recursively
yara -r rules.yar /path/to/samples/
// Show matching strings
yara -s rules.yar suspicious.exe
// Scan running processes
yara -p 4 rules.yar /proc/*/exe 2>/dev/null
๐ก Pro Tip: Start building a personal YARA rule library. Every time you analyze malware, create a YARA rule that would detect it. Over time, you build a powerful detection capability that complements your AV/EDR. Public rule sets like Yara-Rules and Neo23x0/signature-base are great starting points.
uint16(0) == 0x5A4D check for?uint16(0) reads a 16-bit unsigned integer at file offset 0 (the very beginning of the file). 0x5A4D is the little-endian representation of "MZ" โ the magic bytes that identify a Windows PE (Portable Executable) file. This condition ensures the YARA rule only matches PE files, not scripts, documents, or other file types. It's a very common condition in malware YARA rules.Understanding malware categories helps you know what to look for during analysis:
CryptEncrypt, CryptGenKey, BCryptEncrypt), ransom notes. Focus on encryption algorithm and key management.SetWindowsHookEx, GetAsyncKeyState, low-level keyboard hooks.More advanced malware actively tries to detect when it's being analyzed and changes its behavior to avoid detection. Here are the main techniques you'll encounter:
HKLM\...\Services\VBoxGuest (VirtualBox), HKLM\SOFTWARE\VMware, Inc.\VMware Toolsvmtoolsd.exe, VBoxService.exe, VBoxTray.exeC:\Windows\System32\drivers\VBoxMouse.sys, vmhgfs.sysCounter: Remove VM artifacts, use bare-metal analysis for evasive samples.
IsDebuggerPresent(), CheckRemoteDebuggerPresent(), NtQueryInformationProcess(ProcessDebugPort)rdtsc instruction, GetTickCount() before/after code, QueryPerformanceCounter()Counter: Use ScyllaHide plugin, patch anti-debug checks.
Packers compress or encrypt the malware's real code inside a wrapper. When executed, the wrapper unpacks the real code into memory and runs it. This hides the malware's true functionality from static analysis โ you only see the unpacker, not the payload.
upx -d packed_malware.exe # Unpack UPX (most common packer)
Detecting packing: high section entropy (>7.0), small import table (only LoadLibrary + GetProcAddress), and section names like UPX0, .aspack, .themida.
XOR encoding, Base64 encoding, stack strings (built character by character in memory). Tool: FLOSS automatically decodes many obfuscation schemes.
Sleep(300000) โ wait 5 minutes before executingWhen you don't have a local analysis lab, or just want a quick automated analysis, online sandboxes are a great option:
โ ๏ธ IMPORTANT: Files uploaded to public sandboxes are visible to everyone. If analyzing sensitive/targeted malware, use a private sandbox. At minimum, search by hash first before uploading the file.
Let's walk through analyzing a theoretical dropper step by step. This mirrors the real-world analysis workflow:
# Step 1: Initial triage
file dropper.exe
# PE32 executable (GUI) Intel 80386, for MS Windows
sha256sum dropper.exe
# abc123def456...
# Search hash on VirusTotal: 35/72 detections
# Names include: Trojan.GenericKD, Trojan.Downloader, Dropper.Agent
# Step 2: Static analysis
strings dropper.exe | grep -i http
# http://185.234.72.19/stage2.bin
# โ Found the payload URL!
strings dropper.exe | grep -iE "run|startup|service"
# Software\Microsoft\Windows\CurrentVersion\Run
# โ It sets a Run key for persistence!
# Check imports with pefile or pestudio:
# - URLDownloadToFile (download payload from internet)
# - RegSetValueExA (set registry value for persistence)
# - CreateProcessA (execute the downloaded payload)
# - GetTempPathA (get temp directory for staging)
# Step 3: Dynamic analysis (in isolated VM)
# Start ProcMon + Wireshark + FakeNet
# Execute dropper.exe
# Observe:
# 1. DNS query for 185.234.72.19 (captured by FakeNet)
# 2. HTTP GET /stage2.bin (FakeNet returns fake payload)
# 3. File write: C:\Users\user\AppData\Local\Temp\update.exe
# 4. Registry write: HKCU\...\Run\SystemUpdate = "...update.exe"
# 5. Process create: update.exe (the second stage)
# Step 4: Behavioral summary
# The dropper:
# 1. Downloads stage2.bin from 185.234.72.19
# 2. Saves it as %TEMP%\update.exe
# 3. Sets Run key for persistence
# 4. Executes the downloaded payload
# Step 5: Create IOCs
# File hash: abc123def456...
# C2 IP: 185.234.72.19
# URL: http://185.234.72.19/stage2.bin
# Registry: HKCU\Software\Microsoft\Windows\CurrentVersion\Run\SystemUpdate
# File path: %TEMP%\update.exe
# Step 6: Write YARA rule
rule Dropper_Example {
meta:
description = "Detects Example Dropper"
strings:
$url = "185.234.72.19" ascii
$reg = "CurrentVersion\\Run" ascii
$api1 = "URLDownloadToFile" ascii
$api2 = "RegSetValueEx" ascii
condition:
uint16(0) == 0x5A4D and all of them
}
Follow this systematic approach for every malware sample:
Triage (2 min) โ Static Analysis (15-30 min) โ Dynamic Analysis (30-60 min) โ Deep Analysis (optional) โ Reporting
โ Mistake #1: Analyzing malware on your real system.
Even "just looking at strings" can be risky if you accidentally double-click the file. Always use an isolated VM. The 30 minutes to set up a lab saves you from potentially catastrophic consequences.
โ Mistake #2: Uploading sensitive samples to public sandboxes.
Files uploaded to VirusTotal, Any.Run, and Hybrid Analysis are often publicly accessible. If you're analyzing targeted malware from a real incident, the attacker may monitor these services. Search by hash first; only upload if the sample isn't sensitive.
โ Mistake #3: Skipping static analysis and going straight to execution.
Static analysis gives you a roadmap of what to expect during dynamic analysis. If you execute first without understanding the malware, you might miss important behaviors or expose your lab to unexpected threats (like worm propagation).
โ Mistake #4: Not taking VM snapshots.
If you don't snapshot before execution, you can't get your clean lab back. You also can't compare before/after states for behavioral analysis. Snapshot is the single most important step.
โ Mistake #5: Trusting file extensions.
A file named "invoice.pdf" might be an executable. Always usefilecommand or check magic bytes to identify the true file type. Malware frequently uses misleading extensions.
โ Mistake #6: Ignoring packed/encrypted samples.
If entropy is high and the import table is tiny, the binary is packed. Static analysis of the packed code is useless โ you need to unpack first (UPX for common packers, dynamic analysis for custom packers).
You receive a suspicious file named "invoice.pdf.exe".
Running 'strings' reveals:
- "http://192.168.1.100/payload.bin"
- "SOFTWARE\Microsoft\Windows\CurrentVersion\Run"
- "URLDownloadToFileA"
- "cmd.exe /c del %0"