Series: Binary Exploitation | Difficulty: Beginner | Read time: ~30 min
Prerequisites: Part 1 — Inside an ELF Binary
You've looked at the ELF binary from the outside. Now it's time to read the instructions inside it. Every buffer overflow, every ROP chain, every piece of shellcode is ultimately just assembly — instructions the CPU executes one at a time.
You don't need to become an assembly programmer. But you do need to read it, understand what the stack is doing, and know which registers matter. That's what this article covers.
Registers are tiny, fast storage locations inside the CPU. x86-64 has 16 general-purpose registers, each 64 bits (8 bytes) wide. Here are the ones you'll use constantly:
| Register | Lower 32-bit | Lower 16-bit | Common Purpose |
|---|---|---|---|
rax | eax | ax | Return value from functions |
rbx | ebx | bx | Base pointer (callee-saved) |
rcx | ecx | cx | Counter / 4th argument |
rdx | edx | dx | 3rd argument / I/O |
rsi | esi | si | 2nd argument |
rdi | edi | di | 1st argument |
rbp | ebp | bp | Stack base pointer |
rsp | esp | sp | Stack pointer |
rip | eip | ip | Instruction pointer (program counter) |
r8–r15 | r8d–r15d | r8w–r15w | 5th–8th arguments / general use |
rsp (stack pointer) — always points to the top of the stack. Every push, pop, call, and return modifies this.
rip (instruction pointer) — points to the next instruction to execute. You can't directly write to rip, but you can change where it points using jumps, calls, and returns. This is the register we hijack in every control-flow attack.
When a function is called on x86-64 Linux, arguments are passed in specific registers (not on the stack like 32-bit). This is the System V AMD64 ABI:
| Argument # | Register | Example |
|---|---|---|
| 1st arg | rdi | printf("hello") → rdi = pointer to "hello" |
| 2nd arg | rsi | printf("%d", 42) → rsi = 42 |
| 3rd arg | rdx | |
| 4th arg | rcx | |
| 5th arg | r8 | |
| 6th arg | r9 | |
| 7th+ arg | Stack | |
| Return value | rax |
You'll see this pattern over and over when reversing:
mov edi, 0x1 # 1st arg = 1 mov esi, 0x2 # 2nd arg = 2 call some_function # call with rdi=1, rsi=2 # result is now in rax
You don't need to memorize hundreds of instructions. These are the ones that show up in almost every binary you'll encounter:
mov rax, rbx # rax = rbx (copy value) lea rax, [rbx+8] # rax = rbx + 8 (address math, no memory access) xchg rax, rbx # swap rax and rbx
push rax # rsp -= 8; [rsp] = rax pop rax # rax = [rsp]; rsp += 8
push decrements rsp by 8 before storing the value. pop reads from [rsp] before incrementing it. The stack grows downward — toward lower addresses. This is the foundation of every stack-based attack.
add rax, 5 # rax += 5 sub rax, 3 # rax -= 3 inc rax # rax += 1 dec rax # rax -= 1 xor rax, rax # rax = 0 (common way to zero a register) and rax, 0xff # rax = rax & 0xff or rax, rbx # rax = rax | rbx
jmp target # unconditional jump (rip = target) je target # jump if equal (ZF == 1) jne target # jump if not equal jg target # jump if greater jl target # jump if less call function # push rip; jmp function ret # pop rip (return from function)
cmp rax, rbx # sets flags based on rax - rbx (doesn't store result) test rax, rax # sets flags based on rax & rax (check if zero)
You'll see xor eax, eax everywhere. It zeroes a register in fewer bytes than mov eax, 0. Compilers use it constantly. In exploitation, it's used to clear registers before setting them to controlled values.
The stack is a region of memory that grows downward. It's used for:
Here's the stack layout for a simple function call:
Higher addresses ┌─────────────────────────┐ │ ... previous frame ... │ ├─────────────────────────┤ │ 3rd arg (if any) │ ← pushed before call ├─────────────────────────┤ │ return address │ ← pushed by 'call' instruction ├─────────────────────────┤ │ saved rbp │ ← pushed by function prologue ├─────────────────────────┤ │ local variable 1 │ ← rsp points here initially ├─────────────────────────┤ │ local variable 2 │ ├─────────────────────────┤ │ ... │ ← rsp (stack pointer) └─────────────────────────┘ Lower addresses
Every function follows the same pattern when it starts and ends:
push rbp # save caller's base pointer mov rbp, rsp # set new base pointer sub rsp, 0x20 # allocate 32 bytes for local variables
leave # equivalent to: mov rsp, rbp; pop rbp ret # pop return address into rip
The leave instruction is a shortcut that restores the stack pointer and base pointer in one step. After ret, the CPU jumps to the return address that was saved on the stack.
The return address sits on the stack, right above the saved rbp. If you can overflow a local buffer that sits below the return address, you can overwrite it with an address of your choosing. When the function returns, ret pops your address into rip and execution jumps wherever you want. This is a buffer overflow — the topic of Part 3.
You saw objdump in Part 1. Now let's learn to read it properly:
$ objdump -d -M intel ./vuln | head -60
Each line has this format:
ADDRESS: BYTECODE INSTRUCTION OPERANDS 1050: 31 ed xor ebp,ebp 1052: 49 89 d1 mov r9,rdx 1055: 5e pop rsi
The address on the left is the virtual memory address (what rip will be when this instruction runs). The bytecode is the raw machine code — what the CPU actually reads. The instruction is the human-readable mnemonic.
The call instruction is how functions invoke other functions. In a disassembly, it looks like this:
106d: e8 be 00 00 00 call 1130 <__libc_start_main@plt>
The e8 opcode means "near call". The 4 bytes after it (be 00 00 00) are a relative offset — the number of bytes to jump forward (or backward if negative) from the next instruction. The CPU computes: rip (after this instruction) + offset = target address.
GDB is the tool you'll use to step through binaries, inspect registers, and examine memory. Here are the essential commands:
| Command | Short | What it does |
|---|---|---|
run | r | Start the program |
break main | b main | Set breakpoint at main() |
break *0x401050 | b *0x401050 | Set breakpoint at address |
continue | c | Continue execution |
stepi | si | Step one instruction (into calls) |
nexti | ni | Step one instruction (over calls) |
info registers | i r | Show all registers |
x/20x $rsp | Examine 20 hex values at rsp | |
x/s $rdi | Examine string at rdi | |
x/i $rip | Examine instruction at rip | |
disassemble main | disas main | Disassemble a function |
quit | q | Exit GDB |
Let's trace a simple program step by step:
$ cat vuln.c
#include <stdio.h>
int main() {
int x = 42;
printf("x = %d\n", x);
return 0;
}
$ gcc -o vuln vuln.c -no-pie -fno-stack-protector
$ gdb -q ./vuln
Now inside GDB:
(gdb) break main Breakpoint 1 at 0x1149 (gdb) run Starting program: ./vuln Breakpoint 1, 0x0000555555555149 in main () (gdb) disas main Dump of assembler code for function main: 0x0000555555555149 <+0>: push rbp 0x000055555555514a <+1>: mov rbp,rsp 0x000055555555514d <+4>: sub rsp,0x10 0x0000555555555151 <+8>: mov DWORD PTR [rbp-0x4],0x2a 0x0000555555555158 <+15>: mov eax,DWORD PTR [rbp-0x4] 0x000055555555515b <+18>: mov esi,eax 0x000055555555515d <+20>: lea rdi,[rip+0xe9e] 0x0000555555555164 <+27>: mov eax,0x0 0x0000555555555169 <+32>: call 0x555555555030 <printf@plt> 0x000055555555516e <+37>: mov eax,0x0 0x0000555555555173 <+42>: leave 0x0000555555555174 <+43>: ret
Let's trace through it:
(gdb) stepi 0x000055555555514a in main () (gdb) i r rsp rbp rsp 0x7fffffffe3a0 0x7fffffffe3a0 rbp 0x7fffffffe3c0 0x7fffffffe3c0 (gdb) stepi 0x000055555555514d in main () (gdb) i r rsp rbp rsp 0x7fffffffe3a0 0x7fffffffe3a0 rbp 0x7fffffffe3a0 0x7fffffffe3a0 # rbp now = rsp (gdb) stepi 0x0000555555555151 in main () (gdb) i r rsp rsp 0x7fffffffe390 0x7fffffffe390 # rsp decremented by 0x10 (16 bytes)
After sub rsp, 0x10, the stack has room for local variables. The variable x = 42 (0x2a in hex) is stored at [rbp-0x4]:
(gdb) x/1x $rbp-0x4 0x7fffffffe39c: 0x0000002a # 0x2a = 42 in decimal
Here's what the stack looks like during that function, with actual addresses:
Address Contents
─────────────────────────────────────
0x7fffffffe3c0 ┌─────────────────┐ ← old rbp (before push rbp)
│ saved rbp │
0x7fffffffe3b8 ├─────────────────┤
│ return address │
0x7fffffffe3b0 ├─────────────────┤
│ (padding) │
0x7fffffffe3a8 ├─────────────────┤
│ (padding) │
0x7fffffffe3a0 ├─────────────────┤ ← rbp (after mov rbp, rsp)
│ x = 0x2a │ [rbp-0x4]
0x7fffffffe39c ├─────────────────┤
│ (unused) │
0x7fffffffe390 └─────────────────┘ ← rsp (after sub rsp, 0x10)
Look at the stack layout carefully. The return address sits at a higher address than the local variables. This means if you overflow a buffer that starts at a lower address and write upward, you'll hit the saved rbp first, then the return address. That's exactly how buffer overflows work.
Let's trace what happens when main() calls printf():
Before call printf:
rsp → ┌──────────────┐
│ local vars │
└──────────────┘
Step 1: call instruction
push rip+5 (return address) onto stack
rsp -= 8
jmp to printf
rsp → ┌──────────────┐
│ return addr │ ← pushed by 'call'
├──────────────┤
│ local vars │
└──────────────┘
Step 2: printf's prologue
push rbp
mov rbp, rsp
rsp → ┌──────────────┐
│ saved rbp │ ← pushed by 'push rbp'
├──────────────┤
│ return addr │
├──────────────┤
│ local vars │
└──────────────┘
Step 3: printf's epilogue
leave (mov rsp, rbp; pop rbp)
ret (pop rip → return address)
Execution returns to the instruction after 'call printf'
Pin this somewhere. You'll need it.
=== x86-64 CALLING CONVENTION (System V ABI) === rdi = 1st arg rsi = 2nd arg rdx = 3rd arg rcx = 4th arg r8 = 5th arg r9 = 6th arg rax = return value === ESSENTIAL INSTRUCTIONS === mov dst, src # dst = src lea dst, [src] # dst = address calculation push val # rsp -= 8; [rsp] = val pop dst # dst = [rsp]; rsp += 8 call func # push rip; jmp func ret # pop rip leave # mov rsp, rbp; pop rbp xor reg, reg # reg = 0 cmp a, b # set flags based on a - b je / jne / jg / jl # conditional jumps === STACK GROWS DOWNWARD === Higher addresses ↑ push → rsp decreases pop → rsp increases === GDB ESSENTIALS === b main # breakpoint r # run si / ni # step instruction i r # info registers x/20x $rsp # examine stack x/i $rip # examine instruction disas main # disassemble function
Answer these questions to make sure you understand assembly and the stack:
Which register holds the return value of a function on x86-64?
What does the push rbp instruction do to rsp?
In the calling convention, which register holds the 1st argument to a function?
What does the call instruction push onto the stack before jumping?
Which direction does the x86-64 stack grow?
You can now read assembly, understand the stack, and step through code in GDB. In Part 3 — Stack Buffer Overflow, we'll put it all together: overflow a buffer, overwrite the return address, and redirect execution to our shellcode.
Time to break something.