Overview
This challenge consists of two main stages. The first involves analyzing an ECC key generation script to derive an AES key and decrypt an encrypted payload into a Linux ELF binary. The second stage requires reversing the recovered statically-linked ELF and bypassing a tricky null-byte string truncation issue to solve a linear system over GF(2) and recover the flag.
Reconnaissance
Stage 1: Decrypting the Payload
In the workspace, we found several files including an encrypted payload professor.elf.enc and a directory repo/ containing keygen.py.
Looking at keygen.py, it calculates an AES key used for encrypting sensitive data based on ECC (Elliptic Curve Cryptography) parameters:
aes_key = sha256(str(public_key[0]).encode()).digest()The script uses the standard generator point G = (Gx, Gy). By looking closely at the decryption scripts provided or writing a quick one, we realize the AES key was simply derived from the Gx coordinate of the generator point itself.
Using AES-CBC with the key sha256(str(Gx).encode()).digest(), we decrypted professor.elf.enc, manually checking offsets for the \x7fELF magic bytes, and successfully carved out the decrypted executable: professor.elf.
Stage 2: Analyzing professor.elf
The extracted file is a 64-bit, statically linked, and stripped Linux executable. When ran, it simply prompts:
Enter passphrase:By passing different inputs to it via standard input, we notice a few characteristics of its output:
- It prints a hex string
Output: <hex>. - The length of the output hex scales linearly with the length of the input (up to 32 bytes / 64 hex characters).
- The relationship between the input bits and output bits is perfectly linear over GF(2). That is, the transformation can be modeled as F(x) = M * x ⊕ C.
Searching through the binary using strings, we find a 64-character hex string that is practically guaranteed to be our target encrypted flag:a824ed1218c832195fe9da9d23649d9935eb5fcac84de0d718c9669981ecfa9c
Exploitation Strategy
The Null-Byte Trap
Standard black-box GF(2) analysis typically involves feeding an oracle matrix inputs with a single bit set (\x00\x00...\x01, \x00\x00...\x02, etc.) to map out the transformation matrix.
However, doing this against professor.elf fails completely or produces outputs that vary wildly in length. This happens because the internal check likely uses a C string function like strlen(). As soon as it encounters a null byte (\x00), the input is truncated, meaning our 32-byte input becomes a much shorter array, destroying the matrix mapping.
Stage 3: The GF(2) Matrix Oracle Bypass
To bypass the early string truncation, we must establish a full 32-byte baseline input that contains no null bytes. We use an array of 32 0xff bytes.
Let the base vector be B. We compute the output F(B). Then, for every bit position i ∈ [0, 255], we flip that bit in B to create B'_i, and compute F(B'_i). Because the system is affine over GF(2), the i-th column of the transformation matrix M is simply:
Once the 256x256 bit matrix M is constructed, we can derive the constant C. Since B is all 1s, its vector representation has every bit set.
(It turned out that C = 0 in this case.)
Solution Code
import subprocess
def run_elf(b_str):
p = subprocess.Popen(["wsl", "./professor.elf"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, _ = p.communicate(b_str)
hex_out = out.decode().split("Output: ")[1].strip()
if len(hex_out) < 64:
hex_out = "0" * (64 - len(hex_out)) + hex_out
return int(hex_out, 16)
# 1. Establish B without null-bytes
B = bytearray([0xff]*32)
F_B = run_elf(B)
M = []
for i in range(256):
test_vec = bytearray(B)
test_vec[i // 8] ^= (1 << (i % 8))
M.append(F_B ^ run_elf(test_vec))
# 2. Gaussian Elimination against Target
target = int("a824ed1218c832195fe9da9d23649d9935eb5fcac84de0d718c9669981ecfa9c", 16)
# ... matrix construction and reduction logic ...
# 3. Solved flag state recovery
# By writing the solution array back into bytes, we retrieve the correct 32-byte input that, when passed into the binary, satisfies the encryption exactly.Flag
HTB{b0bby_sm1L3s_e5_eL_pr0f3s0r}