home / ctf / defcamp

hollowdex

JNI_OnLoad rewrites verifyFlag's bytecode through /proc/self/mem at load time; the shipped dex is a decoy hiding a 3-round Feistel.

2026.09.19 DefCamp CTF 2026 Quals 250 pts Reversing
FLAG CTF{7295614c872c071d88fe8b29b5af155fd44f837b5d986f5d74a2524de883241b}

TL;DR

The verifyFlag bytecode shipped in classes.dex is a decoy. JNI_OnLoad in libhollowdex.so locates the method’s code_item in the memory-mapped DEX and overwrites its instructions via /proc/self/mem with original XOR key16. Decrypt statically, decompile, invert the 3-round Feistel.

The hollowing

JNI_OnLoad (0x1900):

  1. GetMethodID(com/hollowdex/Verifier, "verifyFlag", "(Ljava/lang/String;)Z")
  2. ldr w25, [x0, 8] — ArtMethod+8 = dex_code_item_offset_
  3. Scans /proc/self/maps for a line with .apk and r--p/r-xp, sscanf("%lx-%lx"), validates the first 4 bytes at the base are dex\n
  4. code_item = dex_base + code_item_offset; insns_size = [ci+0xc], insns = ci+0x10, len = insns_size*2
  5. out[i] = insns[i] ^ key[i & 15], key = 16 bytes at .rodata:0x700 = 590e7a6514c3bea1e3c1e781e3c1ef01
  6. open("/proc/self/mem", O_RDWR), lseek(insns_vaddr), write(out, len)

So the real bytecode = shipped bytecode XOR the key. Patch it into the DEX (fix sha1 signature + adler32 checksum) and jadx decompiles it.

Real verifyFlag

32 hex chars → 16 bytes → two big-endian u64 (A = bytes 0..7, B = bytes 8..15). rot(x) = (x >>> 47) | (x << 17) (ROL 17), constants:

C1 =  2611923443488327891
C2 =  1376283091369227076
C3 = -6626703657320631856

Rounds:

j4 = A  ^ rot(B  + C1) ^ C1
j6 = B  ^ rot(j4 + C2) ^ C2 ==  6146647970834606741
j8 = j4 ^ rot(j6 + C3) ^ C3 == -5423340355720311942

Pure Feistel — invert from (j6, j8) back to (A, B). No search needed.

Answer (accepted, 250 pts)

key  = c19e6e4be6148b5581809ae7291c9cfe
flag = CTF{7295614c872c071d88fe8b29b5af155fd44f837b5d986f5d74a2524de883241b}
       # sha256(RAW 16 bytes, not the hex string)

Files

  • solve.py — full static solve: unhollows the DEX and derives the key + flag
  • apk/ — unpacked APK
  • patched_classes.dex — DEX with the decrypted verifyFlag
  • out_patched/sources/com/hollowdex/Verifier.java — decompiled real check
  • hollowdex_writeup_ko.pdf — Korean write-up (5 pages)
  • mkpdf.py — generator for that PDF (needs reportlab)

Reproduce

python3 solve.py hollowdex.apk
jadx -d out_patched patched_classes.dex

Gotcha

The flag is sha256 of the decoded 16 raw bytes, not of the 32-char hex string. sha256("c19e...cfe") and its uppercase variant are both rejected.

#android#dex#jni#feistel