PE TLS: do not crash on a TLS directory the object does not back - #830
PE TLS: do not crash on a TLS directory the object does not back#830zardus wants to merge 1 commit into
Conversation
Building the thread-local storage initialisation image read the module's initial data with `obj.memory.load(tls_data_start, tls_data_size)`. Clemory raises `KeyError` when nothing backs the start address, so a PE whose TLS directory names memory the image does not map took `Loader.tls.new_thread()` down, and with it `angr.Project` on that binary, because `SimWindows.configure_project` calls it unguarded. cle maps a PE as one blob built by `PE._get_memory_mapped_image`, which ends at the last section's raw data. An object's backed memory can therefore end well before its virtual extent, and a TLS directory pointing into that gap -- the zero-fill tail of a section whose VirtualSize exceeds its SizeOfRawData, say -- reads as unbacked even though the rest of the file is ordinary. Packed and damaged binaries reach the same line from further out, naming a start nowhere near the image at all. Two changes in `ThreadManager.initialization_image`, beside the guards that already reject a negative start and a negative size: - Zero-fill when the range is inside the object but nothing backs it. That gap reads as zeroes at run time. `Clemory.load` already returns a short read for a range that starts in mapped memory and runs out of it; it raises only when the start itself is unbacked, and the two mean the same thing here. Nothing reached on that path is real data, so a tls_block_size that does not fit in the object is skipped rather than allocated. - Skip TLS for the module when the range to be read ends past the object, since it cannot then be describing the object's data. That guard bounds only the range that is read, never tls_block_size: the difference between them is zero fill, which need not lie inside the object at all, and an ELF's .tbss is exactly that.
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Validation record for head Local gateAll ten suites pass; isolation afterwards reports 72 shared objects unchanged in
A green gate that skipped a suite is green over less than it appears to be. Dependency boundary. The gate runs against a nix store environment, not the set The objects that fail today18 Windows PE objects from a corpus that cannot be redistributed. Same interpreter,
11 take the repair path and get an image of 5 to 52 bytes; 7 have a directory whose Nothing already tracked changes
This is what caught a wrong version of the patch, which bounded Reachability of the unbacked pathOver all 873 On the truncated public fixture, with the start unbacked:
So the change trades no Public corpus survey72,664 distinct public PE objects by content hash — 5,784 from a corpus of generated 3,616 carry a TLS directory. Across those, |
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS The reproducer, before and after
Save as import logging, os, tempfile, cle
logging.basicConfig(level=logging.WARNING, format="%(levelname)s | %(name)s | %(message)s")
fd, p = tempfile.mkstemp(suffix=".exe")
os.write(fd, open("binaries/tests/x86/windows/TLS.exe", "rb").read()[:0x8000]); os.close(fd)
ld = cle.Loader(p, auto_load_libs=False)
o = ld.main_object
print("tls_data_start = %#x tls_data_size = %d object span = %#x"
% (o.tls_data_start, o.tls_data_size, o.max_addr - o.mapped_base + 1))
th = ld.tls.new_thread()
print("new_thread() -> %r" % (th,))
print("module TLS data address = %#x" % th.get_tls_data_addr(o.tls_module_id))Before — the full output
After — the module gets its TLS block: full outputThe image is the full 520 bytes the directory asks for, zero-filled, and the Absolute paths in both tracebacks were shortened to repository-relative ones; the The other branchThe function's other new path — skipping TLS when the range to be read ends past |
|
Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_830 |
2 participants
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS
Problem
Loader.tls.new_thread()raisesKeyErroron a PE whose TLS directory points atmemory the image does not map.
SimWindows.configure_projectcalls it unguarded,so
angr.Projectfails on the binary even thoughcle.Loaderitself succeeded.Truncating a fixture
angr/binariesalready tracks reproduces it. Run from thedirectory holding the
binariescheckout, at44a93d4f:110592is0x1b000, which istls_data_startexactly. The output comment hasboth sides in full.
Root cause
initialization_imagecopies a module's initial TLS data out of the module's ownmemory with
obj.memory.load(tls_data_start, tls_data_size). For a PE both valuescome from the TLS directory's
StartAddressOfRawDataandEndAddressOfRawData(
cle/backends/pe/pe.py,_register_tls).Clemory.loadends withraise KeyError(addr)when no backer covers the start.cle maps a PE as one blob built by
PE._get_memory_mapped_image, which ends at thelast section's raw data, so an object's backed memory can stop well short of its
virtual extent. That method replicates pefile's equivalent but adds handling for
partially mapped sections, and it is what skips a section whose raw data runs past
the end of the file. A TLS directory pointing into that gap — the zero-fill tail of a
section whose
VirtualSizeexceeds itsSizeOfRawDatais the usual case — reads asunbacked while the rest of the file is ordinary. Packed and damaged binaries reach
the same line from further out, naming a start hundreds of megabytes outside.
A range that merely runs off the end of a backer does not raise:
Clemory.loadclips and returns a short buffer, which the caller zero-pads. Only an unbacked
start raises, and the two cases mean the same thing.
Fix
Two additions to
initialization_image, beside the guards that already reject anegative start and a negative size:
Zero-fill when the range is inside the object but nothing backs it. That gap
reads as zeroes at run time, so the module gets the template it should have. On
the reproducer above,
new_thread()now returns and the image is the full 520 bytesthe directory asks for.
cle#538gavepack_wordthe same treatment in this package. Nothingreached on that path is real data, so a
tls_block_sizethat does not fit in theobject is skipped there rather than allocated.
Skip TLS for the module when the range to be read ends past the object. It
cannot then be describing the object's data. This is the disposition the two
existing guards already use:
initialization_imagereturnsNone,PETLSObject.__init__skips the module, andget_tls_data_addr()for its indexreturns 0 rather than a pointer.
That second guard bounds only the range that is read, never
tls_block_size. Thedifference between the two is zero fill, which need not lie inside the object at all —
an ELF's
.tbssis exactly that. The bound is exact:tls_data_start + tls_data_size == spanstill yields an image,
span + 1yieldsNone.Deliberately not done: a directory naming a valid, backed range and a
multi-gigabyte zero fill still allocates it. That is master's behaviour and this
change neither widens nor narrows it.
Testing
No new regression test. The reproducer is a truncated copy of a tracked file rather
than compiler output, and I would rather give you the recipe than commit a fixture no
toolchain emits — say the word and I will add one to
angr/binaries.tests/test_tls_resiliency.py::test_tls_pe_incorrect_tls_data_start, which covers thenegative-start guard on
i386/windows/2.exe, still passes, and the cle suite is green.Measured on 18 Windows PE objects from a corpus that cannot be redistributed:
angr.Project(path)raises on all 18 before and constructs on all 18 after.initialization_imageis unchanged for every object with TLS reachable frombinaries/tests, and for every one of the 3,616 TLS-bearing objects in a72,664-object public PE survey — none of which reproduces this, because a linker puts
the TLS template inside a section's raw data.
Validation: #830 (comment)
session: sharpen