Add x64 and Direct3D 11 client support with SDL3 input and selectable JUCE/Miles audio - #21
Open
swgsais wants to merge 156 commits into
Open
Add x64 and Direct3D 11 client support with SDL3 input and selectable JUCE/Miles audio#21swgsais wants to merge 156 commits into
swgsais wants to merge 156 commits into
Conversation
Port the gameplay and God clients to x64 with modern MSBuild, vendored runtime dependencies, a locally buildable Qt 3 tree, deployment scripts, and x64-safe engine fixes.
Show the SDL joystick index alongside localized button, axis, slider, and POV hat names so bindings from multiple controllers can be distinguished. Adapted from Elour's upstream commit b8098ea13f7d8580b0f7c77f441fad6415298897.
Call stack entries are instruction addresses but were stored in uint32 buffers. On the x64 client this truncated every address: win32 DebugHelp::getCallStack() narrowed StackWalk's DWORD64 AddrPC.Offset to 32 bits, so symbol lookup and crash reports were unusable. Widen the storage, the getCallStack()/lookupAddress() signatures and the printing to uint64. RenderWorld and MemoryManager keep their own call stack buffers and are widened to match the new signature; MemoryManager's other path already used uintptr_t. This mirrors the same change on the server, where the identical code was worse than truncation: passing a uint32 buffer to backtrace(), which writes void*, overran every buffer by 2x. Shared files must stay in sync between src and client-tools per README.md.
These specializations wrote the command count as int32 but wrote the baseline count, and read both counts, as size_t. size_t is 8 bytes on both LP64 and LLP64, so on a 64-bit build pack writes 4+8 bytes while unpack reads 8+8 and leaves the stream misaligned by 4 bytes for every subsequent key and value. Switch both counts to int32_t, matching the generic internal_pack/ internal_unpack path and the already-converted PlayerQuestData specialization. The client and server must agree on this format; the matching server change is on Galaxies-Reborn/src.
This branch targets a 64-bit client, but a build that does not specify a platform fell back to Win32. Default Platform to x64 when it is empty. Invocations that pass /p:Platform, including scripts/Build-X64Client.ps1 and any solution build, are unaffected.
DO_TRACK 5 walked five frames of stack on every allocation and DO_GUARDS 1 padded every block with 16-byte guard bands and swept the entire heap every 512 alloc/free. Both were temporary aids for chasing a world-load heap overflow, and both are far too expensive to carry in a build we intend to measure renderer performance against. Re-enable them locally when hunting a heap bug; they are not a shipping configuration.
Design of record for the renderer conversion: 20 architectural decisions, 8 phases with exit criteria and performance gates, the full deletion list for a zero-DX9-artifact tree, and the risk register. Two findings drive the shape of it. First, the renderer already sits behind a DLL boundary -- Graphics::install LoadLibrary's gl%02d_r.dll by rasterMajor and forwards a 118-slot Gl_api function table -- so DX11 is a new backend filling the existing contract and DX9 is then subtracted, never a forked engine path. Second, no instrumentation exists in a shipping build: Direct3d9_Metrics is entirely #ifdef _DEBUG, its report caller sits inside a #if 0, every NP_PROFILER_ macro is a NOP at PRODUCTION==1, and there are zero CreateQuery calls in the tree. Until that is fixed, "no performance degradation" is not a testable claim, so Phase 0 is measurement on DX9 with no DX11 code at all. Also records the prior Direct3D 11 effort found in _client -- a PBR renderer whose source exists in no ref -- and why its binary is quarantined rather than used as a parity reference.
Graphics::lockBackBuffer built its RECT with bottom taken from the rect's x1 and right from its y1, so any caller passing a sub-rect locked a transposed region. There are no callers passing a non-null rect today, which is why it went unnoticed; fix it before the DX11 backend gives the parameter meaning. Texture granted friendship to Direct3d9 and Direct3d9_RenderTarget but only the inner LockData class named Direct3d11_TextureData, so a DX11 render target could not reach Texture's privates. Transform granted Direct3d8, Direct3d9 and OpenGL but not Direct3d11. Both are additive declarations with no layout or ABI effect.
Nineteen individually-buildable commits taking the renderer from an empty project to the shader-corpus conversion, plus the Phase-0 prerequisites split into hard blockers and work that can run in parallel. The ordering deliberately inverts the prior attempt's. Screenshot and back-buffer capture (C4) and Metrics/DebugFlags/the _DEBUG-only Gl_api slots (C5) land before the first geometry (C11), so every later commit has an artifact a pixel diff can consume. The prior attempt implemented neither, which is why none of its parity claims -- including its claims of success -- were falsifiable. Also records the audit of that attempt. Its mechanical layer transplants and several of its findings were independently re-derived by our plan. Its semantic layer does not: retuned lighting constants against DX9's, a knowingly inverted compare table, roughly eighteen Gl_api slots wired to empty bodies that DX9 implements for real (fog leaves the c10 density constant permanently zero), a shader-replacement table matched by unanchored substring search that captured 53 of 130 assembly pixel programs and rewrote most of them with math that is not the file's math, and per-frame diagnostics including a file-writing tracer left live in release code.
Gl_api has three distinct binary layouts: five slots are #ifdef _DEBUG and four
more are #if PRODUCTION == 0. Which layout a build uses is decided by the
DEBUG_LEVEL that also picks the _r/_o/_d suffix in the DLL name, and nothing
checked that the client and the raster DLL agreed. A mismatched pair loads
without complaint and then calls the wrong function through every slot past the
first conditional one -- silent, catastrophic, and the worst failure mode ahead
of us now that a second backend implementation is about to exist.
Each backend now exports GetGlApiStructSize alongside GetApi, and
Graphics::install FATALs on absence or mismatch before it calls through a single
slot. It is a separate export rather than a struct field because Headless
blanket-fills Gl_api as void** over sizeof(Gl_api)/sizeof(void*) and would
overwrite a member.
Two removals ride along, cheap now because only the three DX9 raster DLLs and
Headless exist, and a five-project lockstep edit once a DX11 backend lands:
- Gl_api::setDynamicIndexBufferSize. The DX9 backend never assigned the slot,
so it was permanently null, and Graphics::setDynamicIndexBufferSize called
through it unguarded -- a guaranteed null dereference that survived only
because the facade itself has no callers. Removed the slot, the facade, the
DX9 forwarding function and Direct3d9_DynamicIndexBufferData::setSize, which
that forwarder was the sole caller of.
- Gl_metrics, 68 lines in Graphics.def referenced nowhere in the tree.
Verified: full Release x64 build clean; dumpbin confirms all three of gl05_r,
gl06_r and gl07_r export GetGlApiStructSize.
First commit of the DX11 backend. It is the contract, the size guard and an
install that says no -- no device, no resources, no draws. gl05 remains the
working renderer and is untouched.
The project is authored rather than cloned from Direct3d9.vcxproj on purpose.
That project leads its x64 include path with the vendored directx9 directory
and then the June 2010 DirectX SDK, and that SDK also ships d3d11.h, dxgi.h and
d3d11.lib -- so a clone would have silently bound this backend to D3D11.0 and
DXGI 1.1, losing ID3D11Device1 and IDXGIFactory2, and with them
CreateSwapChainForHwnd and the flip-model swap chain, with nothing reported
anywhere. This project names no legacy DirectX path of any kind and takes
everything from the Windows SDK. Output naming and all intermediate paths are
left to Directory.Build.props/targets rather than hardcoded.
Include set, CRT flavour and DEBUG_LEVEL per configuration match the DX9
projects exactly, minus their legacy DirectX includes and FFP/VSPS switches.
That is a requirement, not tidiness: standard library types cross the DLL
boundary through Gl_api in setLights and getOtherAdapterRects, so
_ITERATOR_DEBUG_LEVEL and the CRT must agree with SwgClient and the other raster
DLLs. Tree-wide quirks in the Optimized configuration are matched rather than
diverged from, for the same reason.
MemoryManagerHook, SetupDll and PaddedVector are byte-identical copies of the
DX9 versions apart from the precompiled-header include, verified programmatically.
Both are load-bearing: MemoryManagerHook routes DLL allocations through the
engine MemoryManager so the engine can safely delete backend-allocated objects,
and SetupDll's delay-load hook redirecting DllExport.dll to the running module
is what lets the DLL load at all.
x64 only. No Win32 solution configuration rows are added, so a Win32 solution
build skips this project rather than failing.
Verified:
- standalone msbuild of Direct3d11.vcxproj with DXSDK_DIR cleared from the
environment produces gl11_r.dll; the project mentions no such variable
- deleting gl11_r.dll and running the full /t:SwgClient build rebuilds it,
proving the solution dependency edge (not a ProjectReference) is what
carries it
- full Release x64 build clean, all six artifacts x64
- dumpbin: exports GetApi and GetGlApiStructSize; imports only KERNEL32 and
the delay-loaded DllExport.dll -- no d3d9, d3dx9 or ddraw
Not yet verified: the in-client load path with rasterMajor=11. Deferred to the
commit that brings up the device, to avoid a modal failure dialog in an
unattended run.
The Headless raster stub has been in the tree for years with no project file,
so [ClientGraphics] rasterMajor=0 -- which SetupClientGraphics uses as the
console default -- has always named a DLL that did not exist. It builds now, as
gl00_{r,o,d}.dll.
Fixing that is the smaller half. The reason to do it before the DX11 backend
grows is that Gl_api gets a second implementation, so a change to the contract
is compile-checked against something other than the backend making the change.
Headless is also DX-free, which makes it the baseline the measurement harness
can attribute against.
Demonstrated rather than asserted: renaming the createTextureData slot in
Gl_dll.def made both implementations fail --
Headless.cpp(73,8): error C2039: 'createTextureData' is not a member of 'Gl_api'
Direct3d9.cpp(1068,11): error C2039: 'createTextureData' is not a member of 'Gl_api'
-- and reverting it returned all three raster projects to a clean build. That is
the property this commit buys, and it is what makes the eventual contract trim
verifiable instead of hopeful.
Authored fresh rather than cloned from a DX9 project. A clone would have linked
d3d9, d3dx9, ddraw and DxErr, pulled in odbc32 and a hardcoded libjpeg-turbo
path, and put the legacy DirectX SDK library directory first -- on a stub that
contains no DirectX at all, which is the include-order hazard this port is
supposed to be eliminating. It links winmm, delayimp and legacy_stdio_definitions
and nothing else. CRT flavour and DEBUG_LEVEL match the other raster DLLs and
SwgClient, which matters more here than elsewhere: Headless blanket-fills Gl_api
as void** over sizeof(Gl_api)/sizeof(void*), so its idea of the layout has to be
exactly the client's.
x64 only, so a Win32 solution build skips it rather than failing.
Verified: full Release x64 build clean with seven artifacts, all x64; dumpbin
shows gl00_r.dll exporting GetApi and GetGlApiStructSize and importing only
KERNEL32 and the delay-loaded DllExport.dll.
gl11 now creates a real device, shows the window and presents. Everything it has
not implemented yet says so by name.
Device. Created deliberately WITHOUT D3D11_CREATE_DEVICE_SINGLETHREADED: resource
creation is not confined to the main thread, because ShaderTemplateList registers
the 'sht' extension with the AsynchronousLoader and shader implementations load
on whatever thread got there first. D3D9 tolerated that only because its device
creation is internally serialised. Feature level 11_1 is preferred for
VSSetConstantBuffers1 with 11_0 as the floor and a FATAL below it; the immediate
context stays main-thread-only and asserts so on entry. A removed device is fatal
rather than recovered, and DXGetErrorString9 is replaced with a local formatter so
no DirectX 9 SDK is needed to explain an HRESULT.
Swap chain. Flip model, three buffers, borderless, MakeWindowAssociation with
NO_ALT_ENTER, and SetFullscreenState is never called -- exclusive fullscreen is
what the entire lost-device apparatus in the DX9 backend exists to survive. The
window work is here because it has to be: Os creates the game window hidden,
WS_POPUP, 640x480 at the origin and never shows it, so a backend that only makes
a swap chain renders correct frames into a window nobody sees. Width, height and
windowed are written back into Gl_install, since the engine builds its UI canvas
size, mouse clip rectangle and viewport bounds checks from those three fields and
nothing else reports them.
Several behaviours are copied because DX11's obvious equivalent is subtly wrong:
- clearViewport clears colour through ClearView with the viewport rectangle.
D3D9's Clear is bounded by viewport and scissor; ClearRenderTargetView is
not, and the heat compositing path clears one primitive's footprint inside a
buffer that already holds other primitives.
- beginScene rebinds the render target, because the flip model unbinds the back
buffer at Present and a draw into nothing is only a debug-layer warning.
- resize deliberately does not resize the OS window. DX9 stretches a smaller
back buffer into the unchanged client area, which is how a cut scene plays a
640x480 video full-window; DXGI_SCALING_STRETCH reproduces it.
- setWindowedMode calls the windowedModeChanged callback unconditionally. It is
the only channel by which Graphics learns the current mode, and DirectInput
is wired to Graphics::isWindowed.
- the clear colour's alpha is forced opaque and the swap chain ignores alpha:
every caller passes a PackedRgb whose alpha byte is zero, which would
composite the window transparent.
Unimplemented slots go through Direct3d11_Unimplemented, which counts each slot,
warns once naming it, DEBUG_FATALs, and reports the total at shutdown. The
resource factories are fatal instead: the engine stores what they return and
dereferences it later, so returning null converts a missing feature into an
access violation with nothing in the log. Fog, alpha fade, scissor and the rest
are therefore loud absences rather than empty bodies that look wired up -- which
is exactly how the prior attempt shipped a renderer whose fog constant was never
uploaded.
install MessageBoxes on failure before returning false. Graphics::install just
returns false, SetupClientGraphics returns false, and ClientMain then skips the
whole game and exits zero -- a silent exit indistinguishable from a crash.
Verified: all three configurations build; dumpbin shows gl11_r.dll importing
d3d11.dll, dxgi.dll, USER32, KERNEL32 and the delay-loaded DllExport, with no
d3d9, d3dx9, ddraw or dxerr; full Release x64 client build clean with all seven
artifacts x64. Building Optimized and Debug caught a missing include that Release
hid, because the DEBUG_FATAL referencing it compiles to nothing there -- the
second implementation and the multi-configuration build are already earning their
keep.
Not yet verified in the client: the runtime load path. Documented in
docs/dx11-standup-sequence.md along with why a cleared frame is not reachable
until the texture and shader factories exist -- SetupClientGraphics loads a
cubemap and preloads shader templates before frame one, so this build stops at
createTextureData with that slot named.
The scene now renders into an offscreen colour and depth-stencil pair, and the
back buffer is written exactly once per frame by a composite pass. One decision
that answers five separate problems the flip model creates: lockBackBuffer gets a
mappable surface where a FLIP_DISCARD back buffer cannot be mapped at all,
screenShot gets a surface that still exists after Present, colour correction gets
somewhere to happen where SetGammaRamp has no working DXGI equivalent, MSAA gets
an obvious resolve point that a multisampled flip-model swap chain does not
allow, and presentToWindow gets something to feed a per-window swap chain from.
This is also the commit that makes the parity work possible at all. Until
something can turn a frame into a file, "identical to DX9" is not a claim anyone
can check, and neither is its negation. The prior DX11 attempt implemented none
of screenShot, writeImage or lockBackBuffer, which is why every parity claim
about it -- including its own reports of success -- was unfalsifiable.
The colour correction curve is reproduced from DX9's arithmetic rather than
reinvented, and the detail that forces this is worth recording. DX9 builds a
256-entry 8-bit table where entry i is driven by i/256, NOT i/255
(Direct3d9.cpp:2080-2099). Verified numerically: at brightness = contrast =
gamma = 1, that table is floor(i * 255 / 256), so 255 of its 256 entries differ
from the identity -- 255 becomes 254, 128 becomes 127, 1 becomes 0. Consequences
that follow, all of them load-bearing:
- the skip test is on the SETTINGS being 1, never on the table being the
identity, because the table never is. There are two pixel shaders, a copy and
a correct, rather than one with a branch, so the copy path is provably free of
arithmetic on any pixel value.
- the curve is applied as a 256-entry lookup sampled POINT, not as a pow() per
pixel, because a float evaluation produces values DX9 cannot produce.
- the sampler is POINT, not LINEAR. A linear sampler on a one-to-one blit is
neither free nor a no-op at the edges.
One intended behaviour difference, stated rather than hidden: DX9 applies its
table to a screenshot only when NOT windowed (Direct3d9.cpp:2731-2733), because
SetGammaRamp does nothing on a windowed swap chain -- so windowed, neither its
screen nor its capture is corrected. Here correction lives in the composite and
is real in both modes, so captures include it in both. At identity settings, the
shipped default and what every parity capture pins, the two backends agree byte
for byte.
TGA is implemented through the same WriteTGA module the DX9 backend uses, so
identical pixels produce identical files. JPG and BMP need the JPEG encoder and
WIC and remain named absences rather than silent ones; the parity harness
captures TGA.
Verified: all three configurations build; full Release x64 client build clean;
the gamma table arithmetic checked against an independent reimplementation of
DX9's loop in float, including the 255-of-256 result quoted above.
Counters that exist in every configuration, including PRODUCTION, and a GPU timing pool. Neither exists today in any form. The DX9 backend's metrics class is wrapped in #ifdef _DEBUG from its first line to its last; its only report caller sits inside a #if 0 in the game loop; every NP_PROFILER_ macro compiles to nothing at PRODUCTION == 1; and there is not one CreateQuery call anywhere in the first-party tree. So no GPU-side number has ever been measured for this renderer, and "no performance degradation" has never been a claim anyone could test -- in either direction. That is what this commit starts to fix. The counters the port's gates are actually written against are the ones required to read zero: state-object, input-layout, constant-buffer and shader-compile creations inside a frame, plus dropped draws, texture-bake readbacks and blocking staging maps. Creating work inside the frame loop is the single most common way a DX11 port ends up slower than the DX9 code it replaced, and it is invisible without exactly these numbers. Binds are counted as calls AND misses, so a cache's hit rate is measured rather than assumed -- DX9's own vertex declaration cache is a permanent 100% miss because forceVertexDeclaration never assigns the shadow it compares against, which went unnoticed for twenty years for want of this pair of numbers. The report goes out through WARNING at shutdown rather than through a DebugFlags report routine. The routine is registered, but DebugFlags::callReportRoutines has exactly one call site in the game loop and it is inside a #if 0, so report routines never fire. A zero-invariant counter that is only visible in a build nobody measures is not instrumentation. GPU timing is triple-buffered and read three frames late with DONOTFLUSH, never waited on. A GetData that blocks the main thread, or that flushes the command buffer, changes the submission pattern being measured. Disjoint frames are dropped rather than averaged in: a frame spanning a clock-rate change has an elapsed count that does not convert to time. Three of the five _DEBUG-only Gl_api slots are now real, against the metrics. Graphics.cpp wraps three of them in NOT_NULL, which is why the prior DX11 attempt -- which never defined any of the five -- could not be loaded by a developer build at all. Points and lines report zero rather than being folded into triangles: the draw paths that would distinguish them do not exist, and a plausible number is worse than an obviously absent one. Scope note. The standup lists an engine-side always-on counter block, handed to the backend through a second out-of-band export, as a prerequisite here. It is deliberately not in this commit: that block has to be populated by the DX9 backend too or the baseline it feeds is not comparable, which makes it part of the Phase 0 instrumentation work on the DX9 side rather than part of bringing up DX11. These counters are useful without it -- they are what the zero-invariant gates read -- and they are wired to publish into it when it lands. Verified: all three configurations build; full Release x64 client build clean.
Immutable blend, depth-stencil, rasterizer and sampler objects, hash-consed and created outside the frame, plus a shadow of what is bound so a redundant bind costs a comparison rather than a driver call. Fill mode, cull mode and scissor are real from here, and supportsScissorRect can finally answer true. The translation tables are copies of the DX9 backend's, index for index, including the entries that look wrong -- because the index IS the contract. Shader effect assets store the integer, not the name, so whatever DX9's table does with an index is what that asset has meant for the whole life of the content. Compare is the case that matters. Direct3d9_ShaderImplementationData.cpp:38-39 maps index 5, named C_GreaterOrEqual in the engine's enum, to NOTEQUAL, and index 6, named C_NotEqual, to GREATEREQUAL. The two are swapped with respect to their names, and every pass authored against either has always got the swapped behaviour. The prior DX11 attempt un-swapped them, found stencil shadows regressed, and reverted -- keeping a wrong translation to mask a symptom rather than recognising the table as an ABI. verifyTables() now asserts the swap at install, so a future tidy-up that "corrects" it stops the client with an explanation instead of quietly changing what shipped assets mean. Two more traps written down where they bite. D3D9's stencil INCR/DECR wrap while its INCRSAT/DECRSAT saturate; D3D11 names them the other way round, so that mapping reads like a rename and is not one -- and the shadow volume counting passes depend on wrapping. And GlCullMode names the winding that is CULLED, not the front face, so with FrontCounterClockwise FALSE, GCM_clockwise is CULL_FRONT; inverting it makes the world invisible rather than visibly wrong. Nothing is created during a frame. The twelve fill x cull x scissor rasterizer combinations are pre-created at install, because those are engine state that changes mid-frame -- Graphics issues setCullMode in save/restore pairs around every shadow volume and every ribbon. Asset-driven state is built where the descriptor is already known. Every creation increments the metrics counter the zero-invariant gate reads. Each cache asserts a capacity rather than growing quietly. Exceeding one means something dynamic has leaked into a key, which is a design defect and not a number to raise. Keys are packed descriptors compared by value: the prior attempt built a std::string from raw descriptor bytes on every lookup -- a heap allocation, a memcpy of up to 264 bytes and a tree walk -- on a path reached from setCullMode. Create failure is fatal, never cached as null. A null depth-stencil state silently reverts the device to depth testing and writing enabled with LESS, which would break every depth-disabled sky pass and every z-fail stencil shadow and would look exactly like a content bug. The composite binds its own state directly and now invalidates the shadow afterwards. A stale shadow makes the next real bind look redundant and get skipped, which is the one way a redundancy cache produces a wrong image instead of merely a slow one. Verified: all three configurations build; full Release x64 client build clean; and the four translation tables were extracted from both backends and compared programmatically -- Compare 8, Blend 11, BlendOperation 5, StencilOperation 8, all corresponding index for index, with the swap present at 5 and 6. Note on the standup's ask for a unit test on the swap: there is no test harness in this tree, and adding one is not this commit's job. verifyTables() is the runtime equivalent and is strictly harder to skip, since it runs at every install rather than only when someone runs the suite.
The three register enumerations move from the Direct3d9 backend to clientGraphics/ShaderConstantRegisters.h, values byte-identical. The DX9 headers become forwards so nothing has to change at once, and ShaderBuilder now includes the promoted header directly instead of reaching across a layering boundary by relative path. These are an asset format, not a backend's private numbering. Every .vsh and .psh in the shipped TRE set was compiled against them, and their correspondence to a D3D constant buffer is exact: a constant declared register(cN) lands at byte offset 16 * N in the auto-generated $Globals buffer, so these numbers ARE the byte offsets divided by sixteen. That is what the DX11 reflection guard will assert against, and it cannot assert against numbers that live inside the backend being replaced. Two problems this fixes rather than just tidies. ShaderBuilder is the only tool that validates a .vsh and writes a .psh, and it included all three headers by relative path into the Direct3d9 directory. Deleting that directory at the end of the port would have broken the tool with no compile-time warning anywhere until someone tried to build it -- and the corpus conversion depends on that tool. Direct3d9_VertexShaderVertexRegisters.h included <d3d9.h> for nothing at all: the file contains no D3D types. Left in place it would have dragged the DirectX 9 header into clientGraphics on promotion. Dropped. One addition beyond relocation. VSCR_MAX is 68, but the shipped registers.inc aliases four scalars onto c95 -- c0_0, c0_5, c1_0 and cLog2e map to c95.x through c95.w -- and nine assembly modules consume them. A constant buffer sized from VSCR_MAX alone leaves every 0.5 bias and every 1.0 constant in those programs reading zero. So c95 is now named, and the buffer row count is derived from it rather than being a literal in a backend. Verified: Direct3d9 builds clean against the forwards, and the full Release x64 client build is clean with all seven artifacts. Not verified, and not verifiable: ShaderBuilder does not build. It has a Project() entry in swg.sln with no ProjectConfigurationPlatforms rows at all, so no solution configuration builds it, and its archive dependency fails on a header that exists at a path the project does not look in. Both predate this change and neither is touched by it -- the edit here is three include lines. The tool needs its own repair before the corpus conversion can rely on it, which is worth knowing now rather than at the point of needing it.
One place where HLSL becomes bytecode, and one place that checks the compiler did what the assets assume. Every shader goes through the same flags, because the numerical-equivalence harness is meaningless if two programs were built differently. Two of those flags are load-bearing far beyond their appearance: ENABLE_BACKWARDS_COMPATIBILITY is mandatory. Without it the corpus does not compile at all -- tex2D is a hard error, sampler declarations are rejected, the legacy POSITION0 output is not mapped to SV_Position. With it, register(cN) lands in $Globals at byte offset 16 * N. PACK_MATRIX_ROW_MAJOR must never be passed. The engine uploads matrix bytes in the order D3D9 wanted, and HLSL's default column-major packing reads those bytes correctly. Forcing row-major transposes every matrix in the corpus at once. Targets are pinned at vs_4_0/ps_4_0. Nothing live needs SM5 -- no tessellation, no compute, no UAVs, highest sampler in use is s4 -- and one profile keeps every comparison against DX9 valid. That 16 * N property is the foundation the whole port stands on: it is why nothing has to be renumbered and why the engine's register-indexed constant setters keep working. It is a property of a compiler flag, not of the language, so it is now verified per shader rather than trusted, and it was verified empirically before being relied on. docs/shader-abi holds the probe, its output on the Windows SDK 10.0.26100.0 compiler, and instructions to reproduce it: c0 -> 0 c4 -> 64 [unused] c8 -> 128 [unused] c9 -> 144 [unused] c10 -> 160 c44 -> 704 [unused] c48 -> 768 [unused] c95 -> 1520 All at 16 * N, and the unreferenced ones keep their offsets -- which is what makes a flat register-file shadow work at all. c95 at 1520 plus 16 is 1536, i.e. 96 rows, matching the VSCR_CBUFFER_ROWS the register header derives. The same probe settled the sampler question, which is a genuine trap. A sampler declared and never sampled is eliminated, and the surviving textures pack densely rather than keeping their sampler's number: mainSampler s0 -> t0 detailSampler s2 -> t1 thirdSampler s3 -> t2 So binding on the assumption that t == s puts textures on the wrong samplers with no error from anything. Reflection reads the real pairing back instead. The prior attempt hit exactly this and its shipped binary still carries the diagnostics from working it out. The guard also guards itself. A check that recognises no constant names would pass every shader forever, which is indistinguishable from working, so the first shader that matches nothing says so loudly and names the file whose table is wrong. The include handler resolves out of the TRE set with DX9's "../../" strip, which is not cosmetic: shader sources carry paths written for a command-line compiler run two directories below the asset root. Includes are cached, with an explicit flush, because the corpus shares a few .inc files across hundreds of programs. DX9 caches only when the engine owns the window -- a way to stop tools serving stale content mid-edit -- so the flush exists rather than making the game path pay for the tools' problem. Compile failure is fatal in a developer build and a warning in PRODUCTION. A warning-only failure yields a null shader and a draw that renders nothing or renders with whatever was bound last, and the wrong image appears far from the cause. Warnings on SUCCESSFUL compiles are logged rather than released unread -- implicit truncation and register packing are precisely what a parity investigation needs. Deferred deliberately: the SHA-256 keyed disk cache and the offline baker. Both key on the content of a converted shader corpus that does not exist yet, so there is nothing to cache and nothing to bake. They land with the corpus. Also not carried: DX9 injects "#define point _pt_lights" into every include, because point is an HLSL reserved word colliding with a LightData field name. The asset conversion renames that field instead, so this compiler does no textual surgery on asset text at all. Verified: all three configurations build; full Release x64 client build clean; the ABI probe reproduced above.
The setters keep D3D9's signature -- a register index, a pointer, a count of
float4 rows -- because that is what lets the light manager and static shader data
port with their upload code unchanged. Everything behind them is different.
D3D9's setters had no shadow, no comparison and no coalescing: every call went
straight to SetVertexShaderConstantF and the driver's register file made that
cheap. D3D11 has no register file, so the same call pattern issued naively becomes
the dominant per-draw cost of the renderer. Constants are now shadowed, compared
before being marked dirty, and flushed at most once per stage per draw over the
dirty span only.
The arithmetic that forces the layout: the two per-draw matrices occupy c0..c7, so
leaving them in the register file dirties its front on every draw and a per-draw
flush moves on the order of 1.5 KB where D3D9 moved 128 bytes. At a few thousand
draws a frame that is megabytes against DX9's hundreds of kilobytes. They go to a
separate buffer at b3 instead, taken from a ring, so a draw costs an offset rather
than an upload. The matching asset change -- dropping the register(c0) and
register(c4) annotations and declaring those matrices in a b3 cbuffer -- lands with
the corpus conversion; until then no shader reads b3, which shows up as
untransformed geometry rather than as silently wrong constants.
The ring needs TWO capabilities, both optional on 11_0 hardware despite living on
the 11_1 interface: ConstantBufferOffsetting to bind a sub-range, and
MapNoOverwriteOnDynamicConstantBuffer to append without renaming. The device
queries both, and the rotating-DISCARD fallback says out loud which one was
missing rather than being quietly slower.
Packing is copied from DX9 and verified rather than transcribed and hoped for.
Every value below was checked by reimplementing DX9's expression in float and
comparing:
c9 viewportData { 2/w, -2/h, -1 - 2x/w, 1 + 2y/h } -- bit-identical to DX9
across six viewport cases including offset and degenerate ones. The Y scale
is negative and the biases asymmetric because screen Y grows downward.
Without this row every 2D vertex collapses to clip zero.
c10 fog { 0, 0, density, density*density }, written ONLY when fog is
being enabled, because that is what DX9 does -- disabling clears a render
state and leaves the density live, so zeroing the row here would change what
every scoped fog disable means.
c48 currentTime { time, 0, 0, 0 }, once per frame before the scene, monotonic
and never wrapped.
c95 literals { 0, 0.5, 1, 1/log(2) } = log2(e) to float precision, which
the fog exponent needs.
c49..c51 unit vectors, with w = ZERO. DX9 uploads a PaddedVector whose pad is
initialised to 0.0f; writing 1.0f there would be invisible and wrong.
Three corrections to the plan, from reading the DX9 sources rather than the
summary of them:
The plan said to drop the extendedLightData upload as having no live references.
It IS uploaded -- c60..c63, four rows, from a real struct. Not dropped.
The plan said the register(bN) light-enable booleans are never written, and that
is right: SetVertexShaderConstantB has zero callers tree-wide, so they hold
D3D9's default of FALSE for the life of the process. Which means the prior DX11
attempt's textual surgery rewriting those declarations to "= true" INVERTED the
shipped behaviour of every shader that reads one.
Only four rows are re-established after a device rebuild -- c95 and the three
unit vectors -- and nothing else, because everything else is rewritten by the
ordinary drawing flow. That set is reproduced exactly rather than reseeding
everything, so both backends agree about what is stale.
setViewport, setFog, update, and the vertex and pixel user constants are real from
here and no longer accounted stubs.
Verified: all three configurations build; full Release x64 client build clean;
c9 packing bit-identical to DX9 across six cases; c95's log2(e) matches to float
precision.
Four corrections from implementing the constant buffers, kept with the plan rather than only in a commit message, because the first one is a trap for whoever writes the light manager next. extendedLightData is uploaded -- four rows at c60..c63 from a real struct with a computed row count. The plan says to drop it as dead. Following that would have deleted a live upload. The bool registers are never written, which confirms the prior attempt's rewrite of those declarations to "= true" inverted rather than neutralised them. Only four constant rows are re-established after a device rebuild, and reproducing exactly that set is what keeps the two backends agreeing about what is stale. And the per-draw constant ring depends on two separately optional capabilities, not the one the plan names.
First half of the buffer work: the format-to-byte-layout map and the two static buffer classes. The dynamic rings, input layouts and draw dispatch follow; these two factories stop being fatal stubs, so the engine gets further before stopping. Both static classes keep a CPU shadow for the buffer's whole lifetime, and that is forced by the engine rather than chosen. StaticVertexBuffer::lockReadOnly and StaticIndexBuffer::lockReadOnly exist and are used at runtime to read back geometry loaded long ago, so the CPU must be able to see current contents on demand. D3D9 got that free from D3DPOOL_MANAGED, which keeps a system-memory copy the runtime hands back on any Lock; D3D11 has no managed pool and a USAGE_DEFAULT buffer cannot be mapped at all. The shadow is not new memory -- D3DPOOL_MANAGED was already paying for it -- it has only become visible. USAGE_DEFAULT rather than IMMUTABLE, despite most of these being written once. IMMUTABLE is impossible for the rest: the engine relocks static buffers to modify geometry, and an immutable resource would have to be recreated, which is an allocation inside a frame. Unlike the prior attempt, unlock uploads only when the lock was writable, so a read costs nothing. Sort keys are monotonic integers, never pointers. DX9 computes reinterpret_cast<int>(the D3D buffer pointer), which cannot compile on x64 without truncating -- and a truncated pointer is not merely ugly, it is not a stable ordering: two live buffers colliding on the low 32 bits silently merge two batches in the sorter. Static keys start at 1 so they can never compare equal to the dynamic buffers' shared key of 0. The descriptor map is a straight port -- the computation contains no D3D at all -- and it is a std::map on purpose. The engine does not copy the descriptor it is handed: StaticVertexBuffer stores its ADDRESS and every vertex iterator dereferences that pointer on every component access, so the entries must never move. A vertex size of zero or above 127 is now fatal rather than silently wrapping, since the descriptor's fields are int8 and a negative stride produces garbage geometry with nothing reported. Engine side: the five buffer headers granted friendship to `class Direct3d11` but not to the per-buffer data classes, where DX9 grants both. Added, mirroring DX9 exactly. Left alone: DynamicIndexBuffer.h grants Direct3d9_StaticIndexBufferData -- the STATIC class -- on the DYNAMIC buffer, which is a copy-paste error in the original. DX9 evidently does not need that access, and changing what an existing backend can reach is not this commit's business. Two corrections to the plan, both from reading the engine rather than the summary: The plan says to resize the dynamic rings from 2 MB to 16 MB and 64 KB to 4 MB. That cannot ride in during a parity port. ShadowVolume::install derives its batch size from the ring: getNumberOfLockableDynamicVertices(true) returns the WHOLE ring divided by the vertex size, and ShadowVolume rounds it down to a multiple of four and uses it to bound every shadow batch. Growing the ring therefore changes shadow batching, which changes draw counts, which is precisely what the geometry-integrity gate measures. It also risks the 16-bit index ceiling: every index in the engine is an `unsigned short`, so a batch above 65535 vertices is unaddressable. DX9 sizes the ring from video memory in tiers of 256 KB to 2 MB; that policy is reproduced instead, and resizing becomes a performance change with its own measurement. The plan also has the buffer-touching Gl_api slots as free functions. They cannot be: only `class Direct3d11` is a friend, and free functions in a namespace have no friend access. Those slots move into the class when the draw path lands. Verified: all three configurations build; the DX9 backend still builds against the changed engine headers; full Release x64 client build clean with seven artifacts.
Second half of the buffer work. Both rings are process-lifetime statics created at install, and the per-buffer objects own nothing at all -- their destructors release nothing, deliberately. That is not a style choice. Eleven engine call sites construct a dynamic vertex or index buffer as a STACK LOCAL, lock it, fill it, unlock it, bind it, and let it die before the draw is issued: ShadowVolume, ClientTerrainSorter, IndexedTriangleListShaderPrimitive, NebulaShellShaderPrimitive, ClientServerProceduralTerrainAppearance, ShaderAppearance, ShadowBlobManager, ReticleManager, DustAppearance, GameCamera and OverheadMap. Anything these objects owned would be freed while the GPU was still about to read it, so every parameter the draw needs gets snapshotted into device state when the buffer is bound instead. lock() returns a pointer into a system-memory mirror and unlock() copies the slice into the ring. Three facts force that rather than mapping directly: The UI holds a lock open across unrelated engine work -- CuiLayerRenderer locks the shared ring, accumulates across many calls including a shader fetch, and unlocks much later. Two dynamic vertex buffers can be locked simultaneously in a release build. CuiLayer_EngineCanvas builds its own while the renderer's batch is still open; the engine DEBUG_FATALs it and ships past it. Dynamic buffers are provably write-only -- the iterator has no getters and the header says so -- so nothing needs to read back what a lock wrote. It also makes unlock(int) trivially correct: copy exactly the vertices claimed, which the UI depends on since it locks all remaining space and returns most of it. Details that are silent when wrong, so they are stated where they are done: Map returns the start of the whole resource where D3D9's Lock returned the start of the locked range. Omitting the offset makes every dynamic draw after the first render the first mesh's geometry. The cursor advances on unlock by the count actually used, never by the count locked, and the round-up-the-cursor helper is shared between lock() and getNumberOfLockableDynamicVertices() -- because the UI asks how many fit without discarding and then locks exactly that many. A one-vertex over-report there discards the ring and destroys geometry already bound for a pending draw. The discard condition is exactly DX9's and nothing is added to it. The window where a spurious discard would corrupt a bound-but-undrawn buffer is real: the sorter draws once per shader pass with other code building its own dynamic buffers in between. Renaming happens at lock time, where DX9 renames, not at unlock. Deferring it would reorder the rename against another buffer's writes when two are locked at once. Ring sizes reproduce DX9's video-memory tiers, 256 KB to 2 MB, and are fixed at install. Not resized, for the reason recorded in the previous commit, and not resized lazily either: a mid-frame reallocation resets the cursor and invalidates the offsets of every buffer already bound and awaiting a draw. The prior DX11 attempt grows its ring inside lock(). Not carried: DX9 tests the mapped pointer with IsBadWritePtr and, if it looks bad, redirects the write to a shared heap block no GPU resource sees -- while still unlocking and still advancing the cursor, so the draw proceeds against stale contents. It is a 2002 driver workaround that converts a fault into silent corruption. Replaced with a fatal on a failed map. Verified: all three configurations build; full Release x64 client build clean with seven artifacts.
…g holder Round-3 numbers (2026-08-16): 30 of 34 callback/lock collisions recovered within 84us -- the drip was photo-finish short holds, as designed. The 4 that survived a full 1ms spin are the other case: a holder PREEMPTED mid-hold, which needs our timeslice to finish -- spinning against it is anti-productive. The wait now runs two phases: _mm_pause for the first 100us (running holder), std::this_thread::yield after (preempted holder), budget 3ms of the ~10ms block period.
…ats PrintScreen The in-game screenshot key produced no file: Windows 11 Snipping Tool registers a low-level keyboard hook on PrintScreen and consumes the key before DirectInput sees it in a foreground non-exclusive acquire, so on a stock Windows 11 install the DIK_SYSRQ bind can never fire. The render- side path (Direct3d11_ImageWriter, ScreenShotHelper, ./screenshots/) is fine -- the event just never arrives. F12 now also triggers the screenshot function. The user can alternatively disable the OS hook (Settings -> Accessibility -> Keyboard) to get PrintScreen back; F12 works without changing OS settings.
…tiates, WritePixels does not convert The first live F12 screenshot came out stretched 4/3 horizontally with period-4 vertical striping and scrambled colour. That triple signature is 32bpp rows interpreted as 24bpp: every 3 source pixels re-sliced into 4, with the alpha byte rotating through the colour channels (measured luminance cycle 146/157/201/173 -- exact period 4, the 12-byte alignment cycle of 4-byte pixels read 3 bytes at a time). The cause is a wrong assumption about WIC, stated in the old comment: "asking for a format the container cannot store makes the encoder convert rather than fail." It does not. SetPixelFormat NEGOTIATES -- it rewrites the requested format to the closest one the container supports (JPEG has no 32bpp encoding, so 32bppBGRA comes back 24bppBGR) and returns success, and WritePixels then interprets the caller's raw bytes as the negotiated format, converting nothing. Fix: wrap the pixels in an IWICBitmap, run them through an IWICFormatConverter to whatever SetPixelFormat negotiated, and hand that to WriteSource. Works for every container uniformly; TGA (WriteTga, no WIC) was never affected -- which is why the debug-frame harness never saw this.
…- the foyer skybox hole The classic interior see-through (standing in the Mos Eisley cantina foyer, the next room renders as raw skybox through the archway) reproduced three times in normal play and was capture-convicted 2026-08-16: pixel history on a void pixel shows clear -> two skybox layers at depth 1.0 -> nothing. No interior fragment was ever submitted, not even a failed one, so the adjacent cell was culled at the dPVS visibility layer -- the identical signature diagnosed and fixed in the sibling tree in July 2026 (CONSULT-64 through -66). This ports those six fixes: 1. dpvsDatabase: update dirty nodes BEFORE the view-frustum test -- the stale-bounds order VF-culled whole nodes on the frame their bounds changed (whole-cell flicker). 2. FreeChaseCamera: derive the camera's cell from its own FINAL position by walking the player->camera segment through the portal graph. The chase camera lags meters behind the player, and the per-frame force-copy of the player's cell rooted dPVS visibility in a cell the eye was not in (movement see-through). The camera is never moved, only re-tagged -- no camera-feel change. 3. DoorObject: meshless doors (open archways) must never close their portals -- a closed portal is invisible to traversal. 4. dpvsImpMeshModel: portal backface EPSILON 0.0 -> -0.05 (the original Umbra comment documents negative leeway as intended) -- a camera eye sitting ON a portal plane flip-flopped the test on float noise. 5. FreeChaseCamera: knife-edge hysteresis -- a portal crossing within 20cm of the eye is not taken, keeping the player-side cell. 6. dpvsDatabase splitInstance: portals never take the exact triangle-vs- AABB child refinement (isStatic && !isPortal) -- a zero-thickness portal quad lying exactly on a BSP split plane failed the zero-epsilon test and silently lost its instance in the child holding most of it, surviving only in a sliver leaf whose clamped bounds excluded it; traversal then VF-culled that node and the portal was never enumerated (the stationary foyer capture's exact shape: tested:0, portal-frame-shaped skybox). Also carried along (same files, self-contained): the dPVS portal-reject counters and query exports the sibling tree used to convict these -- inert here until something reads them, kept so the two dpvs trees stay comparable. Deliberately NOT included: PortalProperty::cellLoaded stays on this tree's asChildObject=true. These fixes operate at the culling layer and are independent of cell ownership semantics; changing one variable at a time keeps the foyer verify attributable. Verify: stand in the cantina foyer looking at the main room (Mos Eisley 3456,5,-4844 -- the capture spot). Pre-fix this reproduced ~1-in-7 logins and healed on eye translation.
…n only grow The shader cache works exactly as designed -- every session logs "198 program(s) baked" and ~185 hits -- but 4-7 programs miss EVERY session (gradient_sky, cloudlayer, occasionally a_scroll_rgb1_a2/a_splitalpha: ~30-48ms of D3DCompile and one ~28ms hitch frame each run), because the original bake session never compiled them and nothing since could add them: a bake run writes a manifest listing only what THAT session compiled, so filling a four-program gap required a session that also re-covered all 198 already-baked programs, or the re-bake would silently SHRINK the cache. The bake branch of install() now seeds ms_baked from the existing manifest before the session starts, so writeManifest emits the union -- a re-bake can only add. Seeding applies the same validation a using run applies (version, compile flags, every recorded include hash against the live data); a manifest failing ANY check is not carried forward, since a stale entry must never be re-listed as current. Seeded programs' blobs already exist on disk and store() skips keys already present, so the run writes only what is genuinely new. The manifest's include list becomes the union of includes served this run and carried lines the seed validated -- carried programs stay covered by the install-time include check even when this run never served their includes. The bake-does-not-use-the-cache rule is unchanged, and so is its rationale: baking still compiles everything it encounters.
…-- the name is the remainder of the line, and the backend's own install-time programs contain spaces
…ded value PortalProperty::cellLoaded had been flipped to attachToObject_p(..., true), reasoning that a cell whose SceneEndBaselines arrives before its building's is skipped by handleEndBaselines' containedBy->isInWorld() check and "nothing later picks them up". Something does, in this very tree: when the BUILDING is added, Object::addToWorld runs the property loop, PortalProperty::addToWorld ends in Container::addToWorld, and that walks the container's CONTENTS -- which the cells are, via containment, independent of the child-attach flag -- and adds every exposed cell. That is the shipped design, it is why retail ran false for two decades of multiplayer, and the sibling tree runs false through byte-identical Object.cpp/Container.cpp live daily, same server, same SceneCreate flow. The flip was not free. asChildObject=true made every network-owned CellObject: - parent-destructor-deleted AND network-deleted: ~Object deletes child objects, and the network layer separately owns and deletes CellObjects it created -- a latent double-delete on building teardown; and - parent-alter-driven instead of world-driven, changing when cell alters run relative to everything else in the frame. If the original flip was compensating for a real missing-interior symptom, the likelier cures were the other two members of the same rework, which both STAY: CellObject::endBaselines' on-demand buildout container load, and the WorldSnapshot POB-CRC proceed-by-default. Verify: log in at a city (server cells + buildout buildings), enter a server-streamed POB -- interiors present, no reload needed. The failure mode this reverts toward, if the old rationale had been right, would be unmissable: buildings with no insides.
…loading frames WorldSnapshot::load parsed the whole .ws node tree, every buildout area table, and the sphere-tree build synchronously inside the GroundScene constructor -- a multi-second single frame, spent BEFORE the loading screen was even enabled. In the sibling tree this was watchdog-convicted at 3.1s and phased in July (CONSULT-60); this is that port. - WorldSnapshotReaderWriter grows beginIncrementalLoad/stepIncrementalLoad: each step parses whole top-level node subtrees under a time budget (a subtree is the only safe suspend point -- Node::load recurses children internally), inserting each subtree into the networkId map as it lands. The synchronous load(filename) path is untouched for server/tools. - WorldSnapshot::load becomes a cheap prologue: unload, open the .ws, begin the incremental parse, and load the buildout-area LIST ( GroundScene::init reads it right after postload -- only the per-area object tables are deferred). The heavy phases -- wsNodes -> buildout (one area per step) -> sphereTree (4096 nodes per batch) -- run in loadStep(), pumped from GroundScene::updateLoading under [ClientGame] worldSnapshotParseBudgetMs (default 40; <= 0 restores the old fully-synchronous load). - donePreloading()/getLoadingPercent() are parse-aware (mid-parse the preload counters are 0/0 and would otherwise read as done), so the loading screen holds until the parse completes. - Exactness valves: any consumer that needs complete snapshot data mid-parse (isClientCached / loadIfClientCached on a map miss, addObject, moveObject, removeObject, detailLevelChanged, findClosestCellIdFromWorldPosition) finishes the remaining parse synchronously -- worst case degrades to the old cost, never to a wrong answer. update() is simply inert until the parse is done (the sphere tree is only populated in the final phase). - unload() cancels an in-flight parse (quit during loading, scene->scene). - The template-preload callback slice drops 1s -> 50ms: a full second of synchronous preloading per loading-screen frame starves the audio mixer's queue; 50ms slices do the same total work across more frames and keep music fed (CONSULT-61 in the sibling tree). Verified in the sibling tree across six weeks of daily play (landed 2026-07-04, "zone-in buttery smooth"); its WorldSnapshot has since grown editor surface this tree does not carry, so this is a hand-port onto this tree's file, not a copy. Verify here: zone in from character select and through a cantina door -- the loading screen should animate smoothly (no multi-second freeze before it appears), music should not hitch during loading, and the world should be complete on entry. [ClientGame] worldSnapshotParseBudgetMs=0 is the kill switch.
…obe-storm elimination Nearly every TreeFile::open resolves inside a TRE/TOC, but loose SearchPath nodes sit ABOVE the TOCs in priority (stage overrides must keep winning), so every open first paid one real CreateFileA/GetFileAttributes kernel round-trip MISS per loose search path. Two layers, ported together from the sibling tree where their A/B measured a 98.6% reduction in real filesystem probes (78,623 -> 1,083 over identical sessions, zero correctness delta): - Negative cache (per node): a fixed-up name that missed once is answered from an in-memory set with no syscall. Misses ONLY -- an existing file never enters the set, so override-wins semantics are unchanged. [SharedFile] searchPathNegativeCache (default true). - File manifest (per node): the negative cache only helps names probed BEFORE; a zone preload or novel asset still paid one real MISS per loose path per NEW name (sampler-convicted: a 643ms space-preload exists() storm). On first probe the node enumerates its directory tree ONCE (names normalized to the fixUpFileName convention) and every later probe answers absent-from-manifest with ZERO syscalls, first-touch included. The negative cache stays underneath as the fallback and self-heals the stale-positive case (file deleted mid-session: the one real probe misses and caches). [SharedFile] searchPathFileManifest (default true). The dangerous direction is a false NEGATIVE, and none is created: TreeFile::forgetMissingFile (call it after WRITING a loose file mid-session) clears the miss cache and inserts into a built manifest on every node, so a freshly written loose file stays visible without a restart. A loose file added mid-session by anything that does not call it stays invisible for already-manifested nodes until restart -- the same caveat both keys' kill switches lift. A/B telemetry: per-node [treefile.probe] REPORT_LOG line (realProbes / manifestFiles / manifestSkips / negCacheSkips) at exit, ExitChain-registered at priority 100 so it runs before the log sink is torn down (equal-priority entries run LIFO, and the log's teardown otherwise precedes TreeFile's). A/B recipe: same binary, one session with searchPathFileManifest=false, one with the default, same route -- realProbes is the number this exists to shrink.
… zone-in music transitions stop snapping Three fixes for the loading-screen -> live-scene audio handoff, ported from the sibling tree where each was ear-verified. The reporter's symptom here: "an attenuation during zone-in that ends abruptly" -- which is exactly the first one. - The duck-release snap (Audio.cpp): the engine ducks background music during zone-in, and the music multiplied by s_globalAudioFadeVolume only while its count-gate said so. When the duck released at load-end, the music instantly inherited the STALE ducked-to-zero global fade and then crawled back up -- a deterministic snap at every loading-screen -> scene handoff. Background music now has its OWN fade volume, ramped continuously toward the same targets, so a gate flip can never step the volume. - The blocking fade pump (SwgCuiManager.cpp): the scene-change listener pumped the title-music fade-out to completion with a blocking 1s Audio::alter/Sleep loop -- a ~1.4s main-thread mega-stall per zone-in. The budgeted-load work (async-loader budget, phased world-snapshot parse, terrain warm-up budget) removed the starvation that loop guarded against, so the fade now rides the main loop's per-frame Audio::alter through the loading screen; the loading screen also appears ~1s sooner. The fade-completes-THEN-new-track sequencing the pump silently provided is preserved explicitly: CuiManager::isMusicPlaying() (true through the stopMusic fade window) and GameMusicManager deferring its track selection while the UI music is audible -- deferred frames consume no state. Old behavior: [ClientUserInterface] blockingSceneChangeMusicFade=true. - The mid-load sunrise (GameMusicManager.cpp): the server's time-of-day sync lands seconds into the (now-fast) load and flips isDay() -- treating that as a real sunrise/sunset killed the planet first-play theme mid-load. Day flips inside the first 15s of a scene latch silently instead of firing E_sunrise/E_sunset.
… buildable and rebuild the vendored lib The eb1b260 port changed dpvs sources that nothing compiles: the exe links the prebuilt deps/x64/lib/dpvs.lib vendored in the original squash, and the checked-in dpvs.vcxproj cannot produce it (Release|x64 is ConfigurationType Utility -- a VC60-upgrade artifact that emits nothing). The ported dpvs-side fixes for the foyer skybox hole were therefore dead source. The hole reproduced in the field today at the exact capture spot (3458,5,-4846) with the ported exe -- on schedule for the ~1-in-7 unfixed rate ("five clean logins" was a 46% coin flip, not a soak pass). - dpvs.vcxproj: replaced with the sibling tree's working project -- v145, StaticLibrary on x64, same 70 sources, importing the committed src/build/win32/x64-platform.props compile block (added here). - dpvsImpObject.cpp: MatrixCache::Entry was 68 bytes on x64 (8-byte m_owner plus the 32-bit-era 2-slot pad), failing the sizeof==64 CT assert that pins the entry stride the scratchpad region is sized for. One pad slot under _M_X64 restores 64. This tree's sources could never have produced the vendored lib as committed. - dpvsRecursionSolver.cpp: the transition hash cast pointers straight to int (C4311, promoted to error by the props guardrail); route through UPTR, low-32 hash intent unchanged. - deps/x64/lib/dpvs.lib: rebuilt from THIS tree's sources -- the first vendored lib that actually contains the six portal fixes (marker: the g_swgDpvsPortalRejects export is now present in the lib). DoorObject.cpp needed no change but was also stale in the binary: the wholesale-copied port files kept their July source mtimes (Copy-Item preserves them), so MSBuild had skipped recompiling it over an 8/15 obj. Touched and rebuilt; the same trap is why the dpvs staleness went unseen. Verify: the foyer capture spot over the coming logins; given the pre-fix odds, ~7+ clean logins is the first real signal.
…he stock-faithful gl05/06/07 stack
The agreed division ("D3D9 is ours -- i stopped messing with 9"): this
tree's Direct3d9 line, including the Direct3d9_LightManager compensation
stack, is replaced by the sibling tree's stock-faithful renderer. All
three plugins (gl05/gl06 FFP/gl07 VS-PS) build from the one source dir
as before. What the replacement brings, beyond stock fidelity:
- x64 is D3DX-FREE: shader compilation goes through d3dcompiler_47
(D3DCompile) with an assembly->HLSL rewrite stage for the pre-SM4
sources D3DX chokes on (this tree's known ps_1_x failure), plus the
gl11-style compiled-bytecode ShaderCache (new files
Direct3d9_ShaderCache.*, Direct3d9_HlslRewrite.*). No d3dx9 link on
x64; Win32 configs keep it.
- D3DCREATE_FPU_PRESERVE on device create (32-bit x87 precision clamp
was a proven see-through-wall root cause).
- The Compare[] GreaterOrEqual<->NotEqual swap and the rest of the
sibling tree's D3D9 fix history, daily-verified there for months.
Boundary kept renderer-scoped: the sibling tree's two Gl_api tail slots
(setFrameCallback/setResizeCallback -- an injected-overlay contract this
repo has no consumer for) are NOT ported; the two accept-and-ignore rows
were stripped from Direct3d9.cpp rather than growing this tree's
Gl_dll.def. sizeof(Gl_api) is the load gate and stays exactly as this
tree's engine expects.
Two clientGraphics x64 sort-key fixes came along as prerequisites (the
new sources compile under /we4311, which caught them): ShaderEffect.h
getShaderImplementationSortKey and StaticShader.cpp
getShaderTemplateSortKey both reinterpret_cast'ed a 64-bit pointer to
int (high-32 truncation, collision risk in the primitive sorter). Both
now hash the full pointer through std::hash<uintptr_t> -- inline body /
.cpp body changes only, int signatures unchanged, no ABI cascade.
Build integration: the three vcxprojs are the sibling tree's (v145, the
committed x64-platform.props block, d3dcompiler+version link, the new
files); jpeg62-x64.lib vendored (import lib -- gl05 needs jpeg62.dll
staged next to it, same libjpeg 6b headers this tree already carries).
Verify: rasterMajor=5 boots to ground on x64; shader compiles appear in
the log as d3dcompiler paths (no D3DX); rasterMajor=6/7 boot; cantina
walls solid (FPU_PRESERVE class); screenshots still write (libjpeg).
…ve's blind-PEXE path was a boot crash on this tree's corpus The replacement renderer created every pixel shader straight from the PEXE bytecode chunk. This tree's shader corpus (the asm2hlsl set) carries the real program as HLSL text in PSRC and only a STUB in PEXE (pixel_program/vertex_color.psh: 4 bytes) -- the DX11 pipeline compiles from source and never needed bytecode. CreatePixelShader on such a stub is an AV inside the d3d9 runtime's shader validator (it walks the token stream off the end of the allocation): crash-dump-confirmed at Direct3DShaderValidatorCreate9+0x9cd loading shader/2d_vertexcolor.sht, identically on both datasets, first pixel shader of the boot. The pre-wave tree already knew this: its pixel ctor recompiled //hlsl PSRC via D3DXCompileShader and used PEXE only for plain-asm programs. That design was load-bearing corpus infrastructure, not compensation -- restored here, with two changes: - The compiler is D3DCompile (d3dcompiler_47, the DLL the vertex path already uses), keeping the x64 line D3DX-free. Include handler ported ID3DXInclude -> ID3DInclude, byte-identical semantics. - Compiled bytecode round-trips through Direct3d9_ShaderCache (the same warm-start cache the vertex path uses), keyed on the exact compile input + a pixel rewriteVersion -- warm sessions create pixel shaders from cached bytes without invoking the compiler. Two inherited fixes ride along verbatim, both still necessary the moment anything compiles from source: the pixel_shader_constants.inc engine- layout include override (the TRE's copy maps textureFactor to a register the engine loads with a NEGATIVE hemisphere term -> black characters), and the texren_copy_c1a1 ps_2_0 substitution (the stock ps.1.1 face-bake asm reads its tint from the wrong register -> black baked faces; ps_1_x asm cannot be recompiled by the modern toolchain). A D3DCompile failure WARNs with the compiler's error text and falls back to PEXE -- on a stub that fallback can still fault, so any such WARNING in the log names a corpus shader that needs attention. Verified: rasterMajor=5 boots to ground on both datasets (previously crashed at the first pixel shader on both).
…t gates on unresolved externals Two build-infrastructure changes, verified by producing this tree's first ever SwgClient_d.exe (the Debug x64 configuration had never compiled, let alone linked). THE GATE (scripts/Build-Client.ps1): SwgClient links with /FORCE, which downgrades unresolved-external errors to warnings and emits a binary anyway -- MSBuild exit 0 has never meant a clean link in this repo. The script now tees the build output to a log and FAILS the build if any 'unresolved external symbol' line appears, printing the offenders. This gate is what surfaced every defect below. THE DEBUG REPAIR, outermost-in: - Tag.h / MeshConstructionHelper.cpp / crypto filters.h: size_t narrowing and template-argument fixes for code that only compiles (or only warnings-as-errors) in Debug. clientAudio / clientDirectInput / crypto had TreatWarningAsError only in their Debug configs -- flipped to match their own working Release configs. - libMozilla.vcxproj: Debug|x64 becomes ConfigurationType Utility -- the same fix Release|x64 already carries, same reason (the legacy XULRunner headers cannot compile under modern x64 SDKs; the link uses the hand-written stub deps\x64\lib\libMozilla.lib). - swg.sln: the dpvs Debug|x64 row mapped to an 'IntelCPP|x64' project configuration that no longer exists; now Debug|x64 -> Debug|x64. - dpvsMemory.cpp: the Debug-only memory-guard tags cast pointers to UINT32 (C4311 under the x64 guardrail); routed through UPTR, low-32 tag semantics unchanged -- same shape as the earlier dpvs x64 fixes. - SwgClient.vcxproj Debug|x64: LinkIncremental off (incremental silently disabled the /FORCE this config sets -- LNK4075); the dependency list drops the legacy x86 third-party soup (Qt 3/4, soePlatform, maya, alienbrain, videocapture... -- x86 libs can never contribute to an x64 link; with the soup gone the REAL unresolved set surfaced); dpvs and lcdui reference the solution-built Debug libs explicitly (the deps dir copies are Release/MT builds and outrank the build dir on the libpath -- they were 130 LNK2038 debug-CRT mismatches); dinput8+dxguid added (were only reachable via the soup). - deps/x64: vivoxSharedWrapper_d.lib -- the wrapper rebuilt /MTd from the in-tree Vivox.cpp (compiles with VIVOX_VERSION=3, matching the in-tree SDK headers); videocapture-stubs_d.lib + its committed source (compat-source/videocapture_stubs.cpp) -- no-op definitions for the VideoCapture/SoeUtil symbols SwgVideoCapture.obj references. The real implementations only exist as x86 libs, so video capture has been structurally dead on x64 since the port; Release never notices because nothing there pulls that object out of clientGraphics.lib. The stubs make the Debug pull harmless (capture reports cStateDead). Verify: Build-Client -Configuration Debug -Architecture x64 -Renderer DX9 completes with zero unresolved externals; SwgClient_d.exe + gl05/06/07_d produced. Boot smoke of the Debug client is the remaining field check.
…out client-assets The stock SWGSource dataset still ships 224 of its 806 shader programs as Direct3D 9 assembly (ui.psh among them), and D3DCompile has no assembler, so a client built from this repository alone draws nothing that uses them -- the first symptom is a black login screen, then interiors with no walls or floors (the tfcl/tfcsl vertex family). The converted HLSL normally arrives via client-assets on the search path; a reviewer pointing a fresh build at a stock install has no such overlay. Direct3d11_EmbeddedShaderCorpus (generated from the corpus drop) carries every converted program -- 127 pixel, 94 vertex, 221 of the 224 -- with the corpus includes inlined into each text. The inlining matters: the stock TRE set carries ASSEMBLY files at the same modules/ include paths (they belong to the asm programs these translations replace), and the include handler resolves TreeFile first, so a #include left in an embedded text picks up the asm copy on a stock dataset and dies with X3000 'm4x4'. Substitution is a fallback by contract: compilePixelShader consults the table only after refuseIfNotHlsl fired, and the vertex side only when parseHeader saw a non-HLSL marker, so a mounted corpus or loose override always wins. The vertex path re-parses the embedded text's own header because the tag block that drives the TEXCOORD mapping belongs to the translation, not the assembly it replaces. Not covered (no corpus translation): membrane.psh, water_pass2_20.psh, water_pass2_25.psh -- ps.1.x variants whose ps20 HLSL siblings exist at stock paths and win implementation validation on this backend (verified: Theed water renders). Verified against a pure stock mount, no overlays, x64 Release DX11: login, character select, Mos Eisley cantina interior, NPCs, Theed water.
…- the corpus copy washed out every interior RenderDoc conviction (Capture240, Mos Eisley cantina ceiling pixel, stock data): the baked vertex colour arrives authored-correct at 0.28 grey, the corpus translation of c_ambient.inc then adds scene ambient -- uploaded as (1,1,1) -- and the sum saturates to full-bright albedo. Every baked-vertex-lit interior renders washed to the diffuse texture while DX9, whose stock asm c_ambient is a bare `mov r7, vColor0`, renders the authored moody image. The corpus drop's `vColor0 + ambient` shape is a leftover compensation from the zeroed-vertex-stream era. The bake IS the cell lighting; adding ambient on top double-counts it on well-formed data. The regenerated table inlines the stock shape into the 27 c_ambient-consuming vertex translations, with the correction documented at the splice. This is also this backend's own stated contract: [Direct3d11] ambientBoost exists to re-add the term opt-in, default off (rendering the data as authored is the default). Two things worth knowing beyond this table: the distributed corpus still ships the boosted c_ambient.inc, so a mounted-corpus client washes interiors the same way; and the ambientBoost include patch searches for the ASM text `mov r7, vColor0`, which cannot match the corpus's HLSL module -- the opt-in gate never engages against the current corpus either way. Verified on stock data, x64 Release DX11: cantina interior now matches the DX9 reference image (moody, lightmapped); Theed water and space (nebulas included) render correctly.
…onsumer frame/resize callbacks Two registration slots appended at the Gl_api tail (sizeof is the load-time compatibility gate, so tail-append only, never reorder): setFrameCallback and setResizeCallback, the no-detour overlay contract from the toolkit's x64 round-2 change request. x64 has no working consumer detour engine, so the injected overlay draws through a provider-invoked callback instead of patching the DXGI vtable. The D3D11 line implements both in the swap-chain class: frame fires in present(), after every provider write to the back buffer (post-composite, post-debug-screenshot) and before Present -- the overlay draws over the finished frame; resize fires in resize() at phase 0 before ResizeBuffers (the consumer must release its back-buffer-referencing views or ResizeBuffers fails) and phase 1 once the new views exist, both with the new client size. Single-slot, last-write-wins, null clears; the invoke sites snapshot the pointer so a concurrent clear cannot fault mid-call. The D3D9 line accepts and ignores with one Release-visible log line each -- a consumer registering there has the wrong rasterMajor, and the line says why its overlay never draws. Graphics forwards both through ms_api with a guard-and-WARN (consumer-driven timing; an early registration degrades loudly instead of crashing the host). sizeof(Gl_api) changes: every plugin and the exe rebuilt as one staged set.
…t v35/165, both platforms The engine-hookpoint advertisement provider, ported as its current state from the sibling tree where it grew v24 through v35 (SWG-Toolkit is the sole consumer; the contract history lives in that tree's commits). The exe exports one undecorated extern "C" GetEngineHookPoints() (in-source dllexport) that hands the injected consumer a versioned, null-checked table of 165 named entry points -- CALLED thunks, DETOURED real-entry addresses (MI-PMF decoded, delta==0 verified per row), and data addresses. When nothing is injected, nothing calls it. What rides with the provider, because the rows reference it: - engine_advertise.cpp + engine_hookpoints.h/.inc (the X-macro contract) + six exe-local forward headers; SwgClient.vcxproj compiles the provider and gains the two missing include dirs (sharedCommandParser, sharedCollision). - Friend-gated shims in the TUs that own private members: Os::WindowProc, DebugHelp::writeMiniDump, SwgCuiChatWindow::createNewWindow address provider, GroundScene init/inputMapUpdate forwarders + update/ handleInputMapEvent real-entry accessors (ENGINE_THIS per-platform seam), CreatureObject setLookAtTarget real-entry + player lookAt-target id shim, ClientEffectManager retrigger/replay (Effects-editor live preview), engine_warpPlayer (the v31 cell-resolving teleport; sendTransform picks the with-parent message so the server hears cell moves). - Engine support the shims call: ExitChain shutdown-phase (process-wide monotonic int -- isRunning() is per-thread and useless to a polling consumer), Game getMainLoopCount/getShutdownPhase/consumer tick callback (invoked at the top of runGameLoopOnce, outside any render call chain), WorldSnapshot save/suppress/generation machinery + the engine_ws* shim family (19 rows) + WorldSnapshotReaderWriter saveFiltered/intern additions, TreeFile::enumerateFiles (SearchTree/SearchTOC TOC-name walk under the node lock), the per-building interior refresh chain (ClientBuildingObjectTemplate::reloadInteriorLayout + TangibleObject::clearClientOnlyInteriorLayoutObjects + CellProperty::clearAppliedInteriorLayout + engine_refreshInteriorLayout), and the WM_SIZE client-rect resize tracking in Os.cpp (embed-panel reparent resizes previously never reached the renderer's resize path; drag-debounced via ENTER/EXITSIZEMOVE). - tools/hookpoints-probe: the boot-free contract gate (LoadLibraryEx DONT_RESOLVE + GetProcAddress + call -- the pre-CRT path the static-init race fix engineered). Friend declarations only -- no data members, no virtuals: every touched class keeps a byte-identical layout, so no shared-header plugin cascade beyond the Gl_api commit below this one. Gate: Probe-HookPoints against the freshly built x64 exe -- version=35 count=165 nulls=0 uniqueNames=165 PASS.
Toolkit advertise
The toolkit advertise surface: GetEngineHookPoints v35/165 + the Gl_api overlay callbacks
…ults Merge pull request #3 from Galaxies-Reborn/toolkit-advertise
Author
|
Awaiting @kennethlong to verify PR MD before submitting for review - cheers. |
AconiteX
self-requested a review
August 19, 2026 15:11
Member
|
I'll own review of this when it's ready and will pull in others as needed |
|
I approve! |
swgsais
marked this pull request as ready for review
August 19, 2026 16:30
|
Pushed Background: the x64 work in this lineage was squashed from a snapshot taken before PR #18 merged, so the squash silently overwrote that change's content while keeping the merge in the branch ancestry — the branch showed "not behind upstream" while the code was absent. We ran a content audit of all post-import upstream commits against this tree; this was the only such loss (everything else is present or intentionally superseded). With this commit, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a maintained 64-bit Windows client path and a new Direct3D 11 renderer while retaining the existing Win32 and Direct3D 9 build paths.
It also introduces:
The phased implementation history has been retained to make the individual x64, renderer, input, audio, and stabilization changes reviewable.
Supported build matrix
x86) and x64gl05,gl06, andgl07gl11gl00The existing Win32/DX9 path remains available. This PR adds new configurations and backends rather than replacing the legacy renderer.
Major changes
x64 build support
Directory.Build.props,Directory.Build.targets, andsrc/build/win32/x64-platform.props.time_tcompatibility flag from x64 builds.Direct3D 11 renderer
Adds a new
Direct3d11project producinggl11_<configuration>.dll.The implementation includes:
The renderer translates the client’s Direct3D 9 assembly shader inputs to HLSL and caches compiled DXBC by content. An embedded shader corpus allows the renderer to operate with the stock shader dataset without requiring a separate client-assets checkout.
Compatibility handling includes:
Direct3D 9 preservation and x64 compatibility
The Direct3D 9 projects remain supported for both architectures:
gl05: standard Direct3D 9gl06: fixed-function pathgl07: vertex/pixel shader pathThe branch restores the stock-faithful renderer project set, updates it for x64, and includes fixes for:
This gives projects a migration path to x64 without requiring an immediate switch to Direct3D 11.
SDL3 multi-controller input
SDL 3.4.10 is integrated for joystick and game-controller input on Win32 and x64.
The implementation supports:
The input-map format advances from version
0006to0007. Existing0006keymaps remain readable and are migrated when saved.SDL controller input is enabled by default. Joystick ownership can be returned to DirectInput with: