Skip to content

Fix shadow rendering for cockpits when raytracing is active - #7685

Open
The-E wants to merge 1 commit into
scp-fs2open:masterfrom
The-E:fix/vulkan-rt-cockpit-shadow-parity
Open

The-E wants to merge 1 commit into
scp-fs2open:masterfrom
The-E:fix/vulkan-rt-cockpit-shadow-parity

Conversation

@The-E

@The-E The-E commented Aug 4, 2026

Copy link
Copy Markdown
Member

Summary

This branch fixes three problems with player cockpit rendering under Vulkan:

  1. The ship hull could show through the cockpit model, because the two shared one depth buffer.
  2. RT shadows on small objects shimmered when the viewer was far from the mission origin.

Fix 1: RT shadows on cockpits

The shadow acceleration structure (TLAS) did not include the cockpit model. It also
did not include OBJ_RAW_POF and OBJ_PROP objects. The shadow ray for the cockpit
pass used the wrong ray origin. Together, these problems meant that cockpits did not
receive shadows.

Changes:

  • Add the cockpit model to the TLAS gather (gatherCockpitShadowCasterInstance()). Add
    OBJ_RAW_POF and OBJ_PROP objects too. This matches the object set that the rasterized
    shadow path already uses. The gate is the new shadows_cockpit_casts_shadow(), which both paths share.
  • Give each TLAS instance a ray-cull mask (TLAS_MASK_* in shadows.h). Tag the viewer's
    own ship hull with TLAS_MASK_VIEWER_HULL. Shadow rays exclude this bit outside the
    cockpit pass, so the hull does not shadow itself. During the cockpit pass,
    ship_render_player_ship_casts_shadow_on_cockpit() still lets the hull cast a shadow onto the cockpit.
  • Add shadow_ray_view_origin and shadow_ray_cull_mask to the shadow uniform block
    (shadow_cascade_static_data, std140, 96 bytes, checked by a static_assert). The
    cockpit pass renders in a camera-relative frame, so the shader cannot rebuild the ray
    origin from inv_view_matrix alone. The shader now uses mat3(inv_view) * viewPos + shadow_ray_view_origin.
  • Add shadow_ray_params. shadow_cascade_params_bind() takes it, so the hull draw and the
    cockpit draw in ship_render_player_ship() each bind their own ray origin and cull mask.
    The old two-argument overload stays for ordinary passes.
  • Add shadow_cascade_params_bind_deferred(). This function removes duplicate
    cascade-selection code from the OpenGL and Vulkan deferred-lighting passes.
  • Add ship_cockpit_render_offset(). This function computes the cockpit's rotated
    and swayed offset in one place. The render path, the rasterized shadow path, and
    the TLAS path all call it, so the three paths cannot drift apart.
  • Update the cascade parameter buffer with a full gr_update_buffer_data() call. The bind
    now runs several times each frame with different content. The Vulkan streaming buffer
    bump-allocates fresh memory only for a full update, so the partial update overwrote data that was still in use.

Fix 2: TLAS-relative coordinates

Float32 has about 1 mm steps at 10 km and about 8 mm at 100 km. Cockpit-scale RT shadows
shimmered far from the mission origin.

  • Translate every TLAS instance by -Shadow_rt_tlas_origin. The TLAS build sets this
    origin to Eye_position once each frame. shadow_rt_relative() converts a world position.
  • The CPU computes shadow_ray_view_origin from exact differences, so the shader never
    adds two large numbers.
  • The detail-box check in the TLAS walk converts the eye position to the same frame.

Fix 3: cockpit depth in Vulkan

OpenGL swaps the depth buffer before it draws the cockpit. The swap keeps the ship
hull's depth values away from the cockpit draw. The Vulkan backend did nothing in
vulkan_post_process_save_zbuffer() and vulkan_post_process_restore_zbuffer(), so
the hull could block the view of the cockpit.

  • Add a backup depth image to VulkanPostProcessor.
  • Add saveSceneDepth(). It copies the scene depth into the backup image. The save
    function then clears the live depth buffer for the cockpit draw, as OpenGL does.
  • Add restoreSceneDepth(). It copies the backup image back after the cockpit draw, so
    later passes that read scene depth (for example lightshafts) see the scene depth.
  • Split the mid-scene depth copy in VulkanRenderer into endScenePassForDepthCopy() and
    resumeScenePassAfterDepthCopy(). The new save and restore code shares them.
    With MSAA, only the scene depth image is parked. The G-buffer pass clears the multisampled
    depth, and the MSAA resolve writes back into the scene depth before the restore.

Known gaps

  • The deferred lighting pass shades the whole G-buffer in one draw call. The pass
    can apply only one ray origin per draw. This is correct for the common
    case. It is not correct when Cockpit_shares_coordinate_space is true, or when a
    ship has no cockpit model. This case needs a per-pixel offset, or the offset must
    move into the G-buffer data itself. This branch does not fix this case.
  • The TLAS walk skips the detail-box check for the cockpit model. The rasterized path uses
    a cockpit-relative eye position. The TLAS walk does not compute this position.

Testing

  • ninja code and ninja Freespace2 build with no errors.
  • Confirmed in-game: player cockpit shadows now render correctly.

@wookieejedi wookieejedi added cleanup A modification or rewrite of code to make it more understandable or easier to maintain. graphics A feature or issue related to graphics (2d and 3d) labels Aug 4, 2026
@The-E
The-E force-pushed the fix/vulkan-rt-cockpit-shadow-parity branch 2 times, most recently from 0181d81 to 580c076 Compare August 15, 2026 11:02
@wookieejedi wookieejedi added this to the Release 26.2 milestone Aug 31, 2026
Comment thread code/graphics/shadows.cpp Outdated
Comment on lines +1101 to +1102
vec3d world_offset = (Lighting_mode == lighting_mode::COCKPIT && Viewer_obj != nullptr) ? Viewer_obj->pos : vmd_zero_vector;
bool allow_viewer_self_shadow = Lighting_mode == lighting_mode::COCKPIT && ship_render_player_ship_casts_shadow_on_cockpit();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested this beyond 10km away from mission origin?
That looks suspiciously like the pattern that will start noticeably and badly vibrating once you reach roughly that far, which is why mapped shadows refuse to render anything for cockpit shadows non-zero offset (you can notice for non-cockpit shadows as well, but only once you're 20, 30km out or you look closely, but it's not as disturbing due to the lower precision patterns that are shaded).

- Introduced `shadow_ray_params` to simplify ray origin and cull mask management.
- Shifted shadow ray origins to TLAS-relative space for improved precision and consistency, especially for cockpit rendering.
- Refactored cascade bind functions, adding `shadow_cascade_params_bind_deferred()` for consolidated offset/count handling in deferred lighting.
- Expanded TLAS to include cockpits and additional object types, ensuring parity with rasterized shadows.
- Introduced per-instance cull masks to address cockpit self-shadowing issues.
- Improved Vulkan depth isolation for cockpit rendering via backup/restore mechanism, ensuring correct depth sampling in later passes.
- Extracted reusable helper functions for cockpit rendering state logic.
@The-E
The-E force-pushed the fix/vulkan-rt-cockpit-shadow-parity branch from 580c076 to 3555180 Compare September 19, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cleanup A modification or rewrite of code to make it more understandable or easier to maintain. graphics A feature or issue related to graphics (2d and 3d)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants