Skip to content

Calibrate stick centre and travel per controller - #3

Open
JoeGeC wants to merge 2 commits into
mainfrom
fix/stick-calibration
Open

Calibrate stick centre and travel per controller#3
JoeGeC wants to merge 2 commits into
mainfrom
fix/stick-calibration

Conversation

@JoeGeC

@JoeGeC JoeGeC commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Fixes the reported "left joystick input seems to be half of the input that D-pad gets".

The bug

ReportMapper.mapStick mapped raw sticks as (v - 2048) * 32767 / 2048, assuming the raw 12-bit value swings the full 0..4095 about a 2048 centre. Neither holds. Measured on hardware:

travel:  full left ~900   full right ~3400   full down ~890   full up ~3360   (half-span ~1250)
rest:    left Joy-Con  x 2080  y 2157        right Joy-Con  x 2014  y 2022

Three separate errors fell out of that:

raw old HID new HID
full left 900 −18,367 (56%) −32,767 (100%)
at rest 2080 +511 (drift) 0
half right 2740 +11,071 +16,383 (50%)
full right 3400 +21,631 (66%) +32,751 (100%)
  • Scale — full deflection only reached 56–66% of the range we advertise, while the D-pad goes out as a digital hat at 100%. That is the reported symptom.
  • Centre — resting off-centre emitted a permanent 4–5% drift.
  • Asymmetry — the wrong centre skewed each half oppositely, hence right/up reading ~65% against left/down's ~56%.

The same 2048 assumption was duplicated in DsuPacketEncoder, StickCard, SidewaysMapper and the CENTER defaults, so DSU output and the on-screen stick were short by the same margin.

The fix

StickCalibrator normalises raw sticks onto a true 2048-centred 0..4095 space where packets are parsed, so every consumer is corrected by one change rather than each fixing its own copy.

  • Centre is learned from the first still window after connect, then frozen. It must be per controller (these two units are 66 LSB apart on X), and it must freeze: unlike gyro bias, a stick held at full deflection is perfectly still, so a "no movement means at rest" test would adopt full tilt as centre.
  • Spans are per direction, since rest is not the midpoint of travel — 1180 below and 1320 above on the left Joy-Con. That is the same centre/below/above triple the controller's factory calibration stores.
  • Spans are seeded at 1150 and only widen, so full tilt is reachable from the first packet and becomes exact once the stick has been rolled.

Second commit clamps the live display's dot inside its ring: now that each axis reaches its own rails, a full diagonal computes to 1.41× the radius on a square mapping. The gate is round, so the magnitude clamps rather than each axis.

Verified

  • 9 new StickCalibratorTest cases; ./gradlew build :konsist:test green.
  • On hardware (AYN Thor): rest reads exactly 2048/2048, full deflection 0/4095, and the dot tracks the ring edge through a full circle.

Notes

  • HID output is deliberately not radially clamped — per-axis clamping to ±32767 is what real gamepads report, and most games do their own normalisation.
  • The rest-window threshold (32 LSB over 30 samples) is set conservatively rather than from measured jitter. If it is too tight the centre simply never gets learned and behaviour falls back to today's, so it fails safe.
  • SPI factory calibration (IEEE-754 floats near 0x013040) remains the proper source and would remove the connect-time learning entirely. Left as follow-up — runtime calibration is needed regardless for the Pro Controller, which we cannot hardware-test.

🤖 Generated with Claude Code

JoeGeC and others added 2 commits September 2, 2026 19:00
The raw 12-bit sticks only travel about +-1250 LSB and rest off-centre, and
both vary per unit and per axis (left Joy-Con x 2080 / y 2157, right x 2014 /
y 2022). Mapping them as (v - 2048) * 32767 / 2048 left full deflection at
56-66% of HID range with a permanent 4-5% drift at rest, so the analog stick
moved characters slower than the d-pad's digital hat.

StickCalibrator learns each axis' centre from the first still window after
connect and scales each direction by its own span. It runs where packets are
parsed, so the live display, HID reports, and DSU all see corrected values
instead of each re-deriving the same wrong assumption.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Normalising each axis against its own travel means a full diagonal reaches 1
on both, which a square mapping draws at 1.41x the ring radius. The stick's
gate is round, so clamp the magnitude rather than each axis, and draw within
the ring instead of centring the dot on it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant