ReFit — align Garmin telemetry to Insta360 video

ReFit finds the Garmin activity that overlaps an Insta360 MP4, estimates the camera-to-Garmin clock offset from visual motion, and writes an Insta360-compatible FIT sidecar. Import that FIT into Insta360 Studio's Stats dashboard to render speed, heart rate, route, and the rest of the original Garmin data.

The output is a minimally patched copy of the source FIT: it retains Garmin messages, developer fields, positions, elevation, heart rate, events, and device information. Only FIT timestamps and the gps_metadata speed samples used by Studio are changed; checksums are recomputed.

Install

ReFit is a Python package with a refit command. It also requires ffmpeg and exiftool on PATH.

python3 -m venv venv
venv/bin/python -m pip install -e .

Use

cd ~/Downloads
/path/to/refit/venv/bin/refit VIDEO.mp4

The command downloads the matching Garmin FIT when none is provided, then writes VIDEO_speed.fit in the working directory. On first use it asks for Garmin Connect credentials and MFA; reusable authentication is stored locally. To supply an already downloaded activity:

/path/to/refit/venv/bin/refit VIDEO.mp4 ACTIVITY.fit

Use --output PATH to override the default <video-stem>_speed.fit output. The existing insta360_video_speed_fit.sh wrapper remains available for compatibility with source-checkout workflows.

The video creation timestamp must include a UTC offset. ReFit intentionally refuses ambiguous camera time rather than inventing a timezone.

For a camera card containing recent footage, use the sequential batch helper:

bash /path/to/refit/batch_refit_recent.sh \
  /Volumes/Untitled/DCIM/Camera01 ~/Downloads

It processes MP4s modified in the last five days and writes one <video>_speed.fit per video. The batch is sequential so camera-card I/O and FFmpeg decoding remain predictable.

What is aligned

The output FIT encodes the nearest whole-second timestamp correction. FIT activity timestamps cannot represent fractional seconds, so the remaining phase is applied by interpolating the gps_metadata speed stream. Thus a correction of $\Delta t$ is represented as

$$ \Delta t = k + r, \qquad k \in \mathbb{Z}, \quad -0.5 \le r < 0.5, $$

where $k$ shifts FIT timestamps and $r$ shifts only the speed samples used by Studio. Garmin coordinates and record-level speed fields are untouched.

Optical-motion alignment

ReFit is a timing estimator, not a monocular speedometer. Optical motion yields a scale-free signal whose shape is aligned to Garmin speed with Pearson correlation; Studio continues to display Garmin's speed scale.

1. Sample at most 1,000 pairs of adjacent original-rate frames, uniformly through the video. Sampling at 4 Hz from 60 fps, for example, measures (0, 1), (15, 16), (30, 31), not frames four video frames apart.
2. Resize frames to 640 px wide, convert them to grayscale, and compute dense Farnebäck optical flow on the unblurred full frame.
3. Interpret the image-coordinate flow on the viewing sphere of a calibrated rectilinear camera model, then take its surface divergence.
4. Take the spatial median from the fixed measurement ROI. On the normal 640×360 analysis frame, this is flow[72:306, 64:576]: 512×234 samples. There is no temporal smoothing.
5. Search only positive offsets from 0 to 45 s, retaining the interior offset that maximizes linear correlation with Garmin gps_metadata speed. A boundary optimum is rejected rather than exported.

Camera geometry

The source video is treated as a rectilinear/pinhole view. ReFit converts the configured horizontal FOV into the equivalent focal length in pixels:

$$ f = \frac{W-1}{2\tan(\theta/2)}. $$

Here, $W$ is the frame width in pixels, $\theta$ is the configured horizontal FOV in radians, and $f$ is the resulting focal length in pixels. For each pixel, the code uses its offset from the frame center and $f$ to form a unit-length viewing direction. That is the usual pinhole projection model. The Ace Pro 2 Bike Mode profile in camera_profiles.json uses 120° horizontal FOV. This is the only FOV used by the calculation, and is a working estimate of the exported, stabilized rectilinear video.

Insta360's published 157° lens FOV is diagonal and describes the physical lens, not the horizontal FOV of this stabilized export. It must not be inserted into horizontal_fov_degrees: even if an unmodified 16:9 rectilinear image really had a 157° diagonal FOV, that would imply roughly 154° horizontal, not 157°. Export FOV selection and Bike Mode/High stabilization can crop or warp the raw lens image further (specification, stabilization guide). The MP4 does not expose an FOV metadata tag, so calibration of the exported image—not the raw-lens marketing number—is the way to replace 120°.

For the next expression, $(u,v)$ means a pixel location, and $\mathbf w=(\dot u,\dot v)$ is the optical-flow displacement there (in pixels per frame). $J(u,v)$ is the local sphere area represented by one image pixel, computed from the FOV conversion above; $S^2$ denotes the unit viewing sphere. The scalar used for alignment is the median of the discrete surface divergence

$$ \mathrm{div}_{S^2}\mathbf w = \frac{1}{J}\left[ \frac{\partial(J\dot u)}{\partial u} + \frac{\partial(J\dot v)}{\partial v} \right]. $$

This compensates for the changing solid angle represented by a pixel away from the image center. A rigid rotation induces a divergence-free tangent field on the viewing sphere; forward camera translation tends to create outward image expansion. ReFit therefore uses divergence directly as a rotation-resistant motion proxy. Depth variation, independently moving objects, stabilization artifacts, and an imperfect FOV calibration can still affect it.

Why turning or nodding has zero ideal divergence

The claim applies to every rigid camera rotation: turning left/right (yaw), nodding (pitch), tilting (roll), or spinning continuously. Let $\mathbf q$ be one unit viewing direction, $\boldsymbol\omega$ be the camera's rotation axis and rate, and $\dot{\mathbf q}$ be that direction's instantaneous movement. For a rigid rotation,

$$ \dot{\mathbf q}=\boldsymbol\omega\times\mathbf q. $$

This moves each direction along the viewing sphere without changing the area of any infinitesimal spherical patch. Thus its surface divergence is zero. The pixels and scene features can still move dramatically—especially at the edge of a wide FOV—but their direction field does not locally spread apart or crowd together after the rectilinear-to-sphere correction. In practice, optical-flow error, stabilization warps, nonrigid scenery, and occlusion make the measured value only approximately rotation-resistant.

Gaussian blur and pre-flow cropping were tested and removed. Full-frame, unblurred flow was both simpler and more accurate on the included validation; the late ROI remains because it is a robust spatial reduction, not a preprocessing shortcut. The exact fixed ROI is left/right 10%/90% and top/bottom 20%/85%; on the standard 640×360 frame those Python end-exclusive bounds are flow[72:306, 64:576]. Farnebäck flow and spherical divergence are calculated on the complete frame before this slice is taken. numpy.gradient therefore uses one-sided differences only at the actual image border (row 0/359 or column 0/639), none of which enter the median. Values at the ROI's own edge still use their immediate full-frame neighbors; the ROI is not a separate numerical domain or a boundary condition.

Demos

make_optical_flow_demo.py uses the same decode, preprocessing, Farnebäck, spherical-divergence, and ROI operations as production. It creates visual stages only for image-valued transformations; scalar reductions are labeled but not padded into artificial video stages.

python make_optical_flow_demo.py VIDEO.mp4 [OUTPUT_DIR] \
  --start 375 --duration 10 --sample-fps 4

To choose the interval where two FIT outputs differ most:

python make_optical_flow_demo.py VIDEO.mp4 --duration 30 \
  --compare-fit DRY.fit OPTICAL.fit

When no output directory is given, the demo uses a deterministic folder under ~/Movies. 99_all_stages_spliced.mp4 shows each visual stage once over its corresponding slice of the chosen interval; it does not restart the clip per stage.

Validation

The repository includes reproducible, modality-selective validators for two off-road datasets:

python tartandrive_validation.py
python sfu_mountain_validation.py --download

See TartanDrive details and SFU Mountain details. Those results evaluate the timing/motion proxy under their stated sensor assumptions; they do not turn monocular optical flow into independently calibrated ground-truth speed.

Experimental highlight scoring

refit-highlights compares three physically grounded rankings using aligned FIT sidecars. All outputs share the same motion analysis, candidate windows, overlap removal, and duration budget:

• bivector (the default score): |velocity ∧ acceleration|
• total: |velocity acceleration| = |velocity||acceleration|
• lateral: |(velocity / |velocity|) ∧ acceleration|, or lateral acceleration

An exported video's FIT file may be named <video-stem>_speed.fit. When names differ, ReFit also matches files such as VID_YYYYMMDD_HHMMSS_*_speed.fit to the recording timestamp embedded in the video metadata.

Highlight scoring uses Garmin's speed magnitude from the already-aligned sidecar; it does not estimate speed from successive GPS positions. ReFit encodes the video/FIT clock correction by shifting FIT timestamps to the nearest whole second and resampling the Garmin gps_metadata speed stream for the remaining fractional phase. The highlight pipeline combines that aligned Garmin speed with the tangent and curvature of the interpolated GPS path. Consequently, renaming an unprocessed FIT file to _speed.fit does not align it—create the sidecar with refit or batch_refit_recent.sh first.

refit-highlights /Volumes/Untitled/DCIM/Camera01 \
  --fit-dir ~/Downloads \
  --recent-days 5 \
  --duration 10m \
  --clip-duration 20s \
  --max-clips-per-source 2 \
  --order interesting \
  --output-prefix ~/Downloads/highlights

This writes highlights_lateral.json, highlights_bivector.json, and highlights_total.json. Each clip also records both geometric components, making the selections directly comparable. The aligned Garmin speed samples are interpolated directly without a separate smoothing pass.

--max-clips-per-source limits how many selected windows may come from any single input video while continuing down the global interest ranking. Candidate windows from the same video never overlap. --order interesting places the strongest selected clip first; --order chronological preserves source and recording order instead.

Sources can be individual files, directories, or a mixture. Passing explicit files is the simplest way to keep only one recording per trail:

refit-highlights \
  "/path/to/albatross 3.mp4" \
  "/path/to/boundary line 2.mp4" \
  "/path/to/super steep 2.mp4" \
  --fit-dir ~/Downloads \
  --duration 2m \
  --clip-duration 20s \
  --max-clips-per-source 2 \
  --order interesting \
  --output-prefix ~/Downloads/trail-highlights

Track geometry is interpolated as a cubic planar path over cumulative GPS distance. Its analytic first and second derivatives provide tangent and signed curvature. Garmin speed is interpolated in time with a shape-preserving cubic; its analytic derivative provides longitudinal acceleration. Interest is sampled at 10 Hz, then each fixed-duration candidate receives the analytic time-average of its shape-preserving interest interpolant.

Render a manifest at its original resolution and frame rate:

refit-render-highlights highlights_bivector.json bivector.mp4 --encoder copy
refit-render-highlights highlights_total.json total.mp4 --encoder copy

Stream-copy rendering retains the source resolution, frame rate, codec, audio, and encoded pixels. Clip boundaries land on nearby source keyframes.

For faster test exports, choose an encoder and output height instead:

refit-render-highlights highlights_total.json preview.mp4 \
  --encoder libx264 --height 720