Repository navigation
feat: Add file loading code for flag overrides - #449
Draft
kinyoklion wants to merge 11 commits into
Draft
kinyoklion wants to merge 11 commits into
kinyoklion wants to merge 11 commits into
Conversation
This was referenced Sep 25, 2026
Adds `LaunchDarkly::Impl::FileData`, the file reading, parsing, merging, and reloading code that the file-based override source defined by the OVERRIDE specification needs. That source must tolerate a missing file, a file that is being written, and a burst of change notifications, and it must keep the last good data when a reload fails. The module provides: - Document parsing for JSON and YAML with validation that the document and its flags, flagValues, and segments members are objects. - An ordered merge of several documents with duplicate keys handling of fail or ignore, flagValues expansion into off flags that serve the value, model deserialization, and per-document entry counts. - A reloader that serializes reloads, debounces change signals, retains the last good result when a reload fails, retries after a bounded delay, reports each distinct failure once, and can skip byte-identical reloads. A configured file that does not exist can be treated as a file with no content. - A stat-based poller that compares existence, modification time, and size, so a file that appears or disappears is a change. - A watcher that uses rb-inotify on Linux to watch the directory of each file without descending into subdirectories, and the listen gem elsewhere, so a file that does not exist yet is picked up when it appears. It retries when a directory does not exist. The existing FDv1 and FDv2 file data sources are not changed. They keep their own implementation and behavior. A new spec pins the behaviors of those sources that their other specs did not assert, so that they stay as they are while this code exists beside them.
kinyoklion
force-pushed
the
rlamb/overrides-ruby-filedata
branch
from
September 28, 2026 20:36
eb26dd7 to
72d5776
Compare
The result is not remembered as applied, on_error receives the exception, the reload is retried, and the worker thread continues.
The inotify watcher tears its watches down when a watched directory is deleted or moved, logs it, retries setup once per second, and signals one change when the watches are in place again.
Compare absolute paths through File.absolute_path, close temp files before deleting them, and backdate modification times instead of sleeping before a write that the pollers must see.
Member
Author
|
bugbot review |
| value.map { |v| symbolize_keys(v) } | ||
| else | ||
| value | ||
| end |
There was a problem hiding this comment.
YAML on key silently disables flags
Medium Severity
YAML.safe_load treats unquoted on as a boolean, and symbolize_keys then stores that field as :true. Full flag definitions never see :on, so FeatureFlag treats them as off. Natural YAML like on: true therefore loads as a disabled flag instead of failing or preserving the on field.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 2e31db3. Configure here.
…pop timeouts Queue#pop accepts a timeout only from Ruby 3.2 on. The sync examples also wait for the poller thread or the listen call before they act, because the source starts change detection after it yields the initial data.
The watcher set up its watches all or nothing. If any directory of the configured paths did not exist, it logged the problem, watched nothing, and retried once per second until every directory existed. A configured file in a directory that never exists is a valid steady state for the override source, which treats a missing file as contributing nothing, so one permanently missing directory meant that changes to files in the directories that do exist were never detected. Each directory is now watched on its own. At start, the directories that exist are watched at once and the others are remembered as missing and retried once per second. A retry adds the watches for the directories that exist by then, and the callback runs once when it added at least one, whether or not other directories are still missing. The directories that are still missing, and any watch that could not be set up, are logged together, so that an unchanged situation repeats at debug level as before. When a watched directory is lost, only its watch is removed and it goes back to the missing set; the other directories keep their watches. With rb-inotify, one notifier and one thread take the watches as they are added, and a lost directory's watch is closed. With the listen gem, there is one listener per directory, and stop ends each of them. The spec that lost a directory as soon as its watches were set up waited for the inotify thread to end as its sign that the loss had been handled. The thread outlives one directory now, so it waits for the warning in the log instead. New specs cover a directory missing at start beside one that exists, a lost directory watched again while another is still missing, and stop while a directory is missing.
Member
Author
|
bugbot review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
1 issue from previous review remains unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 73afbaa. Configure here.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Summary
This is the first of a series that adds flag overrides, as defined by the OVERRIDE specification, to the Ruby server SDK. The existing FDv1 and FDv2 file data sources keep their current behavior: this PR does not change
lib/ldclient-rb/impl/integrations/file_data_source.rb,file_data_source_v2.rb, or their specs, and the override feature is additive throughout the series.The file-based override source must tolerate a missing file, a file that is being written, and a burst of change notifications, and it must keep the last good data when a reload fails. This change adds
LaunchDarkly::Impl::FileData, the file reading, parsing, merging, and reloading code that source needs:flags,flagValues, andsegmentsmembers are objects.failorignore, expansion of eachflagValuesentry into a flag that is on and serves the value by fallthrough (so an evaluation reports the FALLTHROUGH reason kind, as the specification describes), deserialization into the data model classes, and per-document entry counts.rb-inotifyon Linux (a dependency of the optionallistengem) to watch the directory of each file without descending into subdirectories, andlistenelsewhere. It picks up a file that does not exist yet when it appears, and it watches each directory on its own: a directory that does not exist is retried once per second while the others stay watched, and one change is signaled when a retry sets up a watch. Non-recursive watching matters becauselistenscans the whole tree under a watched directory and fails when a subdirectory cannot be read, as happens for a file in the system temporary directory. On Linux, a watched directory that is deleted is watched again once it exists, and one change is signaled so the reload picks up what was written meanwhile.A new spec,
spec/integrations/file_data_source_compatibility_spec.rb, pins the behaviors of the existing file data sources that their other specs did not assert (per-file version numbering, keying by the entry's own key, the on-plus-fallthrough expansion offlagValues, no retry of a failed load, log messages, modification-time-only polling, no reaction to a deleted file when polling, the FDv1 poller's reload cadence after a change, and use oflistenwhen it is present), so that they stay as they are while this code exists beside them.Verification: unit specs for the document parser, merge, poller, reloader (including debounce, retry, last-good retention, sequential reloads, and stop), and watcher (including a directory with an unreadable subdirectory and the inotify thread lifecycle), the new compatibility pins, and the unchanged existing file data source specs. Full suite and RuboCop are clean. Each new spec was checked against a deliberate defect in the code it covers, and each compatibility pin against a deliberate change to the existing source it pins.
SDK-3249
Note
Overview
Introduces
LaunchDarkly::Impl::FileData, shared infrastructure for upcoming file-based flag overrides. It does not change the existing FDv1/FDv2 integrations in this PR.Document handling: JSON/YAML parsing into
flags,flagValues, andsegments, withReadErrorfor per-file failures and helpers to expandflagValuesinto on/fallthrough flags.Multi-file merge: Ordered combination with
failorignoreduplicate-key policy, model deserialization, and per-document counts.Reload pipeline:
Reloaderserializes loads, debounces change signals, retries failures, keeps the last good data on error, optionally skips byte-identical reloads, and can treat missing files as empty.Pollerdetects changes via existence/mtime/size;Watcherusesrb-inotify(Linux, non-recursive) or optionallistenelsewhere, with retries for missing or deleted watch directories.Tests: Unit specs for each component plus
file_data_source_compatibility_spec.rbto lock current FDv1/FDv2 behavior (version numbering, polling semantics, logging, etc.) while the new code lives alongside them.Reviewed by Cursor Bugbot for commit 73afbaa. Bugbot is set up for automated code reviews on this repo. Configure here.