Repository navigation
Conversation
…) to Session and SessionPool The map-based insertTablets methods allow only one tablet per device, because the map is keyed by device id. The new list overloads accept several tablets of the same device and send them in one request. With redirection enabled, each tablet is routed by its own device id, so tablets of devices with the same leader go to that leader together. insertTabletsWithLeaderCache now takes a list of KeyedTablet instead of a map, so a leader cache key may occur several times. The map methods keep routing by their map keys.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Introduce new Session methods:
insertTablets(List<Tablet>)andinsertAlignedTablets(List<Tablet>).The existing methods
insertTablets(Map<String, Tablet>)andinsertAlignedTablets(Map<String, Tablet>)allow only one tablet per device, because the map is keyed by device id.This PR adds
insertTablets(List<Tablet>)andinsertAlignedTablets(List<Tablet>)toISession/SessionandISessionPool/SessionPool. They accept several tablets of the same device and send them in one request. With redirection enabled, each tablet is routed by its own device id, so tablets of devices with the same leader go to that leader together. No behaviour change for existing methods.Why
The devices in our system sample their measurements independently, so the timestamps of their measurements rarely match. Packing such measurements into one tablet per device fills most value cells with nulls. So in our case, the natural representation is one tablet per measurement, i.e. several tablets per device.
TSInsertTabletsReqalready accepts the same device several times; only the client API does not.Workarounds
With the current map-based API, there are two workarounds to send several tablets of one device in one request:
Mapthat contains the same device id several times, one entry per tablet. This routes correctly, but breaks theMapcontract and relies onSessiononly iterating the map. IfSessionever copied it into a real map, all but one tablet per device would be dropped without an error. This is our current workaround, guarded by tests that fail if an IoTDB upgrade breaks it.Changes
Session.insertTablets(List)/insertAlignedTablets(List): with redirection enabled, they key each tablet by its device id and use the existing leader-cache path. Without redirection, they send one request over the default connection, like the map methods.insertTabletsWithLeaderCachenow takes a list ofKeyedTablet(leaderCacheKey, tablet)instead of a map, so a key may occur several times. The map methods use their map keys as leader cache keys, so their routing stays the same.sortedoverloads:sortedis deprecated, and unsorted tablets are sorted anyway.Tests
SessionInsertTabletListTest(new) uses a mocked connection that records every request and, like a DataNode, answers with a redirect when a request contains devices led by another endpoint.test_InsertTablets_WhenLeadersAreUnknown_SendsAllTabletsToDefaultEndpointtest_InsertTablets_WhenLeadersAreKnown_SendsOneRequestPerLeadertest_InsertTablets_WhenRedirectionIsDisabled_SendsOneRequestToDefaultEndpointtest_InsertAlignedTablets_WhenLeadersAreKnown_SendsAlignedRequestPerLeadertest_InsertAlignedTablets_WhenRedirectionIsDisabled_SendsOneAlignedRequestSessionPoolTest:test_InsertTabletList_WhenSessionAcceptsIt_PassesTabletsOnAndPutsSessionBacktest_InsertAlignedTabletList_WhenSessionAcceptsIt_PassesTabletsOnAndPutsSessionBacktest_InsertTabletList_WhenSessionRejectsIt_RethrowsAndPutsSessionBacktest_InsertAlignedTabletList_WhenConnectionBreaks_ClosesSessionAndThrowsThis PR has:
for code coverage.
Key changed/added classes (or packages if there are too many classes) in this PR
ISession,ISessionPool,Session,SessionPool+ tests