Skip to content

Add AOF replication for RangeIndex migration#1911

Open
tiagonapoli wants to merge 6 commits into
mainfrom
tiagonapoli/bftree-migration-replication
Open

Add AOF replication for RangeIndex migration#1911
tiagonapoli wants to merge 6 commits into
mainfrom
tiagonapoli/bftree-migration-replication

Conversation

@tiagonapoli

Copy link
Copy Markdown
Collaborator

What

Adds AOF replication for migrated RangeIndex (BF-Tree) keys, so a migrated
index reaches replicas and survives crash recovery with its full data — not just
an empty tree rebuilt from the stub.

Why

When a node receives a migrated RI key, PublishMigratedIndex created the store
record via a RICREATE RMW that auto-logged only the stub to the AOF. On AOF
replay, HandleRangeIndexCreateReplay rebuilds an empty tree from the stub
config, so the migrated data never reached secondaries or survived recovery
(the TODO(RI) at RangeIndexManager.Migration.cs). A BF-Tree file can also
exceed a single AOF page, so it must be chunked.

How

  • RISTREAM — a new internal, AOF-only pseudo-command (never parsed from the
    network), intercepted in AofProcessor.StoreRMW alongside
    RICREATE/RISET/RIDEL.
  • Primary/publish (ReplicateRangeIndexStream): the reassembled BfTree file is
    streamed into the AOF as chunked RISTREAM entries (reusing
    RangeIndexChunkedSerializer), each entry keyed by the RI key and no larger than
    DefaultMigrationChunkSize; the final chunk is flagged.
  • Replay (HandleRangeIndexStreamReplay): a per-key
    RangeIndexChunkedDeserializer reassembles the file across interleaved AOF
    entries (all chunks for one key hash to a single virtual sublog, so they arrive
    in order on one replay task), then publishes via PublishMigratedIndex — which
    re-streams on a chained replica. Incomplete reassembly (crash mid-stream) is
    dropped by CleanupIncompleteStreamReassembly.
  • RICREATE suppression: the local RICREATE RMW carries the
    StreamedPublishLogArg sentinel so WriteLogRMW skips its auto-log. The
    RISTREAM stream is the single AOF source of truth for a migrated key; keeping a
    competing empty-tree RICREATE entry would cause data loss on replay in either
    ordering (it would block the publish via the KeyExists gate, or clobber the
    reassembled tree).

Source-side deletion of the migrated key already replicates via the normal
DELETE -> AOF path (out of scope here).

Tests

  • ClusterMigrateRangeIndexReplicatesToReplicaViaAof — a migrated DISK-backed RI
    key replicates to the destination primary's replica via AOF; the replica serves
    the full data.
  • ClusterMigrateRangeIndexRecoversFromAof — the destination primary reconstructs
    the migrated key from its own AOF on restart.
  • Full migrate.rangeindex suite (21) green; RangeIndexChunkedSerializerTests
    (57) and RangeIndexMigrationReceiveStateTests (2) green; full solution builds
    with 0 warnings.

Notes

  • MIGRATE REPLACE for RI keys remains unsupported (pre-existing TODO).
  • Preview feature (--enable-range-index-preview).

Draft: opening for review; not yet linked to a tracking issue.

Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Replication.cs Outdated
Comment thread libs/host/Configuration/Options.cs Outdated
Comment thread libs/server/AOF/AofEntryType.cs Outdated
Comment thread libs/host/GarnetServer.cs Outdated
Comment thread libs/server/AOF/AofProcessor.cs Outdated
Comment thread libs/server/AOF/AofProcessor.cs Outdated
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.cs Outdated
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Migration.cs
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Replication.cs Outdated
Comment thread libs/server/Servers/GarnetServerOptions.cs Outdated
Comment thread libs/server/AOF/AofEntryType.cs Outdated
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.cs Outdated
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.cs Outdated
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Replication.cs
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Replication.cs
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Replication.cs Outdated
Comment thread libs/host/GarnetServer.cs Outdated
Comment thread libs/host/Configuration/Options.cs Outdated
Comment thread libs/server/AOF/AofProcessor.cs Outdated
Comment thread libs/server/AOF/AofProcessor.cs Outdated
Comment thread libs/server/AOF/AofProcessor.cs Outdated
Comment thread libs/server/AOF/AofProcessor.cs
@tiagonapoli tiagonapoli force-pushed the tiagonapoli/bftree-migration-replication branch 2 times, most recently from ba9689a to 4c56fc9 Compare July 7, 2026 02:43
Comment thread test/standalone/Garnet.test/RangeIndexStreamReplayTests.cs Outdated
Comment thread test/standalone/Garnet.test/RangeIndexStreamReplayTests.cs Outdated
@tiagonapoli tiagonapoli changed the title Add AOF replication for RangeIndex (BF-Tree) migration Add AOF replication for RangeIndex migration Jul 13, 2026
@tiagonapoli tiagonapoli marked this pull request as ready for review July 13, 2026 04:09
Copilot AI review requested due to automatic review settings July 13, 2026 04:09
@tiagonapoli tiagonapoli force-pushed the tiagonapoli/bftree-migration-replication branch 3 times, most recently from 88caa4a to 2c75f8d Compare July 13, 2026 04:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds end-to-end AOF transport for migrated RangeIndex (BF-Tree) keys by streaming the serialized BF-Tree file into the AOF as chunked entries, then reassembling and publishing the full index during replication replay and crash recovery. This closes the “empty tree on replay” gap for migrated range indexes and adds targeted tests to validate both live replica sync and restart recovery paths.

Changes:

  • Introduces a new AOF entry type for chunked RangeIndex stream replication and adds replay handling that reassembles/publishes the migrated BF-Tree.
  • Suppresses the auto-logged RICREATE stub during migration publish so the stream becomes the single AOF source of truth.
  • Expands cluster migration tests (including multi-chunk and interleaving scenarios) and adds unit tests for replay-side reassembly bookkeeping.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test/standalone/Garnet.test/RangeIndexStreamReplayTests.cs Adds unit tests validating replay-side stream reassembly bookkeeping and cleanup behaviors.
test/standalone/Garnet.test/RangeIndexChunkedSerializerTests.cs Updates RangeIndexMigrationReader construction to match the new parameter naming/API.
test/standalone/Garnet.test/CapturingLogger.cs Adds a structured-log capturing test logger to assert on structured fields (e.g., reason).
test/cluster/Garnet.test.cluster/ClusterTestContext.cs Adds a test hook to force small RangeIndex stream chunk sizes across nodes.
test/cluster/Garnet.test.cluster.migrate.rangeindex/ClusterRangeIndexMigrateTests.cs Adds/extends cluster tests to verify migrated RI replication via AOF and recovery across restarts, including multi-chunk/interleaving/failover.
libs/server/Storage/Functions/MainStore/PrivateMethods.cs Skips auto-AOF logging for migration-publish RICREATE using a sentinel argument.
libs/server/Resp/RangeIndex/RangeIndexReplicationActivities.cs Adds structured activity logging for stream replication and reassembly.
libs/server/Resp/RangeIndex/RangeIndexMigrationReader.cs Refactors reader to accept a general Stream, adds sync chunk read, and introduces a default internal read-buffer size.
libs/server/Resp/RangeIndex/RangeIndexManager.Replication.cs Implements AOF chunk streaming, replay reassembly state, and publish-on-complete handling for migrated indexes.
libs/server/Resp/RangeIndex/RangeIndexManager.Migration.cs Publishes migrated indexes by first streaming snapshot into AOF, then suppressing the competing RICREATE AOF entry.
libs/server/Resp/RangeIndex/RangeIndexManager.cs Ensures incomplete reassembly state is dropped during manager disposal.
libs/server/AOF/AofProcessor.cs Adds replay dispatch for the new RangeIndex stream chunk entry type and cleanup on dispose.
libs/server/AOF/AofEntryType.cs Introduces RangeIndexStreamChunk AOF entry type and marks it replayable/store-type.
libs/host/GarnetServer.cs Adds startup validation that AOF page size can accommodate RangeIndex stream chunks when preview is enabled.
libs/host/Garnet.host.csproj Adds InternalsVisibleTo for cluster tests.
libs/cluster/Session/RangeIndexMigrationReceiveSession.cs Passes AOF handle into migrated-index publish so the target primary can stream to its replicas via AOF.
libs/cluster/Session/ClusterSession.cs Wires append-only file into the RangeIndex migration receive state.
libs/cluster/Server/Migration/RangeIndex/RangeIndexMigrationActivities.cs Minor doc comment formatting change in migration activity tracing.
libs/cluster/Server/Migration/MigrateSession.RangeIndex.cs Updates migration transmit to use the updated snapshot+reader API.

Comment thread libs/server/Resp/RangeIndex/RangeIndexMigrationReader.cs
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Replication.cs
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Migration.cs
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Migration.cs
Signed-off-by: Tiago Napoli <tiagonapoli@microsoft.com>
@tiagonapoli tiagonapoli force-pushed the tiagonapoli/bftree-migration-replication branch from 2c75f8d to 8da6d41 Compare July 13, 2026 04:23
Tiago Napoli and others added 5 commits July 13, 2026 20:14
Assert primary AOF stream chunking, replica reassembly, and crash-recovery
reassembly in the multi-chunk RangeIndex migration test via a per-test
CapturingLogger. Extract repeated endpoint/migrate/ownership patterns into
Endpoint, MigrateSlotsAndWaitCleanup, and MigrateSlotsAndWaitOwnership helpers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Tiago Napoli <tiagonapoli@microsoft.com>
Delete tests whose coverage is subsumed by others: ClusterMigrateRangeIndexSlot
(by SingleBySlot), ClusterMigrateEmptyRangeIndex (by the Replicates variant),
ClusterMigrateRangeIndexBack (by RoundTripThenRestart),
ClusterMigrateRangeIndexThenCheckpointRestart (by CheckpointThenRecover +
ThenCheckpointRestartAndContinue), and the interleaved-writes test. Also remove
ClusterMigrateRangeIndexWithChunkSize, whose chunkSize parameter was never used
(all TestCases ran identically) so it exercised no multi-chunk path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants