Skip to content

fix poller not restarting after indexer mode changes #230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

iuwqyir
Copy link
Collaborator

@iuwqyir iuwqyir commented Jun 26, 2025

TL;DR

Enhanced the Poller component to support dynamic work mode switching between backfill and live modes without requiring a restart.

What changed?

  • Added currentWorkMode field and workModeMutex to the Poller struct to track and safely access the current work mode
  • Modified the polling logic to check the current work mode before processing blocks
  • Updated the work mode handling to allow switching between modes without shutting down the poller
  • Added logic to update the lastPolledBlock when switching from live to backfill mode

How to test?

  1. Start the system in live mode
  2. Switch to backfill mode and verify the poller continues running
  3. Confirm the lastPolledBlock is updated correctly when switching from live to backfill
  4. Switch back to live mode and verify the poller stops processing blocks but remains active

Why make this change?

This change improves the system's flexibility by allowing dynamic switching between work modes without restarting the poller process. This is particularly useful for systems that need to alternate between catching up on historical data (backfill) and processing new blocks in real-time (live mode).

Summary by CodeRabbit

  • New Features

    • Polling now only occurs in backfill mode, providing more controlled and predictable behavior.
    • Mode transitions are now handled smoothly without shutting down the poller, with improved logging of mode changes.
  • Bug Fixes

    • Improved concurrency handling for work mode changes to prevent potential race conditions.

Copy link
Collaborator Author

iuwqyir commented Jun 26, 2025

Copy link

coderabbitai bot commented Jun 26, 2025

"""

Walkthrough

The Poller struct now includes a currentWorkMode field and mutexes to manage concurrent access. Polling iterations check the mode under a read lock and run only in backfill mode. Mode transitions update internal state under a write lock, removing the previous shutdown behavior when switching to live mode.

Changes

File(s) Change Summary
internal/orchestrator/poller.go Added currentWorkMode, lastPolledBlockMutex, and workModeMutex to Poller; polling conditionally runs only in backfill mode; updated main event loop to handle mode transitions with locks and removed poller shutdown on live mode.

Sequence Diagram(s)

sequenceDiagram
    participant MainLoop
    participant Poller
    participant Storage

    MainLoop->>Poller: Change work mode (e.g., to backfill)
    Poller->>Poller: Acquire write lock, update currentWorkMode
    Poller->>Storage: Query last block number (if switching to backfill)
    Storage-->>Poller: Return last block number
    Poller->>Poller: Update lastPolledBlock under mutex

    loop Polling Goroutine
        Poller->>Poller: Acquire read lock, check currentWorkMode
        alt currentWorkMode != Backfill
            Poller->>Poller: Skip polling iteration
        else currentWorkMode == Backfill
            Poller->>Poller: Perform polling
        end
    end
Loading

Possibly related PRs

Suggested reviewers

  • catalyst17
  • nischitpra
  • AmineAfia
    """

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8d99b2f and bc7b24e.

📒 Files selected for processing (1)
  • internal/orchestrator/poller.go (3 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
internal/orchestrator/poller.go (3)
internal/rpc/rpc.go (1)
  • IRPCClient (42-54)
internal/storage/connector.go (1)
  • IStorage (67-71)
internal/orchestrator/work_mode_monitor.go (3)
  • WorkMode (15-15)
  • WorkModeBackfill (21-21)
  • WorkModeLive (20-20)
🔇 Additional comments (5)
internal/orchestrator/poller.go (5)

28-28: LGTM: Thread-safe fields properly added to struct.

The addition of lastPolledBlockMutex and workModeMutex with currentWorkMode correctly implements the concurrency control needed for safe dynamic mode switching. The choice of RWMutex is appropriate as it allows multiple readers for performance optimization.

Also applies to: 33-34


124-131: LGTM: Conditional polling logic correctly implemented.

The polling goroutines now properly check the current work mode under a read lock before processing blocks. This ensures that polling only occurs in backfill mode, which aligns with the PR objective of enabling dynamic mode switching without poller restarts.


164-172: LGTM: Race-safe work mode transition logic.

The work mode change handling correctly uses read locks to check the current mode and write locks to update it. The logic properly prevents unnecessary work when the mode hasn't changed or is empty. The logging provides good visibility into mode transitions.


173-184: LGTM: Proper state update when switching from live to backfill.

The logic correctly handles the transition from live to backfill mode by updating lastPolledBlock from the main storage under mutex protection. This ensures continuity when switching modes and prevents race conditions on the shared lastPolledBlock field.


220-220: Verify thread safety of lastPolledBlock assignment.

The assignment p.lastPolledBlock = endBlock in PollWithoutSaving is not protected by the lastPolledBlockMutex. This could create race conditions with the protected access in the work mode transition logic (lines 178-181).

Let me verify if this field is accessed elsewhere without mutex protection:

#!/bin/bash
# Description: Check for unprotected access to lastPolledBlock field
# Expected: Find all accesses to lastPolledBlock to ensure consistency

echo "=== Searching for lastPolledBlock field access ==="
rg -A 3 -B 3 "lastPolledBlock" --type go

echo -e "\n=== Checking for mutex usage patterns ==="
rg -A 5 -B 5 "lastPolledBlockMutex" --type go
✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@iuwqyir iuwqyir marked this pull request as ready for review June 26, 2025 17:00
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
internal/orchestrator/poller.go (1)

32-33: Consider initializing currentWorkMode with a default value.

The currentWorkMode field will default to an empty string. For better clarity and to avoid potential issues, consider explicitly initializing it to a default mode (e.g., WorkModeLive) in the constructor functions.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9d5d33d and adad370.

📒 Files selected for processing (1)
  • internal/orchestrator/poller.go (3 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
internal/orchestrator/poller.go (1)
internal/orchestrator/work_mode_monitor.go (3)
  • WorkMode (15-15)
  • WorkModeBackfill (21-21)
  • WorkModeLive (20-20)
🔇 Additional comments (1)
internal/orchestrator/poller.go (1)

123-130: Work mode check implementation looks good.

The read lock is properly acquired and released, and the logic correctly skips polling when not in backfill mode. This implementation efficiently prevents unnecessary polling in live mode.

@iuwqyir iuwqyir force-pushed the 06-26-fix_poller_not_restarting_after_indexer_mode_changes branch from adad370 to 8d99b2f Compare June 27, 2025 10:20
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between adad370 and 8d99b2f.

📒 Files selected for processing (1)
  • internal/orchestrator/poller.go (3 hunks)
🔇 Additional comments (2)
internal/orchestrator/poller.go (2)

32-33: LGTM! Well-structured additions for thread-safe work mode management.

The new fields properly implement the requirements for dynamic mode switching with appropriate synchronization.


123-129: LGTM! Proper implementation of conditional polling based on work mode.

The logic correctly prevents polling outside of backfill mode while maintaining thread safety with appropriate read lock usage.

@iuwqyir iuwqyir force-pushed the 06-26-fix_poller_not_restarting_after_indexer_mode_changes branch from 8d99b2f to bc7b24e Compare June 27, 2025 10:39
@iuwqyir iuwqyir merged commit 7d27d53 into main Jun 27, 2025
6 checks passed
@iuwqyir iuwqyir deleted the 06-26-fix_poller_not_restarting_after_indexer_mode_changes branch June 27, 2025 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant