Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(async): dispatcher #1910

Merged
merged 111 commits into from
Aug 20, 2024
Merged

feat(async): dispatcher #1910

merged 111 commits into from
Aug 20, 2024

Conversation

ocnc2
Copy link
Contributor

@ocnc2 ocnc2 commented Aug 16, 2024

Summary by CodeRabbit

  • New Features

    • Improved error handling capabilities with new tailored error types for enhanced clarity.
    • Introduced multiple new error handling functions for more informative timeout messages.
  • Bug Fixes

    • Corrected the description of an existing error type to provide accurate context.
  • Refactor

    • Updated dependency management to facilitate local development and streamline module architecture.
    • Enhanced error handling structure within the middleware for better context and reporting.
  • Chores

    • Updated several dependencies to the latest versions to improve performance and stability.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between a173b27 and 9cf5fad.

Files selected for processing (32)
  • mod/async/pkg/broker/broker.go (1 hunks)
  • mod/async/pkg/dispatcher/dispatcher.go (1 hunks)
  • mod/async/pkg/dispatcher/errors.go (1 hunks)
  • mod/async/pkg/types/dispatcher.go (1 hunks)
  • mod/beacon/block_store/pruner.go (1 hunks)
  • mod/beacon/block_store/service.go (1 hunks)
  • mod/beacon/block_store/types.go (2 hunks)
  • mod/beacon/blockchain/process.go (2 hunks)
  • mod/beacon/blockchain/service.go (6 hunks)
  • mod/beacon/validator/service.go (6 hunks)
  • mod/da/pkg/da/service.go (4 hunks)
  • mod/da/pkg/store/pruner.go (1 hunks)
  • mod/da/pkg/store/pruner_test.go (2 hunks)
  • mod/execution/pkg/deposit/pruner.go (2 hunks)
  • mod/execution/pkg/deposit/service.go (4 hunks)
  • mod/execution/pkg/deposit/sync.go (1 hunks)
  • mod/execution/pkg/deposit/types.go (2 hunks)
  • mod/node-core/pkg/components/availability_store.go (3 hunks)
  • mod/node-core/pkg/components/block_store.go (3 hunks)
  • mod/node-core/pkg/components/deposit_store.go (3 hunks)
  • mod/node-core/pkg/components/events.go (1 hunks)
  • mod/node-core/pkg/components/types.go (3 hunks)
  • mod/primitives/pkg/async/event.go (1 hunks)
  • mod/primitives/pkg/async/id.go (1 hunks)
  • mod/runtime/pkg/middleware/abci.go (7 hunks)
  • mod/runtime/pkg/middleware/middleware.go (4 hunks)
  • mod/storage/pkg/manager/manager_test.go (2 hunks)
  • mod/storage/pkg/manager/types.go (2 hunks)
  • mod/storage/pkg/pruner/mocks/block_event.mock.go (5 hunks)
  • mod/storage/pkg/pruner/pruner.go (3 hunks)
  • mod/storage/pkg/pruner/pruner_test.go (3 hunks)
  • mod/storage/pkg/pruner/types.go (2 hunks)
Additional comments not posted (122)
mod/beacon/block_store/pruner.go (1)

22-31: LGTM! Verify function usage across the codebase.

The change from a pointer to a value type for the event parameter in BuildPruneRangeFn is a positive step towards more efficient event handling. Ensure that all usages of this function in the codebase are compatible with the new signature.

Verification successful

Function signature change verified successfully.

The change from a pointer to a value type for the event parameter in BuildPruneRangeFn is consistently applied across the codebase. The function is used with the new signature in multiple files, and it is also included in test cases, confirming compatibility.

  • Files with updated usage:
    • mod/node-core/pkg/components/availability_store.go
    • mod/node-core/pkg/components/deposit_store.go
    • mod/node-core/pkg/components/block_store.go
    • mod/da/pkg/store/pruner_test.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all function calls to `BuildPruneRangeFn` match the new signature.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type go -A 5 $'BuildPruneRangeFn'

Length of output: 3340

mod/da/pkg/store/pruner.go (1)

24-31: LGTM! Verify function usage across the codebase.

The transition from a pointer to a value type for the event parameter in BuildPruneRangeFn enhances clarity and efficiency. Ensure that all usages of this function in the codebase are updated to align with the new signature.

Verification successful

Function Usage Verified: Consistent with New Signature

The BuildPruneRangeFn function is used consistently with the new signature across the codebase. The transition from a pointer to a value type for the event parameter is reflected in all relevant files.

  • mod/node-core/pkg/components/deposit_store.go
  • mod/node-core/pkg/components/block_store.go
  • mod/node-core/pkg/components/availability_store.go
  • mod/da/pkg/store/pruner_test.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all function calls to `BuildPruneRangeFn` match the new signature.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type go -A 5 $'BuildPruneRangeFn'

Length of output: 3340

mod/storage/pkg/manager/types.go (1)

Line range hint 29-45: Enhancements to BlockEvent interface look good!

The updates to the BlockEvent interface, including the new methods ID and Context, significantly enhance its functionality. Ensure that all implementations of this interface are updated accordingly.

mod/async/pkg/dispatcher/errors.go (1)

21-45: Error handling improvements are well-implemented!

The introduction of error handling functions and global error variables enhances the clarity and context of error reporting. Ensure these improvements are utilized effectively throughout the dispatcher package.

Verification successful

Error handling improvements are effectively utilized in the dispatcher package.

The error handling functions errBrokerNotFound and errBrokerAlreadyExists are used in dispatcher.go, confirming their integration into the package's error management logic. This enhances the clarity and context of error reporting as intended.

  • Usage found in mod/async/pkg/dispatcher/dispatcher.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of new error handling improvements in the dispatcher package.

# Test: Search for usage of `errBrokerNotFound` and `errBrokerAlreadyExists`. Expect: Usage in error handling contexts.
rg --type go 'errBrokerNotFound|errBrokerAlreadyExists'

Length of output: 489

mod/execution/pkg/deposit/pruner.go (1)

Line range hint 24-37: LGTM! Verify function usage across the codebase.

The change from a pointer to a value type for the event parameter in BuildPruneRangeFn aims to improve performance and clarity. Ensure that all usages of this function in the codebase are updated to align with the new signature.

Verification successful

Function Usage Consistency Verified

The BuildPruneRangeFn function's new signature, which uses value types for the event parameter, is consistently used across the codebase. The function calls in various files align with the updated signature, confirming that the necessary adjustments have been made.

  • Files verified:
    • mod/node-core/pkg/components/deposit_store.go
    • mod/node-core/pkg/components/block_store.go
    • mod/node-core/pkg/components/availability_store.go
    • mod/da/pkg/store/pruner_test.go
    • mod/da/pkg/store/pruner.go
    • mod/beacon/block_store/pruner.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all function calls to `BuildPruneRangeFn` match the new signature.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type go -A 5 $'BuildPruneRangeFn'

Length of output: 3340

mod/storage/pkg/pruner/types.go (1)

40-46: Well-structured BlockEvent interface.

The BlockEvent interface is well-designed, providing clear methods for event identification, data retrieval, and context management. This structure enhances the modularity and usability of block event handling.

mod/primitives/pkg/async/id.go (1)

23-49: Effective use of EventID type and constants.

The introduction of EventID and the associated constants provides a clear and type-safe way to manage event identifiers. The topological sorting of events is a thoughtful touch for maintaining order.

mod/beacon/block_store/types.go (2)

42-50: Comprehensive Event interface.

The Event interface provides essential methods for event identification and data access, facilitating robust event management. The design is consistent with Go's interface practices.


52-56: Flexible EventFeed interface.

The EventFeed interface introduces a subscription model that enhances the reactivity of the system. This design supports efficient event-driven programming.

mod/storage/pkg/manager/manager_test.go (1)

43-43: Improved channel handling in tests.

Switching to an unbuffered channel enhances the test's reliability by ensuring events are processed synchronously. This change reduces the risk of blocking issues during testing.

mod/node-core/pkg/components/events.go (1)

29-72: Centralized event publisher management.

The ProvidePublishers function effectively centralizes the creation of event publishers, promoting a clean and organized approach to event management. This setup enhances maintainability and scalability.

mod/primitives/pkg/async/event.go (5)

28-33: Interface BaseEvent is well-defined.

The BaseEvent interface establishes a clear contract for event types, ensuring they provide an ID and context. This abstraction enhances flexibility and modularity in event handling.


35-41: Interface Event extends BaseEvent effectively.

The Event interface builds on BaseEvent by adding methods for data, error handling, and type checking. This design promotes a clean separation of concerns and supports a more robust event-driven architecture.


58-68: Concrete type event correctly implements Event.

The event struct provides a concrete implementation of the Event interface, encapsulating context, ID, data, and error. This implementation is consistent with the interface-driven design.


70-92: Methods on event provide necessary functionality.

The methods ID, Context, Data, Error, and Is on the event struct are straightforward and align with their respective interface requirements. This ensures that the event type is fully compliant with the Event interface.


48-54: Ensure NewEvent usage aligns with the new interface.

The NewEvent function now returns an Event[DataT] instead of a pointer to a struct. Verify that all usages of NewEvent in the codebase are updated to handle the new return type appropriately.

mod/async/pkg/types/dispatcher.go (3)

29-39: Interface Dispatcher provides a comprehensive API.

The Dispatcher interface includes methods for starting the dispatcher, registering brokers, and retrieving the dispatcher name. This design supports a flexible and extensible event-driven system.


41-52: Interface EventDispatcher outlines essential event operations.

The EventDispatcher interface defines methods for publishing and subscribing to events. The TODO comment indicates a missing Unsubscribe method, which should be implemented to complete the event lifecycle.


54-65: Interface Broker supports event management.

The Broker interface provides methods for starting, publishing, subscribing, and unsubscribing events, as well as retrieving the event ID. This design facilitates efficient event management and communication.

mod/execution/pkg/deposit/sync.go (1)

34-40: Refactor enhances depositFetcher function.

The refactored depositFetcher function now directly processes a single event, improving clarity and performance by removing the loop and select statement. This change aligns with an event-driven architecture.

mod/execution/pkg/deposit/types.go (1)

45-56: BlockEvent interface enhances type safety and modularity.

The BlockEvent interface provides a structured approach to managing block events, promoting type safety and modularity. This design supports a more coherent and maintainable event handling framework.

mod/async/pkg/dispatcher/dispatcher.go (7)

33-38: Dispatcher struct facilitates asynchronous communication.

The Dispatcher struct is well-designed to manage asynchronous events, providing a robust framework for event-driven communication between components.


40-48: NewDispatcher initializes the dispatcher effectively.

The NewDispatcher function correctly initializes the dispatcher with a logger and an empty map of brokers, setting up the necessary infrastructure for event handling.


50-57: Publish method dispatches events efficiently.

The Publish method checks for the existence of a broker before publishing an event, ensuring that events are only dispatched to valid brokers. This approach prevents errors and maintains system integrity.


59-70: Subscribe method ensures channel type compatibility.

The Subscribe method verifies the existence of a broker and ensures that the channel type matches the expected event type, promoting type safety and preventing runtime errors.


72-78: Start method initiates brokers concurrently.

The Start method launches each broker in a separate goroutine, enabling concurrent processing and improving the efficiency of the dispatcher.


80-82: Name method returns a static dispatcher name.

The Name method provides a static name for the dispatcher, which can be useful for logging and identification purposes.


84-98: RegisterBrokers method prevents duplicate broker registration.

The RegisterBrokers method checks for existing brokers with the same event ID before registration, preventing duplicates and ensuring consistent event handling.

mod/storage/pkg/pruner/pruner.go (5)

33-33: Import Statement Update.

The import of the async package is appropriate, reflecting the new event handling mechanism. Ensure that all necessary imports are included and unused ones are removed.


45-49: Update to Pruner Struct.

The addition of subBeaconBlockFinalized and pruneRangeFn fields aligns with the new event-driven architecture. Ensure these fields are used consistently across the codebase.


60-68: Constructor Update for New Event Handling.

The NewPruner function now initializes with subBeaconBlockFinalized. This change is consistent with the shift to a more focused event handling strategy.


73-74: Refactor Start Method.

The Start method now uses listen, which is a more descriptive name than start. This improves code readability and clarity.


77-85: Refactor Listen Method for Event Handling.

The listen method effectively handles context cancellation and processes events. Ensure that onFinalizeBlock handles errors and edge cases appropriately.

mod/node-core/pkg/components/deposit_store.go (4)

29-29: Import Statement Update.

The addition of the async package is necessary for the new event handling mechanism. Ensure that all necessary imports are included and unused ones are removed.


67-67: Addition of Dispatcher to DepositPrunerInput.

The inclusion of Dispatcher enhances the event handling capabilities of the DepositPrunerInput. Ensure that the dispatcher is correctly initialized and used.


77-83: Refactor ProvideDepositPruner for Event Subscription.

The ProvideDepositPruner function now subscribes to BeaconBlockFinalizedEvent using the Dispatcher. This aligns with the event-driven architecture and improves scalability.


87-95: Update Return Statement in ProvideDepositPruner.

The return statement now uses subFinalizedBlocks, reflecting the shift to finalized block events. This change improves the accuracy and efficiency of deposit pruning.

mod/storage/pkg/pruner/pruner_test.go (4)

34-34: Import Statement Update.

The import of the async package is consistent with the new event handling mechanism. Ensure that all necessary imports are included and unused ones are removed.


42-42: Update pruneRangeFn Parameter Type.

The parameter type for event in pruneRangeFn is updated to async.Event[BlockT], aligning with the new import. This change maintains functionality while improving consistency.


74-74: Update Channel Type in TestPruner.

The channel type is updated to chan async.Event[pruner.BeaconBlock], ensuring uniformity in event handling across tests. This change is consistent with the new event handling mechanism.


95-97: Update Event Instantiation in TestPruner.

The event instantiation now uses async.NewEvent[pruner.BeaconBlock], reflecting the updated import and ensuring correct event creation.

mod/beacon/block_store/service.go (5)

26-28: Import Statement Update.

The import of async and async1 packages reflects the new event handling mechanism. Ensure that the aliases are used consistently throughout the file.


31-45: Addition of subFinalizedBlkEvents to Service Struct.

The subFinalizedBlkEvents channel is added to handle finalized block events, aligning with the event-driven architecture. Ensure this channel is properly managed to avoid leaks.


55-63: Update NewService Constructor for Event Handling.

The NewService constructor now initializes with dispatcher and sets up subFinalizedBlkEvents. This change supports the new event-driven architecture.


73-87: Refactor Start Method for Event Subscription.

The Start method now subscribes to finalized block events through the dispatcher, improving the service's responsiveness to events.


91-97: Implementation of eventLoop Method.

The eventLoop method replaces the previous listenAndStore function, providing a more structured approach to event processing.

mod/da/pkg/store/pruner_test.go (2)

29-29: Import Statement Update.

The import of the async package is consistent with the new event handling mechanism. Ensure that all necessary imports are included and unused ones are removed.


121-123: Update Event Instantiation in TestBuildPruneRangeFn.

The event instantiation now uses async.NewEvent[MockBeaconBlock], reflecting the updated import and ensuring correct event creation.

mod/node-core/pkg/components/block_store.go (2)

26-31: Import dispatcher package for event-driven architecture.

The inclusion of the dispatcher package signifies a shift towards an event-driven architecture, enhancing modularity and responsiveness in event handling.


77-80: Addition of Dispatcher field in BlockPrunerInput.

The Dispatcher field allows the BlockPrunerInput to utilize a centralized event dispatcher, improving the decoupling of components and enhancing event management.

mod/node-core/pkg/components/availability_store.go (3)

83-83: Event-Driven Design: Dispatcher Integration.

The addition of Dispatcher in AvailabilityPrunerInput reflects a move towards an event-driven architecture, enhancing modularity and responsiveness.


100-101: TODO Comment: Refactor Suggestion.

The TODO comment suggests that the provider should not execute business logic. This indicates a need for further refactoring to separate concerns.


104-108: Subscription Error Handling: Ensure Robustness.

Ensure that the error handling for Dispatcher.Subscribe is robust and consider logging additional context if available. This can help in diagnosing issues more effectively.

mod/beacon/blockchain/process.go (2)

27-28: Import async package for enhanced event handling.

The import of the async package supports the transition to a more structured event handling mechanism using the dispatcher.


85-87: Transition to dispatcher for event publishing.

The change from blkBroker.Publish to dispatcher.Publish reflects an architectural shift towards a more centralized and efficient event handling mechanism.

mod/async/pkg/broker/broker.go (9)

25-27: Import sync package for concurrency control.

The inclusion of the sync package supports the introduction of a mutex for managing concurrent access to the subscriptions map.


33-42: Enhancements to Broker for type safety and concurrency.

The Broker now uses a type constraint for async.BaseEvent, enhancing type safety. The addition of a mutex ensures thread-safe access to the subscriptions map.


57-59: Method EventID for retrieving event identifier.

The EventID method provides access to the broker's event identifier, supporting the event-centric design.


63-77: Refactor Start and start methods for improved operation.

The Start method now launches the broker loop in a goroutine, and the start method manages the loop, enhancing code organization and readability.


82-91: Refactor Publish method for improved message handling.

The Publish method now uses a type-checked message and context for publishing, enhancing flexibility and maintaining type safety.


98-110: Refactor Subscribe method for flexible subscription handling.

The Subscribe method now accepts channels as any type with internal type checks, enhancing flexibility while maintaining type safety.


113-124: Refactor Unsubscribe method for improved client management.

The Unsubscribe method now ensures type safety and proper resource management by closing channels upon unsubscription.


127-135: New broadcast method for message distribution.

The broadcast method encapsulates the logic for sending messages to all clients, improving code organization and readability.


138-144: New shutdown method for graceful resource management.

The shutdown method ensures that all subscriber channels are closed properly, preventing memory leaks and ensuring resource management.

mod/execution/pkg/deposit/service.go (5)

26-28: Import async packages for enhanced event handling.

The import of async packages supports the transition to a more structured and efficient event handling mechanism using the dispatcher.


48-51: Addition of dispatcher and subFinalizedBlockEvents fields in Service.

These fields enhance the service's capability to manage events in an event-driven architecture, improving responsiveness and modularity.


Line range hint 72-88: Refactor NewService constructor for event-driven architecture.

The constructor now accepts a dispatcher, aligning with the event-driven design. This change enhances the service's ability to handle events efficiently.


96-107: Refactor Start method for improved event management.

The Start method now subscribes to events using the dispatcher and launches an eventLoop, enhancing the service's responsiveness to blockchain events.


112-123: New eventLoop method for handling finalized block events.

The eventLoop method decouples event handling from the main processing logic, improving scalability and responsiveness to asynchronous events.

mod/runtime/pkg/middleware/middleware.go (3)

65-76: Centralize event handling with dispatcher.

The transition from multiple brokers to a single dispatcher simplifies the event handling architecture. Ensure the dispatcher is correctly initialized and used consistently across all relevant methods.


Line range hint 94-119: Update constructor parameters and initialization.

The NewABCIMiddleware function now requires a dispatcher instead of multiple brokers. This change improves modularity and reduces complexity. Ensure that all dependencies are correctly passed and initialized.


123-155: Streamline event subscription in Start method.

The Start method now subscribes to events using the dispatcher. This centralizes event management and reduces potential errors. Verify that all necessary events are subscribed to and handled appropriately.

mod/da/pkg/da/service.go (5)

44-47: Transition to dispatcher-based architecture.

The dispatcher replaces the sidecarsBroker, centralizing event management. The addition of subSidecarsReceived and subFinalBlobSidecars channels enhances event handling. Ensure these channels are correctly utilized in the service.


59-72: Update constructor for dispatcher integration.

The NewService function now accepts a dispatcher, aligning with the new event handling paradigm. Ensure the dispatcher is properly initialized and integrated with the service logic.


83-101: Refactor Start method for dispatcher usage.

The Start method now uses the dispatcher for event subscriptions, improving the control flow. Verify that all necessary events are subscribed to and handled correctly.


105-113: Implement eventLoop for event processing.

The eventLoop method processes events from the newly defined channels, enhancing responsiveness. Ensure that the loop handles context cancellation and processes events efficiently.


122-128: Refactor event handlers for clarity and functionality.

The event handlers have been renamed and updated to reflect their responsibilities better. Ensure that the handlers are correctly processing and emitting events as intended.

mod/storage/pkg/pruner/mocks/block_event.mock.go (5)

28-46: Add Context method to BlockEvent mock.

The Context method enhances the mock's ability to simulate context handling. Ensure that the method is correctly implemented and used in tests.


48-73: Implement BlockEvent_Context_Call for mock expectations.

This structure provides a type-safe way to define expectations for the Context method. Ensure that it is correctly integrated into the testing framework.


120-136: Add ID method to BlockEvent mock.

The ID method improves the mock's ability to simulate event identification. Ensure that the method is correctly implemented and used in tests.


138-163: Implement BlockEvent_ID_Call for mock expectations.

This structure provides a type-safe way to define expectations for the ID method. Ensure that it is correctly integrated into the testing framework.


Line range hint 166-178: Update Is method parameter type.

The parameter type for the Is method has been updated to async.EventID, ensuring consistency with the new event handling structure. Verify that this change aligns with the overall architecture.

mod/beacon/validator/service.go (5)

Line range hint 72-92: Integrate dispatcher into Service struct.

The addition of a dispatcher centralizes event management, replacing separate brokers. Ensure the dispatcher is correctly initialized and used throughout the service.


Line range hint 136-159: Update constructor for dispatcher integration.

The NewService function now requires a dispatcher, aligning with the new event handling paradigm. Ensure the dispatcher is properly initialized and integrated with the service logic.


170-183: Refactor Start method for dispatcher usage.

The Start method now uses the dispatcher for event subscriptions, improving the control flow. Verify that all necessary events are subscribed to and handled correctly.


187-195: Implement eventLoop for event processing.

The eventLoop method processes events from the subNewSlot channel, enhancing responsiveness. Ensure that the loop handles context cancellation and processes events efficiently.


200-229: Refactor handleNewSlot for improved clarity and functionality.

The handleNewSlot method now uses the dispatcher to emit events, improving consistency. Ensure that the method processes and emits events correctly.

mod/beacon/blockchain/service.go (5)

Line range hint 64-94: Integrate dispatcher into Service struct.

The addition of a dispatcher centralizes event management, replacing separate brokers. Ensure the dispatcher is correctly initialized and used throughout the service.


Line range hint 124-158: Update constructor for dispatcher integration.

The NewService function now requires a dispatcher, aligning with the new event handling paradigm. Ensure the dispatcher is properly initialized and integrated with the service logic.


169-194: Refactor Start method for dispatcher usage.

The Start method now uses the dispatcher for event subscriptions, improving the control flow. Verify that all necessary events are subscribed to and handled correctly.


198-211: Implement eventLoop for event processing.

The eventLoop method processes events from the newly defined channels, enhancing responsiveness. Ensure that the loop handles context cancellation and processes events efficiently.


222-316: Refactor event handlers for improved clarity and functionality.

The event handlers have been updated to reflect their responsibilities better. Ensure that the handlers are correctly processing and emitting events as intended.

mod/runtime/pkg/middleware/abci.go (10)

70-81: LGTM!

The waitForGenesisProcessed method correctly handles context cancellation and waits for the event.


134-144: LGTM!

The waitForBuiltBeaconBlock method correctly handles context cancellation and waits for the event.


148-158: LGTM!

The waitForBuiltSidecars method correctly handles context cancellation and waits for the event.


162-183: LGTM!

The handleBuiltBeaconBlockAndSidecars method effectively gossips the beacon block and sidecars while handling errors.


Line range hint 185-246: LGTM!

The ProcessProposal method correctly processes proposals, handles errors, and publishes events.


249-260: LGTM!

The waitForBeaconBlockVerification method correctly handles context cancellation and waits for the event.


264-274: LGTM!

The waitForSidecarVerification method correctly handles context cancellation and waits for the event.


Line range hint 276-288: LGTM!

The createProcessProposalResponse method correctly generates responses based on error status.


Line range hint 303-348: LGTM!

The FinalizeBlock method effectively finalizes the block, handles errors, and publishes events.


351-362: LGTM!

The waitForFinalValidatorUpdates method correctly handles context cancellation and waits for the event.

mod/node-core/pkg/components/types.go (20)

425-426: LGTM!

The GenesisDataReceivedEvent type alias enhances semantic clarity for event handling.


428-429: LGTM!

The GenesisDataProcessedEvent type alias enhances semantic clarity for event handling.


431-432: LGTM!

The NewSlotEvent type alias enhances semantic clarity for event handling.


434-435: LGTM!

The BuiltBeaconBlockEvent type alias enhances semantic clarity for event handling.


437-438: LGTM!

The BuiltSidecarsEvent type alias enhances semantic clarity for event handling.


440-441: LGTM!

The BeaconBlockReceivedEvent type alias enhances semantic clarity for event handling.


443-444: LGTM!

The SidecarsReceivedEvent type alias enhances semantic clarity for event handling.


446-447: LGTM!

The BeaconBlockVerifiedEvent type alias enhances semantic clarity for event handling.


449-450: LGTM!

The SidecarsVerifiedEvent type alias enhances semantic clarity for event handling.


452-453: LGTM!

The FinalBeaconBlockReceivedEvent type alias enhances semantic clarity for event handling.


455-456: LGTM!

The FinalSidecarsReceivedEvent type alias enhances semantic clarity for event handling.


458-459: LGTM!

The FinalValidatorUpdatesProcessedEvent type alias enhances semantic clarity for event handling.


461-462: LGTM!

The FinalizedBlockEvent type alias enhances semantic clarity for event handling.


467-468: LGTM!

The BlockMessage type alias enhances semantic clarity for message handling.


470-471: LGTM!

The GenesisMessage type alias enhances semantic clarity for message handling.


473-474: LGTM!

The SidecarMessage type alias enhances semantic clarity for message handling.


476-477: LGTM!

The SlotMessage type alias enhances semantic clarity for message handling.


479-480: LGTM!

The StatusMessage type alias enhances semantic clarity for message handling.


488-489: LGTM!

The BeaconBlockFinalizedPublisher type alias enhances semantic clarity for publisher handling.


496-497: LGTM!

The Dispatcher type alias enhances semantic clarity for dispatcher handling.

mod/storage/pkg/pruner/pruner.go Show resolved Hide resolved
mod/beacon/block_store/service.go Show resolved Hide resolved
mod/node-core/pkg/components/block_store.go Show resolved Hide resolved
mod/node-core/pkg/components/block_store.go Show resolved Hide resolved
mod/async/pkg/broker/broker.go Show resolved Hide resolved
mod/runtime/pkg/middleware/abci.go Show resolved Hide resolved
mod/runtime/pkg/middleware/abci.go Show resolved Hide resolved
@ocnc2 ocnc2 requested a review from itsdevbear August 19, 2024 19:55
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between 9cf5fad and 373e7fa.

Files selected for processing (3)
  • mod/node-core/pkg/components/types.go (3 hunks)
  • mod/runtime/pkg/middleware/middleware.go (4 hunks)
  • mod/runtime/pkg/middleware/types.go (2 hunks)
Additional comments not posted (7)
mod/runtime/pkg/middleware/types.go (1)

51-51: Type alias addition is clear and beneficial.

The addition of the validatorUpdates type alias enhances code clarity by providing a more descriptive reference to transition.ValidatorUpdates.

mod/runtime/pkg/middleware/middleware.go (3)

61-72: Centralized dispatcher integration is a positive change.

Replacing multiple brokers with a single dispatcher simplifies the event management architecture and enhances maintainability.


Line range hint 87-112: Constructor changes align with the new dispatcher model.

The constructor now initializes the dispatcher and subscription channels, reflecting the new centralized event handling approach.


116-148: Ensure all event subscriptions are correctly handled.

The Start method subscribes to various events using the dispatcher. Verify that all event types are correctly subscribed and handled.

Verification successful

Event subscriptions are correctly handled in the Start method.

The Start method in ABCIMiddleware subscribes to all necessary events and appropriately checks for errors during each subscription. This ensures that any issues are caught and handled. No changes are needed.

  • Event types subscribed: async.GenesisDataProcessed, async.BuiltBeaconBlock, async.BuiltSidecars, async.BeaconBlockVerified, async.SidecarsVerified, async.FinalValidatorUpdatesProcessed.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that all event types are correctly subscribed and handled.

# Test: Search for event subscription usage. Expect: Correct event types and handling logic.
rg --type go 'dispatcher.Subscribe'

Length of output: 1047


Script:

#!/bin/bash
# Description: Extract specific event types and their handling logic in the middleware.go file.

# Extract the event types and handling logic in the Start method of ABCIMiddleware.
ast-grep --lang go --pattern $'func (am *ABCIMiddleware[_, _, _, _]) Start($_) error {
  $$$
  am.dispatcher.Subscribe($_, $_)
  $$$
}' --json

Length of output: 146


Script:

#!/bin/bash
# Description: Extract specific event types and their handling logic in the middleware.go file.

# Use a more flexible pattern to match the Start method and extract event subscriptions.
ast-grep --lang go --pattern $'func (am *ABCIMiddleware[$$$]) Start($_) error {
  $$$
}' --json

Length of output: 2261

mod/node-core/pkg/components/types.go (3)

412-450: New event types enhance semantic clarity.

The introduction of new event types provides more descriptive and organized event handling, aligning with an event-driven architecture.


454-468: Message type updates reflect strategic improvements.

The updates to message types align with the new event-driven model, enhancing asynchronous handling.


475-475: Dispatcher type alias addition is beneficial.

The addition of the Dispatcher type alias supports efficient event management and flow control.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between 373e7fa and fa13257.

Files selected for processing (7)
  • mod/beacon/blockchain/execution_engine.go (1 hunks)
  • mod/beacon/blockchain/process.go (2 hunks)
  • mod/beacon/blockchain/service.go (7 hunks)
  • mod/beacon/validator/block_builder.go (1 hunks)
  • mod/beacon/validator/service.go (6 hunks)
  • mod/node-core/pkg/components/types.go (3 hunks)
  • mod/runtime/pkg/middleware/types.go (2 hunks)
Additional comments not posted (9)
mod/runtime/pkg/middleware/types.go (1)

49-49: Type Alias Added for Clarity.

The introduction of the validatorUpdates type alias enhances code readability by providing a clearer reference to transition.ValidatorUpdates.

mod/beacon/blockchain/execution_engine.go (1)

89-89: Enhanced Documentation and Logging.

The added documentation and streamlined error logging improve code clarity and readability.

mod/beacon/blockchain/process.go (1)

85-87: Improved Event Dispatching.

The shift from s.blkBroker.Publish to s.dispatcher.Publish and the renaming of the event to async.BeaconBlockFinalizedEvent enhance clarity and efficiency in event handling.

mod/beacon/validator/service.go (1)

64-65: Unified Event Dispatcher Implementation.

The introduction of a unified dispatcher for event handling streamlines the process, enhancing efficiency and maintainability. The updates to the Service struct and methods are consistent with this new model.

Also applies to: 83-84, 120-120, 142-143, 154-167, 178-179, 184-185, 188-204, 210-212

mod/beacon/blockchain/service.go (1)

59-60: Enhanced Event Handling with Unified Dispatcher.

The introduction of new channels and a unified dispatcher for event handling enhances modularity and maintainability. The updates to the Service struct and methods align with the new event handling paradigm.

Also applies to: 83-90, 115-115, 140-149, 160-185, 189-202, 213-241, 248-269, 276-307

mod/beacon/validator/block_builder.go (1)

39-40: Reordering of type parameters is acceptable.

The reordering of type parameters in the buildBlockAndSidecars function signature does not affect its functionality. This change likely improves readability or logical grouping.

mod/node-core/pkg/components/types.go (3)

409-447: Improved event type clarity and organization.

The renaming and addition of new event types enhance clarity and align naming conventions with their functional roles. This change improves the maintainability and scalability of the codebase.


451-465: Enhanced message type handling.

The introduction of new message types reflects a strategic shift towards improved asynchronous handling, enhancing the application's performance and scalability.


480-481: Addition of Dispatcher type alias is beneficial.

The addition of a Dispatcher type alias is essential for optimizing event and message management, contributing to efficient control flow within the application.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between fa13257 and f962cb8.

Files selected for processing (1)
  • mod/async/pkg/broker/broker.go (2 hunks)
Additional comments not posted (9)
mod/async/pkg/broker/broker.go (9)

33-44: Struct Broker enhancements look good.

The addition of type constraints and concurrency control mechanisms enhance type safety and thread safety.


60-61: Function EventID is correctly implemented.

The function correctly returns the eventID associated with the broker.


Line range hint 68-79: Function start is correctly implemented.

The function correctly handles context cancellation and delegates message broadcasting to the broadcast method.


86-94: Function Publish is correctly implemented.

The function ensures type safety and handles context cancellation appropriately.


104-112: Function Subscribe is correctly implemented.

The function ensures type safety and manages subscriptions with concurrency control.


117-126: Function Unsubscribe is correctly implemented.

The function ensures type safety, manages subscriptions with concurrency control, and closes the channel.


130-150: Function broadcast is correctly implemented.

The function manages concurrency and respects timeouts for each client.


154-159: Function shutdown is correctly implemented.

The function manages resource cleanup and error handling during shutdown.


65-66: Function Start is correctly implemented.

The function correctly initiates the broker loop in a separate goroutine.

Ensure proper context management in the calling code to handle the goroutine lifecycle.

Comment on lines +49 to +55
func New[T async.BaseEvent](eventID string) *Broker[T] {
return &Broker[T]{
clients: make(map[chan T]struct{}),
msgs: make(chan T, defaultBufferSize),
timeout: defaultTimeout,
name: name,
eventID: async.EventID(eventID),
subscriptions: make(map[chan T]struct{}),
msgs: make(chan T, defaultBufferSize),
timeout: defaultBrokerTimeout,
mu: sync.Mutex{},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider Adding Comments for Clarity in New Function.

Adding comments to explain the purpose of each field in the New function could enhance readability, especially for new developers.

// New creates a new broker publishing events of type T for the provided eventID.
func New[T async.BaseEvent](eventID string) *Broker[T] {
	return &Broker[T]{
		eventID:       async.EventID(eventID), // Unique identifier for the event type
		subscriptions: make(map[chan T]struct{}), // Map to manage client subscriptions
		msgs:          make(chan T, defaultBufferSize), // Channel for message publishing
		timeout:       defaultBrokerTimeout, // Timeout for message delivery
		mu:            sync.Mutex{}, // Mutex for concurrency safety
	}
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between f962cb8 and e67ef72.

Files selected for processing (1)
  • mod/async/pkg/broker/broker.go (2 hunks)
Additional comments not posted (10)
mod/async/pkg/broker/broker.go (10)

33-44: Type Safety and Event Management Improvements.

The use of async.BaseEvent as a type constraint enhances type safety. The addition of eventID improves event management. The subscriptions map and msgs channel are well-structured for handling events.


47-55: Consider Adding Comments for Clarity in New Function.

Adding comments to explain the purpose of each field in the New function could enhance readability, especially for new developers.


59-61: LGTM!

The EventID function correctly returns the eventID of the broker.


65-67: LGTM!

The Start function correctly initiates the broker loop in a goroutine and handles context cancellation.


Line range hint 69-79: LGTM!

The start function correctly processes messages and handles shutdown on context cancellation.


84-95: LGTM!

The Publish function ensures type safety and correctly handles message publishing with context cancellation.


100-112: LGTM!

The Subscribe function correctly registers channels with type safety and uses a mutex for concurrency safety.


115-126: LGTM!

The Unsubscribe function correctly handles unsubscription and ensures type safety.


129-139: LGTM!

The broadcast function correctly sends messages to all subscribed channels with a timeout.


142-148: LGTM!

The shutdown function correctly handles the shutdown process by closing all subscribed channels.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between e67ef72 and 117a69a.

Files selected for processing (2)
  • mod/runtime/pkg/middleware/abci.go (7 hunks)
  • mod/runtime/pkg/middleware/errors.go (2 hunks)
Additional comments not posted (17)
mod/runtime/pkg/middleware/errors.go (7)

33-37: Improved clarity in error description.

The updated description for ErrInvalidFinalizeBlockRequestType enhances clarity by accurately describing the error.


40-42: Addition of contextual error handling for genesis timeout.

The ErrInitGenesisTimeout function provides a clear and descriptive error message for genesis data processing timeouts.


44-46: Addition of contextual error handling for beacon block build timeout.

The ErrBuildBeaconBlockTimeout function adds valuable context to errors related to beacon block building timeouts.


48-50: Addition of contextual error handling for sidecars build timeout.

The ErrBuildSidecarsTimeout function enhances error context for timeouts during sidecar building.


52-54: Addition of contextual error handling for beacon block verification timeout.

The ErrVerifyBeaconBlockTimeout function provides detailed context for timeouts during beacon block verification.


56-58: Addition of contextual error handling for sidecars verification timeout.

The ErrVerifySidecarsTimeout function enhances error context for timeouts during sidecar verification.


60-62: Addition of contextual error handling for final validator updates timeout.

The ErrFinalValidatorUpdatesTimeout function provides specific context for timeouts during final validator updates.

mod/runtime/pkg/middleware/abci.go (10)

70-81: Effective handling of context cancellation and timeout.

The waitForGenesisProcessed function correctly handles context cancellation and timeout scenarios.


134-144: Effective handling of context cancellation and timeout.

The waitForBuiltBeaconBlock function correctly manages context cancellation and timeout scenarios.


148-159: Effective handling of context cancellation and timeout.

The waitForBuiltSidecars function correctly manages context cancellation and timeout scenarios.


162-183: Encapsulation of gossiping logic promotes modularity.

The handleBuiltBeaconBlockAndSidecars function effectively encapsulates the logic for gossiping built data, enhancing modularity.


Line range hint 199-246: Simplified asynchronous handling with context timeouts and event notifications.

The ProcessProposal function effectively uses context timeouts and manages event notifications for beacon blocks and sidecars.


249-260: Effective handling of context cancellation and timeout.

The waitForBeaconBlockVerification function correctly manages context cancellation and timeout scenarios.


264-274: Effective handling of context cancellation and timeout.

The waitForSidecarVerification function correctly manages context cancellation and timeout scenarios.


Line range hint 278-289: Effective error-based response creation.

The createProcessProposalResponse function correctly distinguishes between fatal and non-fatal errors to generate appropriate responses.


Line range hint 303-348: Simplified asynchronous handling with context timeouts and event notifications.

The FinalizeBlock function effectively uses context timeouts and manages event notifications for finalizing blocks and validator updates.


351-362: Effective handling of context cancellation and timeout.

The waitForFinalValidatorUpdates function correctly manages context cancellation and timeout scenarios.

Comment on lines +49 to +67
err error
waitCtx, cancel = context.WithTimeout(ctx, AwaitTimeout)
)
defer cancel()
// TODO: in theory the GenesisDataReceived channel should be empty, but we
// should clear it anyways here to ensure that data is valid.

data := new(GenesisT)
if err := json.Unmarshal(bz, data); err != nil {
return nil, err
}
// Send a request to the chain service to process the genesis data.
if err := h.genesisBroker.Publish(ctx, asynctypes.NewEvent(
ctx, events.GenesisDataProcessRequest, *data,
)); err != nil {
if err = json.Unmarshal(bz, data); err != nil {
h.logger.Error("Failed to unmarshal genesis data", "error", err)
return nil, err
}

// Wait for the genesis data to be processed.
g.Go(func() error {
valUpdates, genesisErr = h.waitForGenesisData(ctx)
return genesisErr
})

if err := g.Wait(); err != nil {
if err = h.dispatcher.Publish(
async.NewEvent(ctx, async.GenesisDataReceived, *data),
); err != nil {
return nil, err
}
return valUpdates, nil
return h.waitForGenesisProcessed(waitCtx)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use of context with timeout is appropriate.

The InitGenesis function correctly uses a context with a timeout to manage waiting for genesis data processing.

Consider addressing the TODO comment.

Clearing the GenesisDataReceived channel as suggested in the TODO could prevent processing stale data.

+ // Clear GenesisDataReceived channel
+ select {
+ case <-h.subGenDataProcessed:
+ default:
+ }
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
err error
waitCtx, cancel = context.WithTimeout(ctx, AwaitTimeout)
)
defer cancel()
// TODO: in theory the GenesisDataReceived channel should be empty, but we
// should clear it anyways here to ensure that data is valid.
data := new(GenesisT)
if err := json.Unmarshal(bz, data); err != nil {
return nil, err
}
// Send a request to the chain service to process the genesis data.
if err := h.genesisBroker.Publish(ctx, asynctypes.NewEvent(
ctx, events.GenesisDataProcessRequest, *data,
)); err != nil {
if err = json.Unmarshal(bz, data); err != nil {
h.logger.Error("Failed to unmarshal genesis data", "error", err)
return nil, err
}
// Wait for the genesis data to be processed.
g.Go(func() error {
valUpdates, genesisErr = h.waitForGenesisData(ctx)
return genesisErr
})
if err := g.Wait(); err != nil {
if err = h.dispatcher.Publish(
async.NewEvent(ctx, async.GenesisDataReceived, *data),
); err != nil {
return nil, err
}
return valUpdates, nil
return h.waitForGenesisProcessed(waitCtx)
err error
waitCtx, cancel = context.WithTimeout(ctx, AwaitTimeout)
)
defer cancel()
// TODO: in theory the GenesisDataReceived channel should be empty, but we
// should clear it anyways here to ensure that data is valid.
// Clear GenesisDataReceived channel
select {
case <-h.subGenDataProcessed:
default:
}
data := new(GenesisT)
if err = json.Unmarshal(bz, data); err != nil {
h.logger.Error("Failed to unmarshal genesis data", "error", err)
return nil, err
}
if err = h.dispatcher.Publish(
async.NewEvent(ctx, async.GenesisDataReceived, *data),
); err != nil {
return nil, err
}
return h.waitForGenesisProcessed(waitCtx)

Comment on lines +91 to +131
BeaconBlockT, BlobSidecarsT, _, SlotDataT,
]) PrepareProposal(
ctx context.Context,
slotData SlotDataT,
) ([]byte, []byte, error) {
var (
g errgroup.Group
startTime = time.Now()
beaconBlockErr, sidecarsErr error
beaconBlockBz, sidecarsBz []byte
err error
builtBeaconBlock BeaconBlockT
builtSidecars BlobSidecarsT
startTime = time.Now()
awaitCtx, cancel = context.WithTimeout(ctx, AwaitTimeout)
)
// TODO: clear the built beacon block and sidecars channels, else we may
// end up handling old data from previous slots.
defer cancel()
defer h.metrics.measurePrepareProposalDuration(startTime)

// Send a request to the validator service to give us a beacon block
// and blob sidecards to pass to ABCI.
if err := h.slotBroker.Publish(ctx, asynctypes.NewEvent(
ctx, events.NewSlot, slotData,
)); err != nil {
if err = h.dispatcher.Publish(
async.NewEvent(
ctx, async.NewSlot, slotData,
),
); err != nil {
return nil, nil, err
}

// Wait for the beacon block to be built.
g.Go(func() error {
beaconBlockBz, beaconBlockErr = h.waitforBeaconBlk(ctx)
return beaconBlockErr
})

// Wait for the sidecars to be built.
g.Go(func() error {
sidecarsBz, sidecarsErr = h.waitForSidecars(ctx)
return sidecarsErr
})

// Wait for both processes to complete and then
// return the appropriate response.
return beaconBlockBz, sidecarsBz, g.Wait()
// wait for built beacon block
builtBeaconBlock, err = h.waitForBuiltBeaconBlock(awaitCtx)
if err != nil {
return nil, nil, err
}

// wait for built sidecars
builtSidecars, err = h.waitForBuiltSidecars(awaitCtx)
if err != nil {
return nil, nil, err
}

// gossip the built beacon block and blob sidecars
return h.handleBuiltBeaconBlockAndSidecars(
ctx, builtBeaconBlock, builtSidecars,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified asynchronous handling with context timeouts.

The PrepareProposal function effectively uses context timeouts to manage asynchronous operations.

Consider addressing the TODO comment.

Clearing the built beacon block and sidecars channels as suggested in the TODO could prevent handling stale data.

+ // Clear built beacon block and sidecars channels
+ select {
+ case <-h.subBuiltBeaconBlock:
+ default:
+ }
+ select {
+ case <-h.subBuiltSidecars:
+ default:
+ }
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
BeaconBlockT, BlobSidecarsT, _, SlotDataT,
]) PrepareProposal(
ctx context.Context,
slotData SlotDataT,
) ([]byte, []byte, error) {
var (
g errgroup.Group
startTime = time.Now()
beaconBlockErr, sidecarsErr error
beaconBlockBz, sidecarsBz []byte
err error
builtBeaconBlock BeaconBlockT
builtSidecars BlobSidecarsT
startTime = time.Now()
awaitCtx, cancel = context.WithTimeout(ctx, AwaitTimeout)
)
// TODO: clear the built beacon block and sidecars channels, else we may
// end up handling old data from previous slots.
defer cancel()
defer h.metrics.measurePrepareProposalDuration(startTime)
// Send a request to the validator service to give us a beacon block
// and blob sidecards to pass to ABCI.
if err := h.slotBroker.Publish(ctx, asynctypes.NewEvent(
ctx, events.NewSlot, slotData,
)); err != nil {
if err = h.dispatcher.Publish(
async.NewEvent(
ctx, async.NewSlot, slotData,
),
); err != nil {
return nil, nil, err
}
// Wait for the beacon block to be built.
g.Go(func() error {
beaconBlockBz, beaconBlockErr = h.waitforBeaconBlk(ctx)
return beaconBlockErr
})
// Wait for the sidecars to be built.
g.Go(func() error {
sidecarsBz, sidecarsErr = h.waitForSidecars(ctx)
return sidecarsErr
})
// Wait for both processes to complete and then
// return the appropriate response.
return beaconBlockBz, sidecarsBz, g.Wait()
// wait for built beacon block
builtBeaconBlock, err = h.waitForBuiltBeaconBlock(awaitCtx)
if err != nil {
return nil, nil, err
}
// wait for built sidecars
builtSidecars, err = h.waitForBuiltSidecars(awaitCtx)
if err != nil {
return nil, nil, err
}
// gossip the built beacon block and blob sidecars
return h.handleBuiltBeaconBlockAndSidecars(
ctx, builtBeaconBlock, builtSidecars,
)
BeaconBlockT, BlobSidecarsT, _, SlotDataT,
]) PrepareProposal(
ctx context.Context,
slotData SlotDataT,
) ([]byte, []byte, error) {
var (
err error
builtBeaconBlock BeaconBlockT
builtSidecars BlobSidecarsT
startTime = time.Now()
awaitCtx, cancel = context.WithTimeout(ctx, AwaitTimeout)
)
// Clear built beacon block and sidecars channels
select {
case <-h.subBuiltBeaconBlock:
default:
}
select {
case <-h.subBuiltSidecars:
default:
}
// TODO: clear the built beacon block and sidecars channels, else we may
// end up handling old data from previous slots.
defer cancel()
defer h.metrics.measurePrepareProposalDuration(startTime)
if err = h.dispatcher.Publish(
async.NewEvent(
ctx, async.NewSlot, slotData,
),
); err != nil {
return nil, nil, err
}
// wait for built beacon block
builtBeaconBlock, err = h.waitForBuiltBeaconBlock(awaitCtx)
if err != nil {
return nil, nil, err
}
// wait for built sidecars
builtSidecars, err = h.waitForBuiltSidecars(awaitCtx)
if err != nil {
return nil, nil, err
}
// gossip the built beacon block and blob sidecars
return h.handleBuiltBeaconBlockAndSidecars(
ctx, builtBeaconBlock, builtSidecars,
)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between 117a69a and 3cb0bfb.

Files selected for processing (1)
  • mod/runtime/pkg/middleware/errors.go (2 hunks)
Additional comments not posted (8)
mod/runtime/pkg/middleware/errors.go (8)

23-23: LGTM! Custom errors package import.

The import statement has been updated to use a custom errors package, which likely enhances error handling capabilities.


34-38: LGTM! Improved error description.

The description for ErrInvalidFinalizeBlockRequestType has been updated to accurately reflect the error context.


41-45: LGTM! New error function for genesis timeout.

The function ErrInitGenesisTimeout provides a clear and descriptive error message for timeout issues during genesis data processing.


47-51: LGTM! New error function for beacon block build timeout.

The function ErrBuildBeaconBlockTimeout provides a clear and descriptive error message for timeout issues during beacon block building.


53-57: LGTM! New error function for sidecars build timeout.

The function ErrBuildSidecarsTimeout provides a clear and descriptive error message for timeout issues during sidecars building.


60-64: LGTM! New error function for beacon block verification timeout.

The function ErrVerifyBeaconBlockTimeout provides a clear and descriptive error message for timeout issues during beacon block verification.


66-70: LGTM! New error function for sidecars verification timeout.

The function ErrVerifySidecarsTimeout provides a clear and descriptive error message for timeout issues during sidecars verification.


72-76: LGTM! New error function for final validator updates timeout.

The function ErrFinalValidatorUpdatesTimeout provides a clear and descriptive error message for timeout issues during final validator updates.

@ocnc2 ocnc2 changed the title feat(async): dispatcher fr this time feat(async): dispatcher Aug 20, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between 3cb0bfb and e5bad5b.

Files ignored due to path filters (7)
  • mod/async/go.sum is excluded by !**/*.sum
  • mod/beacon/go.sum is excluded by !**/*.sum
  • mod/da/go.sum is excluded by !**/*.sum
  • mod/execution/go.sum is excluded by !**/*.sum
  • mod/node-core/go.sum is excluded by !**/*.sum
  • mod/runtime/go.sum is excluded by !**/*.sum
  • mod/storage/go.sum is excluded by !**/*.sum
Files selected for processing (8)
  • mod/async/go.mod (1 hunks)
  • mod/beacon/go.mod (2 hunks)
  • mod/da/go.mod (1 hunks)
  • mod/execution/go.mod (1 hunks)
  • mod/node-core/go.mod (3 hunks)
  • mod/runtime/go.mod (3 hunks)
  • mod/runtime/pkg/middleware/errors.go (2 hunks)
  • mod/storage/go.mod (2 hunks)
Additional comments not posted (9)
mod/async/go.mod (1)

7-24: Dependencies updated successfully.

The addition of new dependencies enhances the module's functionality, particularly in error handling and logging. Ensure that these dependencies are used effectively in the codebase.

mod/runtime/pkg/middleware/errors.go (1)

Line range hint 23-75: Enhanced error handling is well-implemented.

The introduction of specific timeout error functions and the update to ErrInvalidFinalizeBlockRequestType improve clarity and robustness in error reporting. Ensure these new error functions are integrated and tested in the relevant parts of the codebase.

Run the following script to verify the usage of new error functions:

Verification successful

Error functions are correctly integrated in relevant contexts.

The new error functions for handling timeouts are used appropriately in the mod/runtime/pkg/middleware/abci.go file, confirming their integration into the codebase. Each function is applied in a context that aligns with its intended purpose.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of new error functions in the codebase.

# Test: Search for the usage of each new error function. Expect: Functions are used in relevant contexts.
rg --type go -A 5 'ErrInitGenesisTimeout|ErrBuildBeaconBlockTimeout|ErrBuildSidecarsTimeout|ErrVerifyBeaconBlockTimeout|ErrVerifySidecarsTimeout|ErrFinalValidatorUpdatesTimeout'

Length of output: 4671

mod/beacon/go.mod (1)

14-14: Dependency version updated.

The errors module version update ensures compatibility with recent changes. Ensure that any new features or fixes in this version are utilized.

mod/execution/go.mod (1)

13-16: Dependency versions updated.

The updates to the errors, log, and primitives modules ensure compatibility with recent changes. Ensure that any new features or fixes in these versions are utilized.

mod/da/go.mod (1)

11-17: Dependency versions updated.

The updates to the async, errors, and log modules ensure compatibility with recent changes. Ensure that any new features or fixes in these versions are utilized.

mod/storage/go.mod (1)

8-8: Verify the local replacement path for primitives.

Ensure that the local path ../primitives exists and is accessible for development.

Run the following script to verify the existence of the local replacement:

Verification successful

Local replacement path for primitives verified successfully.

The directory mod/primitives/ exists, confirming that the local replacement path ../primitives is valid and accessible for development.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the existence of the local replacement for `primitives`.

# Test: Check if the local directory exists.
test -d ../primitives && echo "Local replacement for primitives exists."

Length of output: 23


Script:

#!/bin/bash
# Search for the `primitives` directory within the repository to verify its existence.
fd -t d primitives

Length of output: 125

mod/runtime/go.mod (2)

13-14: Verify the local replacement paths for async and primitives.

Ensure that the local paths ../async and ../primitives exist and are accessible for development.

Run the following script to verify the existence of the local replacements:


23-24: Check compatibility of updated dependencies errors and log.

Ensure that the updates to github.com/berachain/beacon-kit/mod/errors and github.com/berachain/beacon-kit/mod/log are compatible with the rest of the codebase.

Run the following script to check for references to the updated dependencies:

mod/node-core/go.mod (1)

35-42: Check compatibility of updated dependencies.

Ensure that the updates to engine-primitives, errors, log, primitives, runtime, and async are compatible with the rest of the codebase.

Run the following script to check for references to the updated dependencies:

Comment on lines +5 to +9
replace (
github.com/berachain/beacon-kit/mod/async => ../async
github.com/berachain/beacon-kit/mod/engine-primitives => ../engine-primitives
github.com/berachain/beacon-kit/mod/primitives => ../primitives
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Local paths do not exist

The local directories specified in the replace directives (../async, ../engine-primitives, ../primitives) do not exist. Please verify the paths or ensure that the necessary directories are created for successful local development and testing.

  • Ensure that the paths in the replace directives are correct.
  • Create the directories if they are missing.
Analysis chain

Local development paths added successfully.

The replace directives for local paths facilitate easier local development and testing. Ensure that these paths are correct and accessible.

Run the following script to verify the existence of local directories:

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the local paths for `async`, `engine-primitives`, and `primitives` exist.

# Test: Check if the directories exist. Expect: Directories should exist.
[ -d "../async" ] && echo "Async directory exists" || echo "Async directory does not exist"
[ -d "../engine-primitives" ] && echo "Engine-primitives directory exists" || echo "Engine-primitives directory does not exist"
[ -d "../primitives" ] && echo "Primitives directory exists" || echo "Primitives directory does not exist"

Length of output: 329

Comment on lines +5 to +8
replace (
github.com/berachain/beacon-kit/mod/async => ../async
github.com/berachain/beacon-kit/mod/primitives => ../primitives
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Local paths for replace directives do not exist.

The replace directives in the go.mod file point to local directories ../async and ../primitives, which do not exist. This will prevent the intended local development and testing. Please ensure these paths are correct and the directories are accessible.

  • ../async directory does not exist.
  • ../primitives directory does not exist.
Analysis chain

Local development paths added successfully.

The replace directives for local paths facilitate easier local development and testing. Ensure that these paths are correct and accessible.

Run the following script to verify the existence of local directories:

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the local paths for `async` and `primitives` exist.

# Test: Check if the directories exist. Expect: Directories should exist.
[ -d "../async" ] && echo "Async directory exists" || echo "Async directory does not exist"
[ -d "../primitives" ] && echo "Primitives directory exists" || echo "Primitives directory does not exist"

Length of output: 200

Comment on lines +5 to +8
replace (
github.com/berachain/beacon-kit/mod/async => ../async
github.com/berachain/beacon-kit/mod/primitives => ../primitives
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Local directories for replace directives do not exist.

The local paths specified in the replace directives for async and primitives do not exist. This will prevent proper local development and testing. Please ensure that these directories are created or correctly linked.

  • ../async directory does not exist.
  • ../primitives directory does not exist.
Analysis chain

Local development paths added successfully.

The replace directives for local paths facilitate easier local development and testing. Ensure that these paths are correct and accessible.

Run the following script to verify the existence of local directories:

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the local paths for `async` and `primitives` exist.

# Test: Check if the directories exist. Expect: Directories should exist.
[ -d "../async" ] && echo "Async directory exists" || echo "Async directory does not exist"
[ -d "../primitives" ] && echo "Primitives directory exists" || echo "Primitives directory does not exist"

Length of output: 200

@@ -5,6 +5,7 @@ go 1.23.0
replace (
cosmossdk.io/api => cosmossdk.io/api v0.7.3-0.20240806152830-8fb47b368cd4
cosmossdk.io/core => cosmossdk.io/core v0.12.1-0.20240806152830-8fb47b368cd4
github.com/berachain/beacon-kit/mod/primitives => ../primitives
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Potential issues due to removal of async package

The async package is still being referenced in multiple files across the codebase. Removing it from go.mod without updating these references could lead to errors. Please ensure that all dependencies on the async package are resolved or updated accordingly.

  • Files with references to async include:
    • mod/storage/pkg/pruner/pruner.go
    • mod/storage/pkg/pruner/pruner_test.go
    • mod/storage/pkg/pruner/types.go
    • mod/runtime/pkg/middleware/middleware.go
    • mod/runtime/pkg/middleware/abci.go
    • mod/node-core/pkg/components/dispatcher.go
    • mod/node-core/pkg/components/deposit_store.go
    • mod/node-core/pkg/components/types.go
    • mod/node-core/pkg/components/availability_store.go
    • mod/node-core/pkg/components/events.go
    • mod/node-core/pkg/components/block_store.go
    • mod/async/pkg/types/dispatcher.go
    • mod/execution/pkg/deposit/pruner.go
    • mod/execution/pkg/deposit/sync.go
    • mod/execution/pkg/deposit/types.go
    • mod/execution/pkg/deposit/service.go
    • mod/async/pkg/dispatcher/errors.go
    • mod/async/pkg/dispatcher/dispatcher.go
    • mod/async/pkg/broker/broker.go
    • mod/da/pkg/da/service.go
    • mod/da/pkg/store/pruner_test.go
    • mod/da/pkg/store/pruner.go
    • mod/beacon/blockchain/process.go
    • mod/beacon/blockchain/service.go
    • mod/beacon/validator/service.go
    • mod/beacon/block_store/types.go
    • mod/beacon/block_store/service.go
    • mod/beacon/block_store/pruner.go
Analysis chain

Check the impact of removing async.

The removal of github.com/berachain/beacon-kit/mod/async might affect the module's functionality. Ensure that any dependent code has been updated accordingly.

Run the following script to check for references to the removed dependency:

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for references to the removed `async` dependency.

# Test: Search for references to `async` in the codebase.
rg --type go 'async'

Length of output: 19473

Comment on lines +10 to +20
github.com/berachain/beacon-kit/mod/async => ../async
github.com/berachain/beacon-kit/mod/beacon => ../beacon
github.com/berachain/beacon-kit/mod/da => ../da
github.com/berachain/beacon-kit/mod/execution => ../execution
github.com/berachain/beacon-kit/mod/node-api => ../node-api
github.com/berachain/beacon-kit/mod/node-api/engines => ../node-api/engines
github.com/berachain/beacon-kit/mod/payload => ../payload
github.com/berachain/beacon-kit/mod/primitives => ../primitives
github.com/berachain/beacon-kit/mod/runtime => ../runtime
github.com/berachain/beacon-kit/mod/state-transition => ../state-transition
github.com/berachain/beacon-kit/mod/storage => ../storage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Local replacement paths do not exist

The local replacement paths specified in the go.mod file do not exist in the current setup. Please verify the paths and ensure that the necessary directories are initialized or checked out correctly.

  • async does not exist at /home/jailuser/git/async
  • beacon does not exist at /home/jailuser/git/beacon
  • da does not exist at /home/jailuser/git/da
  • execution does not exist at /home/jailuser/git/execution
  • node-api does not exist at /home/jailuser/git/node-api
  • node-api/engines does not exist at /home/jailuser/git/node-api/engines
  • payload does not exist at /home/jailuser/git/payload
  • primitives does not exist at /home/jailuser/git/primitives
  • runtime does not exist at /home/jailuser/git/runtime
  • state-transition does not exist at /home/jailuser/git/state-transition
  • storage does not exist at /home/jailuser/git/storage

Please ensure these directories are correctly set up for development.

Analysis chain

Verify the local replacement paths for multiple modules.

Ensure that the local paths for async, beacon, da, execution, node-api, node-api/engines, payload, primitives, runtime, state-transition, and storage exist and are accessible for development.

Run the following script to verify the existence of the local replacements:

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the existence of local replacements for multiple modules.

# Test: Check if the local directories exist.
for module in async beacon da execution node-api node-api/engines payload primitives runtime state-transition storage; do
  test -d "../$module" && echo "Local replacement for $module exists."
done

Length of output: 1566


Script:

#!/bin/bash
# Description: Verify the existence of local replacements for multiple modules using absolute paths.

# Find the base directory of the repository
BASE_DIR=$(pwd)

# Test: Check if the local directories exist using absolute paths.
for module in async beacon da execution node-api node-api/engines payload primitives runtime state-transition storage; do
  if [ -d "$BASE_DIR/$module" ]; then
    echo "Local replacement for $module exists at $BASE_DIR/$module."
  else
    echo "Local replacement for $module does not exist at $BASE_DIR/$module."
  fi
done

Length of output: 3647

@itsdevbear itsdevbear merged commit 55c4ff7 into main Aug 20, 2024
15 checks passed
@itsdevbear itsdevbear deleted the dispatcher-fr branch August 20, 2024 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants