-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rollup node manager #21
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added comments inline.
.github/workflows/lint.yaml
Outdated
@@ -117,7 +117,7 @@ jobs: | |||
include: | |||
- type: wasm | |||
target: wasm32-unknown-unknown | |||
exclude: engine,scroll-wire,scroll-bridge,scroll-network | |||
exclude: scroll-engine,scroll-wire,scroll-bridge,scroll-network,scroll-rollup-node |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently all our creates are being skipped.
alloy-rpc-types-engine = { version = "0.11.0", default-features = false } | ||
|
||
# scroll-alloy | ||
scroll-alloy-provider = { git = "https://github.com/scroll-tech/reth.git", branch = "feat/add-deref-blanket-engine-api" } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
before merge we should merge feat/add-deref-blanket-engine-api
into scroll
and then revert back to default on these dependency definitions.
@@ -0,0 +1,228 @@ | |||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently blocks 2 - 5 can not be read as this dump does not encode the data using serde. It is instead sourced from the rpc response using cast. We should replace this with serde data such that we can deserialize in testing.
crates/scroll-wire/Cargo.toml
Outdated
|
||
# scroll | ||
reth-scroll-primitives = { workspace = true, features = ["serde"] } | ||
|
||
# misc | ||
eyre.workspace = true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, I think the refactor of the Networking make the code flow clearer! Just couple of nits and questions
@@ -110,7 +102,7 @@ mod tests { | |||
} | |||
|
|||
#[test] | |||
fn test_matching_payloads() -> Result<()> { | |||
fn test_matching_payloads() -> eyre::Result<()> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
instead of custom errors you mean? I don't have a strong opinion on this, if it simplifies the code let's use eyre everywhere
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a remark on the engine, otherwise lgtm!
let payload_status = self.new_payload(execution_payload).await?; | ||
|
||
// Invoke the FCU with the new state. | ||
let fcu = self.forkchoice_updated(fcs, None).await?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we do fcs.safe_block_hash = unsafe_block_info.hash;
before issuing the forkchoice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The logic below defines what is currently implemented to manage fcs. We get the current fcs from the node manager and then set the head block hash to the block hash of the unsafe block received from the network when defining the future for importing the block. Do you think we should be setting the safe block hash when we receive a new block over p2p?
rollup-node/crates/node/src/lib.rs
Lines 145 to 178 in 04a5f44
// Send the block to the engine to validate the correctness of the block. | |
let mut fcs = self.get_alloy_fcs(); | |
let engine = self.engine.clone(); | |
let future = Box::pin(async move { | |
trace!(target: "scroll::node::manager", "handling block import future for block {:?}", block.hash_slow()); | |
// convert the block to an execution payload and update the forkchoice state | |
let execution_payload: ExecutionPayload = | |
ExecutionPayloadV1::from_block_slow(&block).into(); | |
let unsafe_block_info: BlockInfo = (&execution_payload).into(); | |
fcs.head_block_hash = unsafe_block_info.hash; | |
// process the execution payload | |
// TODO: needs testing | |
let (unsafe_block_info, import_outcome) = | |
match engine.handle_execution_payload(execution_payload, fcs).await { | |
Err(EngineDriverError::InvalidExecutionPayload) => ( | |
Some(unsafe_block_info), | |
Some(Err(BlockImportError::Validation(BlockValidationError::InvalidBlock))), | |
), | |
Ok((PayloadStatusEnum::Valid, PayloadStatusEnum::Valid)) => ( | |
Some(unsafe_block_info), | |
Some(Ok(BlockValidation::ValidBlock { | |
new_block: NewBlock { | |
block, | |
signature: signature.serialize_compact().to_vec().into(), | |
}, | |
})), | |
), | |
_ => (None, None), | |
}; | |
(unsafe_block_info, import_outcome.map(|result| BlockImportOutcome { peer, result })) | |
}); |
Upon block import result if success we update the fcs of the node manager.
rollup-node/crates/node/src/lib.rs
Lines 197 to 214 in 04a5f44
/// Handles a block import outcome. | |
fn handle_block_import_outcome( | |
&mut self, | |
unsafe_block_info: Option<BlockInfo>, | |
outcome: Option<BlockImportOutcome>, | |
) { | |
trace!(target: "scroll::node::manager", ?outcome, "handling block import outcome"); | |
// If we have an unsafe block info, update the forkchoice state. | |
if let Some(unsafe_block_info) = unsafe_block_info { | |
self.forkchoice_state.update_unsafe_block_info(unsafe_block_info); | |
} | |
// If we have an outcome, send it to the network manager. | |
if let Some(outcome) = outcome { | |
self.network.handle().block_import_outcome(outcome); | |
} | |
} |
This works ok but I'm thinking we should introduce a State
type that can be shared across async tasks (type State = Arc<Mutex<InnerState>>
* init bridge implementation * extend bridge implementation * refactor and clean up dependencies * add .vscode to .gitignore * lint * spacing * cleanup * wrap inner block import in bridge * add bridge integration test * refactor and clean up * add comments * migrate shared dependencies to workspace * feature propogation * lints and feature fix * remove scroll feature * add scroll feature * integrate reth upstream changes * address PR feedback * stash * add systemd service file * update systemd file * update systemd file * address comments * rollup node manager * merge main * add Cargo.lock * lint * lint * lint * lint * ci remove no-dev-deps * fix serde dep - scroll-wire * skip no_std on scroll-engine and scroll-rollup-node * skip no_std on scroll-engine and scroll-rollup-node * address comments * lints * cleanup * tsets * lint + refactor * refactor engine api call ordering * lint * fix fcu * lint * remove syncing error return on new payload * update forkchoice state after succesfully new payload * update reth following chainspec fork id logic change * PR feedback * lints * lints
* ci: add dprint * feat: initial work + handle_latest_block * Rollup node manager (#21) * init bridge implementation * extend bridge implementation * refactor and clean up dependencies * add .vscode to .gitignore * lint * spacing * cleanup * wrap inner block import in bridge * add bridge integration test * refactor and clean up * add comments * migrate shared dependencies to workspace * feature propogation * lints and feature fix * remove scroll feature * add scroll feature * integrate reth upstream changes * address PR feedback * stash * add systemd service file * update systemd file * update systemd file * address comments * rollup node manager * merge main * add Cargo.lock * lint * lint * lint * lint * ci remove no-dev-deps * fix serde dep - scroll-wire * skip no_std on scroll-engine and scroll-rollup-node * skip no_std on scroll-engine and scroll-rollup-node * address comments * lints * cleanup * tsets * lint + refactor * refactor engine api call ordering * lint * fix fcu * lint * remove syncing error return on new payload * update forkchoice state after succesfully new payload * update reth following chainspec fork id logic change * PR feedback * lints * lints * feat(watcher): reorg detection * feat(primitives): add batch and l1 transaction * feat(watcher): contracts and constants * feat(watcher): imports * feat(watcher): add error types * feat(watcher): handle l1 messages and commit logs * feat: handle batch commit and finalize * test: handle_latest_block * test: wip events decoding * test: commit and finalize batch logs * test: utils * feat: tracing * test: wip reorg integration * test: reorg detection integration test * test: gaps * fix: lints * feat: point deps to default reth branch * fix: security issue on ring * fix: bump alloy * feat: move abi to separate crate * chore: remove ArbitraryTxBuilder * fix: answer comments * fix: lints * fix: no std issues * fix: cargo features check * feat: bounded vec * fix: lints --------- Co-authored-by: frisitano <35734660+frisitano@users.noreply.github.com>
* ci: add dprint * feat: initial work + handle_latest_block * Rollup node manager (#21) * init bridge implementation * extend bridge implementation * refactor and clean up dependencies * add .vscode to .gitignore * lint * spacing * cleanup * wrap inner block import in bridge * add bridge integration test * refactor and clean up * add comments * migrate shared dependencies to workspace * feature propogation * lints and feature fix * remove scroll feature * add scroll feature * integrate reth upstream changes * address PR feedback * stash * add systemd service file * update systemd file * update systemd file * address comments * rollup node manager * merge main * add Cargo.lock * lint * lint * lint * lint * ci remove no-dev-deps * fix serde dep - scroll-wire * skip no_std on scroll-engine and scroll-rollup-node * skip no_std on scroll-engine and scroll-rollup-node * address comments * lints * cleanup * tsets * lint + refactor * refactor engine api call ordering * lint * fix fcu * lint * remove syncing error return on new payload * update forkchoice state after succesfully new payload * update reth following chainspec fork id logic change * PR feedback * lints * lints * feat(watcher): reorg detection * feat(primitives): add batch and l1 transaction * feat(watcher): contracts and constants * feat(watcher): imports * feat(watcher): add error types * feat(watcher): handle l1 messages and commit logs * feat: handle batch commit and finalize * test: handle_latest_block * test: wip events decoding * test: commit and finalize batch logs * test: utils * feat: tracing * test: wip reorg integration * test: reorg detection integration test * test: gaps * fix: lints * feat: point deps to default reth branch * fix: security issue on ring * fix: bump alloy * feat: move abi to separate crate * chore: remove ArbitraryTxBuilder * init database * add database iterator * add test * update streaming implementation * fix: answer comments * fix: lints * fix: no std issues * fix: cargo features check * extend and refactor database, migrations and batch primitive * lint * lint * lint * improve database implementation * lint * lint * database transaction refactor * lint: * use GAT for DatabaseConnectionProvider trait * refactor & lint * refactor databse interface for insert_batch_input * lint: * lint --------- Co-authored-by: Gregory Edison <gregory.edison1993@gmail.com>
* ci: add dprint * feat: initial work + handle_latest_block * Rollup node manager (#21) * init bridge implementation * extend bridge implementation * refactor and clean up dependencies * add .vscode to .gitignore * lint * spacing * cleanup * wrap inner block import in bridge * add bridge integration test * refactor and clean up * add comments * migrate shared dependencies to workspace * feature propogation * lints and feature fix * remove scroll feature * add scroll feature * integrate reth upstream changes * address PR feedback * stash * add systemd service file * update systemd file * update systemd file * address comments * rollup node manager * merge main * add Cargo.lock * lint * lint * lint * lint * ci remove no-dev-deps * fix serde dep - scroll-wire * skip no_std on scroll-engine and scroll-rollup-node * skip no_std on scroll-engine and scroll-rollup-node * address comments * lints * cleanup * tsets * lint + refactor * refactor engine api call ordering * lint * fix fcu * lint * remove syncing error return on new payload * update forkchoice state after succesfully new payload * update reth following chainspec fork id logic change * PR feedback * lints * lints * feat(watcher): reorg detection * feat(primitives): add batch and l1 transaction * feat(watcher): contracts and constants * feat(watcher): imports * feat(watcher): add error types * feat(watcher): handle l1 messages and commit logs * feat: handle batch commit and finalize * test: handle_latest_block * test: wip events decoding * test: commit and finalize batch logs * test: utils * feat: tracing * test: wip reorg integration * test: reorg detection integration test * test: gaps * fix: lints * feat: point deps to default reth branch * fix: security issue on ring * fix: bump alloy * feat: move abi to separate crate * chore: remove ArbitraryTxBuilder * init database * add database iterator * add test * update streaming implementation * init indexer * fix: answer comments * fix: lints * fix: no std issues * fix: cargo features check * extend indexer * extend and refactor database, migrations and batch primitive * lint * lint * lint * improve database implementation * lint * lint * database transaction refactor * lint: * use GAT for DatabaseConnectionProvider trait * refactor & lint * merge database and refactor * clippy fix * skip rolllup-node-indexer no_std * refactor databse interface for insert_batch_input * refactor * lint * refactor indexer * refactor indexer * lint: * lint --------- Co-authored-by: Gregory Edison <gregory.edison1993@gmail.com>
* ci: add dprint * feat: initial work + handle_latest_block * Rollup node manager (#21) * init bridge implementation * extend bridge implementation * refactor and clean up dependencies * add .vscode to .gitignore * lint * spacing * cleanup * wrap inner block import in bridge * add bridge integration test * refactor and clean up * add comments * migrate shared dependencies to workspace * feature propogation * lints and feature fix * remove scroll feature * add scroll feature * integrate reth upstream changes * address PR feedback * stash * add systemd service file * update systemd file * update systemd file * address comments * rollup node manager * merge main * add Cargo.lock * lint * lint * lint * lint * ci remove no-dev-deps * fix serde dep - scroll-wire * skip no_std on scroll-engine and scroll-rollup-node * skip no_std on scroll-engine and scroll-rollup-node * address comments * lints * cleanup * tsets * lint + refactor * refactor engine api call ordering * lint * fix fcu * lint * remove syncing error return on new payload * update forkchoice state after succesfully new payload * update reth following chainspec fork id logic change * PR feedback * lints * lints * feat(watcher): reorg detection * feat(primitives): add batch and l1 transaction * feat(watcher): contracts and constants * feat(watcher): imports * feat(watcher): add error types * feat(watcher): handle l1 messages and commit logs * feat: handle batch commit and finalize * test: handle_latest_block * test: wip events decoding * test: commit and finalize batch logs * test: utils * feat: tracing * test: wip reorg integration * test: reorg detection integration test * test: gaps * fix: lints * feat: point deps to default reth branch * fix: security issue on ring * fix: bump alloy * feat: move abi to separate crate * chore: remove ArbitraryTxBuilder * init database * add database iterator * add test * update streaming implementation * init indexer * fix: answer comments * fix: lints * fix: no std issues * fix: cargo features check * extend indexer * integrate and refactor * extend and refactor database, migrations and batch primitive * lint * lint * lint * improve database implementation * lint * lint * database transaction refactor * lint: * use GAT for DatabaseConnectionProvider trait * refactor & lint * merge database and refactor * clippy fix * skip rolllup-node-indexer no_std * refactor databse interface for insert_batch_input * lint and refactor * refactor and lint * lint * refactor * lint * merge indexer refactor * refactor indexer * refactor indexer * chore: integrate upstream * fix: fix no_std for scroll-wire * chore: point reth dep and rename rollup-node binary * lint * fix: re-add scroll-codec to riscV skip lint --------- Co-authored-by: Gregory Edison <gregory.edison1993@gmail.com>
No description provided.