Stateless transition #31532

gballet · 2025-03-31T07:42:40Z

This is an implementation of the state conversion process described in EIP-7612.

What this PR does:

It creates a special type of "facade pattern" tree, called TransitionTree, that is in charge of abstracting which one of the underlying trees (the "base" MPT tree or the "overlay" verkle tree) values should be read from/written to.
At the end of the block execution, a tree-sweeping process goes over the snapshot and converts N "leaves" (either a single slot or a full account header + code). In order to resume where the previous block stopped, a TransitionState structure is maintained in the db, keyed by the pre-state root of the block. This helps recovering these pointers in case the process is ran multiple times (e.g. block production, followed by insertion).

gballet · 2025-03-31T07:45:18Z

go.mod

@@ -22,7 +22,7 @@ require (
 	github.com/donovanhide/eventsource v0.0.0-20210830082556-c59027999da0
 	github.com/dop251/goja v0.0.0-20230605162241-28ee0ee714f3
 	github.com/ethereum/c-kzg-4844 v1.0.0
-	github.com/ethereum/go-verkle v0.2.2
+	github.com/ethereum/go-verkle v0.2.1


This is due to a bug being found in 0.2.2 that breaks the Holesky shadowfork. I will revert this in a further PR, when I have time to perform a deeper investigation of the bug. v0.2.2 is a db-size optimization and has no functional impact.

gballet · 2025-03-31T16:28:22Z

core/state/database.go

+	// StartVerkleTransition marks the start of the verkle transition
+	StartVerkleTransition(originalRoot, translatedRoot common.Hash, chainConfig *params.ChainConfig, verkleTime *uint64, root common.Hash)
+
+	// EndVerkleTransition marks the end of the verkle transition
+	EndVerkleTransition()
+
+	// InTransition returns true if the verkle transition is currently ongoing
+	InTransition() bool
+
+	// Transitioned returns true if the verkle transition has ended
+	Transitioned() bool
+
+	InitTransitionStatus(bool, bool, common.Hash)
+
+	SetCurrentSlotHash(common.Hash)
+
+	GetCurrentAccountAddress() *common.Address
+
+	SetCurrentAccountAddress(common.Address)
+
+	GetCurrentAccountHash() common.Hash
+
+	GetCurrentSlotHash() common.Hash
+
+	SetStorageProcessed(bool)
+
+	GetStorageProcessed() bool
+
+	GetCurrentPreimageOffset() int64
+
+	SetCurrentPreimageOffset(int64)
+
+	AddRootTranslation(originalRoot, translatedRoot common.Hash)
+
+	SetLastMerkleRoot(common.Hash)
+
+	SaveTransitionState(common.Hash)
+
+	LoadTransitionState(common.Hash)
+
+	LockCurrentTransitionState()
+
+	UnLockCurrentTransitionState()


@rjl493456442 these are helper functions to handle the transition. In my own branch, this is the most convenient way to access them. But maybe they could be moved to a verkle-specific TrieDB if it's cleaner ... what do you think?

gballet · 2025-03-31T16:30:20Z

trie/transition.go

+	if len(val) != 0 {
+		return val, nil
+	}
+	// TODO also insert value into overlay


That's contrary to the spec

Suggested change

// TODO also insert value into overlay

gballet · 2025-03-31T16:31:03Z

trie/transition.go

+		}
+		return data, nil
+	}
+	// TODO also insert value into overlay


Same thing as above

Suggested change

// TODO also insert value into overlay

gballet · 2025-03-31T16:34:26Z

trie/transition.go

+	return t.base
+}
+
+// TODO(gballet/jsign): consider removing this API.


note to self: remove that if it's no longer needed

Signed-off-by: Guillaume Ballet <[email protected]>

gballet

I pushed a second commit in order to explore the idea I mentioned earlier, which is to move as much of the transition management code to the TransitionTrie itself. The code is still incomplete and doesn't compile, but reading my messages will help explain some aspects of it - and hopefully we can converge towards a working and reliable design.

gballet · 2025-04-02T19:06:32Z

core/state/database.go

+	StartVerkleTransition(originalRoot, translatedRoot common.Hash, chainConfig *params.ChainConfig, verkleTime *uint64, root common.Hash)
+
+	// EndVerkleTransition marks the end of the verkle transition
+	EndVerkleTransition()
+
+	// InTransition returns true if the verkle transition is currently ongoing
+	InTransition() bool
+
+	// Transitioned returns true if the verkle transition has ended
+	Transitioned() bool
+
+	InitTransitionStatus(bool, bool, common.Hash)
+
+	// SetCurrentSlotHash provides the next slot to be translated
+	SetCurrentSlotHash(common.Hash)
+
+	// GetCurrentAccountAddress returns the address of the account that is currently being translated
+	GetCurrentAccountAddress() *common.Address
+
+	SetCurrentAccountAddress(common.Address)
+
+	GetCurrentAccountHash() common.Hash
+
+	GetCurrentSlotHash() common.Hash
+
+	SetStorageProcessed(bool)
+
+	GetStorageProcessed() bool
+
+	GetCurrentPreimageOffset() int64
+
+	SetCurrentPreimageOffset(int64)
+
+	AddRootTranslation(originalRoot, translatedRoot common.Hash)
+
+	SetLastMerkleRoot(common.Hash)
+
+	SaveTransitionState(common.Hash)
+
+	LoadTransitionState(common.Hash)
+
+	LockCurrentTransitionState()
+
+	UnLockCurrentTransitionState()


These are helper functions managing the conversion pointers (current slot being considered, current account, as well as methods to save and restore the state from the db).

In the source branch, these are stored in the unique disk database. Having to partially rewrite the code means that we can consider other methods. For instance, I am exploring the possibility to store the pointers in the MPT or verkle DB (if it's not present there, it means that the transition hasn't started) - and I am thinking it'd be better if the transition tree were the one deciding where the data is - I have a hunch this might simplify the design.

Activation of the forks would still be explicit (no way to work around that anyway, we need a header for chainConfig.IsVerkle()) but the interface would be that of the tree.

gballet · 2025-04-02T19:08:30Z

core/state/database.go

+	// Transition-specific fields
+	CurrentTransitionState *TransitionState
+	TransitionStatePerRoot lru.BasicLRU[common.Hash, *TransitionState]
+	transitionStateLock    sync.Mutex
+	addrToPoint            *utils.PointCache
+	baseRoot               common.Hash // hash of last read-only MPT base tree


These could be moved to the TransitionTree, if it makes the interface/diff simpler (see previous comment)

gballet · 2025-04-02T19:11:11Z

core/state/database.go

 }

 // NewDatabase creates a state database with the provided data sources.
 func NewDatabase(triedb *triedb.Database, snap *snapshot.Tree) *CachingDB {
 	return &CachingDB{
 		disk:          triedb.Disk(),
 		triedb:        triedb,
+		verkledb:      nil, // XXX change the interface, but it's a big change so wait for the PR to be reviewed by Gary


The question of how this gets initialized is still open. I have a hunch we could simply create it no matter what, and keep it empty as long as the transition hasn't happened.

I'm not a 100% clear on how this would work in a fully-verkle tesnet, but I guess that in this case, we could have an empty MPT db with just the info that the transition has completed and therefore, that there is no need to go through it.

Gary: pass verkle db as parameter

and initialize both DBs at the beginning so that we support verkle at genesis networks

gballet · 2025-04-02T19:14:09Z

core/state/reader.go

-	if !db.IsVerkle() {
-		tr, err = trie.NewStateTrie(trie.StateTrieID(root), db)
+	// get the transition status from the MPT triedb
+	ts, err := db.LoadTransitionState(root)


note that this method doesn't currently exist, I'm proposing that we move it from (*state.Database).LoadTransitionState) and keep it inside the tree somehow.

gballet · 2025-04-02T19:14:41Z

core/state/reader.go

 	}
+	tr = trie.NewTransitionTree(mptr, vktr, false)


so as a follow-up to the message above, this would be NewTransitionTree(mptr, vktr, false, ts)

gballet · 2025-04-02T19:16:42Z

core/state_processor.go

Not present in this PR yet, but the call to StartVerkleTransition could be called at the start ofProcess, since the state should not be needed beforehand.

gballet · 2025-04-02T19:18:04Z

core/state_processor.go

+			log.Error("error performing the transition", "err", err)
+			panic(fmt.Sprintf("error performing the transition: %s", err))
+		}
+	}


Note to self: this is also where the pointers would be saved at the db. And the equivalent "start / save" would have to be added to the miner for block production.

gballet · 2025-04-03T07:33:47Z

core/state/database.go

 	tr, err := trie.NewStateTrie(trie.StateTrieID(root), db.triedb)
 	if err != nil {
 		return nil, err
 	}
 	return tr, nil
 }

+// OpenTrie opens the main account trie at a specific root hash.
+func (db *CachingDB) OpenTrie(root common.Hash) (Trie, error) {


Gary: read the pointers directly from ethdb.KeyValueStore

and store it in the cachingdb, no lock should be needed

gballet · 2025-04-03T07:44:59Z

core/state/database.go

+	// Transition-specific fields
+	CurrentTransitionState *TransitionState
+	TransitionStatePerRoot lru.BasicLRU[common.Hash, *TransitionState]
+	transitionStateLock    sync.Mutex


Gary: the cache itself has an internal lock, this should not be necessary

core/blockchain.go

core, trie: implement stateless transition

f50b254

gballet added the verkle label Mar 31, 2025

gballet force-pushed the stateless-transition branch from 5f4b22f to 16cb4fa Compare March 31, 2025 15:50

gballet commented Apr 1, 2025

View reviewed changes

core/state, trie: implement stateless transition

8e8d10a

Signed-off-by: Guillaume Ballet <[email protected]>

gballet force-pushed the stateless-transition branch 2 times, most recently from 8e8d10a to b9884bf Compare April 2, 2025 13:08

gballet commented Apr 2, 2025

View reviewed changes

gballet commented Apr 3, 2025

View reviewed changes

save current state after discussion with Gary

7942e52

gballet force-pushed the stateless-transition branch from 500abed to 7942e52 Compare April 3, 2025 18:45

gballet commented Apr 7, 2025

View reviewed changes

core/blockchain.go Outdated Show resolved Hide resolved

Update core/blockchain.go

8efb8b6

gballet mentioned this pull request Apr 14, 2025

core/state: introduce the TransitionState object (verkle transition part 1) #31634

Open

Stateless transition #31532

Are you sure you want to change the base?

Stateless transition #31532

Conversation

gballet commented Mar 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gballet left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

gballet commented Mar 31, 2025 •

edited

Loading