Skip to content

feat(checklists): add EIPChecklist enum to specify EIP checklist items #1718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jun 16, 2025

Conversation

marioevz
Copy link
Member

@marioevz marioevz commented Jun 7, 2025

🗒️ Description

This PR introduces the EIPChecklist class which contains a list of all EIP checklist items for easier marking of tests that cover an item in the EIP checklist.

Calling any value in this class directly returns the marker which can be used in the tests or marks parameter of the parametrize object:

import pytest
from ethereum_test_tools import StateTestFiller
from ethereum_test_checklists import EIPChecklist

@EIPChecklist.TransactionType.Test.IntrinsicValidity.GasLimit.Exact()
def test_exact_intrinsic_gas(state_test: StateTestFiller):
    """Test transaction with exact intrinsic gas limit."""
    # Test implementation
    pass

It also fixes a breaking bug in the checklist generation code that marked all EIPs for a given test id when a marker was used anywhere in any test in any folder (whoops).

Suggestions for improvement of the usage of this new class are welcome.

🔗 Related Issues

✅ Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

@marioevz marioevz requested a review from danceratopz June 7, 2025 13:41
@marioevz marioevz force-pushed the eip-checklist-enum branch from 36ad5a4 to 3ad8a32 Compare June 9, 2025 15:02
Copy link
Member

@danceratopz danceratopz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nested classes are insane 😄 but LGTM, thanks!

@danceratopz danceratopz added type:feat type: Feature scope:fill Scope: fill command labels Jun 10, 2025
@danceratopz danceratopz changed the title feat(checklists): Add EIPChecklist enum to access EIP Checklist Items feat(checklists): add EIPChecklist enum to specify EIP checklist items Jun 10, 2025
@danceratopz danceratopz added the scope:checklists Scope: ethereum_test_checklists package label Jun 10, 2025
Copy link
Member

@danceratopz danceratopz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something's not quite right when running fill.

@danceratopz
Copy link
Member

Something's not quite right when running fill.
@marioevz I tried out the markers within the tests and found two issues:

Issue 1: Marker Without Parentheses Not Working

Problem: Using @EIPChecklist.Opcode.Test.StackOverflow (without parentheses) didn't work - pytest wouldn't execute the test because the marker wasn't being applied correctly.

Root Cause: The original implementation used __new__ in the ChecklistItem class, which only handled the parentheses case (). When used without parentheses, Python would call the
class directly but the marker wouldn't be applied to the test function.

Solution: Moved the marker logic from ChecklistItem.__new__ to ChecklistItemMeta.__call__ in the metaclass. This allows the metaclass to handle both cases:

  • @EIPChecklist.Item() - called with parentheses, returns a marker
  • @EIPChecklist.Item - called without parentheses as a decorator, applies the marker directly to the function

Issue 2: MyPy Type Checking Errors

Problem: MyPy showed "StackComplexOperations" not callable [operator] error when using @EIPChecklist.Opcode.Test.StackComplexOperations() because mypy couldn't understand that the
metaclass makes classes callable.

Root Cause: MyPy doesn't understand metaclass magic that makes classes callable at runtime. The ChecklistItemMeta metaclass provides __call__ but mypy can't infer this.

Solution: Created a comprehensive .pyi stub file that explicitly tells mypy all EIPChecklist classes are callable by inheriting from _CallableChecklistItem with proper @overload
signatures. Also built a CLI tool to auto-generate the stub file when the class structure changes.

Copy link
Member

@danceratopz danceratopz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yo! LGTM! Thanks, the IDE auto-completion really improves UX!

Copy link
Member

@danceratopz danceratopz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @marioevz, I just fixed-up the CI fails due to incorrect eip checklist ids (hopefully 🤞).

I didn't change the string ID format to the new enum format to avoid conflicts; they're probably best solved in the branches working on these new tests.

@danceratopz
Copy link
Member

I didn't change the string ID format to the new enum format to avoid conflicts; they're probably best solved in the branches working on these new tests.

Well at least to reduce them, if all the test PRs are merged we can of course address them here!

@marioevz marioevz merged commit 97b54c6 into main Jun 16, 2025
27 checks passed
@marioevz marioevz deleted the eip-checklist-enum branch June 16, 2025 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:checklists Scope: ethereum_test_checklists package scope:fill Scope: fill command type:feat type: Feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants