Skip to content

(feat) Add Return and Propagate Fail Correctly #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Apr 15, 2025
Merged

Conversation

AlSchlo
Copy link
Collaborator

@AlSchlo AlSchlo commented Apr 11, 2025

What & Why

This PR adds a return operator in the HIR and in the Engine, and likewise fixes the behavior of Fails to propagate correctly up the call stack.

This has been implemented by adding a field to the Engine, which keeps track of the upper function call continuation.
Existing test cases have been adapted: (related to Fail), and a new one for Return has been added.

     // Define a function:
     // fn test_return(x: I64) = 
     //   let result = if x < 10 {
     //     return "too small"
     //   } else {
     //     "big enough"
     //   }
     //   "result is: " ++ result
     // }

Some Refactoring

  • Made all eval functions member functions of Engine.

Follow-up Work (less urgent)

  • Adding a return keyword in the AST and mapping it to the HIR.
  • Adding the Rust-like ? operator to unwrap option types.
  • For some unknown reason I had to make the Messages clonable, and had to make the one shot channel a shared channel. Any idea to avoid that? It doesn't seem like we really need the Clone bound, but removing it leads to errors.

@AlSchlo AlSchlo requested review from connortsui20, yliang412 and SarveshOO7 and removed request for connortsui20 April 11, 2025 20:12
@codecov-commenter
Copy link

codecov-commenter commented Apr 11, 2025

Codecov Report

Attention: Patch coverage is 90.02193% with 364 lines in your changes missing coverage. Please review.

Project coverage is 79.4%. Comparing base (ad270d3) to head (dc8c3f5).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
optd-dsl/src/analyzer/error.rs 29.6% 211 Missing ⚠️
optd/src/catalog/iceberg.rs 0.0% 24 Missing ⚠️
...td-dsl/src/analyzer/semantic_check/cycle_detect.rs 85.0% 19 Missing ⚠️
optd-dsl/src/analyzer/semantic_check/adt_check.rs 98.0% 15 Missing ⚠️
optd-dsl/src/analyzer/from_ast/expr.rs 98.1% 14 Missing ⚠️
optd-dsl/src/compile.rs 0.0% 13 Missing ⚠️
optd-dsl/src/engine/eval/expr.rs 96.7% 13 Missing ⚠️
optd-dsl/src/cli/main.rs 0.0% 11 Missing ⚠️
optd-dsl/src/analyzer/types.rs 92.3% 10 Missing ⚠️
optd-dsl/src/analyzer/from_ast/pattern.rs 96.3% 9 Missing ⚠️
... and 10 more
Additional details and impacted files
Files with missing lines Coverage Δ
optd-core/src/optimizer/mod.rs 0.0% <ø> (ø)
optd-dsl/src/analyzer/context.rs 89.0% <100.0%> (ø)
optd-dsl/src/analyzer/hir.rs 79.1% <ø> (+6.2%) ⬆️
optd-dsl/src/analyzer/map.rs 96.5% <100.0%> (-0.2%) ⬇️
optd-dsl/src/engine/eval/operator.rs 100.0% <100.0%> (ø)
optd-dsl/src/engine/utils.rs 100.0% <100.0%> (ø)
optd-dsl/src/lexer/lex.rs 97.9% <100.0%> (+<0.1%) ⬆️
optd-dsl/src/utils/error.rs 25.0% <ø> (ø)
optd-dsl/src/utils/tests.rs 91.1% <100.0%> (ø)
optd-dsl/src/analyzer/from_ast/converter.rs 98.4% <98.4%> (+0.5%) ⬆️
... and 19 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AlSchlo AlSchlo changed the title Alexis/add return (feat) Add Return and Propagate Fail Correctly Apr 11, 2025
### Semantic Analysis

Performs the following semantic analysis for our DSL.

### adt_check

In `semantic_check/adt_check.rs`:

1. Detects cyclic ADT definitions. This is best described with some
pictures:


![image](https://github.com/user-attachments/assets/610597c9-3493-4b08-bfda-cd0e110662cb)

or, more elaborate:


![image](https://github.com/user-attachments/assets/54b02716-723f-46e0-9545-a0f6695c64bb)

2. Detects whether `Logical`, `Physical`, `LogicalProperties`, and
`PhysicalProperties` types are defined, and are not the children of any
other types.

3. Checks for duplicate fields in ADTs.


![image](https://github.com/user-attachments/assets/afe3683c-9283-4d73-a468-11c58ba89e3b)

4. Validates the types within the fields of ADTs. Specifically:

-  No costed or stored types.
- Only logical types may contain logical children.
- Only physical types may contain physical children.
- Logical `product` types may only contain [Logical] or Logical fields.
- Physical `product` types may only contain [Physical] or Physical
fields.

### from_ast

In `from_ast`, I made the code process ADT definition first. This allows
us to perform the following extra semantic analysis:

1. Verify if `Constructors` and `Patterns` are properly defined.


![image](https://github.com/user-attachments/assets/19926c84-eea5-47e9-b5a5-12c30994b36b)

2. Verify if all annotated types exist when converting them into their
`HIR` version.

### bug fix & others

- The parser was parsing `None` as a `Constructor`. I have changed the
syntax to `none` lowercase, and fixed the parser code correspondingly.

- Refactored `AnalyzerErrorKind` to be `Boxed`, as clippy was
complaining the type was getting too big.

- Made `subtypes` in `TypeRegistry` a `BTreeMap` so that iterating over
all keys is deterministic across compilations.

- One semantic check I didn't implement is verifying the number of
arguments in constructors, patterns, and function calls. **This will be
addressed in a future PR for patterns and constructors**. For function
calls, this validation will be handled in the type inference PR since
functions are first-class values in our language, making it more
sensible to perform there (especially since calls are also used for
arrays, maps, and tuples).
Eventually, we will want `optd` to be an optimizer service. However, we
would also like it to be an embeddable library. This PR creates a
top-level `optd` crate that will act as the "glue" between all of our
components (which right now is just `optd-core` and `optd-dsl`, but more
will probably be created). Users should be able to import a single
`optd` dependency and have access to all functionality we want to
provide (similar to `tokio`). We do _not_ want a datafusion situation
where there could be 5 different datafusion crates you need to import.

There is another subtle reason we need this now. We would like to add
catalog functionality to `optd`. Both the rule engine and the optimizer
need access to the catalog. However, it is not entirely obvious where
one would put an implementation of a catalog right now. It doesn't
really make sense to put it in `optd-dsl`, but we also can't put it in
`optd-core` because `optd-core` depends on `optd-dsl`, which would
result in a circular dependency.

By creating a top-level `optd` crate, we can put the definition of a
`Catalog` trait in `optd-dsl`, and then add implementations of catalogs
in the `optd` crate. This allows for flexibility in catalog
implementations while not adding (arguably) irrelevant code to the
`optd-dsl` crate.

I removed some unused dependencies and bumped all `Cargo.toml`
dependencies to their latest version. However, I updated the
`Cargo.lock` with `-Z direct-minimal-versions`.

This PR also reintroduces the CI checks that we disabled. The
`twox-hash` problem still exists when taking a dependency on `iceberg`,
so instead of `-Z minimal-versions`, I've changed it to `-Z
direct-minimal-versions`.
@yliang412
Copy link
Member

For some unknown reason I had to make the Messages clonable, and had to make the one shot channel a shared channel. Any idea to avoid that? It doesn't seem like we really need the Clone bound, but removing it leads to errors.

I think it's because Engine<O> and if we want the engine to be clonable, then you have to make the continuation clonable.

AlSchlo and others added 2 commits April 15, 2025 10:18
### main change

Finalizes semantic analysis by checking for field number mismatches in
`Patterns` and `Constructors`.
Displays an error message during the `from_ast` phase, as follows:


![image](https://github.com/user-attachments/assets/0f52d113-9f01-4b47-a160-e67a7a9bac94)

### extra changes

- I added all type annotations I could during the `from_ast` phase, to
simplify type inference.
- Added the `none` type, which inherits from Option<Nothing>.
@AlSchlo AlSchlo merged commit 07f8873 into main Apr 15, 2025
12 checks passed
@AlSchlo AlSchlo deleted the alexis/add-return branch April 15, 2025 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants