Track data only steps

[benjie]

Long term I want to make it so that steps can be "merged" together for execution even if they can't be merged in the plan diagram; e.g.:

graph TD
  Constant1["Constant<101>[1]"]
  Constant2["Constant<102>[2]"]
  Constant3["Constant<103>[3]"]
  Constant4["Constant<104>[4]"]
  Constant1 -->|dataOnly| Process5[["Process<1>[5]"]]
  Constant2 -->|dataOnly| Process6[["Process<2>[6]"]]
  Constant3 -->|dataOnly| Process7[["Process<3>[7]"]]
  Constant4 -->|dataOnly| Process8[["Process<4>[8]"]]
  Process5 & Process6 --> DoSomething
  Process7 & Process8 --> DoSomethingElse

Loading

Assuming all other things are equal, this would require us to run Process 4 separate times. However; if, whilst excluding the "data only" dependencies, all these Process nodes should be able to execute as one. We should be able to combine the 4 constants into one "execution value" and then feed it into Process[5] for execution, and then split the results back out again spreading them across Process[5-8] for the dependent steps to consume.

To make way for this, we've introduced "data only" dependencies, but they're not data only if their dependents can still communicate with them. We need to block that communication

[changeset-bot]

⚠️ No Changeset found

Latest commit: daab0c3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR