💡 Yield Overrides

[arcanis]

Suggestion

🔍 Search Terms

yield, any, generator, saga

✅ Viability Checklist

My suggestion meets these guidelines:

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
• This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion

A new YieldType builtin generic type would instruct TypeScript to ignore the actual generator type in a yield expression, and to instead use whatever is used as generic parameter. The following would thus be true:

declare const val: YieldType<boolean>;

function* generator() {
  const res = yield val;
  assertType<res, boolean>();
}

📃 Motivating Example

Imagine you define an API where the user provides a generator and can interact with your application by the mean of "effects". Something akin to this:

run(function* () {
  const counter = yield select(state => state.counter);
});

Under the current model, TypeScript is forced to type counter as any, because in theory the generator runner may return whatever value it wants to the yield expression. It can be somewhat improved by adding an explicit type to the generator, but then all yield expressions will return the same type - which is obviously a problem when doing multiple select calls, each returning a different value type:

run(function* (): Generator<any, any, number | string> {
  const counter = yield select(state => state.counter);
  const firstName = yield select(state => state.firstName);
});

Not only does it prevent return type inference and yield value checks, but it also unnecessarily widens the type of both counter and firstName into number | string. The alternative is to write the expressed values at callsite:

run(function* () {
  const counter: number = yield select(state => state.counter);
  const firstName: string = yield select(state => state.firstName);
});

But it then requires the user code to have an explicit requirement on the exact types, which prevents accessing various refactoring tools and generally leads to worse UX (after all that's why TS has type inference in the first place). The last option is to change the API:

run(function* () {
  const counter = yield* selectS(state => state.counter);
  const firstName = yield* selectS(state => state.firstName);
});

By using a select variant supporting yield*, library code would be able to define a return type that TS would be able to use. However, it requires all libraries to adopt a non-idiomatic runtime pattern just for the sake of TypeScript, which is especially problematic considering that it would lead to worse performances.

The main point is that in all those cases, the library code already knows what's the value returned by the yield select(...) expression. All we need is a way to transmit this information to TypeScript.

💻 Use Cases

Redux-Saga (21.5K ⭐, 6 years old)

The redux-saga library heavily use generators (there are various reasons why async/await isn't enough, you can see them here). As a result, its collection of effects are all typed as any, leaving their users mostly unprotected on segments of code that would significantly benefit from types (for instance the select utility mentioned above can return any state slice from a larger one - typing the return as any leaves the door open to refactoring mistakes, typos, etc).

Additionally, under noImplicitAny, TS will put red underline on the whole select call, hiding important errors that could arise within the selector itself. In some of my own application files, about a third of the lines have red squiggles because of this issue:

MobX (23.8K ⭐)

https://mobx.js.org/actions.html#using-flow-instead-of-async--await-

Note that the flowResult function is only needed when using TypeScript. Since decorating a method with flow, it will wrap the returned generator in a promise. However, TypeScript isn't aware of that transformation, so flowResult will make sure that TypeScript is aware of that type change.

Others

• Ember Concurrency (663 ⭐)

  Due to limitations in TypeScript's understanding of generator functions, it is not possible to express the relationship between the left and right hand side of a yield expression. As a result, the resulting type of a yield expression inside a task function must be annotated manually:

• Effection (138 ⭐)

• GenSync (29 ⭐)

💻 Try it Out

I've implemented this feature (minus tests, and I'm not a TS contributor so some adjustments may be needed):

master...arcanis:mael/yield-override

Here's a playground: Before | After ✨

cc @Andarist

[nmain]

Why is this better than using : Generator<..., ..., T>? That can be enforced at call sites too:

Typescript Playground

Your motivating example could be rewritten to type run as something like run: (f: (props) => Generator<...>) => void and the type would be inferred inside the anonymous function* as well.

[Andarist]

@nmain in your demo it's not possible to receive a type dependent on the yielded type. This can be done when using yield delegates (yield *) but not with yield alone which is used to create "effects" system (like redux-saga, koa etc)

[nmain]

I see, yes, that is useful. Unfortunately your proposal needs a call thunk to do that. For instance, if my generator runner communicates back a string when yielded a number, and a number when yielded a string, there's no clean way to express that:

function* foo() {
  // not so pretty
  var a = yield (5 as number & YieldOverride<string>);
  var b = yield ("foo" as string & YieldOverride<number>);
}

[arcanis]

Exactly, which defers the responsibility of providing the types to the user (at every callsite!), which is what this proposal attempts to avoids. The YieldOverride primitive would be interpreted by TypeScript as "when it goes through a yield expression, this value must resolve to T, regardless what the generator thinks".

The following would thus work as expected: TypeScript playground.

[Andarist]

That is right but also - maybe not a super common use case. Thunks are usually pretty common for those kinds of patterns.

However, this is a valid criticism and maybe an alternative approach could be figured out. For instance, we could "dispatch" to the injected type using conditional types like this:

type Cond<T> = T extends string ? number : T extends number ? string : never;

function* foo(): YieldOverride<Cond> {
  var a: string = yield 5;
  var b: number = yield "foo";
}

[vaukalak]

@Andarist I think this is right direction. However I'd suggest to keep all 3 params of generator (T, TReturn and TNext). The only addition needed here is to allow TNext to be a resolver type for T:

type InferResolve<Yielded> =
  Yielded extends CallEffect ?
    CallEffectResolve<Yielded> :
    Yielded extends SelectEffect ?
      SelectEffectResolve<Yielded> :
      never;

type ReduxSaga = Generator<
  CallEffect | SelectEffect,
  void,
  InferResolve // ts complains that should be parametrized.
>;

function* betterFoo(): ReduxSaga {
  const value = (yield sampleCallEffect); // should be "string" one day
}

full example

Imho if Generator generic third param could be (take yielded value and infer resolved type), that would be enough.

[nmain]

One use case for this is Mobx's flow, which uses generator functions in which yield foo has a runtime type equal to await foo.

[arcanis]

@andrewbranch I've implemented this feature (here), do you mind if I open a PR?

Current usage:

type SelectEffect = {type: `select`};
declare function select<T>(): SelectEffect & YieldType<T>;

// Just some test utilities
type AssertEqual<T, Expected> = [T, Expected] extends [Expected, T] ? true : false;
const assertEqual = <U>() => <V>(val: V, expected: AssertEqual<U, V>) => {};

function* foo() {
  const x: SelectEffect = select<number>();
  assertEqual<SelectEffect>()(select<number>(), true);
  assertEqual<number>()(yield select<number>(), true);
}

(to be clear, suggestions in this thread have their own merits, however I have no idea how I would implement them; the YieldType approach is simple enough, works for most real-life cases, and has the benefit to have a precedent in the form of ThisType)

[andrewbranch]

I think since we haven’t decided on a direction to go for this issue, it’s unlikely that your PR will get much attention from the team, which might mean it gets closed after a while. This issue is still the best place to discuss the problem and try to promote it from “Awaiting More Feedback” to something more committed. In my view, having a prototype implementation doesn’t help its chances, since it doesn’t fall into the category of “it’s a good idea but we don’t know how or don’t have time to implement it.” But generally, it’s not something you need to ask permission for as long as you’re aware that it may not go anywhere. We tend to advise against implementing uncommitted things just to avoid wasted work, but that’s not relevant here since you’ve already done it 😁

[arcanis]

Makes sense! What would you say is the current action item, however? Your team hasn't weighted yet, so it's difficult to know what's the type of feedback you're looking for. Is it a case of getting the idea upvoted? Or is there a concrete problem that should be addressed in the proposal?

[andrewbranch]

At one point I jotted down this progression:

• Awaiting More Feedback → In Discussion
  • ↑ problem clarity
  • ↑ popularity (volume of 👍 and positive comments)
  • ↑ applicability (breadth of use cases positively impacted)
  • ↑ utility (inverse of the level of pain caused by the problem)
  • ↓ replaceability (other ways of solving the problem)
• In Discussion → Design meeting
  • ↑ solution clarity (everyone understands the high level proposal)
  • ↑ feasibility (we can probably implement it without huge tradeoffs)

I think in this case, the problem clarity is good, the popularity is lacking, the applicability is not yet proven (does anyone outside redux-saga and koa care about this?), the level of pain the problem causes could be elaborated a bit (so all the yield expressions in redux-saga are typed as any, but how often do people need to use their results?), and the replaceability of the proposed solution is unclear to me, but I see other ideas being discussed in this thread so I think the design space seems very open still. Maybe @rbuckton can help give some specific guidance, because any shortcomings of Generator typings is something I guarantee he’s thought about a lot.

[Andarist]

so all the yield expressions in redux-saga are typed as any, but how often do people need to use their results?

Most of the time - most of the yielded expressions return something meaningful to the "caller".