[RISCV] How to model chaining in RVV scheduler?

[wangpc-pp]

IIUC, we have already considered LMUL/SEW in our RVV scheduler, but not chaining. So, we may overestimate the latency of RVV instructions. For example:

vsetvli a0, zero, e8, m8, ta, ma
vadd.vv v3, v2, v1
vmul.vv v5, v4, v3

The part of v3 can be read by vmul.vv before the commit of vadd.vv, so the whole latency is lower than the sum of latencies of these two instructions.

We have ReadAdvance to bypass some cycles, but now we don't apply this mechanism to existing models. Instead, we set the Latency to LMUL_1's cycles and occupy the vector unit during the whole execution via setting AcquireAtCycles and ReleaseAtCycles (correct me if I understand it wrong). I don't know if this way is OK or good enough.

So some questions here:

1. Is current approach good enough?
2. Any other thoughts about modeling chaining?

@michaelmaitland @mshockwave

[llvmbot]

@llvm/issue-subscribers-backend-risc-v

Author: Pengcheng Wang (wangpc-pp)

IIUC, we have already considered LMUL/SEW in our RVV scheduler, but not chaining. So, we may overestimate the latency of RVV instructions. For example: ``` vsetvli a0, zero, e8, m8, ta, ma vadd.vv v3, v2, v1 vmul.vv v5, v4, v3 ``` The part of v3 can be read by `vmul.vv` before the commit of `vadd.vv`, so the whole latency is lower than the sum of latencies of these two instructions.

We have ReadAdvance to bypass some cycles, but now we don't apply this mechanism to existing models. Instead, we set the Latency to LMUL_1's cycles and occupy the vector unit during the whole execution via setting AcquireAtCycles and ReleaseAtCycles (correct me if I understand it wrong). I don't know if this way is OK or good enough.

So some questions here:

1. Is current approach good enough?
2. Any another thoughts about modeling chaining?

@michaelmaitland @mshockwave

[mshockwave]

For SiFive's scheduling models primarily measured by llvm-exegesis -- like P400 and P600 -- since the latency are measured, (theoretically) it already accounts for any chaining. But if you want to explicitly account for chaining, I think using ReadAdvance is still the proper way to do so.

occupy the vector unit during the whole execution via setting AcquireAtCycles and ReleaseAtCycles

I'm not sure if I follow you here: generally speaking if we set the duration of AcquireAtCycle & ReleaseAtCycle the same as latency, that means the instruction is un-pipelined -- which is pretty common for all kinds of division instructions, being vector or not.

[michaelmaitland]

For SiFive scheduling models, we are modeling the latency for just the first unit of data that can be chained. By doing this, we account for chaining. When an instruction can't chain, we have to increase its latency in the model.

[topperc]

For SiFive's scheduling models primarily measured by llvm-exegesis -- like P400 and P600 -- since the latency are measured, (theoretically) it already accounts for any chaining. But if you want to explicitly account for chaining, I think using ReadAdvance is still the proper way to do so.

occupy the vector unit during the whole execution via setting AcquireAtCycles and ReleaseAtCycles

I'm not sure if I follow you here: generally speaking if we set the duration of AcquireAtCycle & ReleaseAtCycle the same as latency, that means the instruction is un-pipelined -- which is pretty common for all kinds of division instructions, being vector or not.

I think there's a misunderstanding of "whole execution" here. What the SiFive schedulers do is set the AcquireAtCycles and ReleaseAtCycles based on how long the first stage of the vector pipeline is occupied. For fully pipelined instructions that operate on one DLEN per cycle this is LMUL*(VLEN/DLEN) cycles.

[wangpc-pp]

For SiFive's scheduling models primarily measured by llvm-exegesis -- like P400 and P600 -- since the latency are measured, (theoretically) it already accounts for any chaining. But if you want to explicitly account for chaining, I think using ReadAdvance is still the proper way to do so.

occupy the vector unit during the whole execution via setting AcquireAtCycles and ReleaseAtCycles

I'm not sure if I follow you here: generally speaking if we set the duration of AcquireAtCycle & ReleaseAtCycle the same as latency, that means the instruction is un-pipelined -- which is pretty common for all kinds of division instructions, being vector or not.

I think there's a misunderstanding of "whole execution" here. What the SiFive schedulers do is set the AcquireAtCycles and ReleaseAtCycles based on how long the first stage of the vector pipeline is occupied. For fully pipelined instructions that operate on one DLEN per cycle this is LMUL*(VLEN/DLEN) cycles.

For example, llvm/lib/Target/RISCV/RISCVSchedSiFiveP600.td:

// 1 Micro-Op per cycle.
class SiFiveP600GetLMulCycles<string mx> {
  int c = !cond(
    !eq(mx, "M1") : 1,
    !eq(mx, "M2") : 2,
    !eq(mx, "M4") : 4,
    !eq(mx, "M8") : 8,
    !eq(mx, "MF2") : 1,
    !eq(mx, "MF4") : 1,
    !eq(mx, "MF8") : 1
  );
}

// 11. Vector Integer Arithmetic Instructions
foreach mx = SchedMxList in {
  defvar LMulLat = SiFiveP600GetLMulCycles<mx>.c;
  defvar IsWorstCase = SiFiveP600IsWorstCaseMX<mx, SchedMxList>.c;
  let Latency = 2, ReleaseAtCycles = [LMulLat] in {
    defm "" : LMULWriteResMX<"WriteVExtV",    [SiFiveP600VectorArith], mx, IsWorstCase>;
    defm "" : LMULWriteResMX<"WriteVICmpV",   [SiFiveP600VectorMask],  mx, IsWorstCase>;
    defm "" : LMULWriteResMX<"WriteVICmpX",   [SiFiveP600VectorMask],  mx, IsWorstCase>;
    defm "" : LMULWriteResMX<"WriteVICmpI",   [SiFiveP600VectorMask],  mx, IsWorstCase>;
  }
  let ReleaseAtCycles = [LMulLat] in {
    let Latency = 6 in {
      defm "" : LMULWriteResMX<"WriteVIMulV",    [SiFiveP600VectorArith], mx, IsWorstCase>;
      defm "" : LMULWriteResMX<"WriteVIMulX",    [SiFiveP600VectorArith], mx, IsWorstCase>;
      defm "" : LMULWriteResMX<"WriteVIMulAddV", [SiFiveP600VectorArith], mx, IsWorstCase>;
      defm "" : LMULWriteResMX<"WriteVIMulAddX", [SiFiveP600VectorArith], mx, IsWorstCase>;
    }

When LMUL is 8, the LMulLat is 8. So, there are cases in which Latency is smaller than ReleaseAtCycles. From my understanding, in such cases, Latency is the latency considering chaining, and ReleaseAtCycles is the duration of occupying a vector unit. I think this is also what @topperc was saying.