Skip to content

Commit

Permalink
[RISCV] Preserve tail agnostic policy in foldVMV_V_V (#105788)
Browse files Browse the repository at this point in the history
This patch helps avoid regressions in an upcoming patch by making sure
we don't accidentally lose a tail agnostic policy when folding a vmv.v.v
into its source.

The previous comment about RISCVInsertVSETVLI relaxing the policy didn't
take into account the fact that there's a policy operand on vmv.v.v,
which can be tail agnostic.

If the tail is agnostic (via either the policy operand or the passthru
being undef) and vmv.v.v's VL <= Src's VL, then Src's tail can be made
agnostic.
  • Loading branch information
lukel97 authored Sep 4, 2024
1 parent 9fef09f commit 6c607cf
Show file tree
Hide file tree
Showing 2 changed files with 49 additions and 4 deletions.
11 changes: 7 additions & 4 deletions llvm/lib/Target/RISCV/RISCVVectorPeephole.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -529,10 +529,13 @@ bool RISCVVectorPeephole::foldVMV_V_V(MachineInstr &MI) {
*Src->getParent()->getParent()));
}

// Use a conservative tu,mu policy, RISCVInsertVSETVLI will relax it if
// passthru is undef.
Src->getOperand(RISCVII::getVecPolicyOpNum(Src->getDesc()))
.setImm(RISCVII::TAIL_UNDISTURBED_MASK_UNDISTURBED);
// If MI was tail agnostic and the VL didn't increase, preserve it.
int64_t Policy = RISCVII::TAIL_UNDISTURBED_MASK_UNDISTURBED;
bool TailAgnostic = (MI.getOperand(5).getImm() & RISCVII::TAIL_AGNOSTIC) ||
Passthru.getReg() == RISCV::NoRegister;
if (TailAgnostic && isVLKnownLE(MI.getOperand(3), SrcVL))
Policy |= RISCVII::TAIL_AGNOSTIC;
Src->getOperand(RISCVII::getVecPolicyOpNum(Src->getDesc())).setImm(Policy);

MRI->replaceRegWith(MI.getOperand(0).getReg(), Src->getOperand(0).getReg());
MI.eraseFromParent();
Expand Down
42 changes: 42 additions & 0 deletions llvm/test/CodeGen/RISCV/rvv/vmv.v.v-peephole.mir
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,45 @@ body: |
%y:gpr = ADDI $x0, 1
%z:vr = PseudoVMV_V_V_M1 %passthru, %x, 4, 5 /* e32 */, 0 /* tu, mu */
...
---
name: tail_agnostic
body: |
bb.0:
liveins: $v8
; CHECK-LABEL: name: tail_agnostic
; CHECK: liveins: $v8
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: %passthru:vr = COPY $v8
; CHECK-NEXT: %x:vr = PseudoVADD_VV_M1 %passthru, $noreg, $noreg, 4, 5 /* e32 */, 1 /* ta, mu */
%passthru:vr = COPY $v8
%x:vr = PseudoVADD_VV_M1 %passthru, $noreg, $noreg, 4, 5 /* e32 */, 0 /* tu, mu */
%y:vr = PseudoVMV_V_V_M1 %passthru, %x, 4, 5 /* e32 */, 1 /* ta, mu */
...
---
name: tail_agnostic_larger_vl
body: |
bb.0:
liveins: $v8
; CHECK-LABEL: name: tail_agnostic_larger_vl
; CHECK: liveins: $v8
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: %passthru:vr = COPY $v8
; CHECK-NEXT: %x:vr = PseudoVADD_VV_M1 %passthru, $noreg, $noreg, 4, 5 /* e32 */, 0 /* tu, mu */
%passthru:vr = COPY $v8
%x:vr = PseudoVADD_VV_M1 %passthru, $noreg, $noreg, 4, 5 /* e32 */, 0 /* tu, mu */
%y:vr = PseudoVMV_V_V_M1 %passthru, %x, 5, 5 /* e32 */, 1 /* ta, mu */
...
---
name: undef_passthru_src_undef_passthru
body: |
bb.0:
liveins: $v8
; CHECK-LABEL: name: undef_passthru_src_undef_passthru
; CHECK: liveins: $v8
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: %passthru:vr = COPY $v8
; CHECK-NEXT: %x:vr = PseudoVADD_VV_M1 $noreg, $noreg, $noreg, 4, 5 /* e32 */, 1 /* ta, mu */
%passthru:vr = COPY $v8
%x:vr = PseudoVADD_VV_M1 $noreg, $noreg, $noreg, 4, 5 /* e32 */, 0 /* tu, mu */
%y:vr = PseudoVMV_V_V_M1 $noreg, %x, 4, 5 /* e32 */, 0 /* tu, mu */
...

0 comments on commit 6c607cf

Please sign in to comment.