Enregister multireg lclVars #36862

CarolEidt · 2020-05-22T03:03:45Z

Allow struct lclVars that are returned in multiple registers to be
enregistered, as long as the fields are a match for the registers.

Fix #34105

sandreenko

runtime/src/coreclr/src/jit/morph.cpp

Lines 10586 to 10592 in e858984

    
           // Can't use field by field assignment if the src is a call. 
        
           if (src->OperGet() == GT_CALL) 
        
           { 
        
               JITDUMP(" src is a call"); 
        
               // C++ style CopyBlock with holes 
        
               requiresCopyBlock = true; 
        
           }

I was expecting changes in this part that is currently blocking independent struct promotion for ASG(LCL_VAR struct, call struct). Could you please explain how your change avoids that block?

src/coreclr/src/jit/codegenarmarch.cpp

src/coreclr/src/jit/codegenxarch.cpp

src/coreclr/src/jit/codegencommon.cpp

sandreenko · 2020-05-23T00:31:32Z

src/coreclr/src/jit/codegenlinear.cpp

+{
+    // This should only be called for multireg lclVars.
+    assert(compiler->lvaEnregMultiRegVars);
+    assert(tree->IsMultiRegLclVar() || (tree->gtOper == GT_COPY));


I am confused about copy, won't it be better to have tree->IsMultiReg that recognizes both IsMultiRegLclVar and copy->GetRegCount() > 1?

src/coreclr/src/jit/codegenlinear.cpp

src/coreclr/src/jit/importer.cpp

src/coreclr/src/jit/lclvars.cpp

src/coreclr/src/jit/morph.cpp

sandreenko · 2020-05-23T07:30:47Z

src/coreclr/src/jit/morph.cpp

+                if (!dest->IsMultiRegLclVar() || (blockWidth != destLclVar->lvExactSize) ||
+                    (destLclVar->lvCustomLayout && destLclVar->lvContainsHoles))
+                {
+                    // Mark it as DoNotEnregister.


Will it mark struct A { bool a; int b; } as DoNotEnregister?

CarolEidt · 2020-06-01T16:57:22Z

The liveness model for multi-reg lclVar stores is challenging, since they both use and define multiple registers. Multireg calls (i.e. that define multiple registers), and multireg returns (that consume multiple registers don't suffer from the same issues as all the defs (in the former case) or the uses (in the latter case) can be modeled as occuring simultaneously. This will also be the case (eventually) for intrinsics that define multiple registers.

For multi-reg lclVar stores, we don't want to model them as occuring simultaneously, as then we must guarantee that all uses and defs have no conflicts, and without adding non-trivial complexity to the register allocator, that would actually mean that we couldn't support an assignment that reuses the source registers without spill. Take the following example with a 2-register return:

t1 = CALL
STORE V1<V10,V11> = t2
STORE V2<V12,V13> = V1<V10,V11>
RETURN V2

What we want this to generate is a simple call and return with no register spills or copies. While we want to ensure that the second field doesn't occupy the first return register at the point of the RETURN, we want the first field to do so.

So, the model for a STORE_LCL_VAR is as follows:
- First, use reg#0 of the source, including any reload or copy and updating liveness and GC info
- Then define reg#0 of the lclVar, including spilling if necessary and updating liveness and GC info
- Repeat for all registers

Getting this right is a bit tricky and requires factoring out some of the liveness, spill and GC updates.

src/coreclr/src/jit/gentree.cpp

CarolEidt · 2020-06-02T23:46:12Z

Regarding fgMorphCopyBlock():

I was expecting changes in this part that is currently blocking independent struct promotion for ASG(LCL_VAR struct, call struct). Could you please explain how your change avoids that block?

With these changes, that will remain a full struct assignment, with the destination lclVar being marked with GTF_VAR_MULTIREG. Here's an example:
Before and after morph (unchanged):

               [000060] -AC-G-------              *  ASG       simd12 (copy)
               [000058] M------N----              +--*  LCL_VAR   simd12<System.Numerics.Vector3>(P) V08 tmp2         
                                                  +--*    float  V08.X (offs=0x00) -> V72 tmp66        
                                                  +--*    float  V08.Y (offs=0x04) -> V73 tmp67        
                                                  +--*    float  V08.Z (offs=0x08) -> V74 tmp68        
               [000057] --C-G-------              \--*  CALL      struct VectorTest.F2_v3,NA,NA
               [000056] ------------ arg0            \--*  CNS_DBL   float  0.60000002384185791

After register allocation:

N299 ( 17,  7) [000057] --CXG-------        t57 = *  CALL      struct VectorTest.F2_v3 REG d0,d1,d2 $201
                                                  /--*  t57    struct 
N301 ( 21, 10) [000060] MA-XG-------              *  STORE_LCL_VAR simd16<System.Numerics.Vector3>(P) V08 tmp2          d16
                                                  *    float  V08.X (offs=0x00) -> V72 tmp66         d16
                                                  *    float  V08.Y (offs=0x04) -> V73 tmp67         d17
                                                  *    float  V08.Z (offs=0x08) -> V74 tmp68         d18 REG d16

Code Generated:

IN0063:                           bl      VectorTest:F2_v3(float):System.Numerics.Vector3
IN0064:                           fmov    s16, s0
IN0065:                           fmov    s17, s1
IN0066:                           fmov    s18, s2

CarolEidt · 2020-06-02T23:49:51Z

@sandreenko - I think this is ready for another round of review. I'm not sure why all the test builds (not the test runs) failed for the jitstressregs leg, and similarly the perf test failures didn't seem related. I'm attempting to re-run them.

CarolEidt · 2020-06-03T16:45:56Z

@dnceng @dotnet/jit-contrib - Can someone help me figure out how to determine what's going wrong with the perf runs? The log shows:

2020/06/03 15:37:29][INFO] Razor build server (process 7389) failed to shut down: The shutdown command failed: The application to execute does not exist: '/home/helixbot/work/BCB909DD/w/B34D09AF/e/.dotnet5273/sdk/5.0.100-preview.6.20266.3/Sdks/Microsoft.NET.Sdk.Razor/tools/netcoreapp3.0/rzc.dll'
[2020/06/03 15:37:29][INFO] 
[2020/06/03 15:37:30][INFO] MSBuild server shut down successfully.
[2020/06/03 15:37:30][INFO] $ popd
[2020/06/03 15:37:30][ERROR] Process exited with status 1
Traceback (most recent call last):
  File "/home/helixbot/work/A7A00953/p/performance/scripts/benchmarks_ci.py", line 264, in <module>
    __main(sys.argv[1:])
  File "/home/helixbot/work/A7A00953/p/performance/scripts/benchmarks_ci.py", line 250, in __main
    dotnet.shutdown_server(verbose)
  File "/home/helixbot/work/A7A00953/p/performance/scripts/dotnet.py", line 656, in shutdown_server
    get_repo_root_path())
  File "/home/helixbot/work/A7A00953/p/performance/scripts/performance/common.py", line 200, in run
    returncode, quoted_cmdline)
subprocess.CalledProcessError: Command '$ dotnet build-server shutdown' returned non-zero exit status 1.
+ export _commandExitCode=1

It's the same error for both the "Linux x64 release coreclr net5.0" and "Linux x64 release mono net5.0". It seems unlikely that this is an issue introduced with my PR. In the past I've found it possible to miss actual failures in these perf runs, but it reports 578 benchmarks run, and there are 578 instances of "Process xxx exited with code 0", so it doesn't appear to be an execution failure.

BruceForstall · 2020-06-03T17:53:18Z

@dotnet/runtime-infrastructure @DrewScoggins Can you answer @CarolEidt 's question about problems with the dotnet-runtime-perf runs?

CarolEidt · 2020-06-03T17:58:44Z

@dotnet/dnceng @dotnet/jit-contrib - I'm also having failures in the managed test build for the jitstressregs pipeline. The only error I can find in the log is here:

2020-06-03T17:47:39.3180740Z /Users/runner/runners/2.169.1/work/1/s/.dotnet/sdk/5.0.100-preview.6.20266.3/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(234,5): error NETSDK1004: Assets file '/Users/runner/runners/2.169.1/work/1/s/artifacts/obj/coreclr/ILVerification/project.assets.json' not found. Run a NuGet package restore to generate this file. [/Users/runner/runners/2.169.1/work/1/s/src/coreclr/src/tools/ILVerification/ILVerification.csproj]
2020-06-03T17:47:39.3313690Z ##[error].dotnet/sdk/5.0.100-preview.6.20266.3/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(234,5): error NETSDK1004: Assets file '/Users/runner/runners/2.169.1/work/1/s/artifacts/obj/coreclr/ILVerification/project.assets.json' not found. Run a NuGet package restore to generate this file.

BruceForstall · 2020-06-03T18:24:28Z

@agocke @jkotas added the ILVerification stuff just recently, and maybe can help with that error.

MattGal · 2020-06-03T21:51:26Z

@dotnet/dnceng @dotnet/jit-contrib - I'm also having failures in the managed test build for the jitstressregs pipeline. The only error I can find in the log is here:

2020-06-03T17:47:39.3180740Z /Users/runner/runners/2.169.1/work/1/s/.dotnet/sdk/5.0.100-preview.6.20266.3/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(234,5): error NETSDK1004: Assets file '/Users/runner/runners/2.169.1/work/1/s/artifacts/obj/coreclr/ILVerification/project.assets.json' not found. Run a NuGet package restore to generate this file. [/Users/runner/runners/2.169.1/work/1/s/src/coreclr/src/tools/ILVerification/ILVerification.csproj]
2020-06-03T17:47:39.3313690Z ##[error].dotnet/sdk/5.0.100-preview.6.20266.3/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(234,5): error NETSDK1004: Assets file '/Users/runner/runners/2.169.1/work/1/s/artifacts/obj/coreclr/ILVerification/project.assets.json' not found. Run a NuGet package restore to generate this file.

Sorry I missed this, thought it was your previous dnceng tag :) Taking a peek.

MattGal · 2020-06-03T22:06:06Z

Perf issue is Razor build server (process 27970) failed to shut down: The shutdown command failed: The application to execute does not exist sdk#10573 - looks like @ooooolivia is actively looking into workarounds but not something introduced by your change.
I'm pretty sure that the build failures are indeed something funky about ILVerify being added to the repo and we should get @agocke and @jkotas responses.

sandreenko · 2020-06-05T09:49:53Z

I am currently trying your changes with JitDoOldStructRetyping == false and there are some failures, I need more time to understand if they should be fixed here or in a separate change. I hope to finish that and review the PR today.

DrewScoggins · 2020-06-05T17:24:34Z

The perf run issue is because of a bug in dotnet.exe, we just tracked it down yesterday and checked in a workaround dotnet/performance#1346. You should not see that behavior on performance runs any longer.

sandreenko · 2020-06-05T22:23:39Z

src/coreclr/src/jit/lower.cpp

+                CheckMultiRegLclVar(op1->AsLclVar(), &retTypeDesc);
+            }
+        }
+#else // !FEATURE_MULTIREG_RET


why was this block put under !FEATURE_MULTIREG_RET?
It breaks compDoOldStructRetyping == false logic.

I've changed it; I'm not sure why I made that change.

sandreenko · 2020-06-07T05:33:48Z

src/coreclr/src/jit/codegencommon.cpp

+//
+// Arguments:
+//    tree - the GT_COPY node
+//    multiRegIndex - The index of the register to be copied


Nit: formatting in this header is not consistent, asingle register, The index, -when the source, to the register allocated to the register.

sandreenko · 2020-06-07T05:46:30Z

src/coreclr/src/jit/codegencommon.cpp

+regNumber CodeGen::genRegCopy(GenTree* treeNode, unsigned multiRegIndex)
+{
+    assert(treeNode->OperGet() == GT_COPY);
+    GenTree* op1 = treeNode->AsOp()->gtOp1;


Suggested change

GenTree* op1 = treeNode->AsOp()->gtOp1;

GenTree* op1 = copyNode ->gtGetOp1();

Changed to treeNode->gtGetOp1()

sandreenko · 2020-06-07T05:51:50Z

src/coreclr/src/jit/codegencommon.cpp

+    assert(op1->IsMultiRegNode());
+
+    GenTreeCopyOrReload* copyNode = treeNode->AsCopyOrReload();
+    // GenTreeCopyOrReload only reports the highest index that has a valid register.


Nit: this comment repeats a few lines below and regCount is used only in an assert, maybe delete this block?

sandreenko · 2020-06-07T07:04:25Z

src/coreclr/src/jit/codegenarm64.cpp

-    // var = call, where call returns a multi-reg return value
-    // case is handled separately.
-    if (data->gtSkipReloadOrCopy()->IsMultiRegCall())
+    // Multi-reg nodes are handled separately.


IsMultiReg is only for GenTreeLclVarCommon, IsMultiRegNode and IsMultiRegLclVar are for all tree, is it correct?
Maybe rename so:
GenTree has IsMultiRegNode, IsMultiRegLclVar(virtual)
GenTreeLclVarCommon has IsMultiRegLclVar.
or rename tree to lclVar in this function.

Also, I think this comment could be confusing, maybe Stores from a multi-reg source are handled separately?
What does tree->IsMultiReg() return when data->gtSkipReloadOrCopy()->IsMultiRegNode() == true?

IsMultiReg is only for GenTreeLclVarCommon, IsMultiRegNode and IsMultiRegLclVar are for all tree, is it correct?

Actually, isMultiReg is only for GenTreeLclVar (it must be GT_LCL_VAR not other variants).

Maybe rename so:
GenTree has IsMultiRegNode, IsMultiRegLclVar(virtual)
GenTreeLclVarCommon has IsMultiRegLclVar.

I don't think we really have much of a "standard" for this, but since we don't really support virtual methods on GenTree, I believe we generally keep the names distinct.

I'll rename tree to lclNode (I think that lclVar is confusing because one might expect it to be a LclVarDsc*. I'll make the same change to the version in codegenxarch.cpp.

sandreenko · 2020-06-07T07:10:22Z

src/coreclr/src/jit/codegenarmarch.cpp

+            // mov dst[i], reg[0]
+            // This effectively moves from `reg[0]` to `dst[i]`, leaving other dst bits unchanged till further
+            // iterations
+            // For the case where reg == dst, if we iterate so that we write dst[0] last, we eliminate the need for


Could we save a mov when reg == dst?

Possibly, but this is pre-existing code (factored out of genMultiRegStoreToLocal) so I'd prefer not to make that change here.

sandreenko · 2020-06-07T07:30:13Z

src/coreclr/src/jit/codegencommon.cpp

+    //    use reg #1 from src, including any reload or copy
+    //    define reg #1
+    // If we defined it as using all the source registers, there would be more
+    // conflicts and higher register pressure. In addition, it complicates the


Could you please explain why we will have higher register pressure?

I've added a comment; perhaps it's overkill there or should be moved somewhere else.

Thanks, I forgot that we have not had an algorithm to find an optimal move sequence for such chains.

sandreenko · 2020-06-07T09:12:39Z

src/coreclr/src/jit/codegenlinear.cpp

+//
+regNumber CodeGen::genConsumeReg(GenTree* tree, unsigned multiRegIndex)
+{
+    if (tree->OperGet() == GT_COPY)


off-topic: I prefer OperIs(GT_COPY) because it is usually shorter and doesn't not need additional brackets in && conditions.

sandreenko · 2020-06-07T09:39:38Z

src/coreclr/src/jit/liveness.cpp

-void Compiler::fgComputeLifeUntrackedLocal(VARSET_TP&           life,
+//
+// Returns:
+//    `true` if the node is a dead store (i.e. all fields are dead); `false` otherwise.


does it currently return true somewhere?

Sorry - I meant to respond to this. Yes, it returns true under the if (isDef) condition:

// None of the fields were live, so this is a dead store. if (!opts.MinOpts()) { // keepAliveVars always stay alive VARSET_TP keepAliveFields(VarSetOps::Intersection(this, fieldSet, keepAliveVars)); noway_assert(VarSetOps::IsEmpty(this, keepAliveFields)); // Do not consider this store dead if the parent local variable is an address exposed local. return !varDsc.lvAddrExposed; }

sandreenko

LGTM in general, one issue in Lower with !compDoOldStructRetyping support and a few questions/nits.

sandreenko · 2020-06-07T21:09:48Z

src/coreclr/src/jit/lower.cpp

+                    (comp->lvaGetPromotionType(varDsc) != Compiler::PROMOTION_TYPE_INDEPENDENT) ||
+                    (varDsc->lvFieldCnt > MAX_MULTIREG_COUNT))
+                {
+                    lclNode->ClearMultiReg();


Could we check (varDsc->lvFieldCnt > MAX_MULTIREG_COUNT) during importation and avoid setting MultiReg for such nodes?

I'd like to see us make these decisions in Lowering, since eventually we'd like to be able to promote register-passed structs with more than MAX_MULTIREG_COUNT fields (i.e. multiple fields packed into a single register).

sandreenko · 2020-06-07T21:17:32Z

src/coreclr/src/jit/lower.cpp

+//
+// Arguments:
+//   lclNode     - the GT_LCL_VAR node
+//   retTypeDesc - a return type descriptor for the consuming node


retTypeDesc can be both a user of this LCL_VAR: (RET(LCL_VAR) and the source STORE_LCL_VAR(call), is it correct?

Yes, it can come from a GT_CALL source or a GT_RETURN user.

I've updated the comments.

sandreenko · 2020-06-08T06:24:37Z

src/coreclr/src/jit/gentree.cpp

-                        if (fieldVarDsc->lvTracked && fgLocalVarLivenessDone && // Includes local variable liveness
-                            ((tree->gtFlags & GTF_VAR_DEATH) != 0))
+                        if (fieldVarDsc->lvTracked && fgLocalVarLivenessDone &&
+                            tree->AsLclVar()->IsLastUse(i - varDsc->lvFieldLclStart))


It should probably be AsLclVarCommon or it will fail for GT_LCL_FLD.

Edit: or check that it is a LCL_VAR, as I see Common does not have IsLastUse.

I added a check for tree->IsMultiRegLclVar(), as that's a more precise condition that it has the last-use bits.

sandreenko · 2020-06-09T14:01:01Z

src/coreclr/src/jit/importer.cpp

@@ -1440,6 +1440,19 @@ GenTree* Compiler::impAssignStructPtr(GenTree*             destAddr,
    }
    else if (compDoOldStructRetyping())
    {
+        if (dest->OperIs(GT_LCL_VAR) &&


Git has missed this merge conflict, I changed the condition from:

else { dest->gtType = asgType; }

to

else if (compDoOldStructRetyping()) { dest->gtType = asgType; }

and you added code under the original else // no condition that should not be under the change condition.

Could you please move if (compDoOldStructRetyping()) to dest->gtType = asgType;?

Allow struct lclVars that are returned in multiple registers to be enregistered, as long as the fields are a match for the registers. Fix dotnet#34105

Undo change to `fgMorphBlkNode()`

Extract common code for `genMultiRegStoreToLocal` Fix last use for multireg when extending lifetimes Fix call dump

CarolEidt · 2020-06-10T23:09:40Z

The jitstressregs leg has no new failures.
@sandreenko - there are 4 new commits since you last reviewed. The first was to fix the merge issue you pointed out, the second is just formatting, the third fixes an issue found by jitStressRegs, and the last fixes an issue that was exposed by #37280. Could you have a look?

sandreenko · 2020-06-10T23:30:39Z

@sandreenko - there are 4 new commits since you last reviewed. The first was to fix the merge issue you pointed out, the second is just formatting, the third fixes an issue found by jitStressRegs, and the last fixes an issue that was exposed by #37280. Could you have a look?

Looks good, do you have diffs for your changes?

CarolEidt · 2020-06-10T23:38:00Z

Here are the diffs:

Arch	OS	What	Delta	Methods Improved	Methods Regressed
Arm	Windows	Crossgen fx+tests	-11420 (-0.01%)	282	36
Arm64	Windows	Crossgen fx+tests	-477648 (-0.19%)	265	11
x64	Linux	Crossgen fx+tests	-1086482 (-0.55%)	765	9

No diffs for x64/windows or x86.

The regressions are cases where we promote where we didn't previously and which weren't mitigated by my earlier struct improvements. I expect we'll recover many/most of those when we enable enregistering of incoming arguments.

Dotnet-GitSync-Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label May 22, 2020

sandreenko reviewed May 23, 2020

View reviewed changes

CarolEidt force-pushed the EnregMultiRegVars branch 2 times, most recently from 54ae2d7 to c912772 Compare May 27, 2020 22:34

sandreenko mentioned this pull request May 28, 2020

Struct getters are generating unneccessary instructions on x64 when struct contains floats #7200

Closed

CarolEidt force-pushed the EnregMultiRegVars branch 3 times, most recently from 3742328 to b87798d Compare June 1, 2020 16:38

sandreenko reviewed Jun 2, 2020

View reviewed changes

src/coreclr/src/jit/gentree.cpp Show resolved Hide resolved

CarolEidt force-pushed the EnregMultiRegVars branch 2 times, most recently from 9396207 to 8dfb4fc Compare June 2, 2020 20:35

CarolEidt force-pushed the EnregMultiRegVars branch from 8dfb4fc to a031193 Compare June 3, 2020 14:08

CarolEidt mentioned this pull request Jun 3, 2020

Bmi2.MultiplyNoFlags issues #11782

Open

CarolEidt mentioned this pull request Jun 3, 2020

Change Http2Connection.StartWriteAsync to use a callback model #37353

Merged

jashook mentioned this pull request Jun 3, 2020

Add ILVerify from the corert repo #35038

Merged

sandreenko reviewed Jun 5, 2020

View reviewed changes

sandreenko reviewed Jun 7, 2020

View reviewed changes

sandreenko approved these changes Jun 7, 2020

View reviewed changes

CarolEidt force-pushed the EnregMultiRegVars branch from a031193 to 37f9dd3 Compare June 8, 2020 01:32

sandreenko reviewed Jun 8, 2020

View reviewed changes

CarolEidt force-pushed the EnregMultiRegVars branch from 12e673e to 8e6a875 Compare June 9, 2020 10:04

sandreenko reviewed Jun 9, 2020

View reviewed changes

CarolEidt added 10 commits June 10, 2020 09:07

Enregister multireg lclVars

2818bd7

Allow struct lclVars that are returned in multiple registers to be enregistered, as long as the fields are a match for the registers. Fix dotnet#34105

Bug fix and PR feedback

97c318e

Undo change to `fgMorphBlkNode()`

Consume/reload/copy multi-reg lclVar uses separately

a9cea3d

Extract common code for `genMultiRegStoreToLocal` Fix last use for multireg when extending lifetimes Fix call dump

More PR feedback

e1c14ae

More PR feedback

594c08e

JitStressRegs fixes

9ff1aa0

Fix merge error

6e00497

Formatting

694404e

Another jitStressRegs fix

cf4f3d5

Fix issue exposed by dotnet#37280

90f1990

CarolEidt force-pushed the EnregMultiRegVars branch from df4ae36 to 90f1990 Compare June 10, 2020 17:07

CarolEidt merged commit 7ef49af into dotnet:master Jun 11, 2020

tannergooding mentioned this pull request Jun 11, 2020

System.Numerics.Vectors.Tests failing on checked Linux arm with assert failure #37761

Closed

This was referenced Jun 15, 2020

Enable promotion of structs passed in multiple registers with matching fields #37924

Closed

Enable multireg long args on 32-bit targets #37925

Open

This was referenced Oct 16, 2020

Avoid marking tmp as DoNotEnregister in tmp=GT_CALL() where call returns a enregisterable struct in two return registers #5112

Closed

JIT: inefficient codegen for calls returning 16-byte structs on Linux x64 / arm64 #8571

Closed

CarolEidt deleted the EnregMultiRegVars branch December 4, 2020 04:12

ghost locked as resolved and limited conversation to collaborators Jan 3, 2021

	// Can't use field by field assignment if the src is a call.
	if (src->OperGet() == GT_CALL)
	{
	JITDUMP(" src is a call");
	// C++ style CopyBlock with holes
	requiresCopyBlock = true;
	}

	GenTree* op1 = treeNode->AsOp()->gtOp1;
	GenTree* op1 = copyNode ->gtGetOp1();

Enregister multireg lclVars #36862

Enregister multireg lclVars #36862

Conversation

CarolEidt commented May 22, 2020

sandreenko left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CarolEidt commented Jun 1, 2020

CarolEidt commented Jun 2, 2020

CarolEidt commented Jun 2, 2020

CarolEidt commented Jun 3, 2020

BruceForstall commented Jun 3, 2020

CarolEidt commented Jun 3, 2020

BruceForstall commented Jun 3, 2020

MattGal commented Jun 3, 2020

MattGal commented Jun 3, 2020

sandreenko commented Jun 5, 2020

DrewScoggins commented Jun 5, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandreenko left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandreenko Jun 8, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CarolEidt commented Jun 10, 2020

sandreenko commented Jun 10, 2020

CarolEidt commented Jun 10, 2020

sandreenko Jun 8, 2020 •

edited

Loading