FrustumCullIsVisible() incorrectly culls geometry

[christianaguilera-foundry]

Description of Issue

We had experienced that objects disappear/reappear randomly as the camera moves around the scene. After some investigation, we've narrowed it down to FrustumCullIsVisible(...) in frustumCull.glslfx.

    // This prim is visible if it wasn't culled and at least one tiny prim test
    // failed. Test two axes here because size is measured in screen space.
    // We front-load this test because it saves quite a bit of compute: in one
    // test the framerate went from 7.7 to 9.0 FPS.
    if (FrustumCullIsTinyPrim(p[0], p[7]) && FrustumCullIsTinyPrim(p[2], p[5]))
        return false;

That bit of code promises to increase performance, but it's incorrectly culling geometry from the scene. Fortunately, the performance boost hack can be disabled (which makes us think that someone already noticed that it's acting up).

Workarounds

Option 1: Set HD_ENABLE_GPU_TINY_PRIM_CULLING environment variable to 0.
Option 2: Edit FrustumCullIsVisible(...) in frustumCull.glslfx to remove (or comment out) the block of code above.

[jtran56]

Filed as internal issue #152909.