Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clamp the damage rect to the destination rect #449

Conversation

cmeissl
Copy link
Collaborator

@cmeissl cmeissl commented Jan 6, 2022

this fixes issues when the damage rect is greater
than the destination rect, like providing i32::Max as
the damage size

@Drakulix @vberger
I have added two functions in geometry.rs, if they are not entirely clear or do not fit i can remove them.

Copy link
Member

@Drakulix Drakulix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I think that looks just fine 👍

Comment on lines 1288 to 1304
.map(|rect| {
let rect = rect.to_f64();

let rect_constrained_loc = rect
.loc
.constrain(Rectangle::from_extemities((0f64, 0f64), dest.size.to_point()));
let rect_clamped_size = rect.size.clamp((0f64, 0f64), dest.size);

[
rect.loc.x as f32 / dest.size.w as f32,
rect.loc.y as f32 / dest.size.h as f32,
rect.size.w as f32 / dest.size.w as f32,
rect.size.h as f32 / dest.size.h as f32,
(rect_constrained_loc.x / dest.size.w) as f32,
(rect_constrained_loc.y / dest.size.h) as f32,
(rect_clamped_size.w / dest.size.w) as f32,
(rect_clamped_size.h / dest.size.h) as f32,
]
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is that different from this?

let viewport = Rectangle::from_extemities((0f64, 0f64), dest.size.to_point());

let damage = damage
    .iter()
    .flat_map(|rect| rect.to_f64().intersection(viewport).map(|rect_constrained| {
        [
            (rect_constrained.loc.x / dest.size.w) as f32,
            (rect_constrained.loc.y / dest.size.h) as f32,
            (rect_constrained.size.w / dest.size.w) as f32,
            (rect_constrained.size.h / dest.size.h) as f32,
        ]
    }))
    .flatten()
    .collect::<Vec<_>>();

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to use intersection and added a debug_assert! that the intersection is not None.
This could prevent some bugs where the submitted damage is completely outside of the surface.

@cmeissl cmeissl force-pushed the fix/desktop_abstraction_damage_clamping branch from 9f02bea to f1dce26 Compare January 6, 2022 21:16
this fixes issues when the damage rect is greater
than the destination rect, like providing i32::Max as
the damage size
@cmeissl cmeissl force-pushed the fix/desktop_abstraction_damage_clamping branch from f1dce26 to e9904ce Compare January 6, 2022 21:24
(rect_constrained.size.h / dest.size.h) as f32,
]
});
debug_assert!(damage.is_some());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this debug assert? Do we make it a fatal error to provide damage that is outside the viewport?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The damage is what is used for rendering in render_texture defining the instances for instanced rendering. If the damage is outside of the damage_viewport in the worst case nothing will be drawn. Imo this is an error in the calling code but may be hard to find which this debug_assert! aims to make easier.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's your opinion on that @Drakulix ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our code should never do this and it would be nice to catch cases early.

I would say we leave it in, until someone opens an issue with a legit use-case for providing non-overlapping damage values. Because I can't think of any.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright

@Drakulix Drakulix merged commit 753d386 into Smithay:feature/desktop_abstractions Jan 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants