Filled convex polygon shapes have jagged edges

[robbert-vdh]

Describe the bug
The edges around the filled area of a egui::Shape::convex_polygon() don't have proper anti-aliasing and will appear jagged. For semi-transparent fill colors this looks as if some form of anti aliasing is taking place, but the anti-aliased pixels are drawn without the alpha channel. Opaque colors look like they are not anti aliased at all. (see the screenshots below for examples of both of these things)

To Reproduce

1. Take the hello_world.rs example, and add the following add the end of the CentralPanel:

   diff --git a/eframe/examples/hello_world.rs b/eframe/examples/hello_world.rs
   index 7ec61fb..d3619e8 100644
   --- a/eframe/examples/hello_world.rs
   +++ b/eframe/examples/hello_world.rs
   @@ -35,6 +35,22 @@ impl epi::App for MyApp {
                    *age += 1;
                }
                ui.label(format!("Hello '{}', age {}", name, age));
   +
   +            let painter = ui.painter();
   +            painter.rect_filled(
   +                egui::Rect::from_x_y_ranges(0.0..=260.0, 0.0..=60.0),
   +                egui::Rounding::none(),
   +                egui::Color32::BLACK,
   +            );
   +            painter.add(egui::Shape::convex_polygon(
   +                vec![
   +                    egui::pos2(10.0, 40.0),
   +                    egui::pos2(250.0, 50.0),
   +                    egui::pos2(10.0, 10.0),
   +                ],
   +                egui::Color32::GRAY,
   +                egui::Stroke::none(),
   +            ));
            });
    
            // Resize the native window to be just the size we need it to be:

2. Run the example with cargo run --example hello_world.

Expected behavior
The edges should appear smooth, just like drawn lines or shape strokes currently do.

Screenshots
The modified hello_world example mentioned above:

The same example, but with a pure pure fill color without any transparency:

And the same problem with a different backend (egui-baseview):

Desktop (please complete the following information):

• OS: Arch Linux

Additional context
Tested with the released egui 0.16.0 and with the current master branch (0.16.0-102-g2802e035).

[emilk]

You need to pass in the arguments in the correct winding order. Switching two of the points:

                vec![
                    egui::pos2(10.0, 40.0),
                    egui::pos2(10.0, 10.0),
                    egui::pos2(250.0, 50.0),
                ],

Gives us this:

This is much better, but the bottom line still doesn't look great.

Rotating the points:

                vec![
                    egui::pos2(10.0, 10.0),
                    egui::pos2(250.0, 50.0),
                    egui::pos2(10.0, 40.0),
                ],

Gives us this:

The jagged edge has moved! Rotating again:

                vec![
                    egui::pos2(250.0, 50.0),
                    egui::pos2(10.0, 40.0),
                    egui::pos2(10.0, 10.0),
                ],

It is back! I'm guessing something goes wrong with the first and last edges of the polygon.

[robbert-vdh]

Yeah I assumed that the points had to be either clockwise or counterclockwise, but both gave me jaggies. So I used the most offensive looking order for the screenshot here. 😉

[emilk]

I figured out why and it has to do with how feathering (anti-aliasing) is performed. In short: the shape is both expanded and contracted, with a one-pixel wide border between. This border fades out, causing a soft edge. The contraction is the problem: for some corners it can cause self-intersecting geometry. This mostly happens for very sharp corners, which the tessellator "cuts off" so that they don't extend to much. This cutting-off worsens the problem.

I need to rewrite parts of the path-tessellation to fix this properly.

[robbert-vdh]

You know way more about this than I do so I can't really help you with that, but let me know if you need any help testing things.

[emilk]

I pushed a few fixes now. The last fix comes at the cost of having sharp edges extrude weirdly, and extrude further the sharper they are.

Right now I think this is less of a problem than broken anti-aliasing.

A proper fix that solves both problem will have to wait until it becomes a pressing issue for someone.

[emilk]

I pushed a few fixes now. The last fix comes at the cost of having sharp edges extrude weirdly, and extrude further the sharper they are.

Right now I think this is less of a problem than broken anti-aliasing.

A proper fix that solves both problem will have to wait until it becomes a pressing issue for someone.