Broken images in Optimizations section of the book

[ttencate]

At least the following sections seem to have broken/missing images (either a tiny square image or a plain old 404):

• https://pola-rs.github.io/polars-book/user-guide/optimizations/lazy/predicate-pushdown.html#in-comes-optimization
• https://pola-rs.github.io/polars-book/user-guide/optimizations/lazy/projection-pushdown.html#break-it-down
• https://pola-rs.github.io/polars-book/user-guide/optimizations/lazy/projection-pushdown.html#optimized-query

[Trippy3]

I have investigated the above.
About "projection-pushdown", it seems to be simply a matter of specifying the code path.

-![](./../outputs/projection_pushdown/graph.png)
+![](../../outputs/projection_pushdown/graph.png)

About "predicate-pushdown", I am wondering if the return value of to_dot is as expected.
I did a debug print in frame.py and the results are below. 'graph polars_query {\n\n\n}'

        dot = _ldf.to_dot(optimized)
        print(f"{dot=}")

(.venv) $ python -m user_guide.src.examples.predicate_pushdown
dot='graph  polars_query {\n"FILTER BY col(\\"name\\").str.contains(... [(0, 0)]" -- "FILTER BY (col(\\"link_karma\\")) > (0i... [(0, 1)]"\n"FILTER BY (col(\\"link_karma\\")) > (0i... [(0, 1)]" -- "FILTER BY (col(\\"comment_karma\\")) > ... [(0, 2)]"\n"FILTER BY (col(\\"comment_karma\\")) > ... [(0, 2)]" -- "CSV SCAN data/reddit.csv;\nπ */6;\nσ -; [(0, 3)]"\n\n"FILTER BY (col(\\"link_karma\\")) > (0i... [(0, 1)]"[label="FILTER BY (col(\\"link_karma\\")) > (0i..."]\n"CSV SCAN data/reddit.csv;\nπ */6;\nσ -; [(0, 3)]"[label="CSV SCAN data/reddit.csv;\nπ */6;\nσ -;"]\n"FILTER BY col(\\"name\\").str.contains(... [(0, 0)]"[label="FILTER BY col(\\"name\\").str.contains(..."]\n"FILTER BY (col(\\"comment_karma\\")) > ... [(0, 2)]"[label="FILTER BY (col(\\"comment_karma\\")) > ..."]\n\n}'
dot='graph  polars_query {\n\n\n}'
dot='graph  polars_query {\n"FILTER BY (((col(\\"comment_karma\\")) ... [(0, 0)]" -- "CSV SCAN data/reddit.csv;\nπ */6;\nσ -; [(0, 1)]"\n\n"FILTER BY (((col(\\"comment_karma\\")) ... [(0, 0)]"[label="FILTER BY (((col(\\"comment_karma\\")) ..."]\n"CSV SCAN data/reddit.csv;\nπ */6;\nσ -; [(0, 1)]"[label="CSV SCAN data/reddit.csv;\nπ */6;\nσ -;"]\n\n}'
dot='graph  polars_query {\n\n\n}'

I guess I need to trace to_dot in dot.rs any further.

[Trippy3]

I did some more research on the above.　This issue seems to be relevant as well. pola-rs/polars#5570

Print Logical Plan

polars/polars-lazy/src/dot.rs: println!("{:?}", &logical_plan);

• optimized=False

  FILTER col("name").str.contains() FROM
    FILTER [(col("link_karma")) > (0i64)] FROM
      FILTER [(col("comment_karma")) > (0i64)] FROM
        CSV SCAN data/reddit.csv
        PROJECT */6 COLUMNS
        SELECTION: None

• optimized=True

  CSV SCAN data/reddit.csv
  PROJECT */6 COLUMNS
  SELECTION: Some([([(col("name").str.contains()) & ([(col("link_karma")) > (0i64)])]) & ([(col("comment_karma")) > (0i64)])])

dot

Get a dot language representation of the LogicalPlan.
When id and branch are 0, node evaluation is not performed.

polars/polars-lazy/polars-plan/src/dot.rs: write_dot

        if current_node.id == 0 && current_node.branch == 0 {
            writeln!(acc_str, "graph  polars_query {{")
        } else {
            let fmt_prev_node = prev_node.fmt.replace('"', r#"\""#);
            let fmt_current_node = current_node.fmt.replace('"', r#"\""#);


            let id_prev_node = format!(
                "\"{} [{:?}]\"",
                &fmt_prev_node,
                (prev_node.branch, prev_node.id)
            );
            let id_current_node = format!(
                "\"{} [{:?}]\"",
                &fmt_current_node,
                (current_node.branch, current_node.id)
            );


            writeln!(acc_str, "{} -- {}", &id_prev_node, &id_current_node)?;


            id_map.insert(id_current_node, fmt_current_node);
            id_map.insert(id_prev_node, fmt_prev_node);


            Ok(())
        }

If the first Plan is Selection, then write_dot will call dot recursively after write_dot, so the "dot language" is correctly represented.

                self.write_dot(acc_str, prev_node, current_node, id_map)?;
                input.dot(acc_str, (branch, id + 1), current_node, id_map)
            }

On the other hand, if the first Plan is CsvScan, write_dot is called only once and returns only "graph polars_query {{".

                self.write_dot(acc_str, prev_node, current_node, id_map)
            }

Therefore, I think that either of the following should be addressed

• Fix write_dot.
• Fix node_to_lp (if an unexpected Logical Plan is returned in the first place)

[braaannigan]

Closed by #211