non-cosmetic difference detected if semicolons present

[djrobstep]

I get a non-cosmetic difference detected when I try and use pgpp on a file with eg the following contents:

select * from t;

The same query, and other queries are fine without the semicolon.

What's happening here? Perhaps pgpp doesn't support multiple statements?

Happy to take a look at fixing if you'd like to point me in the right direction?

[lelit]

You can see what's happening by looking at pgpp -t output for the two statements, and spotting the difference: this is basically what the safety belt does, ie it compares the AST of the input statement with that of the prettified one, and if there is any discrepancy, it raises an exception.
A quick test revealed that, when there is a semicolon, the underlying PG parser emits a stmt_len slot, not present otherwise:

$ diff -u <(echo "select * from i" | pgpp -t) <(echo "select * from i;" | pgpp -t)
--- /dev/fd/63	2018-03-25 10:51:54.356248860 +0200
+++ /dev/fd/62	2018-03-25 10:51:54.356248860 +0200
@@ -32,7 +32,8 @@
             }
           ]
         }
-      }
+      },
+      "stmt_len": 15
     }
   }
 ]

I'll try to understand the meaning of that stmt_len: AFAICT, it should be ignored when comparing the ASTs.

Thank you for reporting the issue!

[lelit]

Here there is a brief explanation of the meaning of stmt_len.

Also libpg_query seems to explicitly ignore it to compute statement's fingerprint.

So yes, I think it's just a matter of removing it from the structure before comparison, as we already do for the location field.

[lfittl]

@lelit Just in case it helps, here is the full reference for the fingerprinting logic in pg_query itself: https://github.com/lfittl/libpg_query/wiki/Fingerprinting#version-20-based-on-postgresql-10

(the stmt_len is a Postgres 10 addition, and if I understand right I would ignore it for any comparisons between ASTs in your library as well)

[lelit]

Thanks Lukas for the confirmation!

[lelit]

I just release version 0.25 that fixes this.