Scientific/exponential format silently behaves unexpectedly

[dan-robertson]

What happened?

Possibly a duplicate of #1118 (though this issue seems different in that these inputs were rejected before).

Entering numbers with scientific notation silently produces the wrong number.

I think instead such inputs should either be rejected or passed through to sql.

PRQL input

from numbers
select [ exponential = 1e9 ]

SQL output

SELECT
  19 AS exponential
FROM
  numbers

Expected SQL output

SELECT
  1E9 AS exponential
FROM
  numbers
-- Or the prql compiler rejects the input

MVCE confirmation

• Minimal example
• New issue

Anything else?

This was on the playground. Perhaps it is different in other versions.

[eitsupi]

Hi, thank you for reporting this!
I can reproduce with the latest dev version.

[max-sixty]

Some options for the implementation:

• Convert to float in rust, then convert to scientific when writing. But not trivial, likely requires heuristics on when to use scientific notation, and some float approximation.
  • We could also store smaller numbers as integers, up to 2^32 for i32...
• Store as string
• Raise an error

Probably string is easiest, float is best. Any thoughts from others?

[max-sixty]

Actually converting to a float was easiest, in #2865

It currently will write the whole number out, which isn't ideal with huge numbers, but fixing that is a bit more work, as it requires writing a heuristic for when to express fully vs in scientific notation. (Contributions welcome if anyone wants to take a swing)