add parameter to import functions to limit or suppress data output

[dimitri-b]

Currently when using import_table or import_json, on completion the comand will output 1,000 rows of imported data. It would be good to have a function parameter to control or suppress that, so that users don't have to scroll past 1,000 lines of text for their next step.
Maybe have a default param set to 1,000, but can be overriden with any number or 0, and if 0 - it will output "imported N rows" or something, to indicate success.

Another option might be to have it as a config setting.

Clearing the terminal or Jupyter cell output also works, but isn't ideal.

[erezsh]

Can you explain how you're getting 1000 rows?

In the terminal it's limited to 16 rows. In Jupyter to 160 rows, but it's supposed to be shown as a small cell + scrolling. Is that what you're referring to, or is there some other effect?

[dimitri-b]

Sorry, I got confused by the output row count indicator. It looks like there are about 160 rows, but I get fully expanded display, no scrollbar.

[erezsh]

Okay, just wanted to make sure.

I understand that sometimes it's inconvenient to see the preview. I will add a way to hide it.

As a temporary hack, you can add a "null" statement after it, in the same cell, like

# In the same Jupyter cell:
covid = import_table(...)
null

(Preql always previews the last statement)

[erezsh]

I added a function to master called set_setting.

You can all it like this:

set_setting("TABLE_PREVIEW_SIZE_HTML", 20)

It will limit the next preview to 20 rows.

Another setting there is MAX_AUTO_COUNT (default: 10000), which limits the maximum count on the preview, since in some situations it might be a linear calculation.

[dimitri-b]

Good news,
I assume it will apply only to the current session, is that correct?

[erezsh]

Yes, you'll have to re-set it every session. But I can add a more permanent option too.