Feature Request: csvlook to wrap long columns

[Tabea-K]

Quite often I have csv files that I want to quickly scan, that have one or two columns with very long strings (i.e. >2000 characters). csvlook will then wrap them over many lines.

What I would suggest is to add an argument with which the user can specify a maximum length for each column to display. If, for example, I set this to 50, then for each row, all columns that contain more than 50 characters are truncated.

At first, I thought that the option -z would do exactly that, but apparently it doesn't.

[mkauzlar]

I vote for this too.
I tried to use the -z option the same way before realising the mistake. It seems that the -z option only checks if fields are larger than x.

If possible I would suggest to add 2 options:

1. Truncate the text in a column: If a text in a column exeeds say 10 then is truncated at 10

Original Output:

-----------------------------------------+------------------+--------------+
column1                                           column2          column3
-----------------------------------------+------------------+--------------+
aaaaaaaaaaaaaaaaaaaaaaaaa        bbbbb             cccc
-----------------------------------------+------------------+--------------+

Column1 Truncated at 10 Output:

-------------------+------------------+--------------+
column1                 column2          column3
-------------------+------------------+--------------+
aaaaaaaaaa          bbbbb             cccc
-------------------+------------------+--------------+

1. Wrap the text in a column row: If a text in a column exeeds say 10 than text is wrapped at lenght 10:

Original Output:

-----------------------------------------+------------------+--------------+
column1                                           column2          column3
-----------------------------------------+------------------+--------------+
aaaaaaaaaaaaaaaaaaaaaaaaa        bbbbb             cccc

Column1 text Wrapped at 10 Output:

-------------------+------------------+--------------+
column1                 column2          column3
-------------------+------------------+--------------+
aaaaaaaaaa          bbbbb             cccc
aaaaaaaaaa
aaaaa
-------------------+------------------+--------------+

Regards

[themiurgo]

Needed wrapping mode today, so I came up with this implementation. Let me know how to improve it so that it can be included.

themiurgo@aaf18e9

[mkauzlar]

themirugo: works as requested thanks.

An improvement would be to be able to set wrap length per column. We have now a perl script that does that:
cat test5.csv | perl col.pl -s +20,5,10,-5,-20
so if the csv has 5 columns each column can be sized with a number and + or - in front means that text is aligned left or right.
Here is an example output:

EDIT: (unfortunately there is no support for fixed width so it doesn't look good pasting the result here)

With the above features one can format the output table to be nice and readable without being too wide for some screens.

[themiurgo]

@mkauzlar That wouldn't be hard to implement that in csvlook. However specifying the charwidth for each columns seems overkill for most of the situations and complicates the CLI. From what I've understood csvlook is not meant to be the ultimate csv viewer but just a "good enough" for most use cases. Providing a max width for all the columns seems a good trade off between ease of use and functionality. Let's hear @onyxfish's opinion on this, perhaps he can provide a good solution.

[themiurgo]

I'm not even sure that '-w / --wrap' is the right flag for this. As an alternative, csvlook might use of the -z flag for this. Right now it just raises an error when a field is over a certain width.

[themiurgo]

Needed this again today. I'll send a PR.

[synapticarbors]

+1 for the ability to set a max width to truncate a column at

[pesterhazy]

It would be very useful just to have a setting to set a max width for each column, so that a single wide value doesn't make the whole file unreadable.

[themiurgo]

Setting a single max value for all the columns seems a good trade off between having a complicated CLI and being able to quickly view datasets with long fields.

By the way, I don't think this PR will be processed quickly / at all. The project seems to have a backlog of PRs dating back to a year ago.

[Tabea-K]

I created pull request #482 to fix this

[themiurgo]

@Tabea-K, there is already an older PR that fixes this by wrapping #429 . However the maintainer(s) seem to be busy at the moment.

[jpmckinney]

The implementation of this functionality is moving out of the csvkit repository into its new dependency agate (#515). This is the file that would need to be modified. I've opened an issue there referring to all issues and PRs related to this feature request.

[jpmckinney]

Re-opening as this can now be implemented thanks to the issue I opened on agate being fixed.

[themiurgo]

The agate commits just introduced truncation, not wrapping.

[onyxfish]

@themiurgo I apologize for your pull request not getting due attention back when it was fresh. As you noticed this project has been on ice for a while as the backend was being reimplemented as agate. That being the case, I would merge a wrapping pull request for agate (eg. table.print_table(max_column_width=20, wrap_columns=True)), if you feel like reimplementing it there.

The one caveat I have is this seems like a fairly unusual case and I don't want it at the cost of making the implementation tremendously complicated. If there is a way to implement it that's straightforward then great.

[jpmckinney]

Ah, sorry, I had reopened this based on the issue title ("truncate") not the description ("wrapping"). Re-closing as before.

[jpmckinney]

I'll re-open a new issue just for truncating: #561

[jugglinmike]

Hey folks, I'm having trouble understanding the resolution of this feature request. The referenced pull requests concern the "agate" dependency, but I can't find any patches which implement the functionality here in csvkit. Neither the csvlook documentation nor the csvkit's changelog mention wrapping, either. Can csvkit format output so that individual cells are displayed across multiple lines?

[jpmckinney]

There's no wrapping, only truncation. To avoid confusion, open a new issue on agate.

[jugglinmike]

Got it. Here's the issue wireservice/agate#773. Thanks for the quick response!