Add option for DataFrame.to_html() to render URL data as links (#2679) #23715

benjaminarjun · 2018-11-15T06:11:16Z

closes Improvement: DataFrame.to_html() to create hyperlinks #2679
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

Two lines were added to tests that exceed the 79 character limit.

pep8speaks · 2018-11-15T06:11:22Z

Hello @benarthur91! Thanks for updating the PR.

There are no PEP8 issues in the file pandas/core/frame.py !
There are no PEP8 issues in the file pandas/io/formats/format.py !
There are no PEP8 issues in the file pandas/io/formats/html.py !
There are no PEP8 issues in the file pandas/tests/io/formats/test_to_html.py !

Comment last updated on November 26, 2018 at 05:41 Hours UTC

pandas/core/frame.py

pandas/core/config_init.py

pandas/io/formats/html.py

pandas/tests/io/formats/test_to_html.py

pandas/core/frame.py

gfyoung · 2018-11-26T05:44:14Z

@benarthur91 : This is looking good! Next up: could you resolve the merge conflicts?

benjaminarjun · 2018-11-26T05:50:59Z

Thanks for the feedback everyone, I've pushed the requested changes. I'm unfamiliar with the PR workflow, so please let me know if I've done anything incorrectly.

Edit: Thanks @gfyoung, I'm assuming this means rebase onto master, resolve conflicts, and force push that branch over the current?

gfyoung · 2018-11-26T05:52:30Z

I'm assuming this means rebase onto master, resolve conflicts, and force push that branch over the current?

Yep, that's correct.

WillAyd · 2018-11-26T10:51:33Z

Typically preferred if you merge in the latest master instead of rebasing. Keeps github comment history and doesn’t require a force push

benjaminarjun · 2018-11-26T21:26:20Z

Gotcha, sorry about that. I still have the pre-rebase branch with original commits, would it be helpful if I pushed that one up instead?

gfyoung · 2018-11-26T21:29:14Z

Gotcha, sorry about that. I still have the pre-rebase branch with original commits, would it be helpful if I pushed that one up instead?

@benarthur91 : Just leave it as is and work off what you just pushed. It's fine.

benjaminarjun · 2018-11-29T01:48:01Z

Does anyone know what might have happened? Getting a seemingly unrelated error in pandas/tests/plotting/test_datetimelike.py, I couldn't see any obvious reason that would have happened.

gfyoung · 2018-11-29T05:46:10Z

Might be a weird Travis thing. Don't forget to fix the isort error on the other Travis build though.

benjaminarjun · 2018-11-29T06:09:19Z

Ah, I didn't see a clear error message and assumed it was related to the other failure. As you mention isort, I'm now thinking it's either a) imports are in the wrong order or b) it doesn't like that I'm importing a private method?

Where can I look to find the proper order of imports? Also I ran flake8 on frame.py locally and it reported no issues (even when I changed the order of imports), would you expect it to?

gfyoung · 2018-11-29T06:10:19Z

@benarthur91 : You need to install isort via pip and run the command itself (isort )

benjaminarjun · 2018-11-29T06:31:01Z

Pretty neat :) Thanks for clarifying, I just pushed that change.

codecov · 2018-11-29T08:24:32Z

Codecov Report

Merging #23715 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #23715      +/-   ##
==========================================
+ Coverage   92.22%   92.22%   +<.01%     
==========================================
  Files         162      162              
  Lines       51777    51785       +8     
==========================================
+ Hits        47751    47759       +8     
  Misses       4026     4026

Flag	Coverage Δ
#multiple	`90.62% <100%> (ø)`	⬆️
#single	`43.01% <11.11%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/frame.py	`96.91% <ø> (ø)`	⬆️
pandas/io/formats/format.py	`97.98% <100%> (ø)`	⬆️
pandas/io/formats/html.py	`98.64% <100%> (+0.03%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7ee829...9f3e058. Read the comment docs.

benjaminarjun · 2018-12-01T07:27:22Z

Is there anything else I should do to clean this up right now?

WillAyd

Minor comments otherwise lgtm

pandas/tests/io/formats/test_to_html.py

pandas/io/formats/html.py

pandas/tests/io/formats/test_to_html.py

jreback

can you add a whatsnew note, New Features is prob good. Also we have some docs on writing .to_html() in io.rst, can you add an example of this there (with a versionadded tag)

benjaminarjun · 2018-12-09T04:40:10Z

Added note to whatsnew and also a short description and example in io.rst

I also checked the last two boxes in the original (autogenerated) comment - I'm assuming that's for me to do and not someone else, but not sure. Let me know if there's anything else I can do.

WillAyd · 2018-12-09T08:13:00Z

pandas/tests/io/formats/test_to_html.py

+            },
+        ]
+        df = DataFrame(data, columns=['foo', 'bar', None],
+                       index=range(len(data)))


You don't need to pass index here as this is the default behavior

pandas/tests/io/formats/test_to_html.py

doc/source/io.rst

jreback · 2018-12-09T16:19:56Z

doc/source/io.rst

+.. ipython:: python
+   :suppress:
+
+   write_html(url_df, 'render_links', render_links=True)


what is this for?

This runs the write_html() call without including its input or output in the notebook cells that appear in the doc. This makes available the static file rendered on line 2625.

I copied a pattern used elsewhere in this file, example "bold_rows" section, starting line 2584.

What's the downside of just printing it? I think creating files like this is hard to keep track of within our codebase so less than ideal

The generated HTML is also printed in a notebook cell. The lines pointed out by @jreback write that HTML to a separate file that can be rendered.

Here's what it looks like after building:

That said, if having the rendered sample is not worth the overhead of the extra file, I'm happy to remove.

hmm i am not sure we actually need to show the html, showing the rendered version is enough. but since we are doing it for all of these, ok

jreback

just some small comments. ping on green.

jreback · 2018-12-12T13:34:15Z

pandas/core/frame.py

@@ -2067,6 +2067,12 @@ def to_html(self, buf=None, columns=None, col_space=None, header=True,
            A css id is included in the opening `<table>` tag if specified.

            .. versionadded:: 0.23.0
+
+        render_links : boolean, default False


i think our standard is for bool, @datapythonista should probably validate this generally as I seem mixed usage all over the place.

yes, we still need to fix many cases before we can validate this, but if you run ./scripts/validate_docstrings.py pandas.DataFrame.to_html it will report this, and any other formatting issue.

jreback · 2018-12-12T13:34:52Z

pandas/tests/io/formats/data/render_links_false.html

+      <td>pydata.org</td>
+    </tr>
+  </tbody>
+</table>


can you add a linefeed here (and on the one below)

jreback · 2018-12-13T00:52:26Z

doc/source/whatsnew/v0.24.0.rst

@@ -27,6 +27,7 @@ New features
 - :meth:`DataFrame.corr` and :meth:`Series.corr` now accept a callable for generic calculation methods of correlation, e.g. histogram intersection (:issue:`22684`)
 - :func:`DataFrame.to_string` now accepts ``decimal`` as an argument, allowing the user to specify which decimal separator should be used in the output. (:issue:`23614`)
 - :func:`DataFrame.read_feather` now accepts ``columns`` as an argument, allowing the user to specify which columns should be read. (:issue:`24025`)
+- :func:`DataFrame.to_html` now accepts ``render_links`` as an argument, allowing the user to generate HTML with links to any URLs that appear in the DataFrame. (:issue:`2679`)


can you add a ref to the new docs that you added.

Need a link here or just a mention of the addition to docs?

If you add a link I can approve and merge

@jreback @WillAyd Link added and all green.

jreback · 2018-12-13T00:53:40Z

minor point, pls ping on green. @WillAyd over to you.

benjaminarjun · 2018-12-13T01:52:07Z

Just got in front of keyboard and was about to ping you 😃 thanks for approving, will wait to hear from @WillAyd.

WillAyd · 2018-12-13T07:09:49Z

Thanks @benarthur91 - nice change!

benjaminarjun · 2018-12-13T14:30:11Z

Thanks to everyone for all the help!

…s-dev#2679) (pandas-dev#23715)

WillAyd requested changes Nov 15, 2018

View reviewed changes

gfyoung added Enhancement IO HTML read_html, to_html, Styler.apply, Styler.applymap labels Nov 15, 2018

jreback requested changes Nov 15, 2018

View reviewed changes

pandas/core/frame.py Show resolved Hide resolved

ENH: Add render_links option to DataFrame to_html()

0702824

benjaminarjun force-pushed the feature/render-df-link-issue2679 branch from 974908d to 0702824 Compare November 26, 2018 15:31

benjaminarjun added 4 commits November 27, 2018 20:50

Remove usage of dedent

61c22a3

Merge branch 'master' into feature/render-df-link-issue2679_test

67cfe5f

HTML writer to write unicode

793006d

Merge branch 'master' into feature/render-df-link-issue2679_test

ccffd5f

Move _is_url import to correct position

13e20ea

WillAyd requested changes Dec 1, 2018

View reviewed changes

pandas/tests/io/formats/test_to_html.py Outdated Show resolved Hide resolved

pandas/io/formats/html.py Outdated Show resolved Hide resolved

pandas/io/formats/html.py Outdated Show resolved Hide resolved

benjaminarjun added 3 commits December 2, 2018 13:10

A few style fixes

db31c1d

Merge branch 'master' into feature/render-df-link-issue2679_test

13bbca4

Merge branch 'master' into feature/render-df-link-issue2679_test

61737ed

WillAyd reviewed Dec 3, 2018

View reviewed changes

pandas/tests/io/formats/test_to_html.py Outdated Show resolved Hide resolved

benjaminarjun added 2 commits December 3, 2018 18:51

Move render_links test expectations to separate files

14b79e1

Merge branch 'master' into feature/render-df-link-issue2679_test

dd83afa

jreback requested changes Dec 4, 2018

View reviewed changes

benjaminarjun added 4 commits December 4, 2018 21:41

Add render_links description and example to whatsnew and IO docs

2f02e28

Merge branch 'master' into feature/render-df-link-issue2679_test

7ecbeb6

Minor style fix in io.rst

7e0de08

Merge branch 'master' into feature/render-df-link-issue2679_test

8683d56

WillAyd reviewed Dec 9, 2018

View reviewed changes

pandas/tests/io/formats/test_to_html.py Outdated Show resolved Hide resolved

benjaminarjun added 2 commits December 9, 2018 08:13

Style fix on render_links test

7b5b323

Merge branch 'master' into feature/render-df-link-issue2679_test

505a2f0

jreback requested changes Dec 9, 2018

View reviewed changes

Add versionadded in io.rst

05f17f2

jreback added this to the 0.24.0 milestone Dec 12, 2018

jreback requested changes Dec 12, 2018

View reviewed changes

benjaminarjun added 2 commits December 12, 2018 12:38

boolean -> bool and newlines for render_links_*.html

bb637a8

Merge branch 'master' into feature/render-df-link-issue2679_test

c01306d

jreback requested changes Dec 13, 2018

View reviewed changes

jreback approved these changes Dec 13, 2018

View reviewed changes

benjaminarjun added 2 commits December 12, 2018 21:32

Add link to IO docs from render_links whatsnew entry

78ed4b9

Merge branch 'master' into feature/render-df-link-issue2679_test

9f3e058

WillAyd approved these changes Dec 13, 2018

View reviewed changes

WillAyd merged commit 2f6d682 into pandas-dev:master Dec 13, 2018

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

Add option for DataFrame.to_html() to render URL data as links (panda…

3cb580a

…s-dev#2679) (pandas-dev#23715)

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

Add option for DataFrame.to_html() to render URL data as links (panda…

f411e73

…s-dev#2679) (pandas-dev#23715)

Add option for DataFrame.to_html() to render URL data as links (#2679) #23715

Add option for DataFrame.to_html() to render URL data as links (#2679) #23715

Conversation

benjaminarjun commented Nov 15, 2018 • edited Loading

pep8speaks commented Nov 15, 2018 • edited Loading

Comment last updated on November 26, 2018 at 05:41 Hours UTC

gfyoung commented Nov 26, 2018

benjaminarjun commented Nov 26, 2018 • edited Loading

gfyoung commented Nov 26, 2018

WillAyd commented Nov 26, 2018

benjaminarjun commented Nov 26, 2018

gfyoung commented Nov 26, 2018 • edited Loading

benjaminarjun commented Nov 29, 2018

gfyoung commented Nov 29, 2018 • edited Loading

benjaminarjun commented Nov 29, 2018

gfyoung commented Nov 29, 2018

benjaminarjun commented Nov 29, 2018

codecov bot commented Nov 29, 2018 • edited Loading

Codecov Report

benjaminarjun commented Dec 1, 2018

WillAyd left a comment

Choose a reason for hiding this comment

jreback left a comment

Choose a reason for hiding this comment

benjaminarjun commented Dec 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjaminarjun Dec 9, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjaminarjun Dec 11, 2018 • edited Loading

Choose a reason for hiding this comment

jreback Dec 12, 2018 • edited Loading

Choose a reason for hiding this comment

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Dec 13, 2018

benjaminarjun commented Dec 13, 2018

WillAyd commented Dec 13, 2018

benjaminarjun commented Dec 13, 2018

benjaminarjun commented Nov 15, 2018 •

edited

Loading

pep8speaks commented Nov 15, 2018 •

edited

Loading

benjaminarjun commented Nov 26, 2018 •

edited

Loading

gfyoung commented Nov 26, 2018 •

edited

Loading

gfyoung commented Nov 29, 2018 •

edited

Loading

codecov bot commented Nov 29, 2018 •

edited

Loading

benjaminarjun Dec 9, 2018 •

edited

Loading

benjaminarjun Dec 11, 2018 •

edited

Loading

jreback Dec 12, 2018 •

edited

Loading