Improved python tree plots #2304

CharlesAuguste · 2019-08-01T16:17:05Z

In this pull request, we tried to improve the way trees are plotted by the function plot_tree . Visually digestible trees are essential for interpreting, debugging, and improving tree-based methods. LightGBM's current implementation of plot_tree produces trees which are very hard to mentally process and which, in our opinion, lack some useful information, like which nodes are constrained and how much data flows down through each node.

This PR makes several changes to the plot_tree function which increases the amount of information displayed in a given tree whilst, at the same time, making the tree more digestible.

Moreover, we add monotonically constrained features to the model's meta information so that this information is accessible through the model.

The figures below show how the output from the current plot_tree function compares with the output from our proposed plot_tree function, and, demonstrate how the proposed output contains significantly more information than the current output, yet, is much easier to interpret.

Here is a list of the new features from this pull request:

Trees are now plotted horizontally and not vertically, and are more compact, which allows for a better visualization of big trees;
Features and thresholds are now written in a more compact yet intelligible way;
The default precision of the information printed is now 3;
The user can now print the data percentage in nodes and leaves;
Nodes splitted on a monotone feature are now colored in light green for a monotone increasing feature, and in light red for a monotone decreasing feature, and a legend is added to explain what these features correspond to;
Important information is printed in bold;
Other minor changes.

Current LightGBM tree:

Tree generated by the code of this pull request

…aints is passed.

This reverts commit dd8bf14a3ba604b0dfae3b7bb1c64b6784d15e03.

…ees.

… exception instead of doing nothing.

msftclas · 2019-08-01T16:17:19Z

All CLA requirements met.

aldanor · 2019-08-01T16:22:05Z

+@redditur

CharlesAuguste · 2019-08-08T09:41:25Z

Initially we only had features and leaf values in bold because we thought they were the most important information. But as you said, it is not consistent. In my opinion, a good compromise would be this (thresholds in bold added).

I don't think it makes sense to put "leaf x" in bold, because it is not an important information. But if you still prefer it that way I will make the necessary changes.

StrikerRUS

@CharlesAuguste Totally agree with you! Plots look really amazing now! Thanks a lot for your contribution!

Only one remaining thing has some room for improvements, I think. In my opinion, legend can be somehow clarified. At least, some words are needed about that Increasing/Decreasing is related to monotone constraints. Imagine, someone uses tree plot in their slides. To be honest, without any additional info (or code around a picture), it's not very clear, what means Increasing/Decreasing...

CharlesAuguste · 2019-08-09T08:25:00Z

Would you like this better ("Monotone constraints" added on top of the legend)?

StrikerRUS · 2019-08-09T10:56:27Z

@CharlesAuguste Nice, thanks! I think it looks clearer now.

StrikerRUS · 2019-09-01T11:30:15Z

@henry0312 @chivee @guolinke @jameslamb Can you please help and give a second review to this PR?

jameslamb

I don't know enough C++ to approve the config.h changes but looked over the rest and left a few minor stylistic comments. Otherwise, this is good from my perspective.

jameslamb · 2019-09-02T02:25:30Z

python-package/lightgbm/plotting.py

            if root['decision_type'] == '<=':
-                l_dec, r_dec = '<=', '>'
+                operator = "&#8804;"


I think the code would be easier to understand if this was put in a named variable, e.g.

lte_symbol = "≤" operator = lte_symbol

jameslamb · 2019-09-02T02:26:16Z

python-package/lightgbm/plotting.py

-                elif info == 'internal_count':
-                    label += r'\n{0}: {1}'.format(info, root[info])
-            graph.node(name, label=label)
+            l_dec, r_dec = 'yes', "no"


in my opinion, simple assignments like this should be on their own lines. Could you change this to?

l_dec = 'yes' r_dec = 'no'

jameslamb · 2019-09-02T02:28:11Z

python-package/lightgbm/plotting.py

-    precision : int or None, optional (default=None)
+        'split_gain', 'internal_value', 'internal_count', 'internal_weight',
+        'leaf_count', 'leaf_weight', 'data_percentage'.
+    precision : int or None, optional (default=3)


Could you expand this documentation please? What are the units and what is the implication of setting None?

@jameslamb This is nice catch, but I think this is beyond of this PR. precision was introduced in #1424 and is used in several functions (e.g. in plot_importance() (#1777)), so I'll address your comment in a separate PR after merging this one.

makes sense!

jameslamb · 2019-09-02T02:29:03Z

python-package/lightgbm/plotting.py

+    if 'monotone_constraints' in model:
+        monotone_constraints = model['monotone_constraints']
+    else:
+        monotone_constraints = None


could you please simplify this?

monotone_constraints = model.get('monotone_constraints', None)

StrikerRUS · 2019-09-05T13:26:06Z

Close-reopen to re-trigger Appveyor.

StrikerRUS · 2019-09-07T22:01:23Z

@jameslamb Can you please review the changes made?

jameslamb

I took another look, looks good to me! This is an awesome improvement.

aldanor · 2019-09-08T16:35:13Z

Great stuff, thanks guys 👍 (now we just need to figure out what to do with the other big PR to move it forward...)

Charles Auguste and others added 25 commits August 1, 2019 11:06

Some basic changes to the plot of the trees to make them readable.

442db7e

Squeezed the information in the nodes.

0ade18f

Added colouring when a dictionnary mapping the features to the constr…

3655199

…aints is passed.

Fix spaces.

1a8ec2c

Added data percentage as an option in the nodes.

7bef020

Squeezed the information in the leaves.

6d0de1a

Important information is now in bold.

7e1f705

Added a legend for the color of monotone splits.

0e713b7

Changed "split_gain" to "gain" and "internal_value" to "value".

49ca769

Sqeezed leaves a bit more.

8cb62a9

Changed description in the legend.

45f4d0e

Revert "Sqeezed leaves a bit more."

41bec87

This reverts commit dd8bf14a3ba604b0dfae3b7bb1c64b6784d15e03.

Increased the readability for the gain.

4f7ec67

Tidied up the legend.

a8ba552

Added the data percentage in the leaves.

8968b30

Added the monotone constraints to the dumped model.

6cb343d

Monotone constraints are now specified automatically when plotting tr…

f30cacc

…ees.

Raise an exception instead of the bug that was here before.

0e8e51d

Removed operators on the branches for a clearer design.

59f3694

Small cleaning of the code.

77bcccc

Setting a monotone constraint on a categorical feature now returns an…

a8f4496

… exception instead of doing nothing.

Fix bug when monotone constraints are empty.

79c8cb7

Fix another bug when monotone constraints are empty.

aa7b96b

Variable name change.

cea089b

Added is / isn't on every edge of the trees.

0bb1533

CharlesAuguste requested review from jameslamb, Laurae2 and StrikerRUS as code owners August 1, 2019 16:17

Made information in nodes written in a specific order.

4df0a64

StrikerRUS approved these changes Aug 8, 2019

View reviewed changes

StrikerRUS requested review from guolinke, henry0312 and chivee August 8, 2019 19:51

Added information to clarify legend.

82c6ef0

StrikerRUS mentioned this pull request Aug 17, 2019

v2.3.0 realese #2138

Merged

StrikerRUS requested review from guolinke and jameslamb and removed request for jameslamb and guolinke August 27, 2019 10:24

guolinke approved these changes Sep 2, 2019

View reviewed changes

jameslamb requested changes Sep 2, 2019

View reviewed changes

Code cleaning.

388543b

StrikerRUS closed this Sep 5, 2019

StrikerRUS reopened this Sep 5, 2019

jameslamb approved these changes Sep 7, 2019

View reviewed changes

StrikerRUS merged commit f52be9b into microsoft:master Sep 8, 2019

StrikerRUS mentioned this pull request Sep 8, 2019

[examples] Update example notebook #2391

Merged

CharlesAuguste deleted the improved-python-tree-plots/LightGBM branch September 8, 2019 20:02

CharlesAuguste restored the improved-python-tree-plots/LightGBM branch December 19, 2019 15:53

CharlesAuguste deleted the improved-python-tree-plots/LightGBM branch December 19, 2019 15:54

lock bot locked as resolved and limited conversation to collaborators Mar 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improved python tree plots #2304

Improved python tree plots #2304

CharlesAuguste commented Aug 1, 2019 •

edited

Loading

msftclas commented Aug 1, 2019 •

edited

Loading

aldanor commented Aug 1, 2019

CharlesAuguste commented Aug 8, 2019

StrikerRUS left a comment

CharlesAuguste commented Aug 9, 2019

StrikerRUS commented Aug 9, 2019

StrikerRUS commented Sep 1, 2019

jameslamb left a comment

jameslamb Sep 2, 2019

jameslamb Sep 2, 2019

jameslamb Sep 2, 2019

StrikerRUS Sep 2, 2019 •

edited

Loading

jameslamb Sep 3, 2019

jameslamb Sep 2, 2019

StrikerRUS commented Sep 5, 2019

StrikerRUS commented Sep 7, 2019

jameslamb left a comment

aldanor commented Sep 8, 2019

Improved python tree plots #2304

Improved python tree plots #2304

Conversation

CharlesAuguste commented Aug 1, 2019 • edited Loading

msftclas commented Aug 1, 2019 • edited Loading

aldanor commented Aug 1, 2019

CharlesAuguste commented Aug 8, 2019

StrikerRUS left a comment

Choose a reason for hiding this comment

CharlesAuguste commented Aug 9, 2019

StrikerRUS commented Aug 9, 2019

StrikerRUS commented Sep 1, 2019

jameslamb left a comment

Choose a reason for hiding this comment

jameslamb Sep 2, 2019

Choose a reason for hiding this comment

jameslamb Sep 2, 2019

Choose a reason for hiding this comment

jameslamb Sep 2, 2019

Choose a reason for hiding this comment

StrikerRUS Sep 2, 2019 • edited Loading

Choose a reason for hiding this comment

jameslamb Sep 3, 2019

Choose a reason for hiding this comment

jameslamb Sep 2, 2019

Choose a reason for hiding this comment

StrikerRUS commented Sep 5, 2019

StrikerRUS commented Sep 7, 2019

jameslamb left a comment

Choose a reason for hiding this comment

aldanor commented Sep 8, 2019

CharlesAuguste commented Aug 1, 2019 •

edited

Loading

msftclas commented Aug 1, 2019 •

edited

Loading

StrikerRUS Sep 2, 2019 •

edited

Loading