DOC: update the pandas.DataFrame.plot.density docstring

[jonas-schulze]

This docstring is quite similar to the one of #20041.

Unfortunately, I was not able to compute a kernel estimate of a
two-dimensional random variable. Hence, the example is more of an
analysis of some independent data series.

Checklist for the pandas documentation sprint (ignore this if you are doing
an unrelated PR):

• PR title is "DOC: update the docstring"
• The validation script passes: scripts/validate_docstrings.py <your-function-or-method>
• The PEP8 style check passes: git diff upstream/master -u -- "*.py" | flake8 --diff
• The html version looks good: python doc/make.py --single <your-function-or-method>
• It has been proofread on language by another sprint participant (sort of; it's based on PR20041)

Please include the output of the validation script below between the "```" ticks:

################################################################################
################## Docstring (pandas.DataFrame.plot.density)  ##################
################################################################################

Generate Kernel Density Estimate plot using Gaussian kernels.

In statistics, `kernel density estimation`_ (KDE) is a non-parametric
way to estimate the probability density function (PDF) of a random
variable. This function uses Gaussian kernels and includes automatic
bandwith determination.

.. _kernel density estimation:
    https://en.wikipedia.org/wiki/Kernel_density_estimation

Parameters
----------
bw_method : str, scalar or callable, optional
    The method used to calculate the estimator bandwidth. This can be
    'scott', 'silverman', a scalar constant or a callable.
    If None (default), 'scott' is used.
    See :class:`scipy.stats.gaussian_kde` for more information.
ind : NumPy array or integer, optional
    Evaluation points for the estimated PDF. If None (default),
    1000 equally spaced points are used. If `ind` is a NumPy array, the
    KDE is evaluated at the points passed. If `ind` is an integer,
    `ind` number of equally spaced points are used.
**kwds : optional
    Additional keyword arguments are documented in
    :meth:`pandas.DataFrame.plot`.

Returns
-------
axes : matplotlib.AxesSubplot or np.array of them

See Also
--------
scipy.stats.gaussian_kde : Representation of a kernel-density
    estimate using Gaussian kernels. This is the function used
    internally to estimate the PDF.
Series.plot.kde : Generate a KDE plot for a Series.

Examples
--------
Given several Series of points randomly sampled from unknown
distributions, estimate their distribution using KDE with automatic
bandwidth determination and plot the results, evaluating them at
1000 equally spaced points (default):

.. plot::
    :context: close-figs

    >>> df = pd.DataFrame({
    ...     'x': [1, 2, 2.5, 3, 3.5, 4, 5],
    ...     'y': [4, 4, 4.5, 5, 5.5, 6, 6],
    ... })
    >>> ax = df.plot.kde()

A scalar bandwidth can be specified. Using a small bandwidth value can
lead to overfitting, while using a large bandwidth value may result
in underfitting:

.. plot::
    :context: close-figs

    >>> ax = df.plot.kde(bw_method=0.3)

.. plot::
    :context: close-figs

    >>> ax = df.plot.kde(bw_method=3)

Finally, the `ind` parameter determines the evaluation points for the
plot of the estimated PDF:

.. plot::
    :context: close-figs

    >>> ax = df.plot.kde(ind=[1, 2, 3, 4, 5, 6])

################################################################################
################################## Validation ##################################
################################################################################

Errors found:
	Errors in parameters section
		Parameters {'kwds'} not documented
		Unknown parameters {'**kwds'}

I left the error because David and Israel mentioned doing it either way on Gitter ..

[jorisvandenbossche]

Damn, we again have the problem of miscommunication that this one was already take one of the previous days: #20041
Sorry about that!

cc @dukebody it would be good if one of both of you can combine both PRs and take the best bits of both PRs

[TomAugspurger]

If either of you are comfortable with git, you should be able to cherry pick the other's commit so you'll both show up as authors.

LMK if you need help with the git stuff.

[jonas-schulze]

@jorisvandenbossche and @TomAugspurger thanks for the quick responses! I noticed the other PR during the sprint, but that one improved the function of Series and not DataFrame. I'm not that comfortable in git, I'd appreciate any help. What about the requested changes in the other PR?

[jorisvandenbossche]

Hmm, OK, didn't see it was DataFrame vs Series.
We should probably share the docstrings somehow, but let's do that in a follow-up issue. Can you for now see if can 'steal' ideas of the other one?

[jonas-schulze]

I agree; I had the same idea with sharing the docstrings between DataFrames and Series. Most DataFrame plotting functions just seem to "pass the call" to the Series it's containing.

What do you mean by "steal ideas of the other one"? I basically copied the Series docstring, included all (but See Also) requested changes and extended the example Series to 2-column DataFrames.

[jorisvandenbossche]

Ah, then you already 'stole' the ideas :-)
No, I assumed you wrote it completely independent, and then it would have been nice to combine the different enhancements in both PRs here. But that already happened then

[jorisvandenbossche]

So I think the main difference between the frame vs series methods is that the frame version has x, y as keywords?

[jreback]

this dupes #20041

[jorisvandenbossche]

@jreback see discussion we are having. It's not an exact duplciate, as it is frame vs series. But I agree the docstring is certainly duplicative and can be shared.
We only need to figure out how to exactly do that.

[jreback]

@jorisvandenbossche yep, just noticed. nvm.

[jorisvandenbossche]

OK, I merged the other PR.
@jonas-schulze the comments that were left there (and the ones you added yourself), can you address them here?
You will need to fetch upstream, merge upstream/master and fix the conflicts though

[jonas-schulze]

Yep, I'll do that and squash the commits if I am able to.

It does provide x and y as it passes them to Series.plot or DataFrame.plot (which one? depending on the answer, I have to fix the See Also section), but I wasn't able to figure out what x does ..

[jonas-schulze]

Should I change this one to pandas.DataFrame.plot (copy-and-paste 🙈)? I tried to understand the code but it's not yet clear to me. Both functions eventually call MPLPlot._plot().

[TomAugspurger]

yeah, pandas.DataFrame.plot is fine.

[TomAugspurger]

Just

Series.plot.kde : Generate a KDE plot for a Series.

[jonas-schulze]

I included the changes and updated the very first comment to include the new output of the validation script.

[jonas-schulze]

What do you think of my concern with the docstring / the DataFrame density function?

Unfortunately, I was not able to compute a kernel estimate of a
two-dimensional random variable. Hence, the example is more of an
analysis of some independent data series.

[jorisvandenbossche]

Updates to the existing docstring look good!

What is the difference now exactly between both docstrings? Only the examples section?

[jonas-schulze]

Unless I missed something, that's the only difference.

[jorisvandenbossche]

Can you then maybe pull out the common part into a variable, and use @Appender(_kde_docstring % _kde_series_example) to add it to both functions?

[jorisvandenbossche]

(if that was very cryptic, say so :-))

[jonas-schulze]

I just noticed two more differences:

1. the **kwds arguments include their own class (Series or DataFrame)
2. the See Also sections references the other class (DataFrame or Series)

So there would be some more processing required than just appending the examples .. shall I still proceed? I think it might be better to come up with a more general solution to this phenomenon of "sibling functions" between Series and DataFrame. It should then be applied to all the other plotting functions as well once their PRs got merged, hence I would do it in a later PR.

[pep8speaks]

Hello @jonas-schulze! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on March 11, 2018 at 19:25 Hours UTC

[jonas-schulze]

The new commit for the unified docstrings introduces two new errors in the validation script:

Errors found:
	Closing quotes should be placed in the line after the last text in the docstring (do not close the quotes in the same line as the text, or leave a blank line between the last text and the quotes)
	Use only one blank line to separate sections or paragraphs

I have no idea how to fix them .. can you help me please?

[dukebody]

I think the original was right: what we estimate is the distribution of the random sample. If you say "estimate its distribution" it looks like we are estimating the distribution of the distribution. But I'm not a native English speaker. What do you think?

[dukebody]

Saw your comment in my PR about this.

I personally prefer "this" because I feel it makes clearer that what is estimated is the unknown distribution. Otherwise "its" refers to the Series, which is early in the sentence, and is more confusing to me.

But I'm not a native English speaker. I'm fine with both options, don't like bikeshedding too much :)

[jonas-schulze]

Me neither .. just because we are already thinking of it: changing the whole part to estimate its PDF using KDE might be an even better fit, as this is what we are really doing.

[jonas-schulze]

I just changed it to that.

[dukebody]

@jonas-schulze Great work on combining both docstrings and addressing the comments on my PR! LGTM.

Regarding the issue raised by the validator, can you check the output of the compiled docstring when running it in both methods? It's probably a silly formatting issue when using the Appender.

[dukebody]

@jonas-schulze Great work on combining both docstrings and addressing the comments on my PR! LGTM.

Regarding the issue raised by the validator, can you check the output of the compiled docstring when running it in both methods? It's probably a silly formatting issue when using the Appender.

[jorisvandenbossche]

Perfect!

Added small comment regarding the indentation

The new commit for the unified docstrings introduces two new errors in the validation script:

You can ignore those I think. I don't see the two blank lines, and the issue of the closing quotes is mainly for normal in-code docstrings. For those templated ones, it doesn't really matter (it doesn't affect rendering in sphinx)

[jorisvandenbossche]

You need to give this one level of indentation less (now it 2, should only be 1 to match the level of identation you gave to the template)

(and the same for the dataframe ones)

[jonas-schulze]

I added some indentation to the common part instead. Removing the indentation at this place just doesn't look right. 🙈

[jonas-schulze]

Adding a .strip() to the dictionaries fixes the validation errors. In order to keep the amount of changes low, and because the strings would look very weird otherwise, I added 1 level of indent to the common docstring above class KdePlot.

[jonas-schulze]

Off topic: why doesn't pandas have a panda as its logo/mascot?

[dukebody]

LGTM

[TomAugspurger]

Thanks!