Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose PyFloat_AsDouble at Python level: operator.as_float? #84978

Closed
mdickinson opened this issue May 28, 2020 · 13 comments
Closed

Expose PyFloat_AsDouble at Python level: operator.as_float? #84978

mdickinson opened this issue May 28, 2020 · 13 comments
Labels
3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@mdickinson
Copy link
Member

mdickinson commented May 28, 2020

BPO 40801
Nosy @mdickinson, @zware, @serhiy-storchaka
PRs
  • bpo-40801: Add operator.as_float #20481
  • gh-84978: Add float.from_number() and complex.from_number() #26827
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2020-05-28.09:23:06.760>
    labels = ['type-feature', '3.11']
    title = 'Expose PyFloat_AsDouble at Python level: operator.as_float?'
    updated_at = <Date 2021-06-21.12:00:00.034>
    user = 'https://github.com/mdickinson'

    bugs.python.org fields:

    activity = <Date 2021-06-21.12:00:00.034>
    actor = 'serhiy.storchaka'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = []
    creation = <Date 2020-05-28.09:23:06.760>
    creator = 'mark.dickinson'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 40801
    keywords = ['patch']
    message_count = 11.0
    messages = ['370181', '370189', '370233', '370244', '370247', '370253', '370255', '370258', '370259', '370286', '396234']
    nosy_count = 3.0
    nosy_names = ['mark.dickinson', 'zach.ware', 'serhiy.storchaka']
    pr_nums = ['20481', '26827']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue40801'
    versions = ['Python 3.11']

    Linked PRs

    @mdickinson
    Copy link
    Member Author

    Motivation
    ----------

    Various pieces of Python need to do a duck-typed conversion of an arbitrary float-like object to a float (or a C double) for computation. The math module is the most obvious example - most math-module functions that accept a float also accept float-like things, like np.float32 and Decimal - but it's not the only place that this is needed.

    This conversion is easy at C level, being encapsulated in a single function call: PyFloat_AsDouble. (Plus a PyFloat_FromDouble if you want to go back to Python space, of course.)

    But: it's surprisingly awkward to get an equivalent effect in pure Python code. Options are:

    1. Do an explicit type check to exclude str, bytes and bytearray, and then call the float constructor. But the extra type check is ugly and potentially not future-proof.

    2. Call type(obj).__float__(obj). But this has several problems: __float__ can return an instance of a float subclass rather than a strict float, and then it's hard to convert to an actual float. And in more recent versions of Python, this no longer matches PyFloat_AsDouble because it doesn't account for objects that provide __index__ but not __float__. And calling dunder methods directly should rarely be the Right Way To Do It.

    3. Use the implicit ability of the math module to do this, for example using math.copysign(obj, obj). This works! But it's not a clear expression of the intent.

    This has bitten me in Real Code (TM), where I've needed a way to convert an arbitrary float-like thing to a float, ideally following the same rules that Python uses. And also ideally in such a way that my own code doesn't have to change if Python updates its rules, as for example it did recently to allow things with an __index__ to be considered float-like.

    Proposal
    --------

    Add a new operator function "operator.as_float" which matches Python's duck-typed acceptance of float-like things, in the same way that the existing operator.index matches Python's implicit acceptance of int-like things in various APIs that expect integers.

    Internally, "operator.as_float" would simply call PyFloat_AsDouble followed by PyFloat_FromDouble (possibly with a fast path to pass objects of exact type float through directly).

    Related: bpo-17576.

    @mdickinson mdickinson added 3.10 only security fixes type-feature A feature request or enhancement labels May 28, 2020
    @mdickinson mdickinson changed the title Expose PyFloat_ToDouble at Python level: operator.to_float? Expose PyFloat_ToDouble at Python level: operator.as_float? May 28, 2020
    @mdickinson mdickinson changed the title Expose PyFloat_ToDouble at Python level: operator.to_float? Expose PyFloat_ToDouble at Python level: operator.as_float? May 28, 2020
    @mdickinson
    Copy link
    Member Author

    Proof of concept in #64680

    @zware
    Copy link
    Member

    zware commented May 28, 2020

    operator seems a slightly odd place for this. My naive expectation would be that float(floatlike_obj) should do what you want, but it seems that's not the case (too permissive of input types?). So then, what about an alternate constructor on the float object, float.from_floatlike(obj)? This could be implemented as effectively:

    class float:
        @classmethod
        def from_floatlike(cls, obj):
            return cls(PyFloat_FromDouble(PyFloat_AsDouble(obj)))

    which would work to get an instance of any float subclass after a round-trip through a double. I have no idea whether that's actually useful, though :)

    @mdickinson
    Copy link
    Member Author

    operator seems a slightly odd place for this.

    Yes, it's not ideal. It's by analogy with operator.index, which provides the equivalent duck-typing for integers, and calls __index__. Similarly, operator.as_float is primarily there to call __float__, except that now that PyFloat_AsDouble also makes use of __index__, it'll call that, too.

    My other thought was putting this in math, since it's what the math module is already doing implicitly to most inputs.

    An alternative float constructor could work. Though we then run into the messy question of what it should do for float subclasses ...

    @mdickinson
    Copy link
    Member Author

    On naming, maybe float.from_number or float.from_numeric?

    @serhiy-storchaka
    Copy link
    Member

    I think an alternative constructor is the best option. Some time ago I proposed to add more alternative constructors: https://mail.python.org/archives/list/python-ideas@python.org/thread/5JKQMIC6EUVCD7IBWMRHY7DRTTNSBOWG/

    @serhiy-storchaka serhiy-storchaka changed the title Expose PyFloat_ToDouble at Python level: operator.as_float? Expose PyFloat_AsDouble at Python level: operator.as_float? May 28, 2020
    @serhiy-storchaka serhiy-storchaka changed the title Expose PyFloat_ToDouble at Python level: operator.as_float? Expose PyFloat_AsDouble at Python level: operator.as_float? May 28, 2020
    @serhiy-storchaka
    Copy link
    Member

    The problem with the roundtrip PyFloat_FromDouble(PyFloat_AsDouble(obj)) is that (in contrary to PyNumber_Index()) it is lossy. Converting Decimal, Fraction, float128 to float before using it in expression can lead to loss of precision.

    So such conversion looks to me less useful than operator.index().

    @mdickinson
    Copy link
    Member Author

    So such conversion looks to me less useful than operator.index().

    That may be true, but it's still useful, and currently very hard to write without core Python support. The problem is that duck-typing for something float-like is easy and widespread in the Python C code (in the math module, in float formatting, in *any* function that uses the "d" converter with PyArg_ParseTuple), but it's hard to correctly spell the equivalent in pure Python.

    I've needed this in a few places. Most recently, I needed it for the Enthought Traits library: the "Float" trait type accepts something float-like and needs to convert it to something of exact Python type float. (That float is then often consumed by other third-party libraries that can't reliably be expected to do their own duck-typing.)

    It would also be useful when trying to create pure Python equivalents for modules written in C (e.g., for things like datetime and decimal). Where the C code uses PyFloat_AsDouble, there's really no equivalent that can be used in Python.

    @mdickinson
    Copy link
    Member Author

    Converting Decimal, Fraction, float128 to float before using it in expression can lead to loss of precision.

    My experience is that this loss of precision is hardly ever a practical problem in the real world of scientific development; in practice floating-point numbers are almost universally IEEE 754 doubles (perhaps sometimes single-precision in large datasets, like seismic SEG-Y files; occasionally IBM format hex floats; but IEEE 754 doubles are by far the majority). It's very rare to be using float128 or Decimal or Fraction in practice for scientific data.

    That's not to say that people outside that world won't be using these things, but there's a big ecosystem where float64 is pretty much all you need.

    @mdickinson
    Copy link
    Member Author

    @serhiy-storchaka
    Copy link
    Member

    PR 26827 is a draft for float.from_number() and complex.from_number().

    @serhiy-storchaka serhiy-storchaka added 3.11 only security fixes and removed 3.10 only security fixes labels Jun 21, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @skirpichev
    Copy link
    Member

    1. Call type(obj).__float__(obj). But this has several problems: __float__ can return an instance of a float subclass

    Since 3.7 it was deprecated (#71170) and maybe it's time to remove this (#109311).

    So, it's not an issue in that part.

    calling dunder methods directly should rarely be the Right Way To Do It.

    Then why not export obj.__float__() as operator.as_float. Same for obj.__complex__() (as_complex?) or obj.__int__() (as_int?) - all should return an appropriate type.

    Perhaps, a different constructor could be more discoverable option. But I don't see why we can't expose mentioned above dunder methods just as for the __index__. Naming? (i.e. operator.as_float vs operator.float)

    serhiy-storchaka added a commit that referenced this issue Jul 15, 2024
    They are alternate constructors which only accept numbers
    (including objects with special methods __float__, __complex__
    and __index__), but not strings.
    estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
    …thonGH-26827)
    
    They are alternate constructors which only accept numbers
    (including objects with special methods __float__, __complex__
    and __index__), but not strings.
    @skirpichev
    Copy link
    Member

    @serhiy-storchaka, probably this can be closed?

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants