init=False attributes which depend on keyword-only attributes are impossible

[gabbard]

The initialization of an init=False attribute cannot (straightforwardly) depend on a kw_only=True attribute because there is no legal ordering of these two attributes.

from attr import attrs, attrib


@attrs
class BrokenInitFirst:
    _to_init: str  = attrib(init=False)
    kwarg: str = attrib(kw_only=True)

    @_to_init.default
    def _init_to_init(self) -> str:
        return self.kwarg + "foo"


BrokenInitFirst(kwarg="meep")

produces:

AttributeError: 'BrokenInitFirst' object has no attribute 'kwarg'

while

from attr import attrs, attrib


@attrs
class BrokenKwArgFirst:
    kwarg: str = attrib(kw_only=True)
    _to_init: str = attrib(init=False)

    @_to_init.default
    def _init_to_init(self) -> str:
        return self.kwarg + "foo"


BrokenKwArgFirst(kwarg="meep")

produces

  File "/Users/gabbard/anaconda3/envs/cwc-event/lib/python3.6/site-packages/attr/_make.py", line 904, in attrs
    return wrap(maybe_cls)
  File "/Users/gabbard/anaconda3/envs/cwc-event/lib/python3.6/site-packages/attr/_make.py", line 855, in wrap
    cache_hash,
  File "/Users/gabbard/anaconda3/envs/cwc-event/lib/python3.6/site-packages/attr/_make.py", line 471, in __init__
    cls, these, auto_attribs, kw_only
  File "/Users/gabbard/anaconda3/envs/cwc-event/lib/python3.6/site-packages/attr/_make.py", line 416, in _transform_attrs
    a=a
ValueError: Non keyword-only attributes are not allowed after a keyword-only attribute.  Attribute in question: Attribute(name='_to_init', default=Factory(factory=<function BrokenKwArgFirst._init_to_init at 0x10830df28>, takes_self=True), validator=None, repr=True, cmp=True, hash=None, init=False, metadata=mappingproxy({}), type=<class 'str'>, converter=None, kw_only=False)

These examples can also be found in https://github.com/rgabbard/attrs-kwonly-init-bug

This is related to #448 . Assuming there is some sort of internal ordering of attributes which also controls their initialization order, I think the solution here is to sort kw_only=True attributes after kw_only=False, and init=False last of all.

[gabbard]

At a quick glance it looks like performing the sort mentioned above (first kw_only=False, init=True, then kw_only=True, init=True, last init=False) around https://github.com/python-attrs/attrs/blob/master/src/attr/_make.py#L378 should do the trick for both this and #448 . @hynek , if this seems like the right strategy to you, I can make a PR with it.

[gabbard]

@hynek : bumping the question above, since I just ran into this problem again today. If you can confirm you are happy with the proposed approach to a fix, I am happy to submit a PR with it.

[hynek]

I’m sorry you hit me in the middle of my vacation and I'm having a hard time to catch up with the more complex issues.

And yes I agree that:

1. init=False attributes should never cause a Non keyword-only attributes… error
2. it’s kinda weird that we don’t allow kw_only attributes anywhere. someone should research why we chose that route. they are a lot more useful if you can sprinkle them anywhere.

So feel free to submit a PR if you manage to achieve that without breaking backward compatibility. Even in subclassing scenarios. I tend to think that it might be better to make the condition slightly more complicated instead of re-ordering them because that might have other unforeseen side effects.

[gabbard]

@hynek : After more thought I think you are correct that using a more complex condition is better. I will poke at this as I have free time. :-)

[wsanchez]

@hynek: I don't think you commented on the BrokenInitFirst case. As with #448, it would introduce a (simpler) difference between the order of attributes at the order of arguments to __init__. But the arguments for allowing them anywhere would be similar, no?

[gabbard]

I went through the initial and final PRs as well as the three related issues (#38 , #106 , #335 ). It looks not allowing kw_only attributes anywhere was not a decision that was made very explicitly. The closest thing I see to a discussion of this is @hynek 's comment here that "I think it’s fair to expect that a non-kw_only attribute must not come after a kw_only attribute" and @Tinche 's comment:

Currently the definition order of attributes is the same as the order of the attributes in the generated __init__. This is strictly by @hynek's executive order and I personally agree with the rationale. If it wasn't intentional we would just sort the attributes and you wouldn't need to put arguments with defaults last. The order of definition is used in other places too: the __repr__, comparison methods, attr.fields.

and subsequent discussion.

I think the way forward is:

• to maintain the current order of arguments (that is, declaration order) for purposes of repr, field initialization, etc.
• to alter the condition to not thrown an exception if an init=False attribute follows a kw_only attribute
• to float kw_only attributes to the end of the argument list when generating __init__

[wsanchez]

I think that makes sense. It doesn't address BrokenInitFirst, though, right?

[gabbard]

@wsanchez : I am a little uncertain whether BrokenInitFirst needs to be fixed, since the general rule seems to be that an init=False attribute with a default needs to come later in the attribute ordering than any fields it depends on (me calling it Broken was misleading - I was just trying to illustrate that no legal attribute ordering for this case existed; as long as we fix one of the two options then the particular problem I had in mind is solved).

[wsanchez]

Whoops sorry, I glossed over the dependency there, but mostly wanted to be clear that this case wasn’t addressed. I agree it shouldn’t be.

[gabbard]

@hynek : PR ready at #459