Crash 'NoneType' object has no attribute 'keywords'

[tybug]

Bug description

When parsing the following file (as seen on https://github.com/circleguard/ossapi/blob/master/ossapi/ossapiv2.py):

from typing import Union, TypeVar, Optional, List, _GenericAlias
import logging
import webbrowser
import socket
import pickle
from pathlib import Path
from datetime import datetime
from enum import Enum
from urllib.parse import unquote
import inspect
import json
from keyword import iskeyword
import hashlib
import functools

from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import (BackendApplicationClient, TokenExpiredError,
    AccessDeniedError)
from oauthlib.oauth2.rfc6749.errors import InsufficientScopeError
import osrparse
from typing_utils import issubtype, get_type_hints, get_origin, get_args

from ossapi.models import (Beatmap, BeatmapCompact, BeatmapUserScore,
    ForumTopicAndPosts, Search, CommentBundle, Cursor, Score,
    BeatmapsetSearchResult, ModdingHistoryEventsBundle, User, Rankings,
    BeatmapScores, KudosuHistory, Beatmapset, BeatmapPlaycount, Spotlight,
    Spotlights, WikiPage, _Event, Event, BeatmapsetDiscussionPosts, Build,
    ChangelogListing, MultiplayerScores, MultiplayerScoresCursor,
    BeatmapsetDiscussionVotes, CreatePMResponse, BeatmapsetDiscussions,
    UserCompact)
from ossapi.enums import (GameMode, ScoreType, RankingFilter, RankingType,
    UserBeatmapType, BeatmapDiscussionPostSort, UserLookupKey,
    BeatmapsetEventType, CommentableType, CommentSort, ForumTopicSort,
    SearchMode, MultiplayerScoresSort, BeatmapsetDiscussionVote,
    BeatmapsetDiscussionVoteSort, BeatmapsetStatus, MessageType)
from ossapi.utils import (is_compatible_type, is_primitive_type, is_optional,
    is_base_model_type, is_model_type, is_high_model_type, Expandable)
from ossapi.mod import Mod
from ossapi.replay import Replay

# our ``request`` function below relies on the ordering of these types. The
# base type must come first, with any auxiliary types that the base type accepts
# coming after.
# These types are intended to provide better type hinting for consumers. We
# want to support the ability to pass ``"osu"`` instead of ``GameMode.STD``,
# for instance. We automatically convert any value to its base class if the
# relevant parameter has a type hint of the form below (see ``request`` for
# details).
GameModeT = Union[GameMode, str]
ScoreTypeT = Union[ScoreType, str]
ModT = Union[Mod, str, int, list]
RankingFilterT = Union[RankingFilter, str]
RankingTypeT = Union[RankingType, str]
UserBeatmapTypeT = Union[UserBeatmapType, str]
BeatmapDiscussionPostSortT = Union[BeatmapDiscussionPostSort, str]
UserLookupKeyT = Union[UserLookupKey, str]
BeatmapsetEventTypeT = Union[BeatmapsetEventType, str]
CommentableTypeT = Union[CommentableType, str]
CommentSortT = Union[CommentSort, str]
ForumTopicSortT = Union[ForumTopicSort, str]
SearchModeT = Union[SearchMode, str]
MultiplayerScoresSortT = Union[MultiplayerScoresSort, str]
BeatmapsetDiscussionVoteT = Union[BeatmapsetDiscussionVote, int]
BeatmapsetDiscussionVoteSortT = Union[BeatmapsetDiscussionVoteSort, str]
MessageTypeT = Union[MessageType, str]
BeatmapsetStatusT = Union[BeatmapsetStatus, str]

BeatmapIdT = Union[int, BeatmapCompact]
UserIdT = Union[int, UserCompact]

def request(scope, *, requires_login=False):
    """
    Handles various validation and preparation tasks for any endpoint request
    method.

    This method does the following things:
    * makes sure the client has the requuired scope to access the endpoint in
      question
    * makes sure the client has the right grant to access the endpoint in
      question (the client credentials grant cannot access endpoints which
      require the user to be "logged in", such as downloading a replay)
    * converts parameters to an instance of a base model if the parameter is
      annotated as being a base model. This means, for instance, that a function
      with an argument annotated as ``ModT`` (``Union[Mod, str, int, list]``)
      will have the value of that parameter automatically converted to a
      ``Mod``, even if the user passes a `str`.
    * converts arguments of type ``BeatmapIdT`` or ``UserIdT`` into a beatmap or
      user id, if the passed argument was a ``BeatmapCompact`` or
      ``UserCompact`` respectively.

    Parameters
    ----------
    scope: Scope
        The scope required for this endpoint. If ``None``, no scope is required
        and any authenticated cliient can access it.
    requires_login: bool
        Whether this endpoint requires a "logged-in" client to retrieve it.
        Currently, only authtorization code grants can access these endpoints.
    """
    def decorator(function):
        instantiate = {}
        for name, type_ in function.__annotations__.items():
            origin = get_origin(type_)
            args = get_args(type_)
            if origin is Union and is_base_model_type(args[0]):
                instantiate[name] = args[0]

        arg_names = list(inspect.signature(function).parameters)

        @functools.wraps(function)
        def wrapper(*args, **kwargs):
            self = args[0]
            if scope is not None and scope not in self.scopes:
                raise InsufficientScopeError(f"A scope of {scope} is required "
                    "for this endpoint. Your client's current scopes are "
                    f"{self.scopes}")

            if requires_login and self.grant is Grant.CLIENT_CREDENTIALS:
                raise AccessDeniedError("To access this endpoint you must be "
                    "authorized using the authorization code grant. You are "
                    "currently authorized with the client credentials grant")

            # we may need to edit this later so convert from tuple
            args = list(args)

            def is_id_type(arg_name, arg):
                annotations = function.__annotations__
                if arg_name not in annotations:
                    return False
                arg_type = annotations[arg_name]
                if (not issubtype(BeatmapIdT, arg_type) and not
                    issubtype(UserIdT, arg_type)):
                    return False
                return isinstance(arg, (BeatmapCompact, UserCompact))

            # args and kwargs are handled separately, but in a similar fashion.
            # The difference is that for ``args`` we need to know the name of
            # the argument so we can look up its type hint and see if it's a
            # parameter we need to convert.

            for i, (arg_name, arg) in enumerate(zip(arg_names, args)):
                if arg_name in instantiate:
                    type_ = instantiate[arg_name]
                    args[i] = type_(arg)
                if is_id_type(arg_name, arg):
                    args[i] = arg.id

            for arg_name, arg in kwargs.items():
                if arg_name in instantiate:
                    type_ = instantiate[arg_name]
                    kwargs[arg_name] = type_(arg)
                if is_id_type(arg_name, arg):
                    kwargs[arg_name] = arg.id

            return function(*args, **kwargs)
        return wrapper
    return decorator


class Grant(Enum):
    CLIENT_CREDENTIALS = "client"
    AUTHORIZATION_CODE = "authorization"

class Scope(Enum):
    CHAT_WRITE = "chat.write"
    DELEGATE = "delegate"
    FORUM_WRITE = "forum.write"
    FRIENDS_READ = "friends.read"
    IDENTIFY = "identify"
    PUBLIC = "public"


class OssapiV2:
    """
    A wrapper around osu api v2.

    Parameters
    ----------
    client_id: int
        The id of the client to authenticate with.
    client_secret: str
        The secret of the client to authenticate with.
    redirect_uri: str
        The redirect uri for the client. Must be passed if using the
        authorization code grant. This must exactly match the redirect uri on
        the client's settings page. Additionally, in order for ossapi to receive
        authentication from this redirect uri, it must be a port on localhost.
        So "http://localhost:3914/", "http://localhost:727/", etc are all valid
        redirect uris. You can change your client's redirect uri from its
        settings page.
    scopes: List[str]
        What scopes to request when authenticating.
    grant: Grant or str
        Which oauth grant (aka flow) to use when authenticating with the api.
        Currently the api offers the client credentials (pass "client" for this
        parameter) and authorization code (pass "authorization" for this
        parameter) grants.
        |br|
        The authorization code grant requires user interaction to authenticate
        the first time, but grants full access to the api. In contrast, the
        client credentials grant does not require user interaction to
        authenticate, but only grants guest user access to the api. This means
        you will not be able to do things like download replays on the client
        credentials grant.
        |br|
        If not passed, the grant will be automatically inferred as follows: if
        ``redirect_uri`` is passed, use the authorization code grant. If
        ``redirect_uri`` is not passed, use the client credentials grant.
    strict: bool
        Whether to run in "strict" mode. In strict mode, ossapi will raise an
        exception if the api returns an attribute in a response which we didn't
        expect to be there. This is useful for developers which want to catch
        new attributes as they get added. More checks may be added in the future
        for things which developers may want to be aware of, but normal users do
        not want to have an exception raised for.
        |br|
        If you are not a developer, you are very unlikely to want to use this
        parameter.
    token_directory: str
        If passed, the given directory will be used to store and retrieve token
        files instead of locally wherever ossapi is installed. Useful if you
        want more control over token files.
    token_key: str
        If passed, the given key will be used to name the token file instead of
        an automatically generated one. Note that if you pass this, you are
        taking responsibility for making sure it is unique / unused, and also
        for remembering the key you passed if you wish to eg remove the token in
        the future, which requires the key.
    """
    TOKEN_URL = "https://osu.ppy.sh/oauth/token"
    AUTH_CODE_URL = "https://osu.ppy.sh/oauth/authorize"
    BASE_URL = "https://osu.ppy.sh/api/v2"

    def __init__(self,
        client_id: int,
        client_secret: str,
        redirect_uri: Optional[str] = None,
        scopes: List[Union[str, Scope]] = [Scope.PUBLIC],
        *,
        grant: Optional[Union[Grant, str]] = None,
        strict: bool = False,
        token_directory: Optional[str] = None,
        token_key: Optional[str] = None,
    ):
        if not grant:
            grant = (Grant.AUTHORIZATION_CODE if redirect_uri else
                Grant.CLIENT_CREDENTIALS)
        grant = Grant(grant)

        self.grant = grant
        self.client_id = client_id
        self.client_secret = client_secret
        self.redirect_uri = redirect_uri
        self.scopes = [Scope(scope) for scope in scopes]
        self.strict = strict

        self.log = logging.getLogger(__name__)
        self.token_key = token_key or self.gen_token_key(self.grant,
            self.client_id, self.client_secret, self.scopes)
        self.token_directory = (
            Path(token_directory) if token_directory else Path(__file__).parent
        )
        self.token_file = self.token_directory / f"{self.token_key}.pickle"

        if self.grant is Grant.CLIENT_CREDENTIALS:
            if self.scopes != [Scope.PUBLIC]:
                raise ValueError(f"`scopes` must be ['public'] if the "
                    f"client credentials grant is used. Got {self.scopes}")

        if self.grant is Grant.AUTHORIZATION_CODE and not self.redirect_uri:
            raise ValueError("`redirect_uri` must be passed if the "
                "authorization code grant is used.")

        self.session = self.authenticate()

    @staticmethod
    def gen_token_key(grant, client_id, client_secret, scopes):
        """
        The unique key / hash for the given set of parameters. This is intended
        to provide a way to allow multiple OssapiV2's to live at the same time,
        by eg saving their tokens to different files based on their key.

        This function is also deterministic, to eg allow tokens to be reused if
        OssapiV2 is instantiated twice with the same parameters. This avoids the
        need to reauthenticate unless absolutely necessary.
        """
        grant = Grant(grant)
        scopes = [Scope(scope) for scope in scopes]
        m = hashlib.sha256()
        m.update(grant.value.encode("utf-8"))
        m.update(str(client_id).encode("utf-8"))
        m.update(client_secret.encode("utf-8"))
        for scope in scopes:
            m.update(scope.value.encode("utf-8"))
        return m.hexdigest()

    @staticmethod
    def remove_token(key, token_directory=None):
        """
        Removes the token file associated with the given key. If
        ``token_directory`` is passed, looks there for the token file instead of
        locally in ossapi's install site.

        To determine the key associated with a given grant, client_id,
        client_secret, and set of scopes, use ``gen_token_key``.
        """
        token_directory = (
            Path(token_directory) if token_directory else Path(__file__).parent
        )
        token_file = token_directory / f"{key}.pickle"
        token_file.unlink()

    def authenticate(self):
        """
        Returns a valid OAuth2Session, either from a saved token file associated
        with this OssapiV2's parameters, or from a fresh authentication if no
        such file exists.
        """
        if self.token_file.exists():
            with open(self.token_file, "rb") as f:
                token = pickle.load(f)

            if self.grant is Grant.CLIENT_CREDENTIALS:
                return OAuth2Session(self.client_id, token=token)

            if self.grant is Grant.AUTHORIZATION_CODE:
                auto_refresh_kwargs = {
                    "client_id": self.client_id,
                    "client_secret": self.client_secret
                }
                return OAuth2Session(self.client_id, token=token,
                    redirect_uri=self.redirect_uri,
                    auto_refresh_url=self.TOKEN_URL,
                    auto_refresh_kwargs=auto_refresh_kwargs,
                    token_updater=self._save_token,
                    scope=[scope.value for scope in self.scopes])

        if self.grant is Grant.CLIENT_CREDENTIALS:
            return self._new_client_grant(self.client_id, self.client_secret)

        return self._new_authorization_grant(self.client_id, self.client_secret,
            self.redirect_uri, self.scopes)

    def _new_client_grant(self, client_id, client_secret):
        """
        Authenticates with the api from scratch on the client grant.
        """
        self.log.info("initializing client credentials grant")
        client = BackendApplicationClient(client_id=client_id, scope=["public"])
        session = OAuth2Session(client=client)
        token = session.fetch_token(token_url=self.TOKEN_URL,
            client_id=client_id, client_secret=client_secret)

        self._save_token(token)
        return session

    def _new_authorization_grant(self, client_id, client_secret, redirect_uri,
        scopes):
        """
        Authenticates with the api from scratch on the authorization code grant.
        """
        self.log.info("initializing authorization code")

        auto_refresh_kwargs = {
            "client_id": client_id,
            "client_secret": client_secret
        }
        session = OAuth2Session(client_id, redirect_uri=redirect_uri,
            auto_refresh_url=self.TOKEN_URL,
            auto_refresh_kwargs=auto_refresh_kwargs,
            token_updater=self._save_token,
            scope=[scope.value for scope in scopes])

        authorization_url, _state = (
            session.authorization_url(self.AUTH_CODE_URL)
        )
        webbrowser.open(authorization_url)

        # open up a temporary socket so we can receive the GET request to the
        # callback url
        port = int(redirect_uri.rsplit(":", 1)[1].split("/")[0])
        serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        serversocket.bind(("localhost", port))
        serversocket.listen(1)
        connection, _ = serversocket.accept()
        # arbitrary "large enough" byte receive size
        data = str(connection.recv(8192))
        connection.send(b"HTTP/1.0 200 OK\n")
        connection.send(b"Content-Type: text/html\n")
        connection.send(b"\n")
        connection.send(b"""<html><body>
            <h2>Ossapi has received your authentication.</h2> You
            may now close this tab safely.
            </body></html>
        """)
        connection.close()
        serversocket.close()

        code = data.split("code=")[1].split("&state=")[0]
        token = session.fetch_token(self.TOKEN_URL, client_id=client_id,
            client_secret=client_secret, code=code)
        self._save_token(token)

        return session

    def _save_token(self, token):
        """
        Saves the token to this OssapiV2's associated token file.
        """
        self.log.info(f"saving token to {self.token_file}")
        with open(self.token_file, "wb+") as f:
            pickle.dump(token, f)

    def _request(self, type_, method, url, params={}, data={}):
        params = self._format_params(params)
        try:
            r = self.session.request(method, f"{self.BASE_URL}{url}",
                params=params, data=data)
        except TokenExpiredError:
            # provide "auto refreshing" for client credentials grant. The client
            # grant doesn't actually provide a refresh token, so we can't hook
            # onto OAuth2Session's auto_refresh functionality like we do for the
            # authorization code grant. But we can do something effectively
            # equivalent: whenever we make a request with an expired client
            # grant token, just request a new one.
            if self.grant is not Grant.CLIENT_CREDENTIALS:
                raise
            self.session = self._new_client_grant(self.client_id,
                self.client_secret)
            # redo the request now that we have a valid token
            r = self.session.request(method, f"{self.BASE_URL}{url}",
                params=params, data=data)

        self.log.info(f"made {method} request to {r.request.url}")
        json_ = r.json()
        self.log.debug(f"received json: \n{json.dumps(json_, indent=4)}")
        # TODO this should just be ``if "error" in json``, but for some reason
        # ``self.search_beatmaps`` always returns an error in the response...
        # open an issue on osu-web?
        if len(json_) == 1 and "error" in json_:
            raise ValueError(f"api returned an error of `{json_['error']}` for "
                f"a request to {unquote(url)}")
        return self._instantiate_type(type_, json_)

    def _get(self, type_, url, params={}):
        return self._request(type_, "GET", url, params=params)

    def _post(self, type_, url, data={}):
        return self._request(type_, "POST", url, data=data)

    def _format_params(self, params):
        for key, value in params.copy().items():
            if isinstance(value, list):
                # we need to pass multiple values for this key, so make its
                # value a list https://stackoverflow.com/a/62042144
                params[f"{key}[]"] = []
                for v in value:
                    params[f"{key}[]"].append(self._format_value(v))
                del params[key]
            elif isinstance(value, Cursor):
                new_params = self._format_params(value.__dict__)
                for k, v in new_params.items():
                    params[f"cursor[{k}]"] = v
                del params[key]
            elif isinstance(value, Mod):
                params[f"{key}[]"] = value.decompose()
                del params[key]
            else:
                params[key] = self._format_value(value)
        return params

    def _format_value(self, value):
        if isinstance(value, datetime):
            return 1000 * int(value.timestamp())
        if isinstance(value, Enum):
            return value.value
        return value

    def _resolve_annotations(self, obj):
        """
        This is where the magic happens. Since python lacks a good
        deserialization library, I've opted to use type annotations and type
        annotations only to convert json to objects. A breakdown follows.

        Every endpoint defines a base object, let's say it's a ``Score``. We
        first instantiate this object with the json we received. This is easy to
        do because (almost) all of our objects are dataclasses, which means we
        can pass the json as ``Score(**json)`` and since the names of our fields
        coincide with the names of the api json keys, everything works.

        This populates all of the surface level members, but nested attributes
        which are annotated as another dataclass object will still be dicts. So
        we traverse down the tree of our base object's attributes (depth-first,
        though I'm pretty sure BFS would work just as well), looking for any
        attribute with a type annotation that we need to deal with. For
        instance, ``Score`` has a ``beatmap`` attribute, which is annotated as
        ``Optional[Beatmap]``. We ignore the optional annotation (since we're
        looking at this attribute, we must have received data for it, so it's
        nonnull) and then instantiate the ``beatmap`` attribute the same way
        we instantiated the ``Score`` - with ``Beatmap(**json)``. Of course, the
        variables will look different in the method (``type_(**value)``).

        Finally, when traversing the attribute tree, we also look for attributes
        which aren't dataclasses, but we still need to convert. For instance,
        any attribute with an annotation of ``datetime`` or ``Mod`` we convert
        to a ``datetime`` and ``Mod`` object respectively.

        This code is arguably trying to be too smart for its own good, but I
        think it's very elegant from the perspective of "just add a dataclass
        that mirrors the api's objects and everything works". Will hopefully
        make changing our dataclasses to account for breaking api changes in
        the future trivial as well.

        And if I'm being honest, it was an excuse to learn the internals of
        python's typing system.
        """
        # we want to get the annotations of inherited members as well, which is
        # why we pass ``type(obj)`` instead of just ``obj``, which would only
        # return annotations for attributes defined in ``obj`` and not its
        # inherited attributes.
        annotations = get_type_hints(type(obj))
        override_annotations = obj.override_types()
        annotations = {**annotations, **override_annotations}
        self.log.debug(f"resolving annotations for type {type(obj)}")
        for attr, value in obj.__dict__.items():
            # we use this attribute later if we encounter an attribute which
            # has been instantiated generically, but we don't need to do
            # anything with it now.
            if attr == "__orig_class__":
                continue
            type_ = annotations[attr]
            # when we instantiate types, we explicitly fill in optional
            # attributes with ``None``. We want to skip these, but only if the
            # attribute is actually annotated as optional, otherwise we would be
            # skipping fields that are null which aren't supposed to be, and
            # prevent that error from being caught.
            if value is None and is_optional(type_):
                continue
            self.log.debug(f"resolving attribute {attr}")

            value = self._instantiate_type(type_, value, obj, attr_name=attr)
            if not value:
                continue
            setattr(obj, attr, value)
        self.log.debug(f"resolved annotations for type {type(obj)}")
        return obj

    def _instantiate_type(self, type_, value, obj=None, attr_name=None):
        # ``attr_name`` is purely for debugging, it's the name of the attribute
        # being instantiated
        origin = get_origin(type_)
        args = get_args(type_)

        # if this type is an optional, "unwrap" it to get the true type.
        # We don't care about the optional annotation in this context
        # because if we got here that means we were passed a value for this
        # attribute, so we know it's defined and not optional.
        if is_optional(type_):
            # leaving these assertions in to help me catch errors in my
            # reasoning until I better understand python's typing.
            assert len(args) == 2
            type_ = args[0]
            origin = get_origin(type_)
            args = get_args(type_)

        # validate that the values we're receiving are the types we expect them
        # to be
        if is_primitive_type(type_):
            if not is_compatible_type(value, type_):
                raise TypeError(f"expected type {type_} for value {value}, got "
                    f"type {type(value)}"
                    f" (for attribute: {attr_name})" if attr_name else "")

        if is_base_model_type(type_):
            self.log.debug(f"instantiating base type {type_}")
            return type_(value)

        if origin is list and (is_model_type(args[0]) or
            isinstance(args[0], TypeVar)):
            assert len(args) == 1
            # check if the list has been instantiated generically; if so,
            # use the concrete type backing the generic type.
            if isinstance(args[0], TypeVar):
                # ``__orig_class__`` is how we can get the concrete type of
                # a generic. See https://stackoverflow.com/a/60984681 and
                # https://www.python.org/dev/peps/pep-0560/#mro-entries.
                type_ = get_args(obj.__orig_class__)[0]
            # otherwise, it's been instantiated with a concrete model type,
            # so use that type.
            else:
                type_ = args[0]
            new_value = []
            for entry in value:
                if is_base_model_type(type_):
                    entry = type_(entry)
                else:
                    entry = self._instantiate(type_, entry)
                # if the list entry is a high (non-base) model type, we need to
                # resolve it instead of just sticking it into the list, since
                # its children might still be dicts and not model instances.
                # We don't do this for base types because that type is the one
                # responsible for resolving its own annotations or doing
                # whatever else it needs to do, not us.
                if is_high_model_type(type_):
                    entry = self._resolve_annotations(entry)
                new_value.append(entry)
            return new_value

        # either we ourself are a model type (eg ``Search``), or we are
        # a special indexed type (eg ``type_ == SearchResult[UserCompact]``,
        # ``origin == UserCompact``). In either case we want to instantiate
        # ``type_``.
        if not is_model_type(type_) and not is_model_type(origin):
            return None
        value = self._instantiate(type_, value)
        # we need to resolve the annotations of any nested model types before we
        # set the attribute. This recursion is well-defined because the base
        # case is when ``value`` has no model types, which will always happen
        # eventually.
        return self._resolve_annotations(value)

    def _instantiate(self, type_, kwargs):
        self.log.debug(f"instantiating type {type_}")
        # we need a special case to handle when ``type_`` is a
        # ``_GenericAlias``. I don't fully understand why this exception is
        # necessary, and it's likely the result of some error on my part in our
        # type handling code. Nevertheless, until I dig more deeply into it,
        # we need to extract the type to use for the init signature and the type
        # hints from a ``_GenericAlias`` if we see one, as standard methods
        # won't work.
        override_type = type_.override_class(kwargs)
        type_ = override_type or type_
        signature_type = type_
        try:
            type_hints = get_type_hints(type_)
        except TypeError:
            assert type(type_) is _GenericAlias # pylint: disable=unidiomatic-typecheck

            signature_type = get_origin(type_)
            type_hints = get_type_hints(signature_type)

        # make a copy so we can modify while iterating
        for key in list(kwargs):
            value = kwargs.pop(key)
            # replace any key names that are invalid python syntax with a valid
            # one. Note: this is relying on our models replacing an at sign with
            # an underscore when declaring attributes.
            key = key.replace("@", "_")
            # python classes can't have keywords as attribute names, so if the
            # api has given us a keyword attribute, append an underscore. As
            # above, this is relying on our models to match this by appending
            # an underscore to keyword attribute names.
            if iskeyword(key):
                key += "_"
            kwargs[key] = value

        # if we've annotated a class with ``Optional[X]``, and the api response
        # didn't return a value for that attribute, pass ``None`` for that
        # attribute.
        # This is so that we don't have to define a default value of ``None``
        # for each optional attribute of our models, since the default will
        # always be ``None``.
        for attribute, annotation in type_hints.items():
            if is_optional(annotation):
                if attribute not in kwargs:
                    kwargs[attribute] = None

        # The osu api often adds new fields to various models, and these are not
        # considered breaking changes. To make this a non-breaking change on our
        # end as well, we ignore any unexpected parameters, unless
        # ``self.strict`` is ``True``. This means that consumers using old
        # ossapi versions (which aren't up to date with the latest parameters
        # list) will have new fields silently ignored instead of erroring.
        # This also means that consumers won't be able to benefit from new
        # fields unless they upgrade, but this is a conscious decision on our
        # part to keep things entirely statically typed. Otherwise we would be
        # going the route of PRAW, which returns dynamic results for all api
        # queries. I think a statically typed solution is better for the osu!
        # api, which promises at least some level of stability in its api.
        parameters = list(inspect.signature(signature_type.__init__).parameters)
        kwargs_ = {}

        # Some special classes take arbitrary parameters, so we can't evaluate
        # whether a parameter is unexpected or not until we instantiate it.
        if isinstance(type_, type) and issubclass(type_, Cursor):
            kwargs_ = kwargs
        else:
            for k, v in kwargs.items():
                if k in parameters:
                    kwargs_[k] = v
                else:
                    if self.strict:
                        raise TypeError(f"unexpected parameter `{k}` for type "
                            f"{type_}")
                    self.log.info(f"ignoring unexpected parameter `{k}` from "
                        f"api response for type {type_}")

        # "expandable" models need an api instance to be injected into them on
        # instantiation. There isn't really a clean way to do this
        # unfortunately. If the parameter name on the ``Expandable`` class ever
        # changes from ``_api``, this will also need to be changed. And it's not
        # very transparent as to where this parameter is coming from, from
        # Expandable's perspective. But such is the way of life when abusing
        # python type hints.
        if isinstance(type_, type) and issubclass(type_, Expandable):
            kwargs_["_api"] = self

        try:
            val = type_(**kwargs_)
        except TypeError as e:
            raise TypeError(f"type error while instantiating class {type_}: "
                f"{str(e)}") from e

        return val


    # =========
    # Endpoints
    # =========


    # /beatmaps
    # ---------

    @request(Scope.PUBLIC)
    def beatmap_user_score(self,
        beatmap_id: BeatmapIdT,
        user_id: UserIdT,
        mode: Optional[GameModeT] = None,
        mods: Optional[ModT] = None
    ) -> BeatmapUserScore:
        """
        https://osu.ppy.sh/docs/index.html#get-a-user-beatmap-score
        """
        params = {"mode": mode, "mods": mods}
        return self._get(BeatmapUserScore,
            f"/beatmaps/{beatmap_id}/scores/users/{user_id}", params)

    @request(Scope.PUBLIC)
    def beatmap_scores(self,
        beatmap_id: BeatmapIdT,
        mode: Optional[GameModeT] = None,
        mods: Optional[ModT] = None,
        type_: Optional[RankingTypeT] = None
    ) -> BeatmapScores:
        """
        https://osu.ppy.sh/docs/index.html#get-beatmap-scores
        """
        params = {"mode": mode, "mods": mods, "type": type_}
        return self._get(BeatmapScores, f"/beatmaps/{beatmap_id}/scores",
            params)

    @request(Scope.PUBLIC)
    def beatmap(self,
        beatmap_id: Optional[BeatmapIdT] = None,
        checksum: Optional[str] = None,
        filename: Optional[str] = None,
    ) -> Beatmap:
        """
        combines https://osu.ppy.sh/docs/index.html#get-beatmap and
        https://osu.ppy.sh/docs/index.html#lookup-beatmap
        """
        if not (beatmap_id or checksum or filename):
            raise ValueError("at least one of beatmap_id, checksum, or "
                "filename must be passed")
        params = {"checksum": checksum, "filename": filename, "id": beatmap_id}
        return self._get(Beatmap, "/beatmaps/lookup", params)


    # /beatmapsets
    # ------------

    @request(Scope.PUBLIC)
    def beatmapset_discussion_posts(self,
        beatmapset_discussion_id: Optional[int] = None,
        limit: Optional[int] = None,
        page: Optional[int] = None,
        sort: Optional[BeatmapDiscussionPostSortT] = None,
        user_id: Optional[UserIdT] = None,
        with_deleted: Optional[bool] = None
    ) -> BeatmapsetDiscussionPosts:
        """
        https://osu.ppy.sh/docs/index.html#get-beatmapset-discussion-posts
        """
        params = {"beatmapset_discussion_id": beatmapset_discussion_id,
            "limit": limit, "page": page, "sort": sort, "user": user_id,
            "with_deleted": with_deleted}
        return self._get(BeatmapsetDiscussionPosts,
            "/beatmapsets/discussions/posts", params)

    @request(Scope.PUBLIC)
    def beatmapset_discussion_votes(self,
        beatmapset_discussion_id: Optional[int] = None,
        limit: Optional[int] = None,
        page: Optional[int] = None,
        receiver_id: Optional[int] = None,
        vote: Optional[BeatmapsetDiscussionVoteT] = None,
        sort: Optional[BeatmapsetDiscussionVoteSortT] = None,
        user_id: Optional[UserIdT] = None,
        with_deleted: Optional[bool] = None
    ) -> BeatmapsetDiscussionVotes:
        """
        https://osu.ppy.sh/docs/index.html#get-beatmapset-discussion-votes
        """
        params = {"beatmapset_discussion_id": beatmapset_discussion_id,
            "limit": limit, "page": page, "receiver": receiver_id,
            "score": vote, "sort": sort, "user": user_id,
            "with_deleted": with_deleted}
        return self._get(BeatmapsetDiscussionVotes,
            "/beatmapsets/discussions/votes", params)

    @request(Scope.PUBLIC)
    def beatmapset_discussions(self,
        beatmapset_id: Optional[int] = None,
        beatmap_id: Optional[BeatmapIdT] = None,
        beatmapset_status: Optional[BeatmapsetStatusT] = None,
        limit: Optional[int] = None,
        message_types: Optional[List[MessageTypeT]] = None,
        only_unresolved: Optional[bool] = None,
        page: Optional[int] = None,
        sort: Optional[BeatmapDiscussionPostSortT] = None,
        user_id: Optional[UserIdT] = None,
        with_deleted: Optional[bool] = None,
    ) -> BeatmapsetDiscussions:
        """
        https://osu.ppy.sh/docs/index.html#get-beatmapset-discussions
        """
        params = {"beatmapset_id": beatmapset_id, "beatmap_id": beatmap_id,
            "beatmapset_status": beatmapset_status, "limit": limit,
            "message_types": message_types, "only_unresolved": only_unresolved,
            "page": page, "sort": sort, "user": user_id,
            "with_deleted": with_deleted}
        return self._get(BeatmapsetDiscussions,
            "/beatmapsets/discussions", params)

    # /changelog
    # ----------

    @request(scope=None)
    def changelog_build(self,
        stream: str,
        build: str
    ) -> Build:
        """
        https://osu.ppy.sh/docs/index.html#get-changelog-build
        """
        return self._get(Build, f"/changelog/{stream}/{build}")

    @request(scope=None)
    def changelog_listing(self,
        from_: Optional[str] = None,
        to: Optional[str] = None,
        max_id: Optional[int] = None,
        stream: Optional[str] = None
    ) -> ChangelogListing:
        """
        https://osu.ppy.sh/docs/index.html#get-changelog-listing
        """
        params = {"from": from_, "to": to, "max_id": max_id, "stream": stream}
        return self._get(ChangelogListing, "/changelog", params)

    @request(scope=None)
    def changelog_lookup(self,
        changelog: str,
        key: Optional[str] = None
    ) -> Build:
        """
        https://osu.ppy.sh/docs/index.html#lookup-changelog-build
        """
        params = {"key": key}
        return self._get(Build, f"/changelog/{changelog}", params)


    # /chat
    # -----

    @request(Scope.CHAT_WRITE)
    def create_pm(self,
        user_id: UserIdT,
        message: str,
        is_action: Optional[bool] = False
    ) -> CreatePMResponse:
        """
        https://osu.ppy.sh/docs/index.html#create-new-pm
        """
        data = {"target_id": user_id, "message": message,
            "is_action": is_action}
        return self._post(CreatePMResponse, "/chat/new", data=data)


    # /comments
    # ---------

    @request(Scope.PUBLIC)
    def comments(self,
        commentable_type: Optional[CommentableTypeT] = None,
        commentable_id: Optional[int] = None,
        cursor: Optional[Cursor] = None,
        parent_id: Optional[int] = None,
        sort: Optional[CommentSortT] = None
    ) -> CommentBundle:
        """
        A list of comments and their replies, up to 2 levels deep.

        https://osu.ppy.sh/docs/index.html#get-comments

        Notes
        -----
        ``pinned_comments`` is only included when ``commentable_type`` and
        ``commentable_id`` are specified.
        """
        params = {"commentable_type": commentable_type,
            "commentable_id": commentable_id, "cursor": cursor,
            "parent_id": parent_id, "sort": sort}
        return self._get(CommentBundle, "/comments", params)

    @request(scope=None)
    def comment(self,
        comment_id: int
    ) -> CommentBundle:
        """
        https://osu.ppy.sh/docs/index.html#get-a-comment
        """
        return self._get(CommentBundle, f"/comments/{comment_id}")


    # /forums
    # -------

    @request(Scope.PUBLIC)
    def forum_topic(self,
        topic_id: int,
        cursor: Optional[Cursor] = None,
        sort: Optional[ForumTopicSortT] = None,
        limit: Optional[int] = None,
        start: Optional[int] = None,
        end: Optional[int] = None
    ) -> ForumTopicAndPosts:
        """
        A topic and its posts.

        https://osu.ppy.sh/docs/index.html#get-topic-and-posts
        """
        params = {"cursor": cursor, "sort": sort, "limit": limit,
            "start": start, "end": end}
        return self._get(ForumTopicAndPosts, f"/forums/topics/{topic_id}",
            params)


    # / ("home")
    # ----------

    @request(Scope.PUBLIC)
    def search(self,
        mode: Optional[SearchModeT] = None,
        query: Optional[str] = None,
        page: Optional[int] = None
    ) -> Search:
        """
        https://osu.ppy.sh/docs/index.html#search
        """
        params = {"mode": mode, "query": query, "page": page}
        return self._get(Search, "/search", params)


    # /me
    # ---

    @request(Scope.IDENTIFY)
    def get_me(self,
        mode: Optional[GameModeT] = None
    ):
        """
        https://osu.ppy.sh/docs/index.html#get-own-data
        """
        return self._get(User, f"/me/{mode.value if mode else ''}")


    # /rankings
    # ---------

    @request(Scope.PUBLIC)
    def ranking(self,
        mode: GameModeT,
        type_: RankingTypeT,
        country: Optional[str] = None,
        cursor: Optional[Cursor] = None,
        filter_: RankingFilterT = RankingFilter.ALL,
        spotlight: Optional[int] = None,
        variant: Optional[str] = None
    ) -> Rankings:
        """
        https://osu.ppy.sh/docs/index.html#get-ranking
        """
        params = {"country": country, "cursor": cursor, "filter": filter_,
            "spotlight": spotlight, "variant": variant}
        return self._get(Rankings, f"/rankings/{mode.value}/{type_.value}",
            params=params)

    @request(Scope.PUBLIC)
    def spotlights(self) -> List[Spotlight]:
        """
        https://osu.ppy.sh/docs/index.html#get-spotlights
        """
        spotlights = self._get(Spotlights, "/spotlights")
        return spotlights.spotlights


    # /rooms
    # ------

    # TODO add test for this once I figure out values for room_id and
    # playlist_id that actually produce a response lol
    @request(Scope.PUBLIC)
    def multiplayer_scores(self,
        room_id: int,
        playlist_id: int,
        limit: Optional[int] = None,
        sort: Optional[MultiplayerScoresSortT] = None,
        cursor: Optional[MultiplayerScoresCursor] = None
    ) -> MultiplayerScores:
        """
        https://osu.ppy.sh/docs/index.html#get-scores
        """
        params = {"limit": limit, "sort": sort, "cursor": cursor}
        return self._get(MultiplayerScores,
            f"/rooms/{room_id}/playlist/{playlist_id}/scores", params=params)


    # /users
    # ------

    @request(Scope.PUBLIC)
    def user_kudosu(self,
        user_id: UserIdT,
        limit: Optional[int] = None,
        offset: Optional[int] = None
    ) -> List[KudosuHistory]:
        """
        https://osu.ppy.sh/docs/index.html#get-user-kudosu
        """
        params = {"limit": limit, "offset": offset}
        return self._get(List[KudosuHistory], f"/users/{user_id}/kudosu",
            params)

    @request(Scope.PUBLIC)
    def user_scores(self,
        user_id: UserIdT,
        type_: ScoreTypeT,
        include_fails: Optional[bool] = None,
        mode: Optional[GameModeT] = None,
        limit: Optional[int] = None,
        offset: Optional[int] = None
    ) -> List[Score]:
        """
        https://osu.ppy.sh/docs/index.html#get-user-scores
        """
        params = {"include_fails": include_fails, "mode": mode, "limit": limit,
            "offset": offset}
        return self._get(List[Score], f"/users/{user_id}/scores/{type_.value}",
            params)

    @request(Scope.PUBLIC)
    def user_beatmaps(self,
        user_id: UserIdT,
        type_: UserBeatmapTypeT,
        limit: Optional[int] = None,
        offset: Optional[int] = None
    ) -> Union[List[Beatmapset], List[BeatmapPlaycount]]:
        """
        https://osu.ppy.sh/docs/index.html#get-user-beatmaps
        """
        params = {"limit": limit, "offset": offset}

        return_type = List[Beatmapset]
        if type_ is UserBeatmapType.MOST_PLAYED:
            return_type = List[BeatmapPlaycount]

        return self._get(return_type,
            f"/users/{user_id}/beatmapsets/{type_.value}", params)

    @request(Scope.PUBLIC)
    def user_recent_activity(self,
        user_id: UserIdT,
        limit: Optional[int] = None,
        offset: Optional[int] = None
    ) -> List[Event]:
        """
        https://osu.ppy.sh/docs/index.html#get-user-recent-activity
        """
        params = {"limit": limit, "offset": offset}
        return self._get(List[_Event], f"/users/{user_id}/recent_activity/",
            params)

    @request(Scope.PUBLIC)
    def user(self,
        user: Union[UserIdT, str],
        mode: Optional[GameModeT] = None,
        key: Optional[UserLookupKeyT] = None
    ) -> User:
        """
        https://osu.ppy.sh/docs/index.html#get-user
        """
        params = {"key": key}
        return self._get(User, f"/users/{user}/{mode.value if mode else ''}",
            params)


    # /wiki
    # -----

    @request(scope=None)
    def wiki_page(self,
        locale: str,
        path: str
    ) -> WikiPage:
        """
        https://osu.ppy.sh/docs/index.html#get-wiki-page
        """
        return self._get(WikiPage, f"/wiki/{locale}/{path}")


    # undocumented
    # ------------

    @request(Scope.PUBLIC)
    def score(self,
        mode: GameModeT,
        score_id: int
    ) -> Score:
        return self._get(Score, f"/scores/{mode.value}/{score_id}")

    @request(Scope.PUBLIC, requires_login=True)
    def download_score(self,
        mode: GameModeT,
        score_id: int
    ) -> Replay:
        r = self.session.get(f"{self.BASE_URL}/scores/{mode.value}/"
            f"{score_id}/download")
        replay = osrparse.parse_replay(r.content)
        return Replay(replay, self)

    @request(Scope.PUBLIC)
    def search_beatmapsets(self,
        query: Optional[str] = None,
        cursor: Optional[Cursor] = None
    ) -> BeatmapsetSearchResult:
        # Param key names are the same as https://osu.ppy.sh/beatmapsets,
        # so from eg https://osu.ppy.sh/beatmapsets?q=black&s=any we get that
        # the query uses ``q`` and the category uses ``s``.
        # TODO implement all possible queries, or wait for them to be
        # documented. Currently we only implement the most basic "query" option.
        params = {"cursor": cursor, "q": query}
        return self._get(BeatmapsetSearchResult, "/beatmapsets/search/", params)

    @request(Scope.PUBLIC)
    def beatmapsets_events(self,
        limit: Optional[int] = None,
        page: Optional[int] = None,
        user_id: Optional[UserIdT] = None,
        types: Optional[List[BeatmapsetEventTypeT]] = None,
        min_date: Optional[datetime] = None,
        max_date: Optional[datetime] = None
    ) -> ModdingHistoryEventsBundle:
        """
        Beatmap history

        https://osu.ppy.sh/beatmapsets/events
        """
        # limit is 5-50
        params = {"limit": limit, "page": page, "user": user_id,
            "min_date": min_date, "max_date": max_date, "types": types}
        return self._get(ModdingHistoryEventsBundle, "/beatmapsets/events",
            params)


    # /oauth
    # ------

    def revoke_token(self):
        self.session.delete(f"{self.BASE_URL}/oauth/tokens/current")
        self.remove_token(self.token_key, self.token_directory)

pylint crashed with a AttributeError and with the following stacktrace:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/inference_tip.py", line 28, in _inference_tip_cached
    result = _cache[func, node]
KeyError: (<function infer_dataclass_field_call at 0x7f7fdf559b80>, <Call l.182 at 0x7f7fe306ecd0>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1008, in _check_files
    self._check_file(get_ast, check_astroid_module, file)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1043, in _check_file
    check_astroid_module(ast_node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1180, in check_astroid_module
    retval = self._check_astroid_module(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1227, in _check_astroid_module
    walker.walk(node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pylint/utils/ast_walker.py", line 78, in walk
    self.walk(child)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    callback(astroid)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pylint/checkers/variables.py", line 1248, in visit_importfrom
    module = self._check_module_attrs(node, module, name_parts[1:])
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pylint/checkers/variables.py", line 1995, in _check_module_attrs
    module = next(module.getattr(name)[0].infer())
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/scoped_nodes.py", line 610, in getattr
    result = [self.import_module(name, relative_only=True)]
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/scoped_nodes.py", line 703, in import_module
    return AstroidManager().ast_from_module_name(absmodname)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/manager.py", line 207, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/manager.py", line 120, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/builder.py", line 139, in file_build
    return self._post_build(module, encoding)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/builder.py", line 163, in _post_build
    module = self._manager.visit_transforms(module)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/manager.py", line 97, in visit_transforms
    return self._transform.visit(node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 96, in visit
    return self._visit(module)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 61, in _visit
    visited = self._visit_generic(value)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 68, in _visit_generic
    return [self._visit_generic(child) for child in node]
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 68, in <listcomp>
    return [self._visit_generic(child) for child in node]
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 74, in _visit_generic
    return self._visit(node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 64, in _visit
    return self._transform(node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 45, in _transform
    if predicate is None or predicate(node):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/brain/brain_namedtuple_enum.py", line 550, in _is_enum_subclass
    for klass in cls.mro()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/scoped_nodes.py", line 3030, in mro
    return self._compute_mro(context=context)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/scoped_nodes.py", line 2999, in _compute_mro
    inferred_bases = list(self._inferred_bases(context=context))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/scoped_nodes.py", line 2982, in _inferred_bases
    baseobj = next(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/scoped_nodes.py", line 2982, in <genexpr>
    baseobj = next(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/node_ng.py", line 137, in infer
    for i, result in enumerate(generator):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/decorators.py", line 142, in raise_if_nothing_inferred
    yield next(generator)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/decorators.py", line 111, in wrapped
    for res in _func(node, context, **kwargs):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/bases.py", line 156, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/node_ng.py", line 137, in infer
    for i, result in enumerate(generator):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/decorators.py", line 142, in raise_if_nothing_inferred
    yield next(generator)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/decorators.py", line 111, in wrapped
    for res in _func(node, context, **kwargs):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/inference.py", line 290, in infer_import_from
    module = self.do_import_module()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/mixins.py", line 101, in do_import_module
    return mymodule.import_module(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/scoped_nodes.py", line 703, in import_module
    return AstroidManager().ast_from_module_name(absmodname)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/manager.py", line 207, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/manager.py", line 120, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/builder.py", line 139, in file_build
    return self._post_build(module, encoding)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/builder.py", line 163, in _post_build
    module = self._manager.visit_transforms(module)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/manager.py", line 97, in visit_transforms
    return self._transform.visit(node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 96, in visit
    return self._visit(module)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 61, in _visit
    visited = self._visit_generic(value)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 68, in _visit_generic
    return [self._visit_generic(child) for child in node]
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 68, in <listcomp>
    return [self._visit_generic(child) for child in node]
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 74, in _visit_generic
    return self._visit(node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 64, in _visit
    return self._transform(node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/transforms.py", line 46, in _transform
    ret = transform_func(node)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/brain/brain_dataclasses.py", line 56, in dataclass_transform
    for assign_node in _get_dataclass_attributes(node):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/brain/brain_dataclasses.py", line 113, in _get_dataclass_attributes
    if _is_class_var(assign_node.annotation):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/brain/brain_dataclasses.py", line 362, in _is_class_var
    inferred = next(node.infer())
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    results = tuple(self._explicit_inference(self, context, **kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/inference_tip.py", line 30, in _inference_tip_cached
    result = _cache[func, node] = list(func(*args, **kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/brain/brain_dataclasses.py", line 246, in infer_dataclass_field_call
    default_type, default = _get_field_default(field_call)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/astroid/brain/brain_dataclasses.py", line 337, in _get_field_default
    for keyword in field_call.keywords:
AttributeError: 'NoneType' object has no attribute 'keywords'

~/Desktop λ

Configuration

No response

Command used

not sure, using pylint with vscode

Pylint output

n/a

Expected behavior

n/a

Pylint version

pylint 2.11.1
astroid 2.8.2
Python 3.9.0 (v3.9.0:9cf6752276, Oct  5 2020, 11:29:23) 
[Clang 6.0 (clang-600.0.57)]

OS / Environment

macos

Additional dependencies

alabaster==0.7.12
altgraph==0.17
applescript==2021.2.9
appnope==0.1.2
asgiref==3.3.4
astroid==2.8.2
attrs==20.3.0
autopep8==1.5.6
Babel==2.9.0
backcall==0.2.0
bcrypt==3.2.0
beautifulsoup4==4.9.3
bleach==3.3.0
bs4==0.0.1
cachetools==4.2.1
certifi==2020.12.5
cffi==1.14.4
chardet==4.0.0
charset-normalizer==2.0.6
cheroot==8.5.1
-e git+ssh://git@github.com/circleguard/circlecore.git@3cb63a7ab816d1ed942ea5334c5d8a21861eb5c5#egg=circleguard
-e git+ssh://git@github.com/circleguard/circlevis.git@32bf52b137ea3d94e10f236799a7893a50f93a4a#egg=circlevis
click==8.0.1
colorama==0.4.4
cx-Oracle==8.1.0
cycler==0.10.0
decorator==4.4.2
discord-webhook==0.12.0
Django==3.2.4
django-appconf==1.0.5
django-crispy-forms==1.12.0
django-sass-processor==1.1
django-tables2==2.4.0
djangorestframework==3.12.4
docutils==0.16
doltpy==1.1.10
flake8==3.8.4
furo==2021.3.20b30
fuzzywuzzy==0.18.0
gallery-dl==1.18.4
google-api-core==1.26.1
google-api-python-client==2.0.2
google-auth==1.27.1
google-auth-httplib2==0.1.0
googleapis-common-protos==1.53.0
hentai==3.2.8
httplib2==0.19.0
idna==2.10
imagesize==1.2.0
importlib-metadata==3.7.3
iniconfig==1.1.1
ipython==7.22.0
ipython-genutils==0.2.0
isort==5.7.0
jaraco.functools==3.1.0
jedi==0.18.0
Jinja2==2.11.3
joblib==1.0.1
keyring==23.0.0
kiwisolver==1.3.1
lazy-object-proxy==1.4.3
libsass==0.21.0
line-profiler==3.1.0
livereload==2.6.3
lxml==4.6.2
macholib==1.14
MarkupSafe==1.1.1
matplotlib==3.2.2
mccabe==0.6.1
more-itertools==8.6.0
mpmath==1.2.1
mysql-connector-python==8.0.21
mysqlclient==2.0.3
numpy==1.19.3
oauthlib==3.1.0
-e git+ssh://git@github.com/kszlim/osu-replay-parser.git@753ce0fd79419aa81784c1bd49cbc8259106ff99#egg=osrparse
-e git+ssh://git@github.com/circleguard/ossapi.git@9996a8620b3e5a91cb0addf8bd49815f56b6cedd#egg=ossapi
packaging==20.9
pandas==1.2.1
parso==0.8.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.2.0
pkginfo==1.7.0
platformdirs==2.4.0
pluggy==0.13.1
portalocker==2.2.1
praw==7.4.0
prawcore==2.3.0
prompt-toolkit==3.0.18
protobuf==3.14.0
psutil==5.8.0
psycopg2-binary==2.8.6
ptyprocess==0.7.0
py==1.10.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycodestyle==2.7.0
pycparser==2.20
pyflakes==2.2.0
Pygments==2.8.1
pyinstaller==4.2
pyinstaller-hooks-contrib==2021.1
pylint==2.11.1
pylint-django==2.4.4
pylint-plugin-utils==0.6
PyMySQL==0.10.1
pynput==1.7.3
pyobjc-core==7.3
pyobjc-framework-Cocoa==7.3
pyobjc-framework-Quartz==7.3
pyparsing==2.4.7
PyQt5==5.15.2
PyQt5-sip==12.8.1
pytest==6.2.2
python-dateutil==2.8.1
python-Levenshtein==0.12.2
pytz==2020.5
rcssmin==1.0.6
readme-renderer==29.0
requests==2.26.0
requests-oauthlib==1.3.0
requests-toolbelt==0.9.1
retry==0.9.2
rfc3986==1.4.0
rjsmin==1.1.0
rsa==4.7.2
rstcheck==3.3.1
scikit-learn==0.24.1
scipy==1.6.1
selenium==3.141.0
six==1.15.0
-e git+ssh://git@github.com/llllllllll/slider.git@f2a5666df70073297b685988295171a5dff15e77#egg=slider
snowballstemmer==2.1.0
soupsieve==2.1
Sphinx==3.5.2
sphinx-autobuild==2021.3.14
sphinx-rtd-theme==0.5.1
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
SQLAlchemy==1.3.22
sqlparse==0.4.1
sympy==1.6.2
TableauScraper==0.1.11
threadpoolctl==2.1.0
toml==0.10.2
tornado==6.1
tqdm==4.62.2
traitlets==5.0.5
twine==3.4.1
typing-extensions==3.10.0.2
typing-utils==0.1.0
unidip==0.1.1
update-checker==0.18.0
uritemplate==3.0.1
urllib3==1.26.2
wcwidth==0.2.5
web.py==0.62
webencodings==0.5.1
websocket-client==1.2.1
wrapt==1.12.1
wtc==1.2.1
zipp==3.4.1

[Pierre-Sassoulas]

Thanks a lot for opening the issue.

[DanielNoord]

Isn't this a duplicate of #5030? I feel we fixed the issue with non existent keywords attributes, but I'm on mobile so can't do a full search.

[cdce8p]

Isn't this a duplicate of #5030? I feel we fixed the issue with non existent keywords attributes, but I'm on mobile so can't do a full search.

This one seems to be different. #5030 was fixed in astroid 2.8.2 which was used here.

--
@tybug It would help tremendously if you could narrow which part of your code is causing the error. I.e. try linting the file on its own and subsequently remove more and more parts until it's reasonable isolated.

Furthermore, there should be one extra line just above the traceback. Something like this: Exception on node <...> in file '...'. If you could post it, that would be really helpful, too.

[DanielNoord]

I checked out the repository and the error is reported on the import on line 23:

Exception on node <ImportFrom l.23 at 0x108c04e20> in file '.../ossapi/ossapiv2.py'
1,0,fatal,fatal:Fatal error while checking '.../ossapi/ossapiv2.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '/Users/daniel/Library/Caches/pylint/pylint-crash-2021-10-14-12.txt'.

Edit:
astroid crashes on ossapi/models.py with:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/astroid/inference_tip.py", line 28, in _inference_tip_cached
    result = _cache[func, node]
KeyError: (<function infer_dataclass_field_call at 0x10c9c83a0>, <Call l.179 at 0x10cf36a40>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1154, in get_ast
    return MANAGER.ast_from_file(filepath, modname, source=True)
  File "/usr/local/lib/python3.10/site-packages/astroid/manager.py", line 120, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/usr/local/lib/python3.10/site-packages/astroid/builder.py", line 139, in file_build
    return self._post_build(module, encoding)
  File "/usr/local/lib/python3.10/site-packages/astroid/builder.py", line 163, in _post_build
    module = self._manager.visit_transforms(module)
  File "/usr/local/lib/python3.10/site-packages/astroid/manager.py", line 97, in visit_transforms
    return self._transform.visit(node)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 96, in visit
    return self._visit(module)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 61, in _visit
    visited = self._visit_generic(value)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 68, in _visit_generic
    return [self._visit_generic(child) for child in node]
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 68, in <listcomp>
    return [self._visit_generic(child) for child in node]
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 74, in _visit_generic
    return self._visit(node)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 64, in _visit
    return self._transform(node)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 45, in _transform
    if predicate is None or predicate(node):
  File "/usr/local/lib/python3.10/site-packages/astroid/brain/brain_namedtuple_enum.py", line 550, in _is_enum_subclass
    for klass in cls.mro()
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/scoped_nodes.py", line 3030, in mro
    return self._compute_mro(context=context)
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/scoped_nodes.py", line 2999, in _compute_mro
    inferred_bases = list(self._inferred_bases(context=context))
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/scoped_nodes.py", line 2982, in _inferred_bases
    baseobj = next(
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/scoped_nodes.py", line 2982, in <genexpr>
    baseobj = next(
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 137, in infer
    for i, result in enumerate(generator):
  File "/usr/local/lib/python3.10/site-packages/astroid/decorators.py", line 142, in raise_if_nothing_inferred
    yield next(generator)
  File "/usr/local/lib/python3.10/site-packages/astroid/decorators.py", line 111, in wrapped
    for res in _func(node, context, **kwargs):
  File "/usr/local/lib/python3.10/site-packages/astroid/bases.py", line 156, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 137, in infer
    for i, result in enumerate(generator):
  File "/usr/local/lib/python3.10/site-packages/astroid/decorators.py", line 142, in raise_if_nothing_inferred
    yield next(generator)
  File "/usr/local/lib/python3.10/site-packages/astroid/decorators.py", line 111, in wrapped
    for res in _func(node, context, **kwargs):
  File "/usr/local/lib/python3.10/site-packages/astroid/inference.py", line 290, in infer_import_from
    module = self.do_import_module()
  File "/usr/local/lib/python3.10/site-packages/astroid/mixins.py", line 101, in do_import_module
    return mymodule.import_module(
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/scoped_nodes.py", line 703, in import_module
    return AstroidManager().ast_from_module_name(absmodname)
  File "/usr/local/lib/python3.10/site-packages/astroid/manager.py", line 207, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/usr/local/lib/python3.10/site-packages/astroid/manager.py", line 120, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/usr/local/lib/python3.10/site-packages/astroid/builder.py", line 139, in file_build
    return self._post_build(module, encoding)
  File "/usr/local/lib/python3.10/site-packages/astroid/builder.py", line 163, in _post_build
    module = self._manager.visit_transforms(module)
  File "/usr/local/lib/python3.10/site-packages/astroid/manager.py", line 97, in visit_transforms
    return self._transform.visit(node)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 96, in visit
    return self._visit(module)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 61, in _visit
    visited = self._visit_generic(value)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 68, in _visit_generic
    return [self._visit_generic(child) for child in node]
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 68, in <listcomp>
    return [self._visit_generic(child) for child in node]
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 74, in _visit_generic
    return self._visit(node)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 64, in _visit
    return self._transform(node)
  File "/usr/local/lib/python3.10/site-packages/astroid/transforms.py", line 46, in _transform
    ret = transform_func(node)
  File "/usr/local/lib/python3.10/site-packages/astroid/brain/brain_dataclasses.py", line 56, in dataclass_transform
    for assign_node in _get_dataclass_attributes(node):
  File "/usr/local/lib/python3.10/site-packages/astroid/brain/brain_dataclasses.py", line 113, in _get_dataclass_attributes
    if _is_class_var(assign_node.annotation):
  File "/usr/local/lib/python3.10/site-packages/astroid/brain/brain_dataclasses.py", line 362, in _is_class_var
    inferred = next(node.infer())
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    results = tuple(self._explicit_inference(self, context, **kwargs))
  File "/usr/local/lib/python3.10/site-packages/astroid/inference_tip.py", line 30, in _inference_tip_cached
    result = _cache[func, node] = list(func(*args, **kwargs))
  File "/usr/local/lib/python3.10/site-packages/astroid/brain/brain_dataclasses.py", line 246, in infer_dataclass_field_call
    default_type, default = _get_field_default(field_call)
  File "/usr/local/lib/python3.10/site-packages/astroid/brain/brain_dataclasses.py", line 337, in _get_field_default
    for keyword in field_call.keywords:
AttributeError: 'NoneType' object has no attribute 'keywords'
************* Module ossapi.models
ossapi/models.py:1:0: F0002: <class 'AttributeError'>: 'NoneType' object has no attribute 'keywords' (astroid-error)

The line we're having trouble with is L179:
https://github.com/circleguard/ossapi/blob/91c0d13950e2bc116a998adf560d1f3041ae647b/ossapi/models.py#L179

[cdce8p]

Think I found the issue:

# pylint: disable=missing-docstring
from abc import ABC, abstractmethod
from dataclasses import dataclass, field

@dataclass
class Expandable(ABC):
    """
    A mixin for models which can be "expanded" to a different model which has a
    superset of attributes of the current model. Typically this expansion is
    expensive (requires an additional api call) which is why it is not done by
    default.
    """
    # can't annotate with OssapiV2 or we get a circular import error, this is
    # good enough
    _api: field()

    @abstractmethod
    def expand(self):
        pass

https://github.com/circleguard/ossapi/blob/91c0d13950e2bc116a998adf560d1f3041ae647b/ossapi/utils.py#L169-L180

--
Is there a reason for writing _api: field()? I don't think that is valid to begin with. From pylance:

Illegal type annotation: call expression not allowed

Should that be _api: Any = field() instead?

[tybug]

Should that be _api: Any = field() instead?

indeed it should...thanks for the catch. I've pushed a fix for this to my repo and pylint seems to lint fine. Will leave up to you if you'd like to close this issue - it seems reasonable to me to expect that linting that program would throw a better error than pylint crashing, even though it's not the right usage of dataclass' field.

[cdce8p]

@tybug I've opened pylint-dev/astroid#1212 to fix the crash with an invalid field call. I agree that pylint should ideally emit a better error in that case. See #5159 for some ideas on how to add / improve dataclass tests in pylint.