Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support inheritance #170

Merged
merged 3 commits into from
Jun 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,6 @@ See [the Loading data section](https://mkdocstrings.github.io/griffe/loading/) f

## Todo

- Visitor/Inspector:
- Merging inherited members into class.
Needs to be able to post-process classes,
and to compute their MRO (C3Linearization, see docspec/pydocspec issues).
- Extensions
- Post-processing extensions
- Third-party libraries we could provide support for:
Expand All @@ -111,7 +107,6 @@ See [the Loading data section](https://mkdocstrings.github.io/griffe/loading/) f
- New Markdown-based format? For graceful degradation
- Serializer:
- Flat JSON
- JSON schema
- API diff:
- [ ] Mechanism to cache APIs? Should users version them, or store them somewhere (docs)?
- [ ] Ability to return warnings (things that are not backward-compatibility-friendly)
Expand Down
93 changes: 93 additions & 0 deletions docs/loading.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,96 @@ There are several ways to access members of an object:

Most of the time, you will only use classes from the [`griffe.dataclasses`][griffe.dataclasses]
and [`griffe.docstrings.dataclasses`][griffe.docstrings.dataclasses] modules.


## Class inheritance

TIP: **New in version 0.30**

WARNING: **Inheritance support is experimental**
Inheritance support was recently added, and might need
some corrections before being fully usable.
Don't hesitate to report any issue that arises
from using inheritance support in Griffe.

Griffe supports class inheritance, both when visiting and inspecting modules.

To access members of a class that are inherited from base classes,
use [`Object.inherited_members`][griffe.dataclasses.Object.inherited_members].
If this is the first time you access inherited members, the base classes
of the given class will be resolved and cached, then the MRO (Method Resolution Order)
will be computed for these bases classes, and a dictionary of inherited members
will be built and cached. Next times you access it, you'll get the cached dictionary.
Make sure to only access `inherited_members` once everything is loaded by Griffe,
to avoid computing things too early. Don't access inherited members
in extensions, while visiting or inspecting a module.

**Important:** only classes from already loaded packages
will be used when computing inherited members.
This gives users control over how deep into inheritance to go,
by pre-loading packages from which you want to inherit members.
For example, if `package_c.ClassC` inherits from `package_b.ClassB`,
itself inheriting from `package_a.ClassA`, and you want
to load `ClassB` members only:

```python
from griffe.loader import GriffeLoader

loader = GriffeLoader()
# note that we don't load package_a
loader.load_module("package_b")
loader.load_module("package_c")
```

If a base class cannot be resolved during computation
of inherited members, Griffe logs a DEBUG message.

If you want to access all members at once (both declared and inherited),
use [`Object.all_members`][griffe.dataclasses.Object.all_members].

If you want to access only declared members,
use [`Object.members`][griffe.dataclasses.Object].

Accessing [`Object.attributes`][griffe.dataclasses.Object.attributes],
[`Object.functions`][griffe.dataclasses.Object.functions],
[`Object.classes`][griffe.dataclasses.Object.classes] or
[`Object.modules`][griffe.dataclasses.Object.modules]
will trigger inheritance computation, so make sure to only call it
once everything is loaded by Griffe. Don't access inherited members
in extensions, while visiting or inspecting a module.

### Limitations

Currently, there are two limitations to our class inheritance support:

1. when visiting (static analysis), some objects are not yet properly recognized as classes,
for example named tuples. If you inherit from a named tuple,
its members won't be added to the inherited members of the inheriting class.

```python
MyTuple = namedtuple("MyTuple", "attr1 attr2")


class MyClass(MyTuple):
...
```

2. when inspecting (dynamic analysis), ephemeral base classes won't be resolved,
and therefore their members won't appear in child classes. To circumvent that,
assign these dynamic classes to variables:

```python
# instead of
class MyClass(namedtuple("MyTuple", "attr1 attr2")):
...


# do
MyTuple = namedtuple("MyTuple", "attr1 attr2")


class MyClass(MyTuple):
...
```

We will try to lift these limitations in the future.
14 changes: 9 additions & 5 deletions src/griffe/agents/inspector.py
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,7 @@ def generic_inspect(self, node: ObjectNode) -> None:
else:
child_name = getattr(child.obj, "__name__", child.name)
target_path = f"{child_module_path}.{child_name}"
self.current[child.name] = Alias(child.name, target_path)
self.current.set_member(child.name, Alias(child.name, target_path))
else:
self.inspect(child)

Expand Down Expand Up @@ -295,14 +295,18 @@ def inspect_class(self, node: ObjectNode) -> None:
Parameters:
node: The node to inspect.
"""
bases = [base.__name__ for base in node.obj.__bases__ if base is not object]
bases = []
for base in node.obj.__bases__:
if base is object:
continue
bases.append(f"{base.__module__}.{base.__qualname__}")

class_ = Class(
name=node.name,
docstring=self._get_docstring(node),
bases=bases,
)
self.current[node.name] = class_
self.current.set_member(node.name, class_)
self.current = class_
self.generic_inspect(node)
self.current = self.current.parent # type: ignore[assignment]
Expand Down Expand Up @@ -430,7 +434,7 @@ def handle_function(self, node: ObjectNode, labels: set | None = None) -> None:
docstring=self._get_docstring(node),
)
obj.labels |= labels
self.current[node.name] = obj
self.current.set_member(node.name, obj)

def inspect_attribute(self, node: ObjectNode) -> None:
"""Inspect an attribute.
Expand Down Expand Up @@ -477,7 +481,7 @@ def handle_attribute(self, node: ObjectNode, annotation: str | Name | Expression
docstring=docstring,
)
attribute.labels |= labels
parent[node.name] = attribute
parent.set_member(node.name, attribute)

if node.name == "__all__":
parent.exports = set(node.obj)
Expand Down
8 changes: 8 additions & 0 deletions src/griffe/agents/nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,13 @@ def __init__(self, obj: Any, name: str, parent: ObjectNode | None = None) -> Non
def __repr__(self) -> str:
return f"ObjectNode(name={self.name!r})"

@cached_property
def path(self) -> str:
"""The object's (Python) path."""
if self.parent is None:
return self.name
return f"{self.parent.path}.{self.name}"

@cached_property
def kind(self) -> ObjectKind:
"""Return the kind of this node.
Expand Down Expand Up @@ -508,6 +515,7 @@ def _pick_member(self, name: str, member: Any) -> bool:
and member is not type
and member is not object
and id(member) not in self._ids
and name in vars(self.obj)
pawamoy marked this conversation as resolved.
Show resolved Hide resolved
)


Expand Down
38 changes: 22 additions & 16 deletions src/griffe/agents/visitor.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ def visit_classdef(self, node: ast.ClassDef) -> None:
runtime=not self.type_guarded,
)
class_.labels |= self.decorators_to_labels(decorators)
self.current[node.name] = class_
self.current.set_member(node.name, class_)
self.current = class_
self.generic_visit(node)
self.current = self.current.parent # type: ignore[assignment]
Expand Down Expand Up @@ -304,10 +304,10 @@ def get_base_property(self, decorators: list[Decorator]) -> tuple[Function | Non
property_setter_or_deleter = (
base_function in {"setter", "deleter"}
and base_name in self.current.members
and self.current[base_name].has_labels({"property"})
and self.current.get_member(base_name).has_labels({"property"})
)
if property_setter_or_deleter:
return self.current[base_name], base_function
return self.current.get_member(base_name), base_function
return None, None

def handle_function(self, node: ast.AsyncFunctionDef | ast.FunctionDef, labels: set | None = None) -> None:
Expand Down Expand Up @@ -356,7 +356,7 @@ def handle_function(self, node: ast.AsyncFunctionDef | ast.FunctionDef, labels:
runtime=not self.type_guarded,
)
attribute.labels |= labels
self.current[node.name] = attribute
self.current.set_member(node.name, attribute)
return

base_property, property_function = self.get_base_property(decorators)
Expand Down Expand Up @@ -461,7 +461,7 @@ def handle_function(self, node: ast.AsyncFunctionDef | ast.FunctionDef, labels:
base_property.deleter = function
base_property.labels.add("deletable")
else:
self.current[node.name] = function
self.current.set_member(node.name, function)
if self.current.kind in {Kind.MODULE, Kind.CLASS} and self.current.overloads[function.name]:
function.overloads = self.current.overloads[function.name]
del self.current.overloads[function.name]
Expand Down Expand Up @@ -499,12 +499,15 @@ def visit_import(self, node: ast.Import) -> None:
alias_path = name.name
alias_name = name.asname or alias_path.split(".", 1)[0]
self.current.imports[alias_name] = alias_path
self.current[alias_name] = Alias(
self.current.set_member(
alias_name,
alias_path,
lineno=node.lineno,
endlineno=node.end_lineno, # type: ignore[attr-defined]
runtime=not self.type_guarded,
Alias(
alias_name,
alias_path,
lineno=node.lineno,
endlineno=node.end_lineno, # type: ignore[attr-defined]
runtime=not self.type_guarded,
),
)

def visit_importfrom(self, node: ast.ImportFrom) -> None:
Expand All @@ -528,12 +531,15 @@ def visit_importfrom(self, node: ast.ImportFrom) -> None:
else:
alias_name = name.asname or name.name
self.current.imports[alias_name] = alias_path
self.current[alias_name] = Alias(
self.current.set_member(
alias_name,
alias_path, # type: ignore[arg-type]
lineno=node.lineno,
endlineno=node.end_lineno, # type: ignore[attr-defined]
runtime=not self.type_guarded,
Alias(
alias_name,
alias_path, # type: ignore[arg-type]
lineno=node.lineno,
endlineno=node.end_lineno, # type: ignore[attr-defined]
runtime=not self.type_guarded,
),
)

def handle_attribute(
Expand Down Expand Up @@ -626,7 +632,7 @@ def handle_attribute(
runtime=not self.type_guarded,
)
attribute.labels |= labels
parent[name] = attribute
parent.set_member(name, attribute)

if name == "__all__":
with suppress(AttributeError):
Expand Down
117 changes: 117 additions & 0 deletions src/griffe/c3linear.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
"""Compute method resolution order. Implements `Class.mro` attribute."""

# Copyright (c) 2019 Vitaly R. Samigullin
# Adapted from https://github.com/pilosus/c3linear
# Adapted from https://github.com/tristanlatr/pydocspec
pawamoy marked this conversation as resolved.
Show resolved Hide resolved

from __future__ import annotations

from itertools import islice
from typing import Deque, TypeVar

T = TypeVar("T")


class _Dependency(Deque[T]):
"""A class representing a (doubly-ended) queue of items."""

@property
def head(self) -> T | None:
"""Head of the dependency."""
try:
return self[0]
except IndexError:
return None

@property
def tail(self) -> islice:
"""Tail od the dependency.

The `islice` object is sufficient for iteration or testing membership (`in`).
"""
try:
return islice(self, 1, self.__len__())
except (ValueError, IndexError):
return islice([], 0, 0)


class _DependencyList:
"""A class representing a list of linearizations (dependencies).

The last element of DependencyList is a list of parents.
It's needed to the merge process preserves the local
precedence order of direct parent classes.
"""

def __init__(self, *lists: list[T | None]) -> None:
"""Initialize the list.

Parameters:
*lists: Lists of items.
"""
self._lists = [_Dependency(lst) for lst in lists]

def __contains__(self, item: T) -> bool:
"""Return True if any linearization's tail contains an item."""
return any(item in lst.tail for lst in self._lists)

def __len__(self) -> int:
size = len(self._lists)
return (size - 1) if size else 0

def __repr__(self) -> str:
return self._lists.__repr__()

@property
def heads(self) -> list[T | None]:
"""Return the heads."""
return [lst.head for lst in self._lists]

@property
def tails(self) -> _DependencyList:
"""Return self so that `__contains__` could be called."""
return self

@property
def exhausted(self) -> bool:
"""True if all elements of the lists are exhausted."""
return all(len(x) == 0 for x in self._lists)

def remove(self, item: T | None) -> None:
"""Remove an item from the lists.

Once an item removed from heads, the leftmost elements of the tails
get promoted to become the new heads.
"""
for i in self._lists:
if i and i.head == item:
i.popleft()


def c3linear_merge(*lists: list[T]) -> list[T]:
"""Merge lists of lists in the order defined by the C3Linear algorithm.

Parameters:
*lists: Lists of items.

Returns:
The merged list of items.
"""
result: list[T] = []
linearizations = _DependencyList(*lists) # type: ignore[arg-type]

while True:
if linearizations.exhausted:
return result

for head in linearizations.heads:
if head and (head not in linearizations.tails):
result.append(head) # type: ignore[arg-type]
linearizations.remove(head)

# Once candidate is found, continue iteration
# from the first element of the list.
break
else:
# Loop never broke, no linearization could possibly be found.
raise ValueError("Cannot compute C3 linearization")
Loading
Loading