🎯 Learning Objectives
- Explain the full attribute lookup chain: instance
__dict__, class__dict__, and the MRO - Distinguish data descriptors from non-data descriptors and explain the priority order
- Implement
__get__,__set__, and__delete__to build custom descriptors - Understand how
property,staticmethod,classmethod, andfunctionobjects use the descriptor protocol - Build reusable validated field descriptors for class-level validation
- Understand
__slots__as member descriptors and their effect on__dict__ - Use
__set_name__to make descriptors self-configuring at class creation time
1 · Attribute Lookup: The Full Chain
When you write obj.attr, Python follows a precise lookup order defined by
object.__getattribute__. Understanding this order is the key to understanding every
"magic" attribute behaviour in Python — from property to ORM field validation to
mock objects. The algorithm runs synchronously, in full, every time any attribute is accessed.
Here is the complete algorithm, expressed as pseudocode:
object.__getattribute__(obj, 'attr'):
1. type_mro = type(obj).__mro__ # [type(obj), ..., object]
2. For each cls in type_mro:
if 'attr' in cls.__dict__:
descriptor = cls.__dict__['attr']
if hasattr(descriptor, '__set__') or hasattr(descriptor, '__delete__'):
# DATA DESCRIPTOR — highest priority
return descriptor.__get__(obj, type(obj))
3. if 'attr' in obj.__dict__:
# Instance variable — second priority
return obj.__dict__['attr']
4. For each cls in type_mro:
if 'attr' in cls.__dict__:
descriptor = cls.__dict__['attr']
if hasattr(descriptor, '__get__'):
# NON-DATA DESCRIPTOR — third priority
return descriptor.__get__(obj, type(obj))
else:
# Plain class attribute — fourth priority
return descriptor
5. raise AttributeError(f"'{type(obj).__name__}' object has no attribute 'attr'")
The following example demonstrates this priority order concretely. Notice how the
data descriptor wins even when an instance __dict__ key with the same
name exists, while the non-data descriptor is silently shadowed by the instance key:
class DataDesc:
def __get__(self, obj, objtype=None):
return "data_desc __get__"
def __set__(self, obj, value):
pass # makes this a DATA descriptor
class NonDataDesc:
def __get__(self, obj, objtype=None):
return "non_data_desc __get__"
# no __set__ → NON-DATA descriptor
class MyClass:
data = DataDesc()
non_data = NonDataDesc()
obj = MyClass()
obj.__dict__["data"] = "instance value (data)"
obj.__dict__["non_data"] = "instance value (non_data)"
print(obj.data) # "data_desc __get__" — DATA desc wins over instance __dict__
print(obj.non_data) # "instance value (non_data)" — instance __dict__ wins over non-data desclookup_priority.pyproperty) must be able
to intercept writes and prevent instance __dict__ entries from shadowing them —
otherwise a single assignment (obj.radius = 5) would bypass the property setter
forever. Non-data descriptors (like regular functions) allow instance attributes to shadow them,
which is exactly how self.method = lambda: ... can override a class method on a
per-instance basis.
Priority table
| Priority | Source | Condition |
|---|---|---|
| 1 (highest) | Class / MRO __dict__ | Object has both __get__ and (__set__ or __delete__) — data descriptor |
| 2 | Instance __dict__ | Key present in obj.__dict__ |
| 3 | Class / MRO __dict__ | Object has __get__ only — non-data descriptor |
| 4 (lowest) | Class / MRO __dict__ | Plain class attribute (no __get__) |
| — | nowhere | AttributeError |
2 · The Descriptor Protocol: __get__, __set__, __delete__
A descriptor is any object that defines one or more of:
__get__, __set__, __delete__.
That is the complete definition. There is no base class to inherit, no registration step.
Python discovers descriptors purely by duck-typing: if an object sitting in a class
__dict__ has the right dunder(s), the descriptor protocol is invoked automatically.
The three dunder methods have carefully-specified signatures. __get__ receives
None for obj when the descriptor is accessed via the class
(MyClass.attr) rather than an instance. This lets descriptors return themselves
(or useful metadata) in the class-access case:
class Descriptor:
"""
__get__(self, obj, objtype=None)
- obj is None when accessed via the class: MyClass.attr
- obj is the instance when accessed via instance: my_obj.attr
__set__(self, obj, value)
- called on instance attribute assignment: my_obj.attr = value
__delete__(self, obj)
- called on del my_obj.attr
"""
def __set_name__(self, owner, name):
# Called at class creation time (Python 3.6+)
self.name = name
self.private_name = f"_{owner.__name__}__{name}"
def __get__(self, obj, objtype=None):
if obj is None:
return self # class-level access: return the descriptor itself
return getattr(obj, self.private_name, None)
def __set__(self, obj, value):
setattr(obj, self.private_name, value)
def __delete__(self, obj):
delattr(obj, self.private_name)
class Point:
x = Descriptor()
y = Descriptor()
p = Point()
p.x = 10 # calls Descriptor.__set__(p, 10)
p.y = 20
print(p.x) # calls Descriptor.__get__(p, Point) → 10
del p.x # calls Descriptor.__delete__(p)
print(Point.x) # Descriptor.__get__(None, Point) → the descriptor object itselfdescriptor_basics.py
The __set_name__ hook (Python 3.6+) is called automatically when the class body
finishes execution. Python passes the owning class and the attribute name to the descriptor,
letting it store its own name without any boilerplate in the class definition. Before
__set_name__ existed, library authors had to require users to repeat the name:
x = TypedField("x", int) — redundant and error-prone. Now descriptors are
self-configuring.
__init__ runs first (when Python
evaluates the right-hand side of x = Descriptor() inside the class body), then
__set_name__ runs after the whole class body has been executed. At
__set_name__ time the class object exists and is passed as owner.
3 · Validated Field Descriptors
One of the most practically useful applications of the descriptor protocol is building
reusable field validators that enforce type constraints and value ranges at
assignment time. This is the same pattern used internally by Django model fields, SQLAlchemy
mapped columns, attrs, and Pydantic v1 under the hood. The descriptor lives once in the class
but guards every instance independently by storing per-instance data in the instance's own
__dict__ (via a mangled private name to avoid collisions).
from typing import Any, Type, Callable
class TypedField:
"""A descriptor that enforces a type constraint on assignment."""
def __set_name__(self, owner: type, name: str) -> None:
self.public_name = name
self.private_name = f"_{name}"
def __init__(self, expected_type: type, *, nullable: bool = False) -> None:
self.expected_type = expected_type
self.nullable = nullable
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, self.private_name, None)
def __set__(self, obj, value) -> None:
if value is None and self.nullable:
setattr(obj, self.private_name, value)
return
if not isinstance(value, self.expected_type):
raise TypeError(
f"{self.public_name!r} expects {self.expected_type.__name__}, "
f"got {type(value).__name__}"
)
setattr(obj, self.private_name, value)
def __delete__(self, obj) -> None:
setattr(obj, self.private_name, None)
class RangedField(TypedField):
"""Adds min/max bounds on top of type checking."""
def __init__(self, expected_type: type, *, lo=None, hi=None, **kw) -> None:
super().__init__(expected_type, **kw)
self.lo, self.hi = lo, hi
def __set__(self, obj, value) -> None:
super().__set__(obj, value) # type check first
v = getattr(obj, self.private_name)
if v is not None:
if self.lo is not None and v < self.lo:
raise ValueError(f"{self.public_name!r} must be >= {self.lo}, got {v}")
if self.hi is not None and v > self.hi:
raise ValueError(f"{self.public_name!r} must be <= {self.hi}, got {v}")
class Employee:
name = TypedField(str)
age = RangedField(int, lo=18, hi=120)
salary = RangedField(float, lo=0.0)
def __init__(self, name, age, salary):
self.name = name
self.age = age
self.salary = salary
def __repr__(self):
return f"Employee({self.name!r}, age={self.age}, salary={self.salary})"
e = Employee("Alice", 30, 95000.0)
print(e) # Employee('Alice', age=30, salary=95000.0)
try:
e.age = 15
except ValueError as err:
print(err) # 'age' must be >= 18, got 15
try:
e.name = 123
except TypeError as err:
print(err) # 'name' expects str, got intvalidated_fields.py
Notice the inheritance chain: RangedField.__set__ delegates to
TypedField.__set__ via super(), then adds its own range check on top.
The type check and range check compose cleanly because descriptors are plain Python objects —
you get the full power of inheritance and composition.
TypedField
instance (e.g., Employee.name) is shared across all
Employee instances. Never store per-instance state on self inside a
descriptor — store it on obj instead (via setattr(obj, self.private_name,
value)). If you store on self, every instance will read and overwrite the
same slot.
4 · How property Is Implemented
property is itself a descriptor — one of the most elegant examples in the
standard library. It is implemented in C for performance, but its behaviour can be reproduced
exactly in pure Python. Understanding this implementation dissolves the mystery of decorators
like @radius.setter (they just call a method that returns a new
property object with the setter wired in):
# Rough Python equivalent of the built-in property descriptor
class property_:
def __init__(self, fget=None, fset=None, fdel=None, doc=None):
self.fget = fget
self.fset = fset
self.fdel = fdel
self.__doc__ = doc or (fget.__doc__ if fget else None)
def __set_name__(self, owner, name):
self.__name__ = name
def __get__(self, obj, objtype=None):
if obj is None:
return self # accessed on the class
if self.fget is None:
raise AttributeError(f"unreadable attribute '{self.__name__}'")
return self.fget(obj)
def __set__(self, obj, value):
if self.fset is None:
raise AttributeError(f"can't set attribute '{self.__name__}'")
self.fset(obj, value)
def __delete__(self, obj):
if self.fdel is None:
raise AttributeError(f"can't delete attribute '{self.__name__}'")
self.fdel(obj)
# These methods return a NEW property with the extra function wired in
def getter(self, fget): return type(self)(fget, self.fset, self.fdel, self.__doc__)
def setter(self, fset): return type(self)(self.fget, fset, self.fdel, self.__doc__)
def deleter(self, fdel): return type(self)(self.fget, self.fset, fdel, self.__doc__)
class Circle:
def __init__(self, radius: float):
self._radius = radius
@property
def radius(self) -> float:
return self._radius
@radius.setter
def radius(self, value: float) -> None:
if value < 0:
raise ValueError("Radius must be non-negative")
self._radius = value
@property
def area(self) -> float:
import math
return math.pi * self._radius ** 2
c = Circle(5.0)
print(c.radius) # 5.0
print(c.area) # 78.539...
c.radius = 10.0
print(Circle.radius) # <property object at 0x...> — descriptor itselfproperty_internals.py
The @radius.setter decorator is not magic syntax. It is exactly
equivalent to:
# @radius.setter is sugar for: radius = radius.setter(lambda self, v: ...) # i.e. it calls property.setter(), which returns a brand-new property # object that has both fget and fset populated.
Because property defines both __get__ and __set__
(even if fset is None, the method still exists), it is a
data descriptor. This means it takes priority over the instance
__dict__. Doing obj.__dict__['radius'] = 999 will not bypass the
property — the next access to obj.radius will still invoke fget.
This is crucial for computed and validated attributes.
obj is None in __get__?
When you access a descriptor on the class (e.g., Circle.radius),
__get__ is called with obj=None. Returning self
(the property object) lets you inspect it: Circle.radius.fget,
Circle.radius.__doc__, etc. If you returned fget(None) you would
crash or produce wrong results.
5 · staticmethod and classmethod as Descriptors
Every callable you put in a class body uses the descriptor protocol to handle binding.
staticmethod and classmethod are both non-data
descriptors (they only define __get__, not __set__).
Their __get__ implementations differ in what they return:
staticmethod.__get__returns the raw, unbound function — no binding occurs at all.classmethod.__get__returns a bound method with the class (cls) pre-filled as the first argument.- Plain
function.__get__returns a bound method with the instance (self) pre-filled — this is how every regular method works.
class MyClass:
@staticmethod
def static_fn(x):
return x * 2
@classmethod
def class_fn(cls, x):
return f"{cls.__name__}: {x * 2}"
# staticmethod.__get__ returns the raw function (no binding)
print(MyClass.__dict__["static_fn"]) # <staticmethod object>
print(MyClass.__dict__["static_fn"].__get__(None, MyClass)) # <function static_fn ...>
print(MyClass.static_fn(5)) # 10
# classmethod.__get__ returns a bound method with cls already bound
print(MyClass.__dict__["class_fn"]) # <classmethod object>
print(MyClass.class_fn(5)) # "MyClass: 10"
# Plain function is a non-data descriptor — __get__ returns a bound method
def greet(self):
return f"Hello from {type(self).__name__}"
class Greeter:
hello = greet # assign a function as a class attribute
g = Greeter()
print(g.hello()) # "Hello from Greeter"
print(Greeter.__dict__["hello"]) # <function greet ...>
print(Greeter.__dict__["hello"].__get__(g, Greeter)) # <bound method greet of ...>
# This is exactly how def creates methods in a class body
class MyClass2:
def method(self): # stored as a plain function in __dict__
return "called"
print(type(MyClass2.__dict__["method"])) # <class 'function'>
print(type(MyClass2().method)) # <class 'method'> — bound on accessstaticmethod_classmethod_descriptors.pyself through the descriptor protocol — not through
special syntax or bytecode. When you access obj.method, Python calls
function.__get__(obj, type(obj)), which returns a bound method —
a thin wrapper that prepends obj as the first argument whenever the wrapper is
called. The def keyword inside a class body stores a plain function;
the binding happens lazily at access time via __get__. This is why you can pass
obj.method around as a callable and it still "knows" its instance.
This also explains the subtle performance difference: accessing a method on an instance
creates a new bound-method wrapper object each time. For hot loops you can cache
fn = obj.method and call fn() repeatedly to avoid repeated
descriptor invocations.
6 · __slots__ as Member Descriptors
When you declare __slots__ on a class, Python creates a
member_descriptor for each slot name and stores it in the class
__dict__. These member descriptors are data descriptors — they
define both __get__ and __set__. The actual slot storage is
allocated inline in the object's C struct, bypassing the per-instance __dict__
entirely. The result: smaller objects and faster attribute access.
import sys
class WithDict:
def __init__(self, x, y):
self.x = x
self.y = y
class WithSlots:
__slots__ = ("x", "y")
def __init__(self, x, y):
self.x = x
self.y = y
d_obj = WithDict(1, 2)
s_obj = WithSlots(1, 2)
print(sys.getsizeof(d_obj)) # ~48 bytes (slots) + ~232 bytes (__dict__ overhead)
print(sys.getsizeof(s_obj)) # ~56 bytes — no __dict__ overhead
# __slots__ creates member_descriptor objects in the class dict
print(type(WithSlots.__dict__["x"])) # <class 'member_descriptor'>
print(WithSlots.__dict__["x"]) # <member 'x' of 'WithSlots' objects>
# member_descriptor IS a data descriptor (has __get__ and __set__)
desc = WithSlots.__dict__["x"]
print(desc.__get__(s_obj, WithSlots)) # 1
desc.__set__(s_obj, 99)
print(s_obj.x) # 99
# No __dict__ on slotted objects
try:
print(s_obj.__dict__)
except AttributeError as e:
print(e) # 'WithSlots' object has no attribute '__dict__'
# ── Inheritance: must redeclare __slots__ or __dict__ is added back ──
class Child(WithSlots):
__slots__ = ("z",) # inherits x, y slots from parent + adds z
def __init__(self, x, y, z):
super().__init__(x, y)
self.z = z
c = Child(1, 2, 3)
print(c.x, c.y, c.z) # 1 2 3
# ── weakref support: add __weakref__ to __slots__ ──
class Cached:
__slots__ = ("data", "__weakref__")
def __init__(self, data):
self.data = data
import weakref
obj = Cached([1, 2, 3])
ref = weakref.ref(obj)
print(ref()) # <Cached object at 0x...>slots_descriptors.py__dict__ (i.e., does not define __slots__), the subclass will also
gain a __dict__ even if the subclass declares __slots__. The memory
savings are only guaranteed when every class in the chain defines
__slots__. When in doubt, check: hasattr(obj, '__dict__').
Memory savings compound at scale. A list of one million WithDict objects can
easily consume several hundred megabytes just for the __dict__ overhead; the same
list of WithSlots objects can be 4–8× smaller in practice. This is why
data-intensive libraries like NumPy, pandas, and dataclasses (with
slots=True in Python 3.10+) use slots heavily.
7 · __getattr__, __getattribute__, __setattr__, __delattr__
The descriptor protocol operates at the slot level (inside
object.__getattribute__). But you can intercept attribute access at the
class level too — by overriding the dunder methods on the class itself. These four
hooks form the outermost layer of Python's attribute machinery:
| Hook | When called | Typical use |
|---|---|---|
__getattribute__ | Every single attribute access, always | Logging, access control, transparent proxies |
__getattr__ | Only when __getattribute__ raises AttributeError | Lazy attributes, dynamic attribute generation, proxy fallback |
__setattr__ | Every attribute assignment | Validation, change tracking, immutable objects |
__delattr__ | Every del obj.attr | Cleanup, audit trails |
class LoggedAccess:
"""Every attribute access is logged."""
def __getattribute__(self, name: str):
"""Called for EVERY attribute access (including __dict__, methods, etc.)"""
print(f" __getattribute__({name!r})")
return super().__getattribute__(name)
def __setattr__(self, name: str, value):
"""Called for EVERY attribute assignment."""
print(f" __setattr__({name!r}, {value!r})")
super().__setattr__(name, value)
def __delattr__(self, name: str):
print(f" __delattr__({name!r})")
super().__delattr__(name)
def __getattr__(self, name: str):
"""ONLY called when normal lookup fails — the last resort."""
print(f" __getattr__({name!r}) — not found via normal lookup")
raise AttributeError(name)
class MyObj(LoggedAccess):
def __init__(self):
self.x = 10 # triggers __setattr__
obj = MyObj()
print(obj.x) # triggers __getattribute__
del obj.x # triggers __delattr__
print(obj.missing) # triggers __getattribute__ first, then __getattr__attribute_hooks.py__getattribute__, accessing
any attribute on self will call __getattribute__ again,
causing infinite recursion. Always use super().__getattribute__(name) or
object.__getattribute__(self, name) to break the cycle. The same applies to
__setattr__: use object.__setattr__(self, name, value) when you
need to bypass your own override.
The critical distinction between __getattribute__ and
__getattr__ bears repeating:
__getattribute__is called always, for every access. Override it only when you truly need to intercept everything (e.g., a security wrapper that blocks access to private names). Getting it wrong causes hard-to-debug infinite recursion.__getattr__is called only when normal lookup (including the descriptor protocol and instance__dict__) has already failed. This is the safe fallback hook. Use it for dynamic attribute generation, lazy loading, and proxy objects.
A practical transparent proxy pattern — wraps any target object so all attribute access flows through transparently:
class Proxy:
"""Transparently wrap any object, intercepting all attribute access."""
def __init__(self, target):
object.__setattr__(self, "_target", target) # bypass our own __setattr__
def __getattr__(self, name):
# Called only when the proxy itself doesn't have 'name'
return getattr(object.__getattribute__(self, "_target"), name)
def __setattr__(self, name, value):
if name == "_target":
object.__setattr__(self, name, value)
else:
setattr(object.__getattribute__(self, "_target"), name, value)
class Config:
debug = False
version = "1.0"
proxy = Proxy(Config)
print(proxy.debug) # False — forwarded to Config.debug
proxy.debug = True # forwarded to Config.debug = True
print(Config.debug) # True — the real object was mutatedproxy_pattern.pyobject.__setattr__ trick. When you override
__setattr__ on a class, any assignment inside __init__ (including
self.x = ...) routes through your override. To store bootstrap attributes
(like _target in the proxy above) without triggering infinite recursion or
forwarding logic, call object.__setattr__(self, name, value) directly to bypass
the overridden version.
Lazy & Cached Descriptors
A common pattern is computing an expensive attribute once and caching it — the descriptor handles the first computation and then shadows itself with an instance value.
import time, functools
class lazy_property:
"""
Non-data descriptor: computes value on first access,
then stores it in instance __dict__ so subsequent accesses
bypass the descriptor entirely (instance __dict__ wins over non-data desc).
"""
def __init__(self, func):
self.func = func
self.attrname = None
self.__doc__ = func.__doc__
def __set_name__(self, owner, name):
self.attrname = name
def __get__(self, obj, objtype=None):
if obj is None:
return self
if self.attrname is None:
raise TypeError("lazy_property must be used as a class attribute")
val = obj.__dict__.get(self.attrname, _MISSING := object())
if val is _MISSING:
val = self.func(obj)
obj.__dict__[self.attrname] = val # cache in instance dict
return val
class DataProcessor:
def __init__(self, data: list[int]):
self._data = data
@lazy_property
def stats(self) -> dict:
print(" Computing stats (expensive)…")
time.sleep(0.01) # simulate work
return {
"mean": sum(self._data) / len(self._data),
"min": min(self._data),
"max": max(self._data),
}
dp = DataProcessor(list(range(1000)))
print(dp.stats) # computes and caches
print(dp.stats) # returns cached value — descriptor not called again
print("stats" in dp.__dict__) # True — stored in instance dict
lazy_property.py
functools.cached_property in the standard library —
it works exactly like the pattern above. Prefer it over rolling your own.
Note: cached_property is a non-data descriptor, so it can
be cleared by deleting the instance attribute:
del obj.stats removes the cached value, forcing recomputation on next access.
Descriptor-Based ORM Columns
Django model fields, SQLAlchemy's mapped_column, and attrs/dataclasses
all use the descriptor protocol to attach metadata and behaviour to class attributes.
Here is a simplified version showing the technique:
from typing import Any
import sqlite3
class Column:
"""Descriptor that represents a database column on a Model class."""
def __set_name__(self, owner, name: str) -> None:
self.name = name
# Register on the owner class so the metaclass can find all columns
if not hasattr(owner, "_columns"):
owner._columns = {}
owner._columns[name] = self
def __init__(self, col_type: str, *, primary_key: bool = False,
nullable: bool = True, default=None):
self.col_type = col_type
self.primary_key = primary_key
self.nullable = nullable
self.default = default
self.name = None # filled by __set_name__
def __get__(self, obj, objtype=None):
if obj is None:
return self
return obj.__dict__.get(self.name, self.default)
def __set__(self, obj, value) -> None:
if value is None and not self.nullable:
raise ValueError(f"Column '{self.name}' is NOT NULL")
obj.__dict__[self.name] = value
def sql_definition(self) -> str:
parts = [self.name, self.col_type]
if self.primary_key: parts.append("PRIMARY KEY AUTOINCREMENT")
if not self.nullable: parts.append("NOT NULL")
return " ".join(parts)
class Model:
"""Base class for ORM models — introspects Column descriptors."""
@classmethod
def create_table_sql(cls) -> str:
cols = ", ".join(
desc.sql_definition()
for desc in cls._columns.values()
)
return f"CREATE TABLE IF NOT EXISTS {cls.__name__} ({cols});"
def to_dict(self) -> dict:
return {name: getattr(self, name) for name in self._columns}
class User(Model):
id = Column("INTEGER", primary_key=True, nullable=False)
username = Column("TEXT", nullable=False)
email = Column("TEXT", nullable=False)
age = Column("INTEGER", nullable=True, default=None)
print(User.create_table_sql())
# CREATE TABLE IF NOT EXISTS User (id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
# username TEXT NOT NULL, email TEXT NOT NULL, age INTEGER);
u = User()
u.username = "alice"
u.email = "alice@example.com"
u.age = 30
print(u.to_dict())
# {'id': None, 'username': 'alice', 'email': 'alice@example.com', 'age': 30}
try:
u.username = None # NOT NULL violation
except ValueError as e:
print(e) # Column 'username' is NOT NULL
orm_columns.py
Complete Attribute Lookup Reference
Combining everything: the full priority order for obj.attr lookup,
the special cases, and how to hook into each layer.
| Priority | Source | Condition | How to hook / override |
|---|---|---|---|
| 1 (highest) | Data descriptor in type(obj).__mro__ |
Class attribute has both __get__ and __set__/__delete__ |
Define a data descriptor class; use property, __slots__ |
| 2 | obj.__dict__['attr'] |
Key exists in the instance dictionary | Direct obj.__dict__ access; __setattr__ |
| 3 | Non-data descriptor in type(obj).__mro__ |
Class attribute has __get__ but not __set__/__delete__ |
Regular functions, classmethod, staticmethod, cached_property |
| 4 | Plain class attribute in type(obj).__mro__ |
Any object in the class dict without __get__ |
Simple class-level assignment |
| 5 (fallback) | __getattr__ |
Only called when all above raise AttributeError |
Define __getattr__ for dynamic attributes, proxies |
| Override all | __getattribute__ |
Always called first; implements the above algorithm | Override with super().__getattribute__(name) to extend |
# Demonstration of all six levels in one class
class AllLevels:
# Level 1: data descriptor
data_desc = property(lambda self: "data_desc")
# Level 3: non-data descriptor (regular function)
def method(self): return "method"
# Level 4: plain class attribute
class_attr = "class_attr"
def __getattr__(self, name):
# Level 5: only called when levels 1-4 all fail
return f"__getattr__({name!r})"
obj = AllLevels()
obj.__dict__["data_desc"] = "instance (shadow attempt)" # level 2
obj.__dict__["method"] = "instance method shadow" # level 2
print(obj.data_desc) # "data_desc" — data desc wins over instance dict
print(obj.method) # "instance method shadow" — instance dict wins over non-data desc
print(obj.class_attr) # "class_attr" — falls through to class
print(obj.missing) # "__getattr__('missing')" — final fallback
all_levels.py
Best Practices
- Always implement
__set_name__in custom descriptors — it gives the descriptor its own name for free, enabling clear error messages and correct instance storage keys without extra boilerplate. - Store instance data in
obj.__dict__using a mangled key (e.g.f"_{owner.__name__}__{name}") — never store it on the descriptor itself, or all instances will share the same value. - Use
functools.cached_propertyfor expensive computed attributes — it is a standard-library non-data descriptor that caches in instance__dict__, avoiding repeated computation. - Prefer
__getattr__over__getattribute__for most use cases —__getattr__is only called when normal lookup fails, so it is easier to reason about and avoids infinite recursion traps. - When overriding
__getattribute__, always delegate tosuper()—object.__getattribute__implements the full descriptor/MRO lookup chain; replacing it entirely breaks everything. - Add
__weakref__to__slots__if instances may be stored inWeakValueDictionaryorWeakSet— otherwise weak-referencing them raisesTypeError. - Return
selffrom__get__whenobj is None— this is the class-level access pattern. Returning the descriptor object allowsMyClass.attrto return the descriptor for inspection. - Understand that
propertyis a data descriptor — it always wins over instance__dict__, which is whyobj.__dict__['x'] = 99cannot bypass a@property.
Exercises
Exercise 1 — Observable Descriptor
Build an Observable descriptor that calls registered callbacks
whenever the value changes:
- Implement
__get__,__set__,__set_name__. - On
__set__, call all registered callbacks with(instance, old_value, new_value). - Add a class-level
observe(instance, callback)method that registers a callback for a specific instance. - Test: create a
Sensorclass withtemperature = Observable(float); register a callback that logs changes; verify it fires on assignment. - Ensure multiple instances have independent callback lists.
💡 Hint
from collections import defaultdict
import weakref
class Observable:
def __set_name__(self, owner, name):
self.name = name
self.private = f"_{name}"
def __get__(self, obj, objtype=None):
if obj is None: return self
return getattr(obj, self.private, None)
def __set__(self, obj, value):
old = getattr(obj, self.private, None)
setattr(obj, self.private, value)
key = id(obj)
for cb in self._callbacks.get(key, []):
cb(obj, old, value)
def __init__(self, typ=None):
self.typ = typ
self._callbacks: dict[int, list] = defaultdict(list)
def observe(self, instance, callback):
self._callbacks[id(instance)].append(callback)
class Sensor:
temperature = Observable(float)
s = Sensor()
Sensor.temperature.observe(s, lambda inst, old, new: print(f"Changed: {old} → {new}"))
s.temperature = 22.5 # prints: Changed: None → 22.5
s.temperature = 25.0 # prints: Changed: 22.5 → 25.0
Exercise 2 — __slots__ Benchmark
Quantify the memory and speed benefit of __slots__ over __dict__:
- Create two identical classes —
PointDict(no slots) andPointSlots(with__slots__ = ("x", "y")). - Instantiate 1 000 000 of each; measure peak memory with
tracemalloc. - Benchmark attribute read speed with
timeit(readobj.x10 million times on each). - Print a summary table showing: object size, total memory for 1M objects, attribute read time.
- Inspect
PointSlots.__dict__["x"]to confirm it is amember_descriptor.
💡 Hint
import sys, tracemalloc, timeit
class PointDict:
def __init__(self, x, y): self.x, self.y = x, y
class PointSlots:
__slots__ = ("x", "y")
def __init__(self, x, y): self.x, self.y = x, y
for cls in (PointDict, PointSlots):
tracemalloc.start()
objs = [cls(i, i) for i in range(1_000_000)]
_, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
read_time = timeit.timeit(lambda: objs[0].x, number=10_000_000)
print(f"{cls.__name__:12s} size={sys.getsizeof(objs[0])}B "
f"peak={peak/1e6:.1f}MB read={read_time:.3f}s")
Exercise 3 — Proxy with Audit Trail
Build an AuditProxy that wraps any object and records every
attribute read, write, and delete as a timestamped log entry:
- Override
__getattr__,__setattr__, and__delattr__. - For writes: record
(timestamp, "set", attr, old_value, new_value). - For reads: record
(timestamp, "get", attr, value). - Expose a
get_audit_log()method that returns the full history. - Test with a
Configdataclass; verify the log captures all operations in order. - Ensure accesses to the proxy's own internals (
_target,_log) do NOT appear in the audit log.
💡 Hint
from datetime import datetime, timezone
class AuditProxy:
_INTERNAL = frozenset({"_target", "_log"})
def __init__(self, target):
object.__setattr__(self, "_target", target)
object.__setattr__(self, "_log", [])
def _record(self, entry):
object.__getattribute__(self, "_log").append(entry)
def get_audit_log(self):
return list(object.__getattribute__(self, "_log"))
def __getattr__(self, name):
target = object.__getattribute__(self, "_target")
value = getattr(target, name)
self._record((datetime.now(timezone.utc), "get", name, value))
return value
def __setattr__(self, name, value):
if name in AuditProxy._INTERNAL:
object.__setattr__(self, name, value)
return
target = object.__getattribute__(self, "_target")
old_val = getattr(target, name, "")
setattr(target, name, value)
self._record((datetime.now(timezone.utc), "set", name, old_val, value))
def __delattr__(self, name):
target = object.__getattribute__(self, "_target")
delattr(target, name)
self._record((datetime.now(timezone.utc), "del", name))
from dataclasses import dataclass
@dataclass
class Config:
debug: bool = False
version: str = "1.0"
proxy = AuditProxy(Config())
proxy.debug = True
proxy.version = "2.0"
_ = proxy.debug
for entry in proxy.get_audit_log():
print(entry)