Thursday, April 9, 2015

Higher-Order Mixin Classes, or "Traits, but not quite!"

This post talks about a technique I use to provide traits-like functionality in Python, by (ab)using higher-order mixin classes and computed properties. This technique can also be used to reduce code duplication when using computed properties.

Note that all of the example code below is Python 3, but it shouldn't be too hard to get it running in Python 2.

Computed Properties

Many languages support the idea of "computed properties" on objects: C#, Ruby, JavaScript, and, of course, Python.

Properties are like "dynamic attributes" of an object. They're accessed like normal attributes, but can have custom getter, setter, and deleter functions. By using these functions, the property's value can be dynamically computed when being retrieved, and arbitrarily modified and verified when being set.

For example, here's a Python class called Shape that describes a geometric shape to be drawn (it's the law that any article talking about object-oriented programming languages must use shapes as examples).

This class has a property called color that describes the color used to draw the object on the screen. This property is treated like a normal object attribute, that is, it can be set or retrieved via normal assignment or attribute access. However, we want the user to be able to specify colors "naturally", but there are several "natural" ways to specify colors.

We thus define the property "setter" to accept mutliple color specifications, either a string naming some color from a list of known colors, or an integer describing the RGB components of the color:

class Shape:
    _colors = {
        "black": 0x00000000,
        "white": 0x00ffffff,
        "red":   0x00ff0000,
        "green": 0x0000ff00,
        "blue":  0x000000ff
    }

    @property
    def color(self):
        # We use getattr here to return a default None value
        # if the color has yet to be specified.
        return getattr(self, "_color", None)

    @color.setter
    def color(self, value):
        if isinstance(value, int):
            # Did you know that bool is a subclass of int?
            if isinstance(value, bool):
                raise TypeError("integer colors must be ints")

            if value < 0 or value > 0x00ffffff:
                raise ValueError("integer colors must be 24-bit")

            self._color = value

        elif isinstance(value, str):
            if value not in self._colors:
                raise ValueError("unknown color '{}'".format(value))

            self._color = self._colors[value]

        else:
            raise TypeError("color specifications must be ints or strs")

We can then instantiate the class and treat color as a normal attribute of the instance:

s = Shape()
s.color = 'red'
print(p.color)

This results in 16711680, which is the value of red in our color dictionary. Also note that the color.setter functionality performs some validation on the value we assign:

s.color = 'cornflower blue'

raises a ValueException saying, rightly, that "cornflower blue" is an unknown color.

The property function can be used as above, in decorator form, or directly, as below:

class Shape2:
    _colors = {
        "black": 0x00000000,
        "white": 0x00ffffff,
        "red":   0x00ff0000,
        "green": 0x0000ff00,
        "blue":  0x000000ff
    }

    def _get_color(self):
        # We use getattr here to return a default None value
        # if the color has yet to be specified.
        return getattr(self, "_color", None)

    def _set_color(self, value):
        if isinstance(value, int):
            # Did you know that bool is a subclass of int?
            if isinstance(value, bool):
                raise TypeError("integer colors must be ints")

            if value < 0 or value > 0x00ffffff:
                raise ValueError("integer colors must be 24-bit")

            self._color = value

        elif isinstance(value, str):
            if value not in self._colors:
                raise ValueError("unknown color '{}'".format(value))

            self._color = self._colors[value]

        else:
            raise TypeError("color specifications must be ints or strs")

    color = property(_get_color, _set_color) # This is new.

The results in this case will be exactly the same, though I generally prefer the former.

Properties and Inheritance

Properties on base classes will be inherited as expected on subclasses:

class Shape3(Password):
    pass

s = Shape3()
s.color = "red"
print(s.color)

Will result, as expected, in 16711680, despite the fact that the color property was defined as part of the Shape class and not as part of Shape3.

However, there's a gotcha here:

The getter and setter methods of a property (and its deleter method, which I didn't show in the example above) are captured as part of the property defition, at that point in the object hierarchy. So, take this subclass definition, which would ostensibly allow mixed-case color names to be specificed:

# Notice that we're inheriting from Shape2, with its
# distinct getter and setter functions.
class Shape4(Shape2):
    def _set_color(self, value):
        super()._set_color(value.lower())

This does not work as expected:

p = Shape4()
p.color = 'red' # works
p.color = 'RED' # Fails

This raises a ValueError exception, because "RED" is not a color in our color dictionary, and the lookup function is still the original one as defined in Shape2, not the new, ostensibly overridden one in Shape4.

Multiple Properties

Now let's define a new class, called Canvas. This class lets us draw things using a foreground color on top of a background color. It would be nice if the class could have two properties, foreground and background that functioned like the color property of our Shape class.

Here's a first attempt:

class Canvas:
    _colors = {
        "black": 0x00000000,
        "white": 0x00ffffff,
        "red":   0x00ff0000,
        "green": 0x0000ff00,
        "blue":  0x000000ff
    }

    def _validate_color(self, value):
        # This is just the body of our color setter method,
        # but instead of setting the value, it returns it
        # or raises an exception.

        if isinstance(value, int):
            if isinstance(value, bool):
                raise TypeError("integer colors must be ints")

            if value < 0 or value > 0x00ffffff:
                raise ValueError("integer colors must be 24-bit")

            return value

        elif isinstance(value, str):
            if value not in self._colors:
                raise ValueError("unknown color '{}'".format(value))

            return self._colors[value]

        else:
            raise TypeError("color specifications must be ints or strs")


    @property
    def foreground(self):
        return getattr(self, "_foreground_color", None)

    @foreground.setter
    def foreground(self, value):
        self._foreground_color = self._validate_color(value)

    @property
    def background(self):
        return getattr(self, "_background_color", None)

    @background.setter
    def background(self, value):
        self._background_color = self._validate_color(value)

This works, and isn't too bad, but we can do a little better. Let's try again:

class Canvas:
    _colors = {
        "black": 0x00000000,
        "white": 0x00ffffff,
        "red":   0x00ff0000,
        "green": 0x0000ff00,
        "blue":  0x000000ff
    }

    def _validate_color(self, value):
        # This is just the body of our color setter method,
        # but instead of setting the value, it returns it
        # or raises an exception.

        if isinstance(value, int):
            if isinstance(value, bool):
                raise TypeError("integer colors must be ints")

            if value < 0 or value > 0x00ffffff:
                raise ValueError("integer colors must be 24-bit")

            return value

        elif isinstance(value, str):
            if value not in self._colors:
                raise ValueError("unknown color '{}'".format(value))

            return self._colors[value]

        else:
            raise TypeError("color specifications must be ints or strs")


    def _set_color(self, prop, value):
        setattr(self, prop, self._validate_color(value))

    def _get_color(self, prop):
        return getattr(self, prop)

    foreground = property(lambda self: self._get_color("_fg"),
                          lambda self, value: self._set_color("_fg", value))

    background = property(lambda self: self._get_color("_bg"),
                          lambda self, value: self._set_color("_bg", value))

This is slightly better, because we've genericized the color-setting functionality and can now define an arbitrary number of color properties in our classes fairly simply.

Colors Everywhere!

What happens, though, if we're designing a whole graphics system, where use of color is pervasive? We'll have Window objects with background, foreground, and border colors. There will be Button objects that will have background, foreground, border, and text colors.

Throughout this hypothetical class hierarchy there will be colors, and different classes will have different numbers and names of color properties. What should we do? We certainly don't want to have to redefine the _validate_color function everywhere.

We could create a mixin class, however. Every class that uses colors would inherit from its base class and from this mixin class, and then use the property function to bind them up:

class UsesColors: # Our mixin.
    def _set_color(self, value):
        # Copy the definition from Canvas above.
        pass

    def _get_color(self):
        # Copy the definition from Canvas above.
        pass

class GraphicsObject: # The base class of our hierarchy.
    pass

class Canvas(GraphicsObject):
    pass

class Button(Canvas, UsesColors):
    foreground = property(lambda self: self._getcolor("_fg"),
                          lambda self, value: self._set_color("_fg", value))

    background = property(lambda self: self._getcolor("_bg"),
                          lambda self, value: self._set_color("_bg", value))

    border     = property(lambda self: self._getcolor("_border"),
                          lambda self, value: self._set_color("_border", value))

    text       = property(lambda self: self._getcolor("_text"),
                          lambda self, value: self._set_color("_text", value))

This works, but it's not pretty. We're on to something, though, let's keep on pushing and see what we can come up with.

Colors as Traits

What we really want is the concept of "color" to be something that we can add to classes whenever we want, as many times as we want, and with arbitrary names. We want classes to have color traits, little bundles of functionality that we can compose together.

We already know that properties can be inherited and work more-or-less as expected. Continuing the example above, we can imagine that we want is for Canvas to inherit from some superclass that has color properties called foreground and background, and we want Button to inherit from a class that additionally has border and text properties.

So, here's the trick. Python allows arbitrary multiple inheritance. More importantly, class definitions are executed at runtime, and the list of base classes isn't just a static list - it's a list of expressions that evaluate to classes.

What we're going to do is define a function, HasColor, that takes the name of a color property and returns an anonymous class that in fact has a color property of that name. We can then do this bit of magic:

class Cavas(GraphicsObject,
            HasColor("foreground"),
            HasColor("background")):
    pass

class Button(Canvas, # Inherit Canvas's foreground and background.
             HasColor("border"),
             HasColor("text")):
    pass

c = Canvas()
c.foreground = 'red'

b = Button()
b.foreground = 'black'
b.text = 'green'

Much cleaner, right?

To not hold you in suspense any longer, here's the HasColor function, whose name is capitalized because it's going to work like a class. There's nothing particularly magical about it, it just works:

def HasColor(name, default=None, doc=None):
    """
    Return an anonymous class that has a color property
    named `name`, with a default value of `default` and
    a docstring of `doc`.
    """

    propname = "__color_{}".format(name)
    colors = {
        "black": 0x00000000,
        "white": 0x00ffffff,
        "red":   0x00ff0000,
        "green": 0x0000ff00,
        "blue":  0x000000ff
    }

    def getter(self):
        return getattr(self, propname, default)

    def setter(self, value):
        if isinstance(value, int):
            if isinstance(value, bool):
                raise TypeError("integer colors must be ints")

            if value < 0 or value > 0x00ffffff:
                raise ValueError("integer colors must be 24-bit")

            setattr(self, propname, value)

        elif isinstance(value, str):
            if value not in colors:
                raise ValueError("unknown color '{}'".format(value))

            setattr(self, propname, colors[value])

        else:
            raise TypeError("color specifications must be ints or strs")


    class Inner:
        pass

    setattr(Inner, name, property(getter, setter, None, doc))
    return Inner

See what we did? The function HasColor creates an anonymous class (referred to within the function as Inner), which encapsulates the concept of "color property". This anonymous class has a color property of the given name. We then return the class so that when we invoke it in some class's list of base classes, we get our color-propertied class.

Conclusion

This sort of metaprogramming is what makes working in Python a joy. Often times, normal object-oriented inheritance doesn't save us from having to duplicate code all over the place. Even multiple inheritance can't save us, because we run into problems where what we want to inherit isn't exactly expressable in a normal class. Python, on the other hand, gives us dynamic class creation at runtime, and lets us do cool things like this, almost effortlessly.

Stay tuned for the next post, where we'll tackle Dwemthy's Array in Python (which, I suppose, would make it Dwemthy's List...).

No comments:

Post a Comment