THE MAGIC OF METACLASSES - PART 2
===========================================================================
Metaclasses are so powerful that a single chapter is not enough to make
justice to them ;) In this second chapter on metaclasses I will
unravel their deepest secrets, covering topics such as meta-metaclasses,
anonymous inner metaclasses, global metaclasses and advanced class factories.
Moreover, I will give various magical applications of metaclasses,
in the realm of enhancing the Python language itself. Actually, this is
probably the most idiomatic application of metaclasses (Guido's examples
on the metaclass usage are all in this area). I will show
how metaclasses can be used to enhance the ``super`` cooperatice call
mechanism.
This is not a chapter for the faint of heart.
The secrets of the ``__metaclass__`` hook
------------------------------------------------------------------------
In the previous chapter we have seen how the ``__metaclass__`` hook can
be used as a way of metaclass enhancing pre-existing classes
with a minimal change of the sourcecode.
But it has much deeper secrets.
The first and simplest of them,
is the fact that the hook can be used it can also be defined
at the module level, *outside* the class. This allows a number of neat
tricks, since in presence of a ``__metaclass__`` hook at the module
level *all* the old style classes in the module (including nested ones!)
acquire that hook. A first application is to rejuvenate old style classes
to new style classes.
I remind that old style classes are retained with compability with old
code, but they are a pain in the back, if you want to use features
intended for new style classes only (for instance properties etc.).
Naively, one would expect the conversion from old style classes
to new style to be long and error prone: suppose you have a very large
application with hundreds of old style classes defined in dozens of modules.
Suppose you want to update your application to Python 2.2+ classes in order
to take advantage of the new features I have discussed extensively in this
book: the naive way to go would be to go trough the source, look for
all classes definitions and change
::
Classname: --> Classname(object)
One could solve this problem with a regular expression search and replace
in all modules, but this would require to change *all* the source.
This is againt the spirit of OOP, we must *reuse* old code.
Metaclasses are particularly handy to solve this problem: actually it is
enough to add to your modules the following line as first line:
::
__metaclass__ = type
Then, all your old style classes will have 'type' as their metaclass: this
is akin to say that all the old style classes are *automagically* rejuvenate
to new style classes! And this also works for *nested* classes!!
::
#
__metaclass__ = type # this rejuvanate all the class in the module
class C:
class D: pass
print dir(C) # both C and C.D
print dir(C.D) # are now new style classes
#
This very first example add consistence (if needed) to the
widespread belief that metaclasses have a well deserved reputation of magic.
The explanation is that defining a global metaclass called ``__metaclass__``
automatically makes all old style classes (new style class simply ignore
the existence of the global ``__metaclass__``) defined in you module
instances of the given metaclass; this automatically converts them to
new style classes.
Anonymous inner metaclasses
---------------------------------------------------------------------------
A second, deeper secret of the ``__metaclass__`` hook is that it can be
used to define anonymous *inner metaclasses*. The following example
explain what I mean:
::
#
def totuple(arg):
"Converts the argument to a tuple, if need there is"
if isinstance(arg,tuple): return arg # do nothing
else: return (arg,) # convert to tuple
class BracketCallable(object):
"""Any subclass C(BracketCallable) can be called with the syntax C[t],
where t is a tuple of arguments stored in bracket_args; returns the
class or an instance of it, depending on the flag 'returnclass'."""
returnclass=True
class __metaclass__(type): # anonymous inner metaclass
def __getitem__(cls,args): # non cooperative metamethod
if cls.returnclass:
c=type(cls.__name__,(cls,),{'bracket_args':totuple(args)})
return c # a customized copy of the original class
else:
self=cls(); self.bracket_args=totuple(args)
return self
#
In this code 'BracketCallable.__metaclass__' is the anonymous (actually
it has a special name, ``__metaclass__``) inner metaclass of 'BracketCallable'.
The effect of 'BracketCallable.__metaclass__' is the following: it makes
'BracketCallable' and its descendants callable with brackets. Since
the 'returnclass' flag is set, ``__getitem__`` returns the class
with an attribute 'bracket_args' containing the tuple of the passed
arguments (otherwise it returns an instance of the class).
This works since when Python encounters an expression of kind
``cls[arg]`` it interprets it as ``type(cls).__getitem__(cls,arg)``.
Therefore, if ``cls`` is a subclass of 'BracketCallable', this means that
::
cls[arg] <=> BracketCallable.__metaclass__.__getitem__(cls,arg)
Let me give few examples:
>>> from oopp import BracketCallable
>>> type(BracketCallable)
>>> print type(BracketCallable).__name__ # not really anonymous
__metaclass__
>>> print BracketCallable['a1'].bracket_args
('a1',)
>>> print BracketCallable['a1','a2'].bracket_args
('a1', 'a2')
This syntactical feature is an example of a thing that can be done
*trough metaclasses only*: it cannot be emulated by functions.
Anonymous inner metaclasses are the least verbose manner
of defining metamethods. Moreover, they are a neat trick to define
mix-in classes that, when inherited, can metamagically enhance
an entire multiple inheritance hierarchy.
In the previous example ``__getitem__`` is noncooperative, but nothing
forbids anonymous inner metaclasses from being made cooperative. However,
there is some subtlety one must be aware of.
Let me give an example. My 'WithCounter' class counts how many instances
of 'WithCounter' and its subclasses are generated. However, it does not
distinguishes bewteen different subclasses.
This was correct in the pizza shop example, simple only the total
number of produced pizzas mattered, however, in other situations,
one may want to reset the counter each time a new subclass is created.
This can be done automagically by a cooperative inner metaclass:
::
class WithMultiCounter(WithCounter):
"""Each time a new subclass is derived, the counter is reset"""
class __metaclass__(type):
def __init__(cls,*args):
cls.counter=0
super(cls.__this,cls).__init__(*args)
reflective(__metaclass__)
Notice that the order of execution of this code is subtle:
1) first, the fact that WithMulticounter has a non-trivial metaclass is
registered, but nothing else is done;
2) then, the line ``reflective(__metaclass__)`` is executed: this means
that the inner metaclass (and therefore its instances) get an
attribute ``._metaclass__this`` containing a reference to the
inner metaclass;
3) then, the outer class is passed to its inner metaclass and created
by the inherited metaclass' ``__new__`` method;
4) at this point ``cls`` exists and ``cls.__this`` is inherited from
``__metaclass__._metaclass__this``; this means that the expression
``super(cls.__this,cls).__init__(*args)`` is correctly recognized and
'WithMultiCounter' can be initialized;
5) only after that, the name 'WithMultiCounter' enters in the global
namespace and can be recognized.
Notice in particular that inside ``super``, we could also
use ``cls.__metaclass__`` instead of ``cls.__this``, but this
would not work inside ``__new__``, whereas ``__this`` would be
recognized even in ``__new__``. Moreover, ``cls.__metaclass__``
can be overridden in subclasses (they may have a submetaclass
of ``__metaclass__``) therefore the usage is risky.
Notice also that ``X.__class__`` can be different from
``X.__metaclass__`` if the metaclass ``__new__`` method
is overridden or in various other circumstances.
>>> from oopp import *
>>> print MRO(WithMultiCounter)
1 - WithMultiCounter(WithCounter)[__metaclass__]
2 - WithCounter(object)
3 - object()
For sake of readability, often it is convenient
to give a name even to inner classes:
::
#
class WithMultiCounter(WithCounter):
"""Each time a new subclass is derived, the counter is reset"""
class ResetsCounter(type):
def __init__(cls,*args):
cls.counter=0
super(cls.ResetsCounter,cls).__init__(*args)
__metaclass__=ResetsCounter
#
Notice that inside super we used the expression ``cls.ResetsCounter`` and
not ``WithMultiCounter.ResetsCounter``: doing that would generate a
``NameError: global name 'WithMultiCounter' is not defined`` since at the
time when ``ResetsCounter.__init__`` is called for the first time,
the class ``WithMultiCounter`` exists but is has not yet entered the global
namespace: this will happens only after the initialization in the
``ResetsCounter`` metaclass, as we said before.
Without the metaclass one can reset the counter by hand each time, or
can reset the counter on all the classes of the hierarchy with a
convenient function (akin to the 'traceH' routine defined in chapter 6).
Example:
>>> from oopp import *
>>> class GrandFather(WithMultiCounter): pass
>>> class Father(GrandFather): pass
>>> class Child(Father): pass
>>> GrandFather()
<__main__.GrandFather object at 0x402f7f6c> # first GrandFather instance
>>> Father()
<__main__.Father object at 0x402f79ec> # first Father instance
>>> Father()
<__main__.Father object at 0x402f7d4c> # second Father instance
>>> Child.counter # zero instances
0
>>> Father.counter # two instances
2
>>> GrandFather.counter # one instance
1
I leave as an exercise for the reader to show that the original 'WithCounter'
would fail to count correctly the different subclasses and would put the
total number of instances in 'Child'.
Passing parameters to (meta) classes
-------------------------------------------------------------------------
Calling a class with brackets is a way of passing parameters to it (or
to its instances, if the 'returnclass' flag is not set).
There additional ways for of doing that.
One can control the instantiation syntax of classes by redefining the
``__call__`` method of the metaclass. The point is that when we instantiate
an object with the syntax ``c=C()``, Python looks
at the ``__call__`` method of the metaclass of 'C'; the default behaviour
it is to call ``C.__new__`` and ``C.__init__`` in succession, however, that
behavior can be overridden. Let me give an example without using
anonymous metaclasses (for sake of clarity only).
::
#
class M(type): # this is C metaclass
def __call__(cls):
return "Called M.__call__"
C=M('C',(),{}) # calls type(M).__call__
c=C() # calls type(C).__call__
# attention: c is a string!
print c #=> Called M.__call__
#
In this example, ``M.__call__`` simply
returns the string ``Called M.__call__``, and the class
'C' is *not* instantiated. Overriding the metaclass
``__call__ `` method therefore provides another way to implement
the ``Singleton`` pattern. However, savage overridings as the one in
this example, are not a good idea, since it will confuse everybody.
This is an example where metaclasses change the semantics: whereas
usually the notation ``C()`` means "creates a C instance", the
metaclass can give to the syntax ``C()`` any meaning we want.
Here there is both the power and the danger of metaclasses: they
allows to make both miracles and disasters. Nevertheless, used with
a grain of salt, they provide a pretty nice convenience.
Anyway, overriding the '__call__' method of the metaclass can be
confusing, since parenthesis are usually reserved to mean instantion,
therefore I will prefere to pass arguments trough brackets.
The beauty and the magic of metaclasses stays in the fact that this mechanism
is completely general: since metaclasses themselves are classes, we can
'CallableWithBrackets' to pass arguments to a metaclass, i.e.
'CallableWithBrackets' can also be used as a meta-metaclass!
I leave as an exercise for the reader to figure out
how to define meta-meta-metaclasses, meta-meta-meta-metaclasses, etc.
etc. (there is no limit to the abstraction level you can reach with
metaclasses;-)
Let me show an example: a magical way of making methods cooperative.
This can be done trough a 'Cooperative' metaclass that inherits from
'BracketCallable' and therefore has 'BracketCallable.__metaclass__'
as (meta)metaclass:
::
#
class Cooperative(BracketCallable,type):
"""Bracket-callable metaclass implementing cooperative methods. Works
well for plain methods returning None, such as __init__"""
def __init__(cls,*args):
methods=cls.bracket_args
for meth in methods:
setattr(cls,meth,cls.coop_method(meth,vars(cls).get(meth)))
def coop_method(cls,name,method): # method can be None
"""Calls both the superclass method and the class method (if the
class has an explicit method). Implemented via a closure"""
def _(self,*args,**kw):
getattr(super(cls,self),name)(*args,**kw) # call the supermethod
if method: method(self,*args,**kw) # call the method
return _
#
The code above works for methods returing ``None``, such as ``__init__``.
Here I give a first example of application: a hierarchy where the ``__init__``
methods are automatically called (similar to automatic initialization
in Java).
::
#
from oopp import Cooperative
class B(object):
"""Cooperative base class; all its descendants will automagically
invoke their ancestors __init__ methods in chain."""
__metaclass__=Cooperative['__init__']
def __init__(self,*args,**kw):
print "This is B.__init__"
class C(B):
"Has not explicit __init__"
class D(C):
"""The metaclass makes D.__init__ to call C.__init__ and
therefore B.__init__"""
def __init__(self,*args,**kw):
print "This is D.__init__"
d=D()
print "The metaclass of B is",type(B)
print "The meta-metaclass of B is", type(type(B))
#
Output:
::
This is B.__init__
This is D.__init__
The metaclass of B is
The meta-metaclass of B is
A second example, is the following, an alternative way of
making the paleoanthropological hierarchy of chapter 4 cooperative:
::
#
from oopp import Cooperative,Homo
class HomoHabilis(Homo):
__metaclass__=Cooperative['can']
def can(self):
print " - make tools"
class HomoSapiens(HomoHabilis):
def can(self):
print " - make abstractions"
class HomoSapiensSapiens(HomoSapiens):
def can(self):
print " - make art"
HomoSapiensSapiens().can()
# Output:
# can:
# - make tools
# - make abstractions
# - make art
#
Metaclasses can be used to violate the old good rule "explicit is
better than implicit". Looking at the source code for 'HomoSapiens'
and 'HomoSapiensSapiens' one would never imagine the ``can`` is
somewhat special. That is why in the following I will prefer to
use the anonymous super call mechanism, which is explicit, instead
of the implicit cooperative mechanism.
Meta-functions
---------------------------------------------------------------------
The third and deepest secret of the ``__metaclass__`` hook is that, even if
it is typically used in conjunction with metaclasses, actually the hook
can refer to generic class factories callable with the signature
``(name,bases,dic)``. Let me show a few examples
where ``__metaclass__`` is a function or a generic callable object
instead of being a metaclass:
::
#
from oopp import kwdict
class Callable(object):
def __call__(self,name,bases,dic):
print name,bases,'\n',kwdict(dic)
return type(name,bases,dic)
callableobj=Callable()
class C: __metaclass__=callableobj
print "type of C:",C.__class__
def f(name,bases,dic):
print name,bases,'\n',kwdict(dic)
return type(name,bases,dic)
class D: __metaclass__=f
print "type of D:",D.__class__
class B(object):
def __metaclass__(name,bases,dic):
"""In this form, the __metaclass__ attribute is a function.
In practice, it works as a special static method analogous
to __new__"""
print "name: ", name
print "bases:", bases
print "dic:\n",kwdict(dic)
return type(name,bases,dic)
class E(B): pass
print "type of E:",E.__class__
print "Non-called E.__metaclass__:", E.__metaclass__
#
With output
::
C ()
__metaclass__ =
__module__ = __builtin__
type of C:
D ()
__metaclass__ =
__module__ = __builtin__
type of D:
name: B
bases: (,)
dic:
__metaclass__ =
__module__ = __builtin__
type of E:
Non-called E.__metaclass__:
The advantage/disadvantage of this solution is that the ``__metaclass__``
hook is called only once, i.e. it is not called again if a new class
is derived from the original one. For instance in this example 'E' is
derived from 'B', but the function ``B.__metaclass__`` is *not* called
during the creation of 'E'.
Metafunctions can also be used when one does not want to transmit the
metaclass contraint. Therefore they usage is convenient in exactly
the opposite situation of a cooperative metaclass.
Anonymous cooperative super calls
-----------------------------------------------------------------------
As I noticed in the previous chapters, the ``super``
mechanism has an annoying
problem: one needs to pass explicitely the name of the base class. Typically,
this is simply an
inelegance since it is annoying to be forced to retype the name of the base
class. However, in particular
cases, it can be a problem. This happens for instance if we try to
pass the class's methods to a different class: one cannot do that,
since the methods contains an explicit reference to the original class
and would not work with the new one. Moreover, having named super calls
is annoying in view of refactoring. Consider for
instance the previous ``supernew.py`` script: in the ``__new__`` method
defined inside the class 'B', we called ``Super`` with the syntax
``Super(B,cls)`` by repeating the name of the class 'B'. Now,
if in the following I decide to give to 'B' a more descriptive
name, I have to go trough the source, search all the ``super``
calls, and change them accordingly to the new name. It would be
nice having Python do the job for me. A first solution is to call
``super`` (or ``Super``) with the syntax ``super(self.__this,obj)``,
where the special name ``__this`` is explicitly replaced by the name
of the class where the call is defined by the 'reflective' function
of last chapter. This approach has the disadvantage that each time we
derive a new class, we need to invoke *explicitely* the routine
``reflective``. It would be marvelous to instruct Python to invoke
``reflective`` automatically at each class creation. Actually, this
seems to be deep magic and indeed it is: fortunately, a custom metaclass
can perform this deep magic in few lines:
::
#
class Reflective(type):
"""Cooperative metaclass that defines the private variable __this in
its instances. __this contains a reference to the class, therefore
it allows anonymous cooperative super calls in the class."""
def __init__(cls,*args):
super(Reflective,cls).__init__(*args)
reflective(cls)
#
Now, let me show how 'Reflective' can be used in a practical example.
By deriving new metaclasses from 'Reflective', one can easily
create powerful class factories that generate reflective classes.
Suppose I want to define a handy class
factory with the abilitity of counting the number of its instances.
This can be done by noticing that metaclasses are just classes, therefore
they can be composed with regular classes in multiple inheritance. In
particular one can derive a 'Logged' metaclass from 'WithLogger': in
this way we send a message to a log file each time a new class is created.
This can be done by composing 'WithLogger' with
'WithMultiCounter.__metaclass__' and with 'Reflective':
::
#
class Logged(WithLogger,Reflective):
"""Metaclass that reuses the features provided by WithLogger.
In particular the classes created by Logged are Reflective,
PrettyPrinted and Customizable."""
#WithLogger provides logfile and verboselog
def __init__(cls,*args,**kw):
super(Logged,cls).__init__(*args,**kw)
bases=','.join([c.__name__ for c in cls.__bases__])
print >> cls.logfile, "%s is a child of %s" % (cls,bases)
print >> cls.logfile,'and an instance of %s' % type(cls).__name__
#
The MRO is
>>> print MRO(Logged)
MRO of Logged:
0 - Logged(WithLogger,Reflective)
1 - WithLogger(WithCounter,PrettyPrinted)
2 - WithCounter(object)
3 - PrettyPrinted(object)
4 - Reflective(type)
5 - type(object)
6 - object()
and the inheritance graph can be drawn as follows:
::
_____________________ object 6 ___
/ / \
2 WithCounter 3 PrettyPrinted type 5
\ / /
\ / /
\ / /
\ / /
\ / /
\ / /
1 WithLogger Reflective 4
\ /
\ /
\ /
\ /
Logged 0
:
:
C1
'WithCounter' acts now as a metaclass, since WithCounter.__new__ invokes
type.__new__. Since ``type.__new__`` is non-cooperative,
in the composition of a metaclass with a regular class, the metaclass
should be put first: this guarantees that ``__new__`` derives from
``type.__new__``, thus avoiding the error message.
>>> Logged.verboselog=True
>>> C1=Logged('C1',(),{})
*****************************************************************************
Tue Apr 22 18:47:05 2003
1. Created 'C1'
with accessibile non-special attributes:
_C1__this = 'C1'
'C1' is a child of object
and an instance of Logged
Notice that any instance of 'WithCounterReflective' inherits the 'WithCounter'
attribute ``counter``, that counts the number of classes that have been
instantiated (however it is not retrieved by ``dir``; moreover the
instances of 'WithCounterReflective' instances have no ``counter`` attribute).
>>> C1.counter
1
More on metaclasses as class factories
----------------------------------------------------------------------------
A slight disadvantage of the approach just described,
is that 'Logged' cooperatively invokes the ``type.__new__``
static method, therefore, when we invoke the metaclass, we must explicitly
provide a name, a tuple of base classes and a dictionary, since the
``type.__new__`` staticmethod requires that signature. Actually,
the expression
::
C=Logged(name,bases,dic)
is roughly syntactic sugar for
::
C=Logged.__new__(Logged,name,bases,dic)
assert isinstance(C,Logged)
Logged.__init__(C,name,bases,dic)
If a different interface is desired, the best way is to use a class
factory 'ClsFactory' analogous to the object factory 'Makeobj'
defined in chapter 4. It is convenient to make 'ClsFactory'
bracket-callable.
::
#
class ClsFactory(BracketCallable):
"""Bracket callable non-cooperative class acting as
a factory of class factories.
ClsFactory instances are class factories accepting 0,1,2 or 3 arguments.
. They automatically converts functions to static methods
if the input object is not a class. If an explicit name is not passed
the name of the created class is obtained by adding an underscore to
the name of the original object."""
returnclass=False # ClsFactory[X] returns an *instance* of ClsFactory
def __call__(self, *args):
"""Generates a new class using self.meta and avoiding conflicts.
The first metaobject can be a dictionary, an object with a
dictionary (except a class), or a simple name."""
# default attributes
self.name="CreatedWithClsFactory"
self.bases=()
self.dic={}
self.metas=self.bracket_args
if len(args)==1:
arg=args[0]
if isinstance(arg,str): # is a name
self.name=arg
elif hasattr(arg,'__name__'): # has a name
self.name=arg.__name__+'_'
self.setbasesdic(arg)
elif len(args)==2:
self.name=args[0]
assert isinstance(self.name,str) # must be a name
self.setbasesdic(args[1])
elif len(args)==3: # must be name,bases,dic
self.name=args[0]
self.bases+=args[1]
self.dic.update(args[2])
if len(args)<3 and not self.bases: # creating class from a non-class
for k,v in self.dic.iteritems():
if isfunction(v): self.dic[k]=staticmethod(v)
#return child(*self.bases,**vars(self))
return makecls(*self.metas)(self.name,self.bases,self.dic)
def setbasesdic(self,obj):
if isinstance(obj,tuple): # is a tuple
self.bases+=obj
elif hasattr(obj,'__bases__'): # is a class
self.bases+=obj.__bases__
if isinstance(obj,dict): # is a dict
self.dic.update(obj)
elif hasattr(obj,"__dict__"): # has a dict
self.dic.update(obj.__dict__)
#
'ClsFactory[X]' where 'X' is a metaclass returns callable objects acting as
class factories. For instance
::
#
Class=ClsFactory[type] # generates non-conflicting classes
Mixin=ClsFactory[Reflective] # generates reflective classes
#
can be used as a class factories that automatically provides a default name,
base classes and dictionary, and avoids meta-type conflicts.
'Mixin' generates reflective classes that can be used as mixin in multiple
inheritance hierarchies. Here I give few example of usage of 'Class':
>>> from oopp import *
>>> C1,C2,C3=[Class('C'+str(i+1)) for i in range(3)]
>>> C1
>>> C2
>>> C3
]
>>> Clock=Class('Clock',{'get_time':get_time})
>>> Clock
>>> Clock.get_time()
16:01:02
Another typical usage of 'Class' is the conversion of a module in a class:
for instance
>>> time_=Class(time)
>>> time_
Notice the convention of adding an underscore to the name of the class
generated from the 'time' module.
>>> time_.asctime()
'Mon Jan 20 16:33:21 2003'
Notice that all the functions in the module ``time`` has been magically
converted in staticmethods of the class ``time_``. An advantage of this
approach is that now the module is a class and can be enhanced with
metaclasses: for instance we could add tracing capabilities, debugging
features, etc.
By design, 'Class' and 'Reflective' also works when the first argument
is a class or a tuple of base classes:
>>> ClsFactory_=Class(ClsFactory)
>>> type(ClsFactory_)
>>> ClsFactory_=Mixin(ClsFactory)
>>> type(ClsFactory_) # automagically generated metaclass
Programming with metaclasses
--------------------------------------------------------------------------
In order to how a non-trivial application of metaclasses in real life,
let me come back to the pizza shop example discussed in chapter 4 and 6.
::
#
def Pizza(toppings,**dic):
"""This function produces classes inheriting from GenericPizza and
WithLogger, using a metaclass inferred from Logged"""
toppinglist=toppings.split()
name='Pizza'+''.join([n.capitalize() for n in toppinglist])
dic['toppinglist']=toppinglist
return ClsFactory[Logged](name,
(GenericPizza,WithLogger,WithMultiCounter),dic)
#
>>> from oopp import *
>>> Margherita=Pizza('tomato mozzarella',verboselog=True)
*****************************************************************************
Tue May 13 14:42:17 2003
1. Created 'PizzaTomatoMozzarella'
with accessibile non-special attributes:
ResetsCounter =
_GenericPizza__this =
_WithCounter__this =
_WithLogger__this =
baseprice = 1
counter = 0
formatstring = %s
logfile = ', mode 'w' at 0x402c2058>
price =
sizefactor = {'small': 1, 'large': 3, 'medium': 2}
topping_unit_price = 0.5
toppinglist = ['tomato', 'mozzarella']
toppings_price =
verboselog = True
'PizzaTomatoMozzarella' is a child of GenericPizza,WithLogger,
WithMultiCounter and an instance of _LoggedResetsCounter
Notice the *deep* magic: ``Pizza`` invokes ``ClsFactory[Logged]`` which in
turns calls the class factory ``child`` that creates 'Margherita' from
'GenericPizza', 'WithLogger' and 'WithMultiCounter' by using the
metaclass 'Logged': however, since 'WithMultiCounter', has the internal
metaclass 'ResetsCounter' , there is a metatype conflict:
``child`` *automagically* solves the conflict by creating the metaclass
'_LoggedResetsCounter' that inherits both from 'Logged' and 'ResetsCounter'.
At this point, 'Margherita' can be safely created
by '_LoggedResetsCounter'. As such, the creation of 'Margherita'
will be registered in the log file and 'Margherita' (with all its
children) will continue to be able to recognize the special identifier
``this``.
>>> print Margherita('large')
*****************************************************************************
Tue May 13 14:47:03 2003
1. Created large pizza with tomato,mozzarella, cost $ 6.0
with accessibile non-special attributes:
ResetsCounter =
_GenericPizza__this =
_WithCounter__this =
_WithLogger__this =
baseprice = 1
counter = 1
formatstring = %s
logfile = ', mode 'w' at 0x402c2058>
price = >
size = large
sizefactor = {'small': 1, 'large': 3, 'medium': 2}
topping_unit_price = 0.5
toppinglist = ['tomato', 'mozzarella']
toppings_price = >
verboselog = True
large pizza with tomato,mozzarella, cost $ 6.0
>>> print MRO(Margherita)
MRO of PizzaTomatoMozzarella:
0 - PizzaTomatoMozzarella(GenericPizza,WithLogger)[_LoggedResetsCounter]
1 - GenericPizza(object)
2 - WithLogger(WithCounter,Customizable,PrettyPrinted)
3 - WithMultiCounter(WithCounter)[ResetsCounter]
4 - WithCounter(object)
5 - PrettyPrinted(object)
6 - object()
Notice that
>>> print Margherita
'PizzaTomatoMozzarella'
The power of inheritance in this example is quite impressive, since
I have reused the same class 'WithLogger' (and its children) both in the
metaclass hierarchy and in the regular hierarchy: this means that I have added
logging capabilities both to classes and their instances in a
strike! And there is no confusion between the two. For instance,
there is a ``counter`` attribute for the metaclass 'Logged'
and many independent ``counter`` attributes for any generated class,
i.e. for any kind of pizza.
It is interesting to notice that '' itself is an instance of
its inner metaclass, as ``type()`` would show. This technique
avoids the need for inventing a new name for the metaclass. The inner
metaclass is automatically inherited by classes inheriting from the outer
class.
Metaclass-aided operator overloading
---------------------------------------------------------------------------
As we discussed in chapter 4, inheriting from built-in types is generally
painful. The problem is that if P is a primitive class, i.e. a
Python built-in type, and D=D(P) is a derived class, then the
primitive methods returning P-objects have to be modified (wrapped) in
such a way to return D-objects.
The problem is expecially clear in the context of operator overloading.
Consider for instance the problem of defining a 'Vector' class in the
mathematical sense. Mathematically-speaking, vectors are defined as objects
that can be summed each other and multiplied by numbers; they can be
represented by (finite or infinite) sequences. In the case of finite
sequences, vectors can be represented with lists and a vector class can
be naturally implemented by subclassing ``list``:
::
#
class Vector(list):
"""Implements finite dimensional vectors as lists. Can be instantiated
as Vector([a,b,c,..]) or as Vector(a,b,c ..)"""
def __add__(self,other):
return [el+other[i] for i,el in enumerate(self)]
__radd__=__add__
def __mul__(self,scalar):
return [el*scalar for el in self]
def __rmul__(self,scalar):
return [scalar*el for el in self]
v=Vector([1,0])
w=Vector([0,1])
print v+w, type(v+w)
print 2*v, type(2*v)
print v*2, type(v*2)
#
With output
::
[1, 1]
[2, 0]
[2, 0]
The problem is that the overloaded methods must be wrapped in such a way
to return ``Vector`` object and not ``list`` object; moreover, if
``Vector`` is subclassed (for instance by defining a ``NumericVector``),
the overloaded methods must return instances of the subclass. There is
only one way of doing that automatically: trough the magic of metaclasses.
Here is the solution, involving an ``autowrappedmethod`` descriptor class,
that wraps the overloaded operators and is automatically invoked by
the metaclass ``AutoWrapped``.
::
#
class autowrappedmethod(wrappedmethod):
"""Makes the method returning cls instances, by wrapping its
output with cls"""
klass=None # has to be fixed dynamically from outside
def __init__(self,meth):
super(autowrappedmethod,self).__init__(meth) # cooperative
self.klass=self.klass # class variable -> instance variable
def wrapper(self): # closure
return lambda *args,**kw: self.klass(self.func(*args,**kw))
class AutoWrapped(type):
"""Metaclass that looks at the methods declared in the attributes
builtinlist and wraplist of its instances and wraps them with
autowrappedmethod."""
def __init__(cls,name,bases,dic):
super(AutoWrapped,cls).__init__(name,bases,dic) # cooperative
cls.builtinlist=getattr(cls,'builtinlist',[])
if not hasattr(cls,'diclist') : # true only at the first call
cls.diclist=[(a,vars(bases[0])[a]) for a in cls.builtinlist]
if dic.has_key('wraplist'): # can be true at any call
cls.diclist+=[(a,dic[a]) for a in cls.wraplist]
wrapper=autowrappedmethod.With(klass=cls)
d=dict([(a,wrapper(v)) for a,v in cls.diclist])
customize(cls,**d)
#
Now the ``Vector`` class can be written as
::
#
class Vector(list):
"""Implements finite dimensional vectors as lists. Can be instantiated
as Vector([a,b,c,..]) or as Vector(a,b,c ..)"""
__metaclass__=AutoWrapped
wraplist='__add__ __radd__ __mul__ __rmul__'.split()
def __add__(self,other):
return [el+other[i] for i,el in enumerate(self)]
__radd__=__add__
def __mul__(self,scalar):
return [scalar*el for el in self]
def __rmul__(self,scalar):
return [el*scalar for el in self]
#
Here the ``AutoWrapped`` metaclass wraps the output of ``__add__,
__radd__, __mul__, __rmul__``, guaranteeing that they returns ``Vector``
instances or instances of some subclass of ``Vector``, if ``Vector`` is
subclassed. This is an example of usage:
.. doctest
>>> from oopp import Vector
>>> v=Vector([1,0])
>>> v
>>> w=Vector([0,1])
>>> v+2*w
>>> print v+2*w
[1, 2]
It should be clear by now that metaclasses are the natural framework where
to discuss operator overloading
(at least in languages that have metaclasses ;-). After all, operator
overloading is another kind of (very nice) syntactic sugar and we know
already that metaclasses are very good when we need syntactic sugar.