diff options
Diffstat (limited to 'pypers/meta.txt')
| -rwxr-xr-x | pypers/meta.txt | 1024 |
1 files changed, 1024 insertions, 0 deletions
diff --git a/pypers/meta.txt b/pypers/meta.txt new file mode 100755 index 0000000..31adcf4 --- /dev/null +++ b/pypers/meta.txt @@ -0,0 +1,1024 @@ +THE MAGIC OF METACLASSES - PART I +========================================================================== + + .. line-block:: + + *Metaclasses are deeper magic than 99% of users should ever + worry about. If you wonder whether you need them, you don't + (the people who actually need them know with certainty that + they need them, and don't need an explanation about why).* + --Tim Peters + +Python always had metaclasses, since they are inherent to its object +model. However, before Python 2.2, metaclasses where tricky and their +study could cause the programmer's brain to explode [#]_. Nowadays, +the situation has changed, and the reader should be able to understand +this chapter without risk for his/her brain (however I do not give any +warranty ;) + +Put it shortly, metaclasses give to the Python programmer +complete control on the creation of classes. This simple statement +has far reaching consequences, since the ability of interfering with +the process of class creation, enable the programmer to make miracles. + +In this and in the following chapters, I will show some of these +miracles. + +This chapter will focus on subtle problems of metaclasses in inheritance +and multiple inheritance, including multiple inheritance of metaclasses +with classes and metaclasses with metaclasses. + +The next chapter will focus more on applications. + + +.. [#] Metaclasses in Python 1.5 [A.k.a the killer joke] + http://www.python.org/doc/essays/metaclasses/ + +There is very little documentation about metaclasses, except Guido's +essays and the papers by David Mertz and myself published in IBMdeveloperWorks + + http://www-106.ibm.com/developerworks/library/l-pymeta.html + +Metaclasses as class factories +------------------------------------------------------------------------ + +In the Python object model (inspired from the Smalltalk, that had metaclasses +a quarter of century ago!) classes themselves are objects. +Now, since objects are instances of classes, that means that classes +themselves can be seen as instances of special classes called *metaclasses*. +Notice that things get hairy soon, since by following this idea, one could +say the metaclasses themselves are classes and therefore objects; that +would mean than even metaclasses can be seen as +instances of special classes called meta-metaclasses. On the other hand, +meta-meta-classes can be seen as instances of meta-meta-metaclasses, +etc. Now, it should be obvious why metaclasses have gained such a +reputation of brain-exploders ;). However, fortunately, the situation +is not so bad in practice, since the infinite recursion of metaclasses is +avoided because there is a metaclass that is the "mother of all metaclasses": +the built-in metaclass *type*. 'type' has the property of being its own +metaclass, therefore the recursion stops. Consider for instance the following +example: + + >>> class C(object): pass # a generic class + >>> type(C) #gives the metaclass of C + <type 'type'> + >>> type(type(C)) #gives the metaclass of type + <type 'type'> + +The recursion stops, since the metaclass of 'type' is 'type'. +One cool consequence of classes being instances of 'type', +is that since *type* is a subclass of object, + + >>> issubclass(type,object) + True + +any Python class is not only a subclass of ``object``, but also +an instance of 'object': + + >>> isinstance(C,type) + True + >>> isinstance(C,object) + True + >>> issubclass(C,object) + True + +Notice that 'type' is an instance of itself (!) and therefore of 'object': + + >>> isinstance(type,type) # 'type' is an instance of 'type' + True + >>> isinstance(type,object) # therefore 'type' is an instance of 'object' + True + +As it is well known, ``type(X)`` returns the type of ``X``; however, +``type`` has also a second form in which it acts as a class factory. +The form is ``type(name,bases,dic)`` where ``name`` is the name of +the new class to be created, bases is the tuple of its bases and dic +is the class dictionary. Let me give a few examples: + + >>> C=type('C',(),{}) + >>> C + <class '__main__.C'> + >>> C.__name__ + 'C' + >>> C.__bases__ + (<type 'object'>,) + >>> C.__dict__ + <dict-proxy object at 0x8109054> + +Notice that since all metaclasses inherits from ``type``, as a consequences +all metaclasses can be used as class factories. + +A fairy tale example will help in understanding the concept +and few subtle points on how attributes are transmitted from metaclasses +to their instances. + +Let me start by defining a 'Nobility' metaclass : + + >>> class Nobility(type): attributes="Power,Richness,Beauty" + +instances of 'Nobility' are classes such 'Princes', 'Dukes', 'Barons', etc. + + >>> Prince=Nobility("Prince",(),{}) + +Instances of 'Nobility' inherits its attributes, just as instances of normal +classes inherits the class docstring: + + >>> Prince.attributes + 'Power,Richness,Beauty' + +Nevertheless, 'attributes' will not be retrieved by the ``dir`` function: + + >>> print dir(Prince) + ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', + '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__repr__', + '__setattr__', '__str__', '__weakref__'] + +However, this is a limitation of ``dir``, in reality ``Prince.attributes`` +is there. On the other hand, the situation is different for a specific +'Prince' object + + >>> charles=Prince() + >>> charles.attributes #error + Traceback (most recent call last): + File "<stdin>", line 1, in ? + AttributeError: 'Prince' object has no attribute 'attributes' + +The transmission of metaclass attributes is not transitive: +instances of the metaclass inherits the attributes, but not the instances +of the instances. This behavior is by design and is needed in order to avoid +troubles with special methods. This point will be throughly +explained in the last paragraph. For the moment, I my notice that the +behaviour is reasonable, since the abstract qualities 'Power,Richness,Beauty' +are more qualities of the 'Prince' class than of one specific representative. +They can always be retrieved via the ``__class__`` attribute: + + >>> charles.__class__.attributes + 'Power,Richness,Beauty' + +Le me now define a metaclass 'Froggyness': + + >>> class Frogginess(type): attributes="Powerlessness,Poverty,Uglyness" + +Instances of 'Frogginess' are classes like 'Frog', 'Toad', etc. + + >>> Frog=Frogginess("Frog",(),{}) + >>> Frog.attributes + 'Powerlessness,Poverty,Uglyness' + +However, in Python miracles can happen: + + >>> def miracle(Frog): Frog.__class__=Nobility + >>> miracle(Frog); Frog.attributes + 'Powerlessness,Richness,Beauty' + +In this example a miracle happened on the class 'Frog', by changing its +(meta)class to 'Nobility'; therefore its attributes have changed accordingly. + +However, there is subtle point here. Suppose we explicitly specify the 'Frog' +attributes, in such a way that it can be inherited by one of its specific +representative: + + >>> Frog.attributes="poor, small, ugly" + >>> jack=Frog(); jack.attributes + 'poor, small, ugly' + +Then the miracle cannot work: + + :: + + #<fairytale2.py> + + class Nobility(type): attributes="Power, Richness, Beauty" + Prince=Nobility("Prince",(),{}) + charles=Prince() + + class Frogginess(type): attributes="Inpuissance, Poverty, Uglyness" + Frog=Frogginess("Frog",(),{}) + Frog.attributes="poor, small, ugly" + jack=Frog() + + def miracle(Frog): Frog.__class__=Nobility + + miracle(Frog) + + print "I am",Frog.attributes,"even if my class is",Frog.__class__ + + #</fairytale2.py> + +Output: + + :: + + I am poor, small, ugly even if my class is <class '__main__.Nobility'> + +The reason is that Python first looks at specific attributes of an object +(in this case the object is the class 'Frog') an only if they are not found, +it looks at the attributes of its class (here the metaclass 'Nobility').Since +in this example the 'Frog' class has explicit attributes, the +result is ``poor, small, ugly``. If you think a bit, it makes sense. + +Remark: + +In Python 2.3 there are restrictions when changing the ``__class__`` +attribute for classes: + + >>> C=type('C',(),{}) + >>> C.__class__ = Nobility #error + Traceback (most recent call last): + File "<stdin>", line 1, in ? + TypeError: __class__ assignment: only for heap types + +Here changing ``C.__class__`` is not allowed, since 'C' is an instance +of the built-in metaclass 'type'. This restriction, i.e. the fact that +the built-in metaclass cannot be changed, has been imposed for +security reasons, in order to avoid dirty tricks with the built-in +classes. For instance, if it was possible to change the metaclass +of the 'bool' class, we could arbitrarily change the behavior of +boolean objects. This could led to abuses. +Thanks to this restriction, +the programmer is always sure that built-in classes behaves as documented. +This is also the reason why 'bool' cannot be subclassed: + + >>> print bool.__doc__ # in Python 2.2 would give an error + bool(x) -> bool + Returns True when the argument x is true, False otherwise. + The builtins True and False are the only two instances of the class bool. + The class bool is a subclass of the class int, and cannot be subclassed. + +In any case, changing the class of a class is not a good idea, since it +does not play well with inheritance, i.e. changing the metaclass of a base +class does not change the metaclass of its children: + + >>> class M1(type): f=lambda cls: 'M1.f' #metaclass1 + >>> class M2(type): f=lambda cls: 'M2.f' #metaclass2 + >>> B=M1('B',(),{}) # B receives M1.f + >>> class C(B): pass #C receives M1.f + >>> B.f() + 'M1.f' + B.__class__=M2 #change the metaclass + >>> B.f() #B receives M2.f + 'M2.f' + C.f() #however C does *not* receive M2.f + >>> C.f() + 'M1.f' + >>> type(B) + <class '__main__.M2'> + >>> type(C) + <class '__main__.M1'> + +Metaclasses as class modifiers +---------------------------------------------------------------------- + +The interpretation of metaclasses in terms of class factories is quite +straightforward and I am sure that any Pythonista will be at home +with the concept. However, metaclasses have such a reputation of black +magic since their typical usage is *not* as class factories, but as +*class modifiers*. This means that metaclasses are typically +used to modify *in fieri* classes. The trouble is that the +modification can be utterly magical. +Here there is another fairy tale example showing the syntax +(via the ``__metaclass__`` hook) and the magic of the game: + + :: + + #<oopp.py> + + class UglyDuckling(PrettyPrinted): + "A plain, regular class" + formatstring="Not beautiful, I am %s" + + class MagicallyTransformed(type): + "Metaclass changing the formatstring of its instances" + def __init__(cls,*args): + cls.formatstring="Very beautiful, since I am %s" + + class TransformedUglyDuckling(PrettyPrinted): + "A class metamagically modified" + __metaclass__ = MagicallyTransformed + formatstring="Not beautiful, I am %s" # will be changed + + #</oopp.py> + + >>> from oopp import * + >>> print UglyDuckling() + Not beautiful, I am <UglyDuckling> + +In this example, even if in 'TransformedUglyDuckling' we explicitely +set the formatstring to "Not beautiful, I am %s", the metaclass changes +it to "Very beautiful, even if I am %s" and thus + + >>> print TransformedUglyDuckling() # gives + Very beautiful, since I am <TransformedUglyDuckling> + +Notice that the ``__metaclass__`` hook passes to the metaclass +``MagicallyTransformed`` the name, bases and dictionary of the class +being created, i.e. 'TransformedUglyDucking'. + +Metaclasses, when used as class modifiers, act *differently* +from functions, when inheritance is +involved. To clarify this subtle point, consider a subclass 'Swan' +of 'UglyDuckling': + + + >>> from oopp import * + >>> class Swan(UglyDuckling): + ... formatstring="Very beautiful, I am %s" + >>> print Swan() + Very beautiful, I am <Swan> + +Now, let me define a simple function acting as a class modifier: + + >>> def magicallyTransform(cls): + ... "Modifies the class formatstring" + ... customize(cls,formatstring="Very beautiful, even if I am %s") + ... return cls + +The function works: + + >>> magicallyTransform(UglyDuckling) + >>> print UglyDuckling() + Very beautiful, even if I am <UglyDuckling> + +This approach is destructive, since we cannot have the original +and the transformed class at the same time, and has potentially bad side +effects in the derived classes. Nevertheless, in this case it works +and it is not dangereous for the derived class 'Swan', since 'Swan' +explicitly overrides the 'formatstring' attribute and doesn't care about +the change in 'UglyDuckling.formatstring'. Therefore the output +of + + >>> print Swan() + Very beautiful, I am <Swan> + +is still the same as before the action of the function ``magicallyTransform``. +The situation is quite different if we use the 'MagicallyTransformed' +metaclass: + + >>> from oopp import * + >>> class Swan(TransformedUglyDuckling): + ... formatstring="Very beautiful, I am %s" + + >>> print TransformedUglyDuckling() + Very beautiful, since I am <UglyDuckling> + >>> print Swan() # does *not* print "Very beautiful, I am <Swan>" + Very beautiful, since I am <Swan> + +Therefore, not only the metaclass has magically transformed the +'TransformedUglyDuckling.formatstring', it has also transformed the +'Swan.formatstring'! And that, despite the fact that +'Swan.formatstring' is explicitly set. + +The reason for this behaviour is that since 'UglyDuckling' is a base +class with metaclass 'MagicallyTransformed', and since 'Swan' inherits from +'UglyDuckling', then 'Swan' inherits the metaclass 'MagicallyTransformed', +which is automatically called at 'Swan' creation time. +That's the reason why metaclasses are much more magical and much +more dangerous than +functions: functions do not override attributes in the derived classes, +metaclasses do, since they are automagically called at the time of +creation of the subclass. In other words, functions are explicit, +metaclasses are implicit. Nevertheless, this behavior can be pretty +useful in many circumstances, and it is a feature, not a bug. In the +situations where this behavior is not intended, one should use a function, +not a metaclass. In general, metaclasses are better than functions, +since metaclasses are classes and as such they can inherit one from each +other. This means that one can improve a basic metaclass trough +(multiple) inheritance, with *reuse* of code. + +A few caveats about the usage of metaclasses +------------------------------------------------------------------------ + +Let me start with some caveats about the ``__metaclass__`` hook, which +commonly used and quite powerful, but also quite dangereous. + +Let's imagine a programmer not +knowing about metaclasses and looking at the 'TransformedUglyDuckling' +code (assuming there are no comments): she would probably think +that "__metaclass__" is some special attribute used for introspection +purposes only, with no other effects, and she would probably expect +the output of the script to be "Not much, I am the class +TransformedUglyDucking" whereas it is exacly the contrary! In other +words, when metaclasses are involved, *what you see, is not what you get*. +The situation is even more implicit when the metaclass is inherited +from some base class, therefore lacking also the visual clue of the hook. + +For these reasons, metaclasses are something to be used with great care; +they can easily make your code unreadable and confuse inexpert programmers. +Moreover, it is more difficult to debug programs involving metaclasses, since +methods are magically transformed by routines defined in the metaclass, +and the code you see in the class is *not* what Python sees. I think +the least confusing way of using metaclasses, is to concentrate all +the dynamics on them and to write empty classes except for the +metaclass hook. If you write a class with no methods such as + + :: + + class TransformedUglyDuckling(object): + __metaclass__=MagicallyTransformed + +then the only place to look at, is the metaclass. I have found extremely +confusing to have some of the methods defined in the class and some in +the metaclass, especially during debugging. + +Another point to make, is that the ``__metaclass__`` +hook should not be used to modify pre-existing classes, +since it requires modifying the source code (even if it is enough to +change one line only). Moreover, it is confusing, since adding a +``__metaclass__`` attribute *after* the class creation would not do the job: + + >>> from oopp import UglyDuckling, MagicallyTransformed + >>> UglyDuckling.__metaclass__=MagicallyTransformed + >>> print UglyDuckling() + "Not much, I am the class UglyDuckling" + +The reason is that we have to think of UglyDuckling as an instance of +``type``, the built-in metaclasses; merely adding a ``__metaclass__`` +attribute does not re-initialize the class. +The problem is elegantly solved by avoiding the hook and creating +an enhanced copy of the original class trough ``MagicallyTransformed`` +used as a class factory. + + >>> name=UglyDuckling.__name__ + >>> bases=UglyDuckling.__bases__ + >>> dic=UglyDuckling.__dict__.copy() + >>> UglyDuckling=MagicallyTransformed(name,bases,dic) + +Notice that I have recreated 'UglyDuckling', giving to the new class +the old identifier. + + >>> print UglyDuckling() + Very beautiful, since I am <UglyDuckling>> + +The metaclass of this new 'UglyDuckling' has been specified and will +accompanies all future children of 'UglyDuckling': + + >>> class Swan(UglyDuckling): pass + ... + >>> type(Swan) + <class '__main__.MagicallyTransformed'> + +Another caveat, is in the overridding of `` __init__`` in the metaclass. +This is quite common in the case of metaclasses called trough the +``__metaclass__`` hook mechanism, since in this case the class +has been already defined (if not created) in the class statement, +and we are interested in initializing it, more than in recreating +it (which is still possible, by the way). +The problem is that overriding ``__init__`` has severe limitations +with respect to overriding ``__new__``, +since the 'name', 'bases' and 'dic' arguments cannot be directly +changed. Let me show an example: + + :: + + #<init_in_metaclass.py> + + from oopp import * + + class M(type): + "Shows that dic cannot be modified in __init__, only in __new__" + def __init__(cls,name,bases,dic): + name='C name cannot be changed in __init__' + bases='cannot be changed' + dic['changed']=True + + class C(object): + __metaclass__=M + changed=False + + print C.__name__ # => C + print C.__bases__ # => (<type 'object'>,) + print C.changed # => False + + #</init_in_metaclass.py> + +The output of this script is ``False``: the dictionary cannot be changed in +``__init__`` method. However, replacing ``dic['changed']=True`` with +``cls.changed=True`` would work. Analougously, changing ``cls.__name__`` +would work. On the other hand, ``__bases__`` is a read-only attribute and +cannot be changed once the class has been created, therefore there is no +way it can be touched in ``__init__``. However, ``__bases__`` could be +changed in ``__new__`` before the class creation. + +Metaclasses and inheritance +------------------------------------------------------------------------- + +It is easy to get confused about the difference between a metaclass +and a mix-in class in multiple inheritance, since +both are denoted by adjectives and both share the same idea of +enhancing a hierarchy. Moreover, both mix-in classes and metaclasses +can be inherited in the whole hierarchy. +Nevertheless, they behaves differently +and there are various subtle point to emphasize. We have already +noticed in the first section that attributes of a metaclass +are transmitted to its instances, but not to the instances of the +instances, whereas the normal inheritance is transitive: the +grandfather transmits its attributes to the children and to the grandchild +too. The difference can be represented with the following picture, where +'M' is the metaclass, 'B' a base class, 'C' a children of 'B' +and c an instance of 'C': + + :: + + M (attr) B (attr) + : | + C (attr) C (attr) + : : + c () c (attr) + +Notice that here the relation of instantiation is denoted by a dotted line. + +This picture is valid when C has metaclass M but not base class, on when C +has base class but not metaclass. However, what happens whrn the class C has +both a metaclass M and a base class B ? + + >>> class M(type): a='M.a' + >>> class B(object): a='B.a' + >>> class C(B): __metaclass__=M + >>> c=C() + +The situation can be represented by in the following graph, + + :: + + (M.a) M B (B.a) + : / + : / + (?) C + : + : + (?) c + + +Here the metaclass M and the base class B are fighting one against the other. +Who wins ? C should inherit the attribute 'B.a' from its base B, however, +the metaclass would like to induce an attribute 'M.a'. +The answer is that the inheritance constraint wins on the metaclass contraint: + + >>> C.a + 'B.a' + >>> c.a + 'B.a' + +The reason is the same we discussed in the fairy tale example: 'M.a' is +an attribute of the metaclass, if its instance C has already a specified +attributed C.a (in this case specified trough inheritance from B), then +the attribute is not modified. However, one could *force* the modification: + + >>> class M(type): + ... def __init__(cls,*args): cls.a='M.a' + >>> class C(B): __metaclass__=M + >>> C.a + 'M.a' + +In this case the metaclass M would win on the base class B. Actually, +this is not surprising, since it is explicit. What could be surprising, +had we not explained why inheritance silently wins, is that + + >>> c.a + 'B.a' + +This explain the behaviour for special methods like +``__new__,__init__,__str__``, +etc. which are defined both in the class and the metaclass with the same +name (in both cases,they are inherited from ``object``). + +In the chapter on objects, we learned that the printed representation of +an object can be modified by overring the ``__str__`` methods of its +class. In the same sense, the printed representation of a class can be +modified by overring the ``__str__`` methods of its metaclass. Let me show an +example: + + :: + + #<oopp.py> + + class Printable(PrettyPrinted,type): + """Apparently does nothing, but actually makes PrettyPrinted acting as + a metaclass.""" + + #</oopp.py> + +Instances of 'Printable' are classes with a nice printable representation: + + >>> from oopp import Printable + >>> C=Printable('Classname',(),{}) + >>> print C + Classname + +However, the internal string representation stays the same: + + >>> C # invokes Printable.__repr__ + <class '__main__.Classname'> + +Notice that the name of class 'C' is ``Classname`` and not 'C' ! + +Consider for instance the following code: + + >>> class M(type): + ... def __str__(cls): + ... return cls.__name__ + ... def method(cls): + ... return cls.__name__ + ... + >>> class C(object): + ... __metaclass__=M + >>> c=C() + +In this case the ``__str__`` method in ``M`` cannot override the +``__str__`` method in C, which is inherited from ``object``. +Moreover, if you experiment a little, you will see that + + >>> print C # is equivalent to print M.__str__(C) + C + >>> print c # is equivalent to print C.__str__(c) + <__main__.C object at 0x8158f54> + + +The first ``__str__`` is "attached" to the metaclass and the +second to the class. + +Consider now the standard method "method". It is both attached to the +metaclass + + >>> print M.method(C) + C + +and to the class + + >>> print C.method() #in a sense, this is a class method, i.e. it receives + C #the class as first argument + +Actually it can be seen as a class method of 'C' (cfr. Guido van Rossum +"Unifying types and classes in Python 2.2". When he discusses +classmethods he says: *"Python also has real metaclasses, and perhaps +methods defined in a metaclass have more right to the name "class method"; +but I expect that most programmers won't be using metaclasses"*). Actually, +this is the SmallTalk terminology, Unfortunately, in Python the word +``classmethod`` denotes an attribute descriptor, therefore it is better +to call the methods defined in a metaclass *metamethods*, in order to avoid +any possible confusion. + +The difference between ``method`` and ``__str__`` is that you cannot use the +syntax + + >>> print C.__str__() #error + TypeError: descriptor '__str__' of 'object' object needs an argument + +because of the confusion with the other __str__; you can only use the +syntax + + >>> print M.__str__(C) + +Suppose now I change C's definition by adding a method called "method": + + :: + + class C(object): + __metaclass__=M + def __str__(self): + return "instance of %s" % self.__class__ + def method(self): + return "instance of %s" % self.__class__ + +If I do so, then there is name clashing and the previously working +statement print C.method() gives now an error: + + :: + + Traceback (most recent call last): + File "<stdin>", line 24, in ? + TypeError: unbound method method() must be called with C instance as + first argument (got nothing instead) + +Conclusion: ``__str__, __new__, __init__`` etc. defined in the metaclass +have name clashing with the standard methods defined in the class, therefore +they must be invoked with the extended syntax (ex. ``M.__str__(C)``), +whereas normal methods in the metaclass with no name clashing with the methods +of the class can be used as class methods (ex. ``C.method()`` instead of +``M.method(C)``). +Metaclass methods are always bound to the metaclass, they bind to the class +(receiving the class as first argument) only if there is no name clashing with +already defined methods in the class. Which is the case for ``__str__``, +``___init__``, etc. + +Conflicting metaclasses +---------------------------------------------------------------------------- + +Consider a class 'A' with metaclass 'M_A' and a class 'B' with +metaclass 'M_B'; suppose I derive 'C' from 'A' and 'B'. The question is: +what is the metaclass of 'C' ? Is it 'M_A' or 'M_B' ? + +The correct answer (see "Putting metaclasses to work" for a thought +discussion) is 'M_C', where 'M_C' is a metaclass that inherits from +'M_A' and 'M_B', as in the following graph: + + + .. figure:: fig1.gif + +However, Python is not yet that magic, and it does not automatically create +'M_C'. Instead, it will raise a ``TypeError``, warning the programmer of +the possible confusion: + + >>> class M_A(type): pass + >>> class M_B(type): pass + >>> A=M_A('A',(),{}) + >>> B=M_B('B',(),{}) + >>> class C(A,B): pass #error + Traceback (most recent call last): + File "<stdin>", line 1, in ? + TypeError: metatype conflict among bases + +This is an example where the metaclasses 'M_A' and 'M_B' fight each other +to generate 'C' instead of cooperating. The metatype conflict can be avoided +by assegning the correct metaclass to 'C' by hand: + + >>> class C(A,B): __metaclass__=type("M_AM_B",(M_A,M_B),{}) + >>> type(C) + <class '__main__.M_AM_B'> + +In general, a class A(B, C, D , ...) can be generated without conflicts only +if type(A) is a subclass of each of type(B), type(C), ... + +In order to avoid conflicts, the following function, that generates +the correct metaclass by looking at the metaclasses of the base +classes, is handy: + + :: + + #<oopp.py> + + metadic={} + + def _generatemetaclass(bases,metas,priority): + trivial=lambda m: sum([issubclass(M,m) for M in metas],m is type) + # hackish!! m is trivial if it is 'type' or, in the case explicit + # metaclasses are given, if it is a superclass of at least one of them + metabs=tuple([mb for mb in map(type,bases) if not trivial(mb)]) + metabases=(metabs+metas, metas+metabs)[priority] + if metabases in metadic: # already generated metaclass + return metadic[metabases] + elif not metabases: # trivial metabase + meta=type + elif len(metabases)==1: # single metabase + meta=metabases[0] + else: # multiple metabases + metaname="_"+''.join([m.__name__ for m in metabases]) + meta=makecls()(metaname,metabases,{}) + return metadic.setdefault(metabases,meta) + + #</oopp.py> + +This function is particularly smart since: + + 1. Avoid duplications .. + + 2. Remember its results. + +We may generate the child of a tuple of base classes with a given metaclass +and avoiding metatype conflicts thanks to the following ``child`` function: + + :: + + #<oopp.py> + + def makecls(*metas,**options): + """Class factory avoiding metatype conflicts. The invocation syntax is + makecls(M1,M2,..,priority=1)(name,bases,dic). If the base classes have + metaclasses conflicting within themselves or with the given metaclasses, + it automatically generates a compatible metaclass and instantiate it. + If priority is True, the given metaclasses have priority over the + bases' metaclasses""" + + priority=options.get('priority',False) # default, no priority + return lambda n,b,d: _generatemetaclass(b,metas,priority)(n,b,d) + + #</oopp.py> + +Here is an example of usage: + + >>> class C(A,B): __metaclass__=makecls() + >>> print C,type(C) + <class 'oopp.AB_'> <class 'oopp._M_AM_B'> + +Notice that the automatically generated metaclass does not pollute the +namespace: + + >>> _M_A_M_B #error + Traceback (most recent call last): + File "<stdin>", line 1, in ? + NameError: name '_M_A_M_B' is not defined + +It can only be accessed as ``type(C)``. + +Put it shortly, the ``child`` function allows to generate a child from bases +enhanced by different custom metaclasses, by generating under the hood a +compatibile metaclass via multiple inheritance from the original metaclasses. +However, this logic can only work if the original metaclasses are +cooperative, i.e. their methods are written in such a way to avoid +collisions. This can be done by using the cooperative the ``super`` call +mechanism discussed in chapter 4. + +Cooperative metaclasses +---------------------------------------------------------------------------- + +In this section I will discuss how metaclasses can be composed with +classes and with metaclasses, too. Since we will discuss even +complicated hierarchies, it is convenient to have an utility +routine printing the MRO of a given class: + + :: + + #<oopp.py> + + def MRO(cls): + count=0; out=["MRO of %s:" % cls.__name__] + for c in cls.__mro__: + name=c.__name__ + bases=','.join([b.__name__ for b in c.__bases__]) + s=" %s - %s(%s)" % (count,name,bases) + if type(c) is not type: s+="[%s]" % type(c).__name__ + out.append(s); count+=1 + return '\n'.join(out) + + #</oopp.py> + +Notice that ``MRO`` also prints the metaclass' name in square brackets, for +classes enhanced by a non-trivial metaclass. + +Consider for instance the following hierarchy: + + >>> from oopp import MRO + >>> class B(object): pass + >>> class M(B,type): pass + >>> class C(B): __metaclass__=M + +Here 'M' is a metaclass that inherits from 'type' and the base class 'B' +and 'C' is both an instance of 'M' and a child of 'B'. The inheritance +graph can be draw as + + :: + + object + / \ + B type + | \ / + | M + \ : + \ : + C + +Suppose now we want to retrieve the ``__new__`` method of B's superclass +with respect to the MRO of C: obviously, this is ``object.__new__``, since + + >>> print MRO(C) + MRO of C: + 0 - C(B)[M] + 1 - B(object) + 2 - object() + +This allows to create an instance of 'C' in this way: + + >>> super(B, C).__new__(C) + <__main__.C object at 0x4018750c> + +It is interesting to notice that this would not work in Python 2.2, +due to a bug in the implementation of ``super``, therefore do not +try this trick with older version of Python. + +Notice that everything works +only because ``B`` inherits the ``object.__new__`` staticmethod that +is cooperative and it turns out that it calls ``type.__new__``. However, +if I give to 'B' a non-cooperative method + + >>> B.__new__=staticmethod(lambda cls,*args: object.__new__(cls)) + +things do not work: + + >>> M('D',(),{}) #error + Traceback (most recent call last): + File "<stdin>", line 1, in ? + File "<stdin>", line 1, in <lambda> + TypeError: object.__new__(M) is not safe, use type.__new__() + +A cooperative method would solve the problem: + + >>> B.__new__=staticmethod(lambda m,*args: super(B,m).__new__(m,*args)) + >>> M('D',(),{}) # calls B.__new__(M,'D',(),{}) + <class '__main__.D'> + +Cooperation between classes and metaclasses is pretty tricky. +This is why autosuper is so difficult to get right. + +Consider this example, with object and type instances of MetaSuper:: + + class B(object): + def __init__(self, *args): + print "B.__init__" + self.__super.__init__(*args) + + class M(B, type): + def __init__(self, n, b, d): + print "M.__init__" + self.__super.__init__(n, b, d) + + class C(B): + __metaclass__ = M + def __init__(self): + print "C.__init__" + self.__super.__init__() + +What happens here at C creation? The metaclass _MMetaSuper is generated, +with ancestors M, B, superobject, supertype, MetaSuper, safetype, type, +which calls M.__init_(C, )_ and then B.__init__(C) which calls +the *unbound* method superobject.__init__(C ..) and NOT the *bound* method +MetaSuper.__init__(C, ) which would be the "right" thing to do in this +situation (but not in other situations). So C._C__super is NOT initialized :-( + +Metamethods vs class methods +------------------------------------------------------------------- + +Meta-methods, i.e. methods defined in +a metaclass. + +Python has already few built-in metamethods such as ``.mro()``, + ``__subclass__()``, ``__delattr__`` and possibly others. +Moreover ``__name__`` is an example of meta-descriptor. +These are methods of the metaclass 'type' and there of any of its +sub-metaclasses. + + >>> dir(type) + ['__base__', '__bases__', '__basicsize__', '__call__', '__class__', + '__cmp__', '__delattr__', '__dict__', '__dictoffset__', '__doc__', + '__flags__', '__getattribute__', '__hash__', '__init__', '__itemsize__', + '__module__', '__mro__', '__name__', '__new__', '__reduce__', '__repr__', + '__setattr__', '__str__', '__subclasses__', '__weakrefoffset__', 'mro'] + + + >>> print type.mro.__doc__ + mro() -> list + return a type's method resolution order + >>> print type.__subclasses__.__doc__ + __subclasses__() -> list of immediate subclasses + + >>> class A(object): pass + >>> class B(A): pass + >>> B.mro() + [<class '__main__.B'>, <class '__main__.A'>, <type 'object'>] + >>> A.__subclasses__() + [<class '__main__.B'>] + +Notice that ``mro()`` and ``__subclasses__`` are not retrieved by ``dir``. + +Let me constrast metamethods with the more traditional classmethods. +In many senses, the to concepts are akin: + + >>> class M(type): + ... "Metaclass with a (meta)method mm" + ... def mm(cls): return cls + >>> D=M('D',(),{'cm':classmethod(lambda cls: cls)}) + >>> # instance of M with a classmethod cm + >>> D.mm # the metamethod + <bound method M.mm of <class '__main__.C'>> + >>> D.cm # the classmethod + <unbound method D.<lambda>> + +Notice the similarities between the classmethod and the metamethod: + + >>> D.mm.im_self, D.cm.im_self # the same + (<class '__main__.D'>, <class '__main__.D'>) + >>> D.mm.im_class, D.cm.im_class # still the same + (<class '__main__.M'>, <class '__main__.M'>) + +There are no surprises for ``im_func``: + + >>> D.mm.im_func, D.cm.im_func + (<function mm at 0x402c272c>, <function <lambda> at 0x402c280c>) + +Nevertheless, there are differences: metamethods are not bounded to +instances of the class + + >>> D().cm() # the classmethod works fine + <class '__main__.D'> + >>> D().mm() # the metamethod does not: error + Traceback (most recent call last): + File "<stdin>", line 1, in ? + AttributeError: 'D' object has no attribute 'mm' + +and they are not retrieved by ``dir``: + + >>> from oopp import * + >>> attributes(D).keys() # mm is not retrieved, only cm + ['cm'] + + >>> cm.__get__('whatever') #under Python 2.2.0 would give a serious error + Segmentation fault + >>> cm.__get__(None) #under Python 2.3 there is no error + <bound method type.<lambda> of <type 'NoneType'>> + +Moreover metamethods behaves differently with respect to multiple +inheritance. If a class A define a classmethod cA and a class B +defines a classmethod cB, then the class C(A,B) inherits both the +classmethods cA and cB. In the case of metamethods defined in M_A +and M_B, the same is true only if one resolves the meta-type +conflict by hand, by generating the metaclass M_C(M_A,M_B). In this +sense, classmethods are simpler to use than metamethods. |