1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
|
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.2.9: http://docutils.sourceforge.net/" />
<title>THE SOPHISTICATION OF DESCRIPTORS</title>
<link rel="stylesheet" href="default.css" type="text/css" />
</head>
<body>
<div class="document" id="the-sophistication-of-descriptors">
<h1 class="title">THE SOPHISTICATION OF DESCRIPTORS</h1>
<p>Attribute descriptors are important metaprogramming tools that allows
the user to customize the behavior of attributes in custom classes.
For instance, attribute descriptors (or descriptors for short)
can be used as method wrappers,
to modify or enhance methods (this is the case for the well
known staticmethods and classmethods attribute descriptors); they
can also be used as attribute wrappers, to change or restrict the access to
attributes (this is the case for properties). Finally, descriptors
allows the user to play with the resolution order of attributes:
for instance, the <tt class="literal"><span class="pre">super</span></tt> built-in object used in (multiple) inheritance
hierarchies, is implemented as an attribute descriptor.</p>
<p>In this chapter, I will show how the user can define its own attribute
descriptors and I will give some example of useful things you can do with
them (in particular to add tracing and timing capabilities).</p>
<div class="section" id="motivation">
<h1><a name="motivation">Motivation</a></h1>
<p>Attribute descriptors are a recent idea (they where first introduced in
Python 2.2) nevertheless, under the hood, are everywhere in Python. It is
a tribute to Guido's ability of hiding Python complications that
the average user can easily miss they existence.
If you need to do simple things, you can very well live without
the knowledge of descriptors. On the other hand, if you need difficult
things (such as tracing all the attribute access of your modules)
attribute descriptors, allow you to perform
impressive things.
Let me start by showing why the knowledge of attribute descriptors is
essential for any user seriously interested in metaprogramming applications.
Suppose I want to trace the methods of a clock:</p>
<blockquote>
<pre class="doctest-block">
>>> import oopp
>>> clock=oopp.Clock()
</pre>
</blockquote>
<p>This is easily done with the <tt class="literal"><span class="pre">with_tracer</span></tt> closure of chapter 2:</p>
<blockquote>
<pre class="doctest-block">
>>> oopp.wrapfunctions(clock,oopp.with_tracer)
<oopp.Clock object at 0x4044c54c>
>>> clock.get_time()
[] Calling 'get_time' with arguments
(){} ...
-> '.get_time' called with result: 19:55:07
'19:55:07'
</pre>
</blockquote>
<p>However, this approach fails if I try to trace the entire class:</p>
<blockquote>
<pre class="doctest-block">
>>> oopp.wrapfunctions(oopp.Clock,oopp.with_tracer)
<class 'oopp.Clock'>
>>> oopp.Clock.get_time() # error
Traceback (most recent call last):
File "<stdin>", line 6, in ?
TypeError: unbound method _() must be called with Clock instance
as first argument (got nothing instead)
</pre>
</blockquote>
<p>The reason is that <tt class="literal"><span class="pre">wrapfunctions</span></tt> sets the attributes of 'Clock'
by invoking <tt class="literal"><span class="pre">customize</span></tt>, which uses <tt class="literal"><span class="pre">setattr</span></tt>. This converts
'_' (i.e. the traced version of <tt class="literal"><span class="pre">get_time</span></tt>) in a regular method, not in
a staticmethod!
In order to trace staticmethods, one has to understand the nature
of attribute descriptors.</p>
</div>
<div class="section" id="functions-versus-methods">
<h1><a name="functions-versus-methods">Functions versus methods</a></h1>
<p>Attribute descriptors are essential for the implementation
of one of the most basic Python features: the automatic conversion
of functions in methods. As I already anticipated in chapter 1, there is
a sort of magic when one writes <tt class="literal"><span class="pre">Clock.get_time=lambda</span> <span class="pre">self:</span> <span class="pre">get_time()</span></tt>
and Python automagically converts the right hand side, that is a
function, to a left hand side that is a (unbound) method. In order to
understand this magic, one needs a better comprehension of the
relation between functions and methods.
Actually, this relationship is quite subtle
and has no analogous in mainstream programming languages.
For instance, C is not OOP and has only functions, lacking the concept
of method, whereas Java (as other OOP languages)
has no functions, only methods.
C++ has functions and methods, but functions are completely
different from methods On the other hand, in Python,
functions and methods can be transformed both ways.</p>
<p>To show how it works, let me start by defining a simple printing
function:</p>
<blockquote>
<pre class="literal-block">
#<oopp.py>
import __main__ # gives access to the __main__ namespace from the module
def prn(s):
"""Given an evaluable string, print its value and its object reference.
Notice that the evaluation is done in the __main__ dictionary."""
try: obj=eval(s,__main__.__dict__)
except: print 'problems in evaluating',s
else: print s,'=',obj,'at',hex(id(obj))
#</oopp.py>
</pre>
</blockquote>
<p>Now, let me define a class with a method <tt class="literal"><span class="pre">m</span></tt> equals to the identity
function <tt class="literal"><span class="pre">f</span></tt>:</p>
<blockquote>
<pre class="doctest-block">
>>> def f(x): "Identity function"; return x
...
>>> class C(object):
... m=f
... print m #here m is the function f
<function f at 0x401c2b1c>
</pre>
</blockquote>
<p>We see that <em>inside</em> its defining class, <tt class="literal"><span class="pre">m</span></tt> coincides with the function
<tt class="literal"><span class="pre">f</span></tt> (the object reference is the same):</p>
<blockquote>
<pre class="doctest-block">
>>> f
<function f at 0x401c2b1c>
</pre>
</blockquote>
<p>We may retrieve <tt class="literal"><span class="pre">m</span></tt> from <em>outside</em> the class via the class dictionary <a class="footnote-reference" href="#id2" id="id1" name="id1">[1]</a>:</p>
<blockquote>
<pre class="doctest-block">
>>> C.__dict__['m']
<function prn at 0x401c2b1c>
</pre>
</blockquote>
<p>However, if we invoke <tt class="literal"><span class="pre">m</span></tt> with
the syntax <tt class="literal"><span class="pre">C.m</span></tt>, then it (magically) becomes a (unbound) method:</p>
<blockquote>
<pre class="doctest-block">
>>> C.m #here m has become a method!
<unbound method C.f>
</pre>
</blockquote>
<p>But why it is so? How comes that in the second syntax the function
<tt class="literal"><span class="pre">f</span></tt> is transformed in a (unbound) method? To answer that question, we have
to understand how attributes are really invoked in Python, i.e. via
attribute descriptors.</p>
</div>
<div class="section" id="methods-versus-functions">
<h1><a name="methods-versus-functions">Methods versus functions</a></h1>
<p>First of all, let me point out the differences between methods and
functions. Here, <tt class="literal"><span class="pre">C.m</span></tt> does <em>not</em> coincides with <tt class="literal"><span class="pre">C.__dict__['m']</span></tt>
i.e. <tt class="literal"><span class="pre">f</span></tt>, since its object reference is different:</p>
<blockquote>
<pre class="doctest-block">
>>> from oopp import prn,attributes
>>> prn('C.m')
C.m = <unbound method C.prn> at 0x81109b4
</pre>
</blockquote>
<p>The difference is clear since methods and functions have different attributes:</p>
<blockquote>
<pre class="doctest-block">
>>> attributes(f).keys()
['func_closure', 'func_dict', 'func_defaults', 'func_name',
'func_code', 'func_doc', 'func_globals']
</pre>
</blockquote>
<p>whereas</p>
<blockquote>
<pre class="doctest-block">
>>> attributes(C.m).keys()
['im_func', 'im_class', 'im_self']
</pre>
</blockquote>
<p>We discussed few of the functions attributes in the chapter
on functions. The instance method attributes are simpler: <tt class="literal"><span class="pre">im_self</span></tt>
returns the object to which the method is attached,</p>
<blockquote>
<pre class="doctest-block">
>>> print C.m.im_self #unbound method, attached to the class
None
>>> C().m.im_self #bound method, attached to C()
<__main__.C object at 0x81bf4ec>
</pre>
</blockquote>
<p><tt class="literal"><span class="pre">im_class</span></tt> returns the class to which the
method is attached</p>
<blockquote>
<pre class="doctest-block">
>>> C.m.im_class #class of the unbound method
<class '__main__.C'>
>>> C().m.im_class #class of the bound method,
<class '__main__.C'>
</pre>
</blockquote>
<p>and <tt class="literal"><span class="pre">im_func</span></tt> returns the function equivalent to
the method.</p>
<blockquote>
<pre class="doctest-block">
>>> C.m.im_func
<function m at 0x8157f44>
>>> C().m.im_func # the same
<function m at 0x8157f44>
</pre>
</blockquote>
<p>As the reference manual states, calling
<tt class="literal"><span class="pre">m(*args,**kw)</span></tt> is completely equivalent to calling
<tt class="literal"><span class="pre">m.im_func(m.im_self,</span> <span class="pre">*args,**kw)</span></tt>".</p>
<p>As a general rule, an attribute descriptor is an object with a <tt class="literal"><span class="pre">__get__</span></tt>
special method. The most used descriptors are the good old functions:
they have a <tt class="literal"><span class="pre">__get__</span></tt> special method returning a <em>method-wrapper object</em></p>
<blockquote>
<pre class="doctest-block">
>>> f.__get__
<method-wrapper object at 0x815cdc4>
</pre>
</blockquote>
<p>method-wrapper objects can be transformed in (both bound and unbound) methods:</p>
<blockquote>
<pre class="doctest-block">
>>> f.__get__(None,C)
<unbound method C.f>
>>> f.__get__(C(),C)
<bound method C.f of <__main__.C object at 0x815cdc4>>
</pre>
</blockquote>
<p>The general calling syntax for method-wrapper objects is
<tt class="literal"><span class="pre">.__get__(obj,cls=None)</span></tt>, where the first argument is an
instance object or None and the second (optional) argument is the class (or a
generic superclass) of the first one.</p>
<p>Now we see what happens when we use the syntax <tt class="literal"><span class="pre">C.m</span></tt>: Python interprets
this as a shortcut for <tt class="literal"><span class="pre">C.__dict['m'].__get__(None,C)</span></tt> (if <tt class="literal"><span class="pre">m</span></tt> is
in the 'C' dictionary, otherwise it looks for ancestor dictionaries).
We may check that everything is correct by observing that
<tt class="literal"><span class="pre">f.__get__(None,C)</span></tt> has exactly the same object reference than <tt class="literal"><span class="pre">C.m</span></tt>,
therefore they are the same object:</p>
<blockquote>
<pre class="doctest-block">
>>> hex(id(f.__get__(None,C))) # same as hex(id(C.m))
'0x811095c'
</pre>
</blockquote>
<p>The process works equally well for the syntax <tt class="literal"><span class="pre">getattr</span></tt>:</p>
<blockquote>
<pre class="doctest-block">
>>> print getattr(C,'m'), hex(id(getattr(C,'m')))
<unbound method C.f> 0x811095c
</pre>
</blockquote>
<p>and for bound methods: if</p>
<blockquote>
<pre class="doctest-block">
>>> c=C()
</pre>
</blockquote>
<p>is an instance of the class C, then the syntax</p>
<blockquote>
<pre class="doctest-block">
>>> getattr(c,'m') #same as c.m
<bound method C.f of <__main__.C object at 0x815cdc4>>
</pre>
</blockquote>
<p>is a shortcut for</p>
<blockquote>
<pre class="doctest-block">
>>> type(c).__dict__['m'].__get__(c,C) # or f.__get__(c,C)
<bound method C.f of <__main__.C object at 0x815cdc4>>
</pre>
</blockquote>
<p>(notice that the object reference for <tt class="literal"><span class="pre">c.m</span></tt> and <tt class="literal"><span class="pre">f.__get__(c,C)</span></tt> is
the same, they are <em>exactly</em> the same object).</p>
<p>Both the unbound method C.m and the bound method c.m refer to the same
object at hexadecimal address 0x811095c. This object is common to all other
instances of C:</p>
<blockquote>
<pre class="doctest-block">
>>> c2=C()
>>> print c2.m,hex(id(c2.m)) #always the same method
<bound method C.m of <__main__.C object at 0x815768c>> 0x811095c
</pre>
</blockquote>
<p>One can also omit the second argument:</p>
<blockquote>
<pre class="doctest-block">
>>> c.m.__get__(c)
<bound method ?.m of <__main__.C object at 0x81597dc>>
</pre>
</blockquote>
<p>Finally, let me point out that methods are attribute descriptors too,
since they have a <tt class="literal"><span class="pre">__get__</span></tt> attribute returning a method-wrapper
object:</p>
<blockquote>
<pre class="doctest-block">
>>> C.m.__get__
<method-wrapper object at 0x815d51c>
</pre>
</blockquote>
<p>Notice that this method wrapper is <em>not</em> the same than the <tt class="literal"><span class="pre">f.__get__</span></tt>
method wrapper.</p>
<blockquote>
<table class="footnote" frame="void" id="id2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id1" name="id2">[1]</a></td><td>If <tt class="literal"><span class="pre">C.__dict['m']</span></tt> is not defined, Python looks if <tt class="literal"><span class="pre">m</span></tt> is defined
in some ancestor of C. For instance if <cite>B</cite> is the base of <cite>C</cite>, it
looks in <tt class="literal"><span class="pre">B.__dict['m']</span></tt>, etc., by following the MRO.</td></tr>
</tbody>
</table>
</blockquote>
</div>
<div class="section" id="static-methods-and-class-methods">
<h1><a name="static-methods-and-class-methods">Static methods and class methods</a></h1>
<p>Whereas functions and methods are implicit attribute descriptors,
static methods and class methods are examples of explicit
descriptors. They allow to convert regular functions to
specific descriptor objects. Let me show a trivial example.
Given the identity function</p>
<blockquote>
<pre class="doctest-block">
>>> def f(x): return x
</pre>
</blockquote>
<p>we may convert it to a staticmethod object</p>
<blockquote>
<pre class="doctest-block">
>>> sm=staticmethod(f)
>>> sm
<staticmethod object at 0x4018a0a0>
</pre>
</blockquote>
<p>or to a classmethod object</p>
<blockquote>
<pre class="doctest-block">
>>> cm=classmethod(f)
>>> cm
<classmethod object at 0x4018a0b0>
</pre>
</blockquote>
<p>In both cases the <tt class="literal"><span class="pre">__get__</span></tt> special method returns a method-wrapper object</p>
<blockquote>
<pre class="doctest-block">
>>> sm.__get__
<method-wrapper object at 0x401751ec>
>>> cm.__get__
<method-wrapper object at 0x4017524c>
</pre>
</blockquote>
<p>However the static method wrapper is quite different from the class
method wrapper. In the first case the wrapper returns a function:</p>
<blockquote>
<pre class="doctest-block">
>>> sm.__get__(C(),C)
<function f at 0x4027a8b4>
>>> sm.__get__(C())
<function f at 0x4027a8b4>
</pre>
</blockquote>
<p>in the second case it returns a method</p>
<blockquote>
<pre class="doctest-block">
>>> cm.__get__(C(),C)
<bound method type.f of <class '__main__.C'>>
</pre>
</blockquote>
<p>Let me discuss more in detail the static methods, first.</p>
<p>It is always possible to extract the function from the static method
via the syntaxes <tt class="literal"><span class="pre">sm.__get__(a)</span></tt> and <tt class="literal"><span class="pre">sm.__get__(a,b)</span></tt> with <em>ANY</em> valid
a and b, i.e. the result does not depend on a and b. This is correct,
since static methods are actually function that have nothing to do
with the class and the instances to which they are bound.</p>
<p>This behaviour of the method wrapper makes clear why the relation between
methods and functions is inversed for static methods with respect to
regular methods:</p>
<blockquote>
<pre class="doctest-block">
>>> class C(object):
... s=staticmethod(lambda : None)
... print s
...
<staticmethod object at 0x8158ec8>
</pre>
</blockquote>
<p>Static methods are non-trivial objects <em>inside</em> the class, whereas
they are regular functions <em>outside</em> the class:</p>
<blockquote>
<pre class="doctest-block">
>>> C.s
<function <lambda> at 0x8158e7c>
>>> C().s
<function <lambda> at 0x8158e7c>
</pre>
</blockquote>
<p>The situation is different for classmethods: inside the class they
are non-trivial objects, just as static methods,</p>
<blockquote>
<pre class="doctest-block">
>>> class C(object):
... cm=classmethod(lambda cls: None)
... print cm
...
<classmethod object at 0x8156100>
</pre>
</blockquote>
<p>but outside the class they are methods bound to the class,</p>
<blockquote>
<pre class="doctest-block">
>>> c=C()
>>> prn('c.cm')
<bound method type.<lambda> of <class '__main__.C'>>
0x811095c
</pre>
</blockquote>
<p>and not to the instance 'c'. The reason is that the <tt class="literal"><span class="pre">__get__</span></tt> wrapper method
can be invoked with the syntax <tt class="literal"><span class="pre">__get__(a,cls)</span></tt> which
is only sensitive to the second argument or with the syntax
<tt class="literal"><span class="pre">__get__(obj)</span></tt> which is only sensitive to the type of the first
argument:</p>
<blockquote>
<pre class="doctest-block">
>>> cm.__get__('whatever',C) # the first argument is ignored
<bound method type.f of <class '__main__.C'>>
</pre>
</blockquote>
<p>sensitive to the type of 'whatever':</p>
<blockquote>
<pre class="doctest-block">
>>> cm.__get__('whatever') # in Python 2.2 would give a serious error
<bound method type.f of <type 'str'>>
</pre>
</blockquote>
<p>Notice that the class method is actually bound to C's class, i.e.
to 'type'.</p>
<p>Just as regular methods (and differently
from static methods) classmethods have attributes <tt class="literal"><span class="pre">im_class</span></tt>, <tt class="literal"><span class="pre">im_func</span></tt>,
and <tt class="literal"><span class="pre">im_self</span></tt>. In particular one can retrieve the function wrapped inside
the classmethod with</p>
<blockquote>
<pre class="doctest-block">
>>> cm.__get__('whatever','whatever').im_func
<function f at 0x402c2534>
</pre>
</blockquote>
<p>The difference with regular methods is that <tt class="literal"><span class="pre">im_class</span></tt> returns the
class of 'C' whereas <tt class="literal"><span class="pre">im_self</span></tt> returns 'C' itself.</p>
<blockquote>
<pre class="doctest-block">
>>> C.cm.im_self # a classmethod is attached to the class
<class '__main__.C'>
>>> C.cm.im_class #the class of C
<type 'type'>
</pre>
</blockquote>
<p>Remark: Python 2.2.0 has a bug in classmethods (fixed in newer versions):
when the first argument of __get__ is None, then one must specify
the second argument (otherwise segmentation fault :-()</p>
</div>
<div class="section" id="properties">
<h1><a name="properties">Properties</a></h1>
<p>Properties are a more general kind of attribute descriptors than
staticmethods and classmethods, since their effect can be customized
trough arbitrary get/set/del functions. Let me give an example:</p>
<blockquote>
<pre class="doctest-block">
>>> def getp(self): return 'property' # get function
...
>>> p=property(getp) # property object
>>> p
<property object at 0x815855c>
</pre>
</blockquote>
<p><tt class="literal"><span class="pre">p</span></tt> has a <tt class="literal"><span class="pre">__get__</span></tt> special method returning a method-wrapper
object, just as it happens for other descriptors:</p>
<blockquote>
<pre class="doctest-block">
>>> p.__get__
<method-wrapper object at 0x8158a7c>
</pre>
</blockquote>
<p>The difference is that</p>
<blockquote>
<pre class="doctest-block">
>>> p.__get__(None,type(p))
<property object at 0x4017016c>
>>> p.__get__('whatever')
'property'
>>> p.__get__('whatever','whatever')
'property'
</pre>
</blockquote>
<p>As for static methods, the <tt class="literal"><span class="pre">__get__</span></tt> method wrapper is independent from
its arguments, unless the first one is None: in such a case it returns
the property object, in all other circumstances it returns the result
of <tt class="literal"><span class="pre">getp</span></tt>. This explains the behavior</p>
<blockquote>
<pre class="doctest-block">
>>> class C(object): p=p
>>> C.p
<property object at 0x815855c>
>>> C().p
'property'
</pre>
</blockquote>
<p>Properties are a dangerous feature, since they change the semantics
of the language. This means that apparently trivial operations can have
any kind of side effects:</p>
<blockquote>
<pre class="doctest-block">
>>> def get(self):return 'You gave me the order to destroy your hard disk!!'
>>> class C(object): x=property(get)
>>> C().x
'You gave me the order to destroy your hard disk!!'
</pre>
</blockquote>
<p>Invoking 'C.x' could very well invoke an external program who is going
to do anything! It is up to the programmer to not abuse properties.
The same is true for user defined attribute descriptors.</p>
<p>There are situations in which they are quite handy, however. For
instance, properties can be used to trace the access data attributes.
This can be especially useful during debugging, or for logging
purposes.</p>
<p>Notice that this approach has the problem that now data attributes cannot
no more be called trough their class, but only though their instances.
Moreover properties do not work well with <tt class="literal"><span class="pre">super</span></tt> in cooperative
methods.</p>
</div>
<div class="section" id="user-defined-attribute-descriptors">
<h1><a name="user-defined-attribute-descriptors">User-defined attribute descriptors</a></h1>
<p>As we have seen, there are plenty of predefined attribute descriptors,
such as staticmethods, classmethods and properties (the built-in
<tt class="literal"><span class="pre">super</span></tt> is also an attribute descriptor which, for sake of
convenience, will be discussed in the next section).
In addition to them, the user can also define customized attribute
descriptors, simply trough classes with a <tt class="literal"><span class="pre">__get__</span></tt> special method.
Let me give an example:</p>
<blockquote>
<pre class="literal-block">
#<simpledescr.py>
class ChattyAttr(object):
"""Chatty descriptor class; descriptor objects are intended to be
used as attributes in other classes"""
def __get__(self, obj, cls=None):
binding=obj is not None
if binding:
return 'You are binding %s to %s' % (self,obj)
else:
return 'Calling %s from %s' % (self,cls)
class C(object):
d=ChattyAttr()
c=C()
print c.d # <=> type(c).__dict__['d'].__get__(c,type(c))
print C.d # <=> C.__dict__['d'].__get__(None,C)
#</simpledescr.py>
</pre>
</blockquote>
<p>with output:</p>
<blockquote>
<pre class="literal-block">
You are binding <ChattyAttr object at 0x401bc1cc> to
<C object at 0x401bc2ec>
Calling <ChattyAttr object at 0x401bc1cc> from <class 'C'>
</pre>
</blockquote>
<p>Invoking a method with the syntax <tt class="literal"><span class="pre">C.d</span></tt> or <tt class="literal"><span class="pre">c.d</span></tt> involves calling
<tt class="literal"><span class="pre">__get__</span></tt>. The <tt class="literal"><span class="pre">__get__</span></tt> signature is fixed: it is
`` __get__=__get__(self,obj,cls=None)``, since the notation
<tt class="literal"><span class="pre">self.descr_attr</span></tt> automatically passes <tt class="literal"><span class="pre">self</span></tt> and <tt class="literal"><span class="pre">self.__class__</span></tt> to
<tt class="literal"><span class="pre">__get__</span></tt>.</p>
<p>Custom descriptors can be used to restrict the access to objects in a
more general way than trough properties. For instance, suppose one
wants to raise an error if a given attribute 'a' is accessed, both
from the class and from the instance: a property cannot help here,
since it works only from the instance. The solution is the following
custom descriptor:</p>
<blockquote>
<pre class="literal-block">
#<oopp.py>
class AccessError(object):
"""Descriptor raising an AttributeError when the attribute is
accessed""" #could be done with a property
def __init__(self,errormessage):
self.msg=errormessage
def __get__(self,obj,cls=None):
raise AttributeError(self.msg)
#</oopp.py>
>>> from oopp import AccessError
>>> class C(object):
... a=AccessError("'a' cannot be accessed")
>>> c=C()
>>> c.a #error
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "oopp.py", line 313, in __get__
raise AttributeError(self.msg)
AttributeError: 'a' cannot be accessed
>>> C.a #error
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "oopp.py", line 313, in __get__
raise AttributeError(self.msg)
AttributeError: 'a' cannot be accessed
</pre>
</blockquote>
<p>It is always possibile to convert plain attributes (i.e. attributes
without a "__get__" method) to descriptor objects:</p>
<blockquote>
<pre class="literal-block">
#<oopp.py>
class convert2descriptor(object):
"""To all practical means, this class acts as a function that, given an
object, adds to it a __get__ method if it is not already there. The
added __get__ method is trivial and simply returns the original object,
independently from obj and cls."""
def __new__(cls,a):
if hasattr(a,"__get__"): # do nothing
return a # a is already a descriptor
else: # creates a trivial attribute descriptor
cls.a=a
return object.__new__(cls)
def __get__(self,obj,cls=None):
"Returns self.a independently from obj and cls"
return self.a
#</oopp.py>
</pre>
</blockquote>
<p>This example also shows the magic of <tt class="literal"><span class="pre">__new__</span></tt>, that allows to use a
class as a function. The output of 'convert2descriptor(a)' can be both
an instance of 'convert2descriptor' (in this case 'convert2descriptor' acts as
a normal class, i.e. as an object factory) or 'a' itself
(if 'a' is already a descriptor): in this case 'convert2descriptor' acts
as a function.</p>
<p>For instance, a string is converted to a descriptor</p>
<blockquote>
<pre class="doctest-block">
>>> from oopp import convert2descriptor
>>> a2=convert2descriptor('a')
>>> a2
<oopp.convert2descriptor object at 0x4017506c>
>>> a2.__get__('whatever')
'a'
</pre>
</blockquote>
<p>whereas a function is untouched:</p>
<blockquote>
<pre class="doctest-block">
>>> def f(): pass
>>> f2=convert2descriptor(f) # does nothing
>>> f2
<function f at 0x4019110c>
</pre>
</blockquote>
</div>
<div class="section" id="data-descriptors">
<h1><a name="data-descriptors">Data descriptors</a></h1>
<p>It is also possible to specify a <tt class="literal"><span class="pre">__set__</span></tt> method (descriptors
with a <tt class="literal"><span class="pre">__set__</span></tt> method are typically data descriptors) with
the signature <tt class="literal"><span class="pre">__set__(self,obj,value)</span></tt> as in the following
example:</p>
<blockquote>
<pre class="literal-block">
#<datadescr.py>
class DataDescriptor(object):
value=None
def __get__(self, obj, cls=None):
if obj is None: obj=cls
print "Getting",obj,"value =",self.value
return self.value
def __set__(self, obj, value):
self.value=value
print "Setting",obj,"value =",value
class C(object):
d=DataDescriptor()
c=C()
c.d=1 #calls C.__dict__['d'].__set__(c,1)
c.d #calls C.__dict__['d'].__get__(c,C)
C.d #calls C.__dict__['d'].__get__(None,C)
C.d=0 #does *not* call __set__
print "C.d =",C.d
#</datadescr.py>
</pre>
</blockquote>
<p>With output:</p>
<blockquote>
<pre class="literal-block">
Setting <C object at 0x401bc1ec> value = 1
Getting <C object at 0x401bc42c> value = 1
Getting <class 'C'> value = 1
C.d = 0
</pre>
</blockquote>
<p>With this knowledge, we may now reconsider the clock example given
in chapter 3. #NO!??</p>
<blockquote>
<pre class="doctest-block">
>>> import oopp
>>> class Clock(object): pass
>>> myclock=Clock()
...
>>> myclock.get_time=oopp.get_time # this is a function
>>> Clock.get_time=lambda self : oopp.get_time() # this is a method
</pre>
</blockquote>
<p>In this example, <tt class="literal"><span class="pre">myclock.get_time</span></tt>, which is attached to the <tt class="literal"><span class="pre">myclock</span></tt>
object, is a function, whereas <tt class="literal"><span class="pre">Clock.get_time</span></tt>, which is attached to
the <tt class="literal"><span class="pre">Clock</span></tt> class is a method. We may also check this by using the <tt class="literal"><span class="pre">type</span></tt>
function:</p>
<blockquote>
<pre class="doctest-block">
>>> type(myclock.get_time)
<type 'function'>
</pre>
</blockquote>
<p>whereas</p>
<blockquote>
<pre class="doctest-block">
>>> type(Clock.get_time)
<type 'instance method'>
</pre>
</blockquote>
<p>It must be remarked that user-defined attribute descriptors, just as
properties, allow to arbitrarily change the semantics of the language
and should be used with care.</p>
</div>
<div class="section" id="the-super-attribute-descriptor">
<h1><a name="the-super-attribute-descriptor">The <tt class="literal"><span class="pre">super</span></tt> attribute descriptor</a></h1>
<p>super has also a second form, where it is more used as a descriptor.</p>
<p><tt class="literal"><span class="pre">super</span></tt> objects are attribute descriptors, too, with a <tt class="literal"><span class="pre">__get__</span></tt>
method returning a method-wrapper object:</p>
<blockquote>
<pre class="doctest-block">
>>> super(C,C()).__get__
<method-wrapper object at 0x8161074>
</pre>
</blockquote>
<p>Here I give some example of acceptable call:</p>
<blockquote>
<pre class="doctest-block">
>>> super(C,C()).__get__('whatever')
<super: <class 'C'>, <C object>>
>>> super(C,C()).__get__('whatever','whatever')
<super: <class 'C'>, <C object>>
</pre>
</blockquote>
<p>Unfortunately, for the time being
(i.e. for Python 2.3), the <tt class="literal"><span class="pre">super</span></tt> mechanism has various limitations.
To show the issues, let me start by considering the following base class:</p>
<blockquote>
<pre class="literal-block">
#<oopp.py>
class ExampleBaseClass(PrettyPrinted):
"""Contains a regular method 'm', a staticmethod 's', a classmethod
'c', a property 'p' and a data attribute 'd'."""
m=lambda self: 'regular method of %s' % self
s=staticmethod(lambda : 'staticmethod')
c=classmethod(lambda cls: 'classmethod of %s' % cls)
p=property(lambda self: 'property of %s' % self)
a=AccessError('Expected error')
d='data'
#</oopp.py>
</pre>
</blockquote>
<p>Now, let me derive a new class C from ExampleBaseClass:</p>
<blockquote>
<pre class="doctest-block">
>>> from oopp import ExampleBaseClass
>>> class C(ExampleBaseClass): pass
>>> c=C()
</pre>
</blockquote>
<p>Ideally, we would like to retrieve the methods and attributes of
ExampleBaseClass from C, by using the <tt class="literal"><span class="pre">super</span></tt> mechanism.</p>
<ol class="arabic simple">
<li>We see that <tt class="literal"><span class="pre">super</span></tt> works without problems for regular methods,
staticmethods and classmethods:</li>
</ol>
<blockquote>
<pre class="doctest-block">
>>> super(C,c).m()
'regular method of <C>'
>>> super(C,c).s()
'staticmethod'
>>> super(C,c).c()
"classmethod of <class '__main__.C'>"
</pre>
</blockquote>
<p>It also works for user defined attribute descriptors:</p>
<blockquote>
<pre class="doctest-block">
>>> super(C,c).a # access error
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "oopp.py", line 340, in __get__
raise AttributeError(self.msg)
AttributeError: Expected error
</pre>
</blockquote>
<p>and for properties (only for Python 2.3+):</p>
<blockquote>
<pre class="doctest-block">
>>> ExampleBaseClass.p
<property object at 0x81b30fc>
</pre>
</blockquote>
<p>In Python 2.2 one would get an error, instead</p>
<blockquote>
<pre class="doctest-block">
>>> super(C,c).p #error
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'super' object has no attribute 'p'
</pre>
</blockquote>
<p>3. Moreover, certain attributes of the superclass, such as its
<tt class="literal"><span class="pre">__name__</span></tt>, cannot be retrieved:</p>
<blockquote>
<pre class="doctest-block">
>>> ExampleBaseClass.__name__
'ExampleBaseClass'
>>> super(C,c).__name__ #error
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'super' object has no attribute '__name__'
</pre>
</blockquote>
<ol class="arabic simple" start="4">
<li>There is no direct way to retrieve the methods of the super-superclass
(i.e. the grandmother class, if you wish) or in general the furthest
ancestors, since <tt class="literal"><span class="pre">super</span></tt> does not chain.</li>
<li>Finally, there are some subtle issues with the <tt class="literal"><span class="pre">super(cls)</span></tt> syntax:</li>
</ol>
<blockquote>
<pre class="doctest-block">
>>> super(C).m #(2) error
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'super' object has no attribute 'm'
</pre>
</blockquote>
<p>means <tt class="literal"><span class="pre">super(C).__get__(None,C)</span></tt>, but only
<tt class="literal"><span class="pre">super(C).__get__(c,C).m==super(C,c)</span></tt> works.</p>
<blockquote>
<blockquote>
On the other hand,</blockquote>
<pre class="doctest-block">
>>> super(C).__init__ #(1)
<built-in method __init__ of type object at 0x80e6fc0>
>>> super(C).__new__ #(1)
<built-in method __init__ of type object at 0x80e6fc0>
</pre>
<blockquote>
seems to work, whereas in reality does not. The reason is that since
<tt class="literal"><span class="pre">super</span></tt> objects are instances
of <tt class="literal"><span class="pre">object</span></tt>, they inherit object's methods, and in particular
<tt class="literal"><span class="pre">__init__</span></tt> ; therefore the <tt class="literal"><span class="pre">__init__</span></tt> method in (1) is <em>not</em>
the <tt class="literal"><span class="pre">ExampleBaseClass.__init__</span></tt> method. The point is that <tt class="literal"><span class="pre">super</span></tt>
objects are attribute descriptors and not references to the superclass.</blockquote>
</blockquote>
<p>Probably, in future versions of Python the <tt class="literal"><span class="pre">super</span></tt> mechanism will be
improved. However, for the time being, one must provide a workaround for
dealing with these issues. This will be discussed in the next chapter.</p>
</div>
<div class="section" id="method-wrappers">
<h1><a name="method-wrappers">Method wrappers</a></h1>
<p>One of the most typical applications of attribute descriptors is their
usage as <em>method wrappers</em>.</p>
<p>Suppose, for instance, one wants to add tracing capabilities to
the methods of a class for debugging purposes. The problem
can be solved with a custom descriptor class:</p>
<blockquote>
<pre class="literal-block">
#<oopp.py>
import inspect
class wrappedmethod(Customizable):
"""Customizable method factory intended for derivation.
The wrapper method is overridden in the children."""
logfile=sys.stdout # default
namespace='' # default
def __new__(cls,meth): # meth is a descriptor
if isinstance(meth,FunctionType):
kind=0 # regular method
func=meth
elif isinstance(meth,staticmethod):
kind=1 # static method
func=meth.__get__('whatever')
elif isinstance(meth,classmethod):
kind=2 # class method
func=meth.__get__('whatever','whatever').im_func
elif isinstance(meth,wrappedmethod): # already wrapped
return meth # do nothing
elif inspect.ismethoddescriptor(meth):
kind=0; func=meth # for many builtin methods
else:
return meth # do nothing
self=super(wrappedmethod,cls).__new__(cls)
self.kind=kind; self.func=func # pre-initialize
return self
def __init__(self,meth): # meth not used
self.logfile=self.logfile # default values
self.namespace=self.namespace # copy the current
def __get__(self,obj,cls): # closure
def _(*args,**kw):
if obj is None: o=() # unbound method call
else: o=(obj,) # bound method call
allargs=[o,(),(cls,)][self.kind]+args
return self.wrapper()(*allargs,**kw)
return _ # the wrapped function
# allargs is the only nontrivial line in _; it adds
# 0 - obj if meth is a regular method
# 1 - nothing if meth is a static method
# 2 - cls if meth is a class method
def wrapper(self): return self.func # do nothing, to be overridden
#</oopp.py>
</pre>
</blockquote>
<p>This class is intended for derivation: the wrapper method has to be overridden
in the children in order to introduce the wanted feature. If I want to
implement the capability of tracing methods, I can reuse the <tt class="literal"><span class="pre">with_tracer</span></tt>
closure introduced in chapter 2:</p>
<blockquote>
<pre class="literal-block">
#<oopp.py>
class tracedmethod(wrappedmethod):
def wrapper(self):
return with_tracer(self.func,self.namespace,self.logfile)
#</oopp.py>
</pre>
</blockquote>
<p>Nothing prevents me from introducing timing features by reusing the
<tt class="literal"><span class="pre">with_timer</span></tt> closure:</p>
<blockquote>
<pre class="literal-block">
#<oopp.py>
class timedmethod(wrappedmethod):
iterations=1 # additional default parameter
def __init__(self,meth):
super(timedmethod,self).__init__(self,meth)
self.iterations=self.iterations # copy
def wrapper(self):
return with_timer(self.func,self.namespace,
self.iterations,self.logfile)
#</oopp.py>
</pre>
</blockquote>
<p>Here there is an example of usage:</p>
<p>The dictionary of wrapped functions is then built from an utility function</p>
<blockquote>
<pre class="literal-block">
#<oopp.py>
def wrap(obj,wrapped,condition=lambda k,v: True, err=None):
"Retrieves obj's dictionary and wraps it"
if isinstance(obj,dict): # obj is a dictionary
dic=obj
else:
dic=getattr(obj,'__dict__',{}).copy() # avoids dictproxy objects
if not dic: dic=attributes(obj) # for simple objects
wrapped.namespace=getattr(obj,'__name__','')
for name,attr in dic.iteritems(): # modify dic
if condition(name,attr): dic[name]=wrapped(attr)
if not isinstance(obj,dict): # modify obj
customize(obj,err,**dic)
#</oopp.py>
</pre>
<pre class="literal-block">
#<tracingmethods.py>
from oopp import *
class C(object):
"Class with traced methods"
def f(self): return self
f=tracedmethod(f)
g=staticmethod(lambda:None)
g=tracedmethod(g)
h=classmethod(do_nothing)
h=tracedmethod(h)
c=C()
#unbound calls
C.f(c)
C.g()
C.h()
#bound calls
c.f()
c.g()
c.h()
#</tracingmethods.py>
</pre>
</blockquote>
<p>Output:</p>
<blockquote>
<pre class="literal-block">
[C] Calling 'f' with arguments
(<C object at 0x402042cc>,){} ...
-> 'C.f' called with result: <C object at 0x402042cc>
[C] Calling '<lambda>' with arguments
(){} ...
-> 'C.<lambda>' called with result: None
[C] Calling 'do_nothing' with arguments
(<class 'C'>,){} ...
-> 'C.do_nothing' called with result: None
[C] Calling 'f' with arguments
(<C object at 0x402042cc>,){} ...
-> 'C.f' called with result: <C object at 0x402042cc>
[C] Calling '<lambda>' with arguments
(){} ...
-> 'C.<lambda>' called with result: None
[C] Calling 'do_nothing' with arguments
(<class 'C'>,){} ...
-> 'C.do_nothing' called with result: None
</pre>
</blockquote>
<p>The approach in 'tracingmethods.py' works, but it is far from
being elegant, since I had to explicitly wrap each method in the
class by hand.</p>
<p>Both problems can be avoided.</p>
<blockquote>
<pre class="doctest-block">
>>> from oopp import *
>>> wrap(Clock,tracedmethod)
>>> Clock.get_time()
[Clock] Calling 'get_time' with arguments
(){} ...
-> 'Clock.get_time' called with result: 21:56:52
'21:56:52'
</pre>
</blockquote>
</div>
</div>
<hr class="footer"/>
<div class="footer">
<a class="reference" href="descr.txt">View document source</a>.
Generated on: 2004-02-18 07:07 UTC.
Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
</div>
</body>
</html>
|
