diff options
| author | micheles <micheles@micheles-mac> | 2010-05-22 09:08:40 +0200 |
|---|---|---|
| committer | micheles <micheles@micheles-mac> | 2010-05-22 09:08:40 +0200 |
| commit | 4ffb6bd356fdfd4451402e5141e3deb552340a32 (patch) | |
| tree | 255ae9f8efb6ce1ead5d961da0977edea2c9d976 /decorator/documentation.html | |
| parent | dfc0684ec52f422f209e4651ab8b0937d9cad115 (diff) | |
| download | micheles-4ffb6bd356fdfd4451402e5141e3deb552340a32.tar.gz | |
Version 3.2 of the decorator module
Diffstat (limited to 'decorator/documentation.html')
| -rw-r--r-- | decorator/documentation.html | 1013 |
1 files changed, 1013 insertions, 0 deletions
diff --git a/decorator/documentation.html b/decorator/documentation.html new file mode 100644 index 0000000..2bca550 --- /dev/null +++ b/decorator/documentation.html @@ -0,0 +1,1013 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<meta name="generator" content="Docutils 0.6: http://docutils.sourceforge.net/" /> +<title>The decorator module</title> +<meta name="author" content="Michele Simionato" /> +<style type="text/css"> + +.highlight { background: #f8f8f8; } +.highlight .c { color: #408080; font-style: italic } /* Comment */ +.highlight .err { border: 1px solid #FF0000 } /* Error */ +.highlight .k { color: #008000; font-weight: bold } /* Keyword */ +.highlight .o { color: #666666 } /* Operator */ +.highlight .cm { color: #408080; font-style: italic } /* Comment.Multiline */ +.highlight .cp { color: #BC7A00 } /* Comment.Preproc */ +.highlight .c1 { color: #408080; font-style: italic } /* Comment.Single */ +.highlight .cs { color: #408080; font-style: italic } /* Comment.Special */ +.highlight .gd { color: #A00000 } /* Generic.Deleted */ +.highlight .ge { font-style: italic } /* Generic.Emph */ +.highlight .gr { color: #FF0000 } /* Generic.Error */ +.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */ +.highlight .gi { color: #00A000 } /* Generic.Inserted */ +.highlight .go { color: #808080 } /* Generic.Output */ +.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */ +.highlight .gs { font-weight: bold } /* Generic.Strong */ +.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */ +.highlight .gt { color: #0040D0 } /* Generic.Traceback */ +.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */ +.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */ +.highlight .kp { color: #008000 } /* Keyword.Pseudo */ +.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */ +.highlight .kt { color: #008000; font-weight: bold } /* Keyword.Type */ +.highlight .m { color: #666666 } /* Literal.Number */ +.highlight .s { color: #BA2121 } /* Literal.String */ +.highlight .na { color: #7D9029 } /* Name.Attribute */ +.highlight .nb { color: #008000 } /* Name.Builtin */ +.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */ +.highlight .no { color: #880000 } /* Name.Constant */ +.highlight .nd { color: #AA22FF } /* Name.Decorator */ +.highlight .ni { color: #999999; font-weight: bold } /* Name.Entity */ +.highlight .ne { color: #D2413A; font-weight: bold } /* Name.Exception */ +.highlight .nf { color: #0000FF } /* Name.Function */ +.highlight .nl { color: #A0A000 } /* Name.Label */ +.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */ +.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */ +.highlight .nv { color: #19177C } /* Name.Variable */ +.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */ +.highlight .w { color: #bbbbbb } /* Text.Whitespace */ +.highlight .mf { color: #666666 } /* Literal.Number.Float */ +.highlight .mh { color: #666666 } /* Literal.Number.Hex */ +.highlight .mi { color: #666666 } /* Literal.Number.Integer */ +.highlight .mo { color: #666666 } /* Literal.Number.Oct */ +.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */ +.highlight .sc { color: #BA2121 } /* Literal.String.Char */ +.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */ +.highlight .s2 { color: #BA2121 } /* Literal.String.Double */ +.highlight .se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */ +.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */ +.highlight .si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */ +.highlight .sx { color: #008000 } /* Literal.String.Other */ +.highlight .sr { color: #BB6688 } /* Literal.String.Regex */ +.highlight .s1 { color: #BA2121 } /* Literal.String.Single */ +.highlight .ss { color: #19177C } /* Literal.String.Symbol */ +.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */ +.highlight .vc { color: #19177C } /* Name.Variable.Class */ +.highlight .vg { color: #19177C } /* Name.Variable.Global */ +.highlight .vi { color: #19177C } /* Name.Variable.Instance */ +.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */ + +</style> +</head> +<body> +<div class="document" id="the-decorator-module"> +<h1 class="title">The <tt class="docutils literal">decorator</tt> module</h1> +<table class="docinfo" frame="void" rules="none"> +<col class="docinfo-name" /> +<col class="docinfo-content" /> +<tbody valign="top"> +<tr><th class="docinfo-name">Author:</th> +<td>Michele Simionato</td></tr> +<tr class="field"><th class="docinfo-name">E-mail:</th><td class="field-body"><a class="reference external" href="mailto:michele.simionato@gmail.com">michele.simionato@gmail.com</a></td> +</tr> +<tr><th class="docinfo-name">Version:</th> +<td>3.2.0 (2010-05-22)</td></tr> +<tr class="field"><th class="docinfo-name">Requires:</th><td class="field-body">Python 2.4+</td> +</tr> +<tr class="field"><th class="docinfo-name">Download page:</th><td class="field-body"><a class="reference external" href="http://pypi.python.org/pypi/decorator/3.2.0">http://pypi.python.org/pypi/decorator/3.2.0</a></td> +</tr> +<tr class="field"><th class="docinfo-name">Installation:</th><td class="field-body"><tt class="docutils literal">easy_install decorator</tt></td> +</tr> +<tr class="field"><th class="docinfo-name">License:</th><td class="field-body">BSD license</td> +</tr> +</tbody> +</table> +<div class="contents topic" id="contents"> +<p class="topic-title first">Contents</p> +<ul class="simple"> +<li><a class="reference internal" href="#introduction" id="id3">Introduction</a></li> +<li><a class="reference internal" href="#definitions" id="id4">Definitions</a></li> +<li><a class="reference internal" href="#statement-of-the-problem" id="id5">Statement of the problem</a></li> +<li><a class="reference internal" href="#the-solution" id="id6">The solution</a></li> +<li><a class="reference internal" href="#a-trace-decorator" id="id7">A <tt class="docutils literal">trace</tt> decorator</a></li> +<li><a class="reference internal" href="#decorator-is-a-decorator" id="id8"><tt class="docutils literal">decorator</tt> is a decorator</a></li> +<li><a class="reference internal" href="#blocking" id="id9"><tt class="docutils literal">blocking</tt></a></li> +<li><a class="reference internal" href="#async" id="id10"><tt class="docutils literal">async</tt></a></li> +<li><a class="reference internal" href="#the-functionmaker-class" id="id11">The <tt class="docutils literal">FunctionMaker</tt> class</a></li> +<li><a class="reference internal" href="#getting-the-source-code" id="id12">Getting the source code</a></li> +<li><a class="reference internal" href="#dealing-with-third-party-decorators" id="id13">Dealing with third party decorators</a></li> +<li><a class="reference internal" href="#caveats-and-limitations" id="id14">Caveats and limitations</a></li> +<li><a class="reference internal" href="#compatibility-notes" id="id15">Compatibility notes</a></li> +<li><a class="reference internal" href="#licence" id="id16">LICENCE</a></li> +</ul> +</div> +<div class="section" id="introduction"> +<h1><a class="toc-backref" href="#id3">Introduction</a></h1> +<p>Python decorators are an interesting example of why syntactic sugar +matters. In principle, their introduction in Python 2.4 changed +nothing, since they do not provide any new functionality which was not +already present in the language. In practice, their introduction has +significantly changed the way we structure our programs in Python. I +believe the change is for the best, and that decorators are a great +idea since:</p> +<ul class="simple"> +<li>decorators help reducing boilerplate code;</li> +<li>decorators help separation of concerns;</li> +<li>decorators enhance readability and maintenability;</li> +<li>decorators are explicit.</li> +</ul> +<p>Still, as of now, writing custom decorators correctly requires +some experience and it is not as easy as it could be. For instance, +typical implementations of decorators involve nested functions, and +we all know that flat is better than nested.</p> +<p>The aim of the <tt class="docutils literal">decorator</tt> module it to simplify the usage of +decorators for the average programmer, and to popularize decorators by +showing various non-trivial examples. Of course, as all techniques, +decorators can be abused (I have seen that) and you should not try to +solve every problem with a decorator, just because you can.</p> +<p>You may find the source code for all the examples +discussed here in the <tt class="docutils literal">documentation.py</tt> file, which contains +this documentation in the form of doctests.</p> +</div> +<div class="section" id="definitions"> +<h1><a class="toc-backref" href="#id4">Definitions</a></h1> +<p>Technically speaking, any Python object which can be called with one argument +can be used as a decorator. However, this definition is somewhat too large +to be really useful. It is more convenient to split the generic class of +decorators in two subclasses:</p> +<ul class="simple"> +<li><em>signature-preserving</em> decorators, i.e. callable objects taking a +function as input and returning a function <em>with the same +signature</em> as output;</li> +<li><em>signature-changing</em> decorators, i.e. decorators that change +the signature of their input function, or decorators returning +non-callable objects.</li> +</ul> +<p>Signature-changing decorators have their use: for instance the +builtin classes <tt class="docutils literal">staticmethod</tt> and <tt class="docutils literal">classmethod</tt> are in this +group, since they take functions and return descriptor objects which +are not functions, nor callables.</p> +<p>However, signature-preserving decorators are more common and easier to +reason about; in particular signature-preserving decorators can be +composed together whereas other decorators in general cannot.</p> +<p>Writing signature-preserving decorators from scratch is not that +obvious, especially if one wants to define proper decorators that +can accept functions with any signature. A simple example will clarify +the issue.</p> +</div> +<div class="section" id="statement-of-the-problem"> +<h1><a class="toc-backref" href="#id5">Statement of the problem</a></h1> +<p>A very common use case for decorators is the memoization of functions. +A <tt class="docutils literal">memoize</tt> decorator works by caching +the result of the function call in a dictionary, so that the next time +the function is called with the same input parameters the result is retrieved +from the cache and not recomputed. There are many implementations of +<tt class="docutils literal">memoize</tt> in <a class="reference external" href="http://www.python.org/moin/PythonDecoratorLibrary">http://www.python.org/moin/PythonDecoratorLibrary</a>, +but they do not preserve the signature. +A simple implementation for Python 2.5 could be the following (notice +that in general it is impossible to memoize correctly something +that depends on non-hashable arguments):</p> +<pre class="literal-block"> +def memoize25(func): + func.cache = {} + def memoize(*args, **kw): + if kw: # frozenset is used to ensure hashability + key = args, frozenset(kw.iteritems()) + else: + key = args + cache = func.cache + if key in cache: + return cache[key] + else: + cache[key] = result = func(*args, **kw) + return result + return functools.update_wrapper(memoize, func) +</pre> +<p>Here we used the <a class="reference external" href="http://www.python.org/doc/2.5.2/lib/module-functools.html">functools.update_wrapper</a> utility, which has +been added in Python 2.5 expressly to simplify the definition of decorators +(in older versions of Python you need to copy the function attributes +<tt class="docutils literal">__name__</tt>, <tt class="docutils literal">__doc__</tt>, <tt class="docutils literal">__module__</tt> and <tt class="docutils literal">__dict__</tt> +from the original function to the decorated function by hand).</p> +<p>The implementation above works in the sense that the decorator +can accept functions with generic signatures; unfortunately this +implementation does <em>not</em> define a signature-preserving decorator, since in +general <tt class="docutils literal">memoize25</tt> returns a function with a +<em>different signature</em> from the original function.</p> +<p>Consider for instance the following case:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@memoize25</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">f1</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> +<span class="o">...</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">1</span><span class="p">)</span> <span class="c"># simulate some long computation</span> +<span class="o">...</span> <span class="k">return</span> <span class="n">x</span> +</pre></div> + +</div> +<p>Here the original function takes a single argument named <tt class="docutils literal">x</tt>, +but the decorated function takes any number of arguments and +keyword arguments:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="kn">from</span> <span class="nn">inspect</span> <span class="kn">import</span> <span class="n">getargspec</span> +<span class="o">>>></span> <span class="k">print</span> <span class="n">getargspec</span><span class="p">(</span><span class="n">f1</span><span class="p">)</span> +<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[],</span> <span class="n">varargs</span><span class="o">=</span><span class="s">'args'</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="s">'kw'</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="bp">None</span><span class="p">)</span> +</pre></div> + +</div> +<p>This means that introspection tools such as pydoc will give +wrong informations about the signature of <tt class="docutils literal">f1</tt>. This is pretty bad: +pydoc will tell you that the function accepts a generic signature +<tt class="docutils literal">*args</tt>, <tt class="docutils literal">**kw</tt>, but when you try to call the function with more than an +argument, you will get an error:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="n">f1</span><span class="p">(</span><span class="mf">0</span><span class="p">,</span> <span class="mf">1</span><span class="p">)</span> +<span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> + <span class="o">...</span> +<span class="ne">TypeError</span><span class="p">:</span> <span class="n">f1</span><span class="p">()</span> <span class="n">takes</span> <span class="n">exactly</span> <span class="mf">1</span> <span class="n">argument</span> <span class="p">(</span><span class="mf">2</span> <span class="n">given</span><span class="p">)</span> +</pre></div> + +</div> +</div> +<div class="section" id="the-solution"> +<h1><a class="toc-backref" href="#id6">The solution</a></h1> +<p>The solution is to provide a generic factory of generators, which +hides the complexity of making signature-preserving decorators +from the application programmer. The <tt class="docutils literal">decorator</tt> function in +the <tt class="docutils literal">decorator</tt> module is such a factory:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="kn">from</span> <span class="nn">decorator</span> <span class="kn">import</span> <span class="n">decorator</span> +</pre></div> + +</div> +<p><tt class="docutils literal">decorator</tt> takes two arguments, a caller function describing the +functionality of the decorator and a function to be decorated; it +returns the decorated function. The caller function must have +signature <tt class="docutils literal">(f, *args, **kw)</tt> and it must call the original function <tt class="docutils literal">f</tt> +with arguments <tt class="docutils literal">args</tt> and <tt class="docutils literal">kw</tt>, implementing the wanted capability, +i.e. memoization in this case:</p> +<pre class="literal-block"> +def _memoize(func, *args, **kw): + if kw: # frozenset is used to ensure hashability + key = args, frozenset(kw.iteritems()) + else: + key = args + cache = func.cache # attributed added by memoize + if key in cache: + return cache[key] + else: + cache[key] = result = func(*args, **kw) + return result +</pre> +<p>At this point you can define your decorator as follows:</p> +<pre class="literal-block"> +def memoize(f): + f.cache = {} + return decorator(_memoize, f) +</pre> +<p>The difference with respect to the Python 2.5 approach, which is based +on nested functions, is that the decorator module forces you to lift +the inner function at the outer level (<em>flat is better than nested</em>). +Moreover, you are forced to pass explicitly the function you want to +decorate to the caller function.</p> +<p>Here is a test of usage:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@memoize</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">heavy_computation</span><span class="p">():</span> +<span class="o">...</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">2</span><span class="p">)</span> +<span class="o">...</span> <span class="k">return</span> <span class="s">"done"</span> + +<span class="o">>>></span> <span class="k">print</span> <span class="n">heavy_computation</span><span class="p">()</span> <span class="c"># the first time it will take 2 seconds</span> +<span class="n">done</span> + +<span class="o">>>></span> <span class="k">print</span> <span class="n">heavy_computation</span><span class="p">()</span> <span class="c"># the second time it will be instantaneous</span> +<span class="n">done</span> +</pre></div> + +</div> +<p>The signature of <tt class="docutils literal">heavy_computation</tt> is the one you would expect:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="k">print</span> <span class="n">getargspec</span><span class="p">(</span><span class="n">heavy_computation</span><span class="p">)</span> +<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[],</span> <span class="n">varargs</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="bp">None</span><span class="p">)</span> +</pre></div> + +</div> +</div> +<div class="section" id="a-trace-decorator"> +<h1><a class="toc-backref" href="#id7">A <tt class="docutils literal">trace</tt> decorator</a></h1> +<p>As an additional example, here is how you can define a trivial +<tt class="docutils literal">trace</tt> decorator, which prints a message everytime the traced +function is called:</p> +<pre class="literal-block"> +def _trace(f, *args, **kw): + print "calling %s with args %s, %s" % (f.__name__, args, kw) + return f(*args, **kw) +</pre> +<pre class="literal-block"> +def trace(f): + return decorator(_trace, f) +</pre> +<p>Here is an example of usage:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">f1</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> +<span class="o">...</span> <span class="k">pass</span> +</pre></div> + +</div> +<p>It is immediate to verify that <tt class="docutils literal">f1</tt> works</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="n">f1</span><span class="p">(</span><span class="mf">0</span><span class="p">)</span> +<span class="n">calling</span> <span class="n">f1</span> <span class="k">with</span> <span class="n">args</span> <span class="p">(</span><span class="mf">0</span><span class="p">,),</span> <span class="p">{}</span> +</pre></div> + +</div> +<p>and it that it has the correct signature:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="k">print</span> <span class="n">getargspec</span><span class="p">(</span><span class="n">f1</span><span class="p">)</span> +<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[</span><span class="s">'x'</span><span class="p">],</span> <span class="n">varargs</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="bp">None</span><span class="p">)</span> +</pre></div> + +</div> +<p>The same decorator works with functions of any signature:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="mf">1</span><span class="p">,</span> <span class="n">z</span><span class="o">=</span><span class="mf">2</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> +<span class="o">...</span> <span class="k">pass</span> + +<span class="o">>>></span> <span class="n">f</span><span class="p">(</span><span class="mf">0</span><span class="p">,</span> <span class="mf">3</span><span class="p">)</span> +<span class="n">calling</span> <span class="n">f</span> <span class="k">with</span> <span class="n">args</span> <span class="p">(</span><span class="mf">0</span><span class="p">,</span> <span class="mf">3</span><span class="p">,</span> <span class="mf">2</span><span class="p">),</span> <span class="p">{}</span> + +<span class="o">>>></span> <span class="k">print</span> <span class="n">getargspec</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> +<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[</span><span class="s">'x'</span><span class="p">,</span> <span class="s">'y'</span><span class="p">,</span> <span class="s">'z'</span><span class="p">],</span> <span class="n">varargs</span><span class="o">=</span><span class="s">'args'</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="s">'kw'</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="p">(</span><span class="mf">1</span><span class="p">,</span> <span class="mf">2</span><span class="p">))</span> +</pre></div> + +</div> +<p>That includes even functions with exotic signatures like the following:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">exotic_signature</span><span class="p">((</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span><span class="o">=</span><span class="p">(</span><span class="mf">1</span><span class="p">,</span><span class="mf">2</span><span class="p">)):</span> <span class="k">return</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span> + +<span class="o">>>></span> <span class="k">print</span> <span class="n">getargspec</span><span class="p">(</span><span class="n">exotic_signature</span><span class="p">)</span> +<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[[</span><span class="s">'x'</span><span class="p">,</span> <span class="s">'y'</span><span class="p">]],</span> <span class="n">varargs</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="p">((</span><span class="mf">1</span><span class="p">,</span> <span class="mf">2</span><span class="p">),))</span> +<span class="o">>>></span> <span class="n">exotic_signature</span><span class="p">()</span> +<span class="n">calling</span> <span class="n">exotic_signature</span> <span class="k">with</span> <span class="n">args</span> <span class="p">((</span><span class="mf">1</span><span class="p">,</span> <span class="mf">2</span><span class="p">),),</span> <span class="p">{}</span> +<span class="mf">3</span> +</pre></div> + +</div> +<p>Notice that the support for exotic signatures has been deprecated +in Python 2.6 and removed in Python 3.0.</p> +</div> +<div class="section" id="decorator-is-a-decorator"> +<h1><a class="toc-backref" href="#id8"><tt class="docutils literal">decorator</tt> is a decorator</a></h1> +<p>It may be annoying to write a caller function (like the <tt class="docutils literal">_trace</tt> +function above) and then a trivial wrapper +(<tt class="docutils literal">def trace(f): return decorator(_trace, f)</tt>) every time. For this reason, +the <tt class="docutils literal">decorator</tt> module provides an easy shortcut to convert +the caller function into a signature-preserving decorator: +you can just call <tt class="docutils literal">decorator</tt> with a single argument. +In our example you can just write <tt class="docutils literal">trace = decorator(_trace)</tt>. +The <tt class="docutils literal">decorator</tt> function can also be used as a signature-changing +decorator, just as <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt>. +However, <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt> return generic +objects which are not callable, while <tt class="docutils literal">decorator</tt> returns +signature-preserving decorators, i.e. functions of a single argument. +For instance, you can write directly</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@decorator</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">trace</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> +<span class="o">...</span> <span class="k">print</span> <span class="s">"calling </span><span class="si">%s</span><span class="s"> with args </span><span class="si">%s</span><span class="s">, </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">func_name</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kw</span><span class="p">)</span> +<span class="o">...</span> <span class="k">return</span> <span class="n">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> +</pre></div> + +</div> +<p>and now <tt class="docutils literal">trace</tt> will be a decorator. Actually <tt class="docutils literal">trace</tt> is a <tt class="docutils literal">partial</tt> +object which can be used as a decorator:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="n">trace</span> +<span class="o"><</span><span class="n">function</span> <span class="n">trace</span> <span class="n">at</span> <span class="mf">0</span><span class="n">x</span><span class="o">...></span> +</pre></div> + +</div> +<p>Here is an example of usage:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">func</span><span class="p">():</span> <span class="k">pass</span> + +<span class="o">>>></span> <span class="n">func</span><span class="p">()</span> +<span class="n">calling</span> <span class="n">func</span> <span class="k">with</span> <span class="n">args</span> <span class="p">(),</span> <span class="p">{}</span> +</pre></div> + +</div> +<p>If you are using an old Python version (Python 2.4) the +<tt class="docutils literal">decorator</tt> module provides a poor man replacement for +<tt class="docutils literal">functools.partial</tt>.</p> +</div> +<div class="section" id="blocking"> +<h1><a class="toc-backref" href="#id9"><tt class="docutils literal">blocking</tt></a></h1> +<p>Sometimes one has to deal with blocking resources, such as <tt class="docutils literal">stdin</tt>, and +sometimes it is best to have back a "busy" message than to block everything. +This behavior can be implemented with a suitable family of decorators, +where the parameter is the busy message:</p> +<pre class="literal-block"> +def blocking(not_avail): + def blocking(f, *args, **kw): + if not hasattr(f, "thread"): # no thread running + def set_result(): f.result = f(*args, **kw) + f.thread = threading.Thread(None, set_result) + f.thread.start() + return not_avail + elif f.thread.isAlive(): + return not_avail + else: # the thread is ended, return the stored result + del f.thread + return f.result + return decorator(blocking) +</pre> +<p>Functions decorated with <tt class="docutils literal">blocking</tt> will return a busy message if +the resource is unavailable, and the intended result if the resource is +available. For instance:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@blocking</span><span class="p">(</span><span class="s">"Please wait ..."</span><span class="p">)</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">read_data</span><span class="p">():</span> +<span class="o">...</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">3</span><span class="p">)</span> <span class="c"># simulate a blocking resource</span> +<span class="o">...</span> <span class="k">return</span> <span class="s">"some data"</span> + +<span class="o">>>></span> <span class="k">print</span> <span class="n">read_data</span><span class="p">()</span> <span class="c"># data is not available yet</span> +<span class="n">Please</span> <span class="n">wait</span> <span class="o">...</span> + +<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">1</span><span class="p">)</span> +<span class="o">>>></span> <span class="k">print</span> <span class="n">read_data</span><span class="p">()</span> <span class="c"># data is not available yet</span> +<span class="n">Please</span> <span class="n">wait</span> <span class="o">...</span> + +<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">1</span><span class="p">)</span> +<span class="o">>>></span> <span class="k">print</span> <span class="n">read_data</span><span class="p">()</span> <span class="c"># data is not available yet</span> +<span class="n">Please</span> <span class="n">wait</span> <span class="o">...</span> + +<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">1.1</span><span class="p">)</span> <span class="c"># after 3.1 seconds, data is available</span> +<span class="o">>>></span> <span class="k">print</span> <span class="n">read_data</span><span class="p">()</span> +<span class="n">some</span> <span class="n">data</span> +</pre></div> + +</div> +</div> +<div class="section" id="async"> +<h1><a class="toc-backref" href="#id10"><tt class="docutils literal">async</tt></a></h1> +<p>We have just seen an examples of a simple decorator factory, +implemented as a function returning a decorator. +For more complex situations, it is more +convenient to implement decorator factories as classes returning +callable objects that can be used as signature-preserving +decorators. The suggested pattern to do that is to introduce +a helper method <tt class="docutils literal">call(self, func, *args, **kw)</tt> and to call +it in the <tt class="docutils literal">__call__(self, func)</tt> method.</p> +<p>As an example, here I show a decorator +which is able to convert a blocking function into an asynchronous +function. The function, when called, +is executed in a separate thread. Moreover, it is possible to set +three callbacks <tt class="docutils literal">on_success</tt>, <tt class="docutils literal">on_failure</tt> and <tt class="docutils literal">on_closing</tt>, +to specify how to manage the function call. +The implementation is the following:</p> +<pre class="literal-block"> +def on_success(result): # default implementation + "Called on the result of the function" + return result +</pre> +<pre class="literal-block"> +def on_failure(exc_info): # default implementation + "Called if the function fails" + pass +</pre> +<pre class="literal-block"> +def on_closing(): # default implementation + "Called at the end, both in case of success and failure" + pass +</pre> +<pre class="literal-block"> +class Async(object): + """ + A decorator converting blocking functions into asynchronous + functions, by using threads or processes. Examples: + + async_with_threads = Async(threading.Thread) + async_with_processes = Async(multiprocessing.Process) + """ + + def __init__(self, threadfactory): + self.threadfactory = threadfactory + + def __call__(self, func, on_success=on_success, + on_failure=on_failure, on_closing=on_closing): + # every decorated function has its own independent thread counter + func.counter = itertools.count(1) + func.on_success = on_success + func.on_failure = on_failure + func.on_closing = on_closing + return decorator(self.call, func) + + def call(self, func, *args, **kw): + def func_wrapper(): + try: + result = func(*args, **kw) + except: + func.on_failure(sys.exc_info()) + else: + return func.on_success(result) + finally: + func.on_closing() + name = '%s-%s' % (func.__name__, func.counter.next()) + thread = self.threadfactory(None, func_wrapper, name) + thread.start() + return thread +</pre> +<p>The decorated function returns +the current execution thread, which can be stored and checked later, for +instance to verify that the thread <tt class="docutils literal">.isAlive()</tt>.</p> +<p>Here is an example of usage. Suppose one wants to write some data to +an external resource which can be accessed by a single user at once +(for instance a printer). Then the access to the writing function must +be locked. Here is a minimalistic example:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="n">async</span> <span class="o">=</span> <span class="n">Async</span><span class="p">(</span><span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">)</span> + +<span class="o">>>></span> <span class="n">datalist</span> <span class="o">=</span> <span class="p">[]</span> <span class="c"># for simplicity the written data are stored into a list.</span> + +<span class="o">>>></span> <span class="nd">@async</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">write</span><span class="p">(</span><span class="n">data</span><span class="p">):</span> +<span class="o">...</span> <span class="c"># append data to the datalist by locking</span> +<span class="o">...</span> <span class="k">with</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">():</span> +<span class="o">...</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">1</span><span class="p">)</span> <span class="c"># emulate some long running operation</span> +<span class="o">...</span> <span class="n">datalist</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> +<span class="o">...</span> <span class="c"># other operations not requiring a lock here</span> +</pre></div> + +</div> +<p>Each call to <tt class="docutils literal">write</tt> will create a new writer thread, but there will +be no synchronization problems since <tt class="docutils literal">write</tt> is locked.</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="n">write</span><span class="p">(</span><span class="s">"data1"</span><span class="p">)</span> +<span class="o"><</span><span class="n">Thread</span><span class="p">(</span><span class="n">write</span><span class="o">-</span><span class="mf">1</span><span class="p">,</span> <span class="n">started</span><span class="o">...</span><span class="p">)</span><span class="o">></span> + +<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="o">.</span><span class="mf">1</span><span class="p">)</span> <span class="c"># wait a bit, so we are sure data2 is written after data1</span> + +<span class="o">>>></span> <span class="n">write</span><span class="p">(</span><span class="s">"data2"</span><span class="p">)</span> +<span class="o"><</span><span class="n">Thread</span><span class="p">(</span><span class="n">write</span><span class="o">-</span><span class="mf">2</span><span class="p">,</span> <span class="n">started</span><span class="o">...</span><span class="p">)</span><span class="o">></span> + +<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">2</span><span class="p">)</span> <span class="c"># wait for the writers to complete</span> + +<span class="o">>>></span> <span class="k">print</span> <span class="n">datalist</span> +<span class="p">[</span><span class="s">'data1'</span><span class="p">,</span> <span class="s">'data2'</span><span class="p">]</span> +</pre></div> + +</div> +</div> +<div class="section" id="the-functionmaker-class"> +<h1><a class="toc-backref" href="#id11">The <tt class="docutils literal">FunctionMaker</tt> class</a></h1> +<p>You may wonder about how the functionality of the <tt class="docutils literal">decorator</tt> module +is implemented. The basic building block is +a <tt class="docutils literal">FunctionMaker</tt> class which is able to generate on the fly +functions with a given name and signature from a function template +passed as a string. Generally speaking, you should not need to +resort to <tt class="docutils literal">FunctionMaker</tt> when writing ordinary decorators, but +it is handy in some circumstances. You will see an example shortly, in +the implementation of a cool decorator utility (<tt class="docutils literal">decorator_apply</tt>).</p> +<p><tt class="docutils literal">FunctionMaker</tt> provides a <tt class="docutils literal">.create</tt> classmethod which +takes as input the name, signature, and body of the function +we want to generate as well as the execution environment +were the function is generated by <tt class="docutils literal">exec</tt>. Here is an example:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> <span class="c"># a function with a generic signature</span> +<span class="o">...</span> <span class="k">print</span> <span class="n">args</span><span class="p">,</span> <span class="n">kw</span> + +<span class="o">>>></span> <span class="n">f1</span> <span class="o">=</span> <span class="n">FunctionMaker</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="s">'f1(a, b)'</span><span class="p">,</span> <span class="s">'f(a, b)'</span><span class="p">,</span> <span class="nb">dict</span><span class="p">(</span><span class="n">f</span><span class="o">=</span><span class="n">f</span><span class="p">))</span> +<span class="o">>>></span> <span class="n">f1</span><span class="p">(</span><span class="mf">1</span><span class="p">,</span><span class="mf">2</span><span class="p">)</span> +<span class="p">(</span><span class="mf">1</span><span class="p">,</span> <span class="mf">2</span><span class="p">)</span> <span class="p">{}</span> +</pre></div> + +</div> +<p>It is important to notice that the function body is interpolated +before being executed, so be careful with the <tt class="docutils literal">%</tt> sign!</p> +<p><tt class="docutils literal">FunctionMaker.create</tt> also accepts keyword arguments and such +arguments are attached to the resulting function. This is useful +if you want to set some function attributes, for instance the +docstring <tt class="docutils literal">__doc__</tt>.</p> +<p>For debugging/introspection purposes it may be useful to see +the source code of the generated function; to do that, just +pass the flag <tt class="docutils literal">addsource=True</tt> and a <tt class="docutils literal">__source__</tt> attribute will +be added to the generated function:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="n">f1</span> <span class="o">=</span> <span class="n">FunctionMaker</span><span class="o">.</span><span class="n">create</span><span class="p">(</span> +<span class="o">...</span> <span class="s">'f1(a, b)'</span><span class="p">,</span> <span class="s">'f(a, b)'</span><span class="p">,</span> <span class="nb">dict</span><span class="p">(</span><span class="n">f</span><span class="o">=</span><span class="n">f</span><span class="p">),</span> <span class="n">addsource</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> +<span class="o">>>></span> <span class="k">print</span> <span class="n">f1</span><span class="o">.</span><span class="n">__source__</span> +<span class="k">def</span> <span class="nf">f1</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span> + <span class="n">f</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span> +<span class="o"><</span><span class="n">BLANKLINE</span><span class="o">></span> +</pre></div> + +</div> +<p><tt class="docutils literal">FunctionMaker.create</tt> can take as first argument a string, +as in the examples before, or a function. This is the most common +usage, since typically you want to decorate a pre-existing +function. A framework author may want to use directly <tt class="docutils literal">FunctionMaker.create</tt> +instead of <tt class="docutils literal">decorator</tt>, since it gives you direct access to the body +of the generated function. For instance, suppose you want to instrument +the <tt class="docutils literal">__init__</tt> methods of a set of classes, by preserving their +signature (such use case is not made up; this is done in SQAlchemy +and in other frameworks). When the first argument of <tt class="docutils literal">FunctionMaker.create</tt> +is a function, a <tt class="docutils literal">FunctionMaker</tt> object is instantiated internally, +with attributes <tt class="docutils literal">args</tt>, <tt class="docutils literal">varargs</tt>, +<tt class="docutils literal">keywords</tt> and <tt class="docutils literal">defaults</tt> which are the +the return values of the standard library function <tt class="docutils literal">inspect.getargspec</tt>. +For each argument in the <tt class="docutils literal">args</tt> (which is a list of strings containing +the names of the mandatory arguments) an attribute <tt class="docutils literal">arg0</tt>, <tt class="docutils literal">arg1</tt>, +..., <tt class="docutils literal">argN</tt> is also generated. Finally, there is a <tt class="docutils literal">signature</tt> +attribute, a string with the signature of the original function.</p> +<p>Notice that while I do not have plans +to change or remove the functionality provided in the +<tt class="docutils literal">FunctionMaker</tt> class, I do not guarantee that it will stay +unchanged forever. For instance, right now I am using the traditional +string interpolation syntax for function templates, but Python 2.6 +and Python 3.0 provide a newer interpolation syntax and I may use +the new syntax in the future. +On the other hand, the functionality provided by +<tt class="docutils literal">decorator</tt> has been there from version 0.1 and it is guaranteed to +stay there forever.</p> +</div> +<div class="section" id="getting-the-source-code"> +<h1><a class="toc-backref" href="#id12">Getting the source code</a></h1> +<p>Internally <tt class="docutils literal">FunctionMaker.create</tt> uses <tt class="docutils literal">exec</tt> to generate the +decorated function. Therefore +<tt class="docutils literal">inspect.getsource</tt> will not work for decorated functions. That +means that the usual '??' trick in IPython will give you the (right on +the spot) message <tt class="docutils literal">Dynamically generated function. No source code +available</tt>. In the past I have considered this acceptable, since +<tt class="docutils literal">inspect.getsource</tt> does not really work even with regular +decorators. In that case <tt class="docutils literal">inspect.getsource</tt> gives you the wrapper +source code which is probably not what you want:</p> +<pre class="literal-block"> +def identity_dec(func): + def wrapper(*args, **kw): + return func(*args, **kw) + return wrapper +</pre> +<div class="codeblock python"> +<div class="highlight"><pre><span class="nd">@identity_dec</span> +<span class="k">def</span> <span class="nf">example</span><span class="p">():</span> <span class="k">pass</span> + +<span class="o">>>></span> <span class="k">print</span> <span class="n">inspect</span><span class="o">.</span><span class="n">getsource</span><span class="p">(</span><span class="n">example</span><span class="p">)</span> + <span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> + <span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> +<span class="o"><</span><span class="n">BLANKLINE</span><span class="o">></span> +</pre></div> + +</div> +<p>(see bug report <a class="reference external" href="http://bugs.python.org/issue1764286">1764286</a> for an explanation of what is happening). +Unfortunately the bug is still there, even in Python 2.6 and 3.0. +There is however a workaround. The decorator module adds an +attribute <tt class="docutils literal">.undecorated</tt> to the decorated function, containing +a reference to the original function. The easy way to get +the source code is to call <tt class="docutils literal">inspect.getsource</tt> on the +undecorated function:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="k">print</span> <span class="n">inspect</span><span class="o">.</span><span class="n">getsource</span><span class="p">(</span><span class="n">factorial</span><span class="o">.</span><span class="n">undecorated</span><span class="p">)</span> +<span class="nd">@tail_recursive</span> +<span class="k">def</span> <span class="nf">factorial</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">acc</span><span class="o">=</span><span class="mf">1</span><span class="p">):</span> + <span class="s">"The good old factorial"</span> + <span class="k">if</span> <span class="n">n</span> <span class="o">==</span> <span class="mf">0</span><span class="p">:</span> <span class="k">return</span> <span class="n">acc</span> + <span class="k">return</span> <span class="n">factorial</span><span class="p">(</span><span class="n">n</span><span class="o">-</span><span class="mf">1</span><span class="p">,</span> <span class="n">n</span><span class="o">*</span><span class="n">acc</span><span class="p">)</span> +<span class="o"><</span><span class="n">BLANKLINE</span><span class="o">></span> +</pre></div> + +</div> +</div> +<div class="section" id="dealing-with-third-party-decorators"> +<h1><a class="toc-backref" href="#id13">Dealing with third party decorators</a></h1> +<p>Sometimes you find on the net some cool decorator that you would +like to include in your code. However, more often than not the cool +decorator is not signature-preserving. Therefore you may want an easy way to +upgrade third party decorators to signature-preserving decorators without +having to rewrite them in terms of <tt class="docutils literal">decorator</tt>. You can use a +<tt class="docutils literal">FunctionMaker</tt> to implement that functionality as follows:</p> +<pre class="literal-block"> +def decorator_apply(dec, func): + """ + Decorate a function by preserving the signature even if dec + is not a signature-preserving decorator. + """ + return FunctionMaker.create( + func, 'return decorated(%(signature)s)', + dict(decorated=dec(func)), undecorated=func) +</pre> +<p><tt class="docutils literal">decorator_apply</tt> sets the attribute <tt class="docutils literal">.undecorated</tt> of the generated +function to the original function, so that you can get the right +source code.</p> +<p>Notice that I am not providing this functionality in the <tt class="docutils literal">decorator</tt> +module directly since I think it is best to rewrite the decorator rather +than adding an additional level of indirection. However, practicality +beats purity, so you can add <tt class="docutils literal">decorator_apply</tt> to your toolbox and +use it if you need to.</p> +<p>In order to give an example of usage of <tt class="docutils literal">decorator_apply</tt>, I will show a +pretty slick decorator that converts a tail-recursive function in an iterative +function. I have shamelessly stolen the basic idea from Kay Schluehr's recipe +in the Python Cookbook, +<a class="reference external" href="http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691">http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691</a>.</p> +<pre class="literal-block"> +class TailRecursive(object): + """ + tail_recursive decorator based on Kay Schluehr's recipe + http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691 + with improvements by me and George Sakkis. + """ + + def __init__(self, func): + self.func = func + self.firstcall = True + self.CONTINUE = object() # sentinel + + def __call__(self, *args, **kwd): + CONTINUE = self.CONTINUE + if self.firstcall: + func = self.func + self.firstcall = False + try: + while True: + result = func(*args, **kwd) + if result is CONTINUE: # update arguments + args, kwd = self.argskwd + else: # last call + return result + finally: + self.firstcall = True + else: # return the arguments of the tail call + self.argskwd = args, kwd + return CONTINUE +</pre> +<p>Here the decorator is implemented as a class returning callable +objects.</p> +<pre class="literal-block"> +def tail_recursive(func): + return decorator_apply(TailRecursive, func) +</pre> +<p>Here is how you apply the upgraded decorator to the good old factorial:</p> +<pre class="literal-block"> +@tail_recursive +def factorial(n, acc=1): + "The good old factorial" + if n == 0: return acc + return factorial(n-1, n*acc) +</pre> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="k">print</span> <span class="n">factorial</span><span class="p">(</span><span class="mf">4</span><span class="p">)</span> +<span class="mf">24</span> +</pre></div> + +</div> +<p>This decorator is pretty impressive, and should give you some food for +your mind ;) Notice that there is no recursion limit now, and you can +easily compute <tt class="docutils literal">factorial(1001)</tt> or larger without filling the stack +frame. Notice also that the decorator will not work on functions which +are not tail recursive, such as the following</p> +<pre class="literal-block"> +def fact(n): # this is not tail-recursive + if n == 0: return 1 + return n * fact(n-1) +</pre> +<p>(reminder: a function is tail recursive if it either returns a value without +making a recursive call, or returns directly the result of a recursive +call).</p> +</div> +<div class="section" id="caveats-and-limitations"> +<h1><a class="toc-backref" href="#id14">Caveats and limitations</a></h1> +<p>The first thing you should be aware of, it the fact that decorators +have a performance penalty. +The worse case is shown by the following example:</p> +<pre class="literal-block"> +$ cat performance.sh +python -m timeit -s " +from decorator import decorator + +@decorator +def do_nothing(func, *args, **kw): + return func(*args, **kw) + +@do_nothing +def f(): + pass +" "f()" + +python -m timeit -s " +def f(): + pass +" "f()" +</pre> +<p>On my MacBook, using the <tt class="docutils literal">do_nothing</tt> decorator instead of the +plain function is more than three times slower:</p> +<pre class="literal-block"> +$ bash performance.sh +1000000 loops, best of 3: 0.995 usec per loop +1000000 loops, best of 3: 0.273 usec per loop +</pre> +<p>It should be noted that a real life function would probably do +something more useful than <tt class="docutils literal">f</tt> here, and therefore in real life the +performance penalty could be completely negligible. As always, the +only way to know if there is +a penalty in your specific use case is to measure it.</p> +<p>You should be aware that decorators will make your tracebacks +longer and more difficult to understand. Consider this example:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">f</span><span class="p">():</span> +<span class="o">...</span> <span class="mf">1</span><span class="o">/</span><span class="mf">0</span> +</pre></div> + +</div> +<p>Calling <tt class="docutils literal">f()</tt> will give you a <tt class="docutils literal">ZeroDivisionError</tt>, but since the +function is decorated the traceback will be longer:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="n">f</span><span class="p">()</span> +<span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> + <span class="o">...</span> + <span class="n">File</span> <span class="s">"<string>"</span><span class="p">,</span> <span class="n">line</span> <span class="mf">2</span><span class="p">,</span> <span class="ow">in</span> <span class="n">f</span> + <span class="n">File</span> <span class="s">"<doctest __main__[18]>"</span><span class="p">,</span> <span class="n">line</span> <span class="mf">4</span><span class="p">,</span> <span class="ow">in</span> <span class="n">trace</span> + <span class="k">return</span> <span class="n">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> + <span class="n">File</span> <span class="s">"<doctest __main__[47]>"</span><span class="p">,</span> <span class="n">line</span> <span class="mf">3</span><span class="p">,</span> <span class="ow">in</span> <span class="n">f</span> + <span class="mf">1</span><span class="o">/</span><span class="mf">0</span> +<span class="ne">ZeroDivisionError</span><span class="p">:</span> <span class="n">integer</span> <span class="n">division</span> <span class="ow">or</span> <span class="n">modulo</span> <span class="n">by</span> <span class="n">zero</span> +</pre></div> + +</div> +<p>You see here the inner call to the decorator <tt class="docutils literal">trace</tt>, which calls +<tt class="docutils literal"><span class="pre">f(*args,</span> **kw)</tt>, and a reference to <tt class="docutils literal">File <span class="pre">"<string>",</span> line 2, in f</tt>. +This latter reference is due to the fact that internally the decorator +module uses <tt class="docutils literal">exec</tt> to generate the decorated function. Notice that +<tt class="docutils literal">exec</tt> is <em>not</em> responsibile for the performance penalty, since is the +called <em>only once</em> at function decoration time, and not every time +the decorated function is called.</p> +<p>At present, there is no clean way to avoid <tt class="docutils literal">exec</tt>. A clean solution +would require to change the CPython implementation of functions and +add an hook to make it possible to change their signature directly. +That could happen in future versions of Python (see PEP <a class="reference external" href="http://www.python.org/dev/peps/pep-0362">362</a>) and +then the decorator module would become obsolete. However, at present, +even in Python 3.1 it is impossible to change the function signature +directly, therefore the <tt class="docutils literal">decorator</tt> module is still useful. +Actually, this is one of the main reasons why I keep maintaining +the module and releasing new versions.</p> +<p>In the present implementation, decorators generated by <tt class="docutils literal">decorator</tt> +can only be used on user-defined Python functions or methods, not on generic +callable objects, nor on built-in functions, due to limitations of the +<tt class="docutils literal">inspect</tt> module in the standard library. Moreover, notice +that you can decorate a method, but only before if becomes a bound or unbound +method, i.e. inside the class. +Here is an example of valid decoration:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="k">class</span> <span class="nc">C</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span> +<span class="o">...</span> <span class="nd">@trace</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">meth</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> +<span class="o">...</span> <span class="k">pass</span> +</pre></div> + +</div> +<p>Here is an example of invalid decoration, when the decorator in +called too late:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="k">class</span> <span class="nc">C</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">meth</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> +<span class="o">...</span> <span class="k">pass</span> +<span class="o">...</span> +<span class="o">>>></span> <span class="n">trace</span><span class="p">(</span><span class="n">C</span><span class="o">.</span><span class="n">meth</span><span class="p">)</span> +<span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> + <span class="o">...</span> +<span class="ne">TypeError</span><span class="p">:</span> <span class="n">You</span> <span class="n">are</span> <span class="n">decorating</span> <span class="n">a</span> <span class="n">non</span> <span class="n">function</span><span class="p">:</span> <span class="o"><</span><span class="n">unbound</span> <span class="n">method</span> <span class="n">C</span><span class="o">.</span><span class="n">meth</span><span class="o">></span> +</pre></div> + +</div> +<p>The solution is to extract the inner function from the unbound method:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="n">trace</span><span class="p">(</span><span class="n">C</span><span class="o">.</span><span class="n">meth</span><span class="o">.</span><span class="n">im_func</span><span class="p">)</span> +<span class="o"><</span><span class="n">function</span> <span class="n">meth</span> <span class="n">at</span> <span class="mf">0</span><span class="n">x</span><span class="o">...></span> +</pre></div> + +</div> +<p>There is a restriction on the names of the arguments: for instance, +if try to call an argument <tt class="docutils literal">_call_</tt> or <tt class="docutils literal">_func_</tt> +you will get a <tt class="docutils literal">NameError</tt>:</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> +<span class="o">...</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">_func_</span><span class="p">):</span> <span class="k">print</span> <span class="n">f</span> +<span class="o">...</span> +<span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> + <span class="o">...</span> +<span class="ne">NameError</span><span class="p">:</span> <span class="n">_func_</span> <span class="ow">is</span> <span class="n">overridden</span> <span class="ow">in</span> +<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">_func_</span><span class="p">):</span> + <span class="k">return</span> <span class="n">_call_</span><span class="p">(</span><span class="n">_func_</span><span class="p">,</span> <span class="n">_func_</span><span class="p">)</span> +</pre></div> + +</div> +<p>Finally, the implementation is such that the decorated function contains +a <em>copy</em> of the original function dictionary +(<tt class="docutils literal">vars(decorated_f) is not vars(f)</tt>):</p> +<div class="codeblock python"> +<div class="highlight"><pre><span class="o">>>></span> <span class="k">def</span> <span class="nf">f</span><span class="p">():</span> <span class="k">pass</span> <span class="c"># the original function</span> +<span class="o">>>></span> <span class="n">f</span><span class="o">.</span><span class="n">attr1</span> <span class="o">=</span> <span class="s">"something"</span> <span class="c"># setting an attribute</span> +<span class="o">>>></span> <span class="n">f</span><span class="o">.</span><span class="n">attr2</span> <span class="o">=</span> <span class="s">"something else"</span> <span class="c"># setting another attribute</span> + +<span class="o">>>></span> <span class="n">traced_f</span> <span class="o">=</span> <span class="n">trace</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="c"># the decorated function</span> + +<span class="o">>>></span> <span class="n">traced_f</span><span class="o">.</span><span class="n">attr1</span> +<span class="s">'something'</span> +<span class="o">>>></span> <span class="n">traced_f</span><span class="o">.</span><span class="n">attr2</span> <span class="o">=</span> <span class="s">"something different"</span> <span class="c"># setting attr</span> +<span class="o">>>></span> <span class="n">f</span><span class="o">.</span><span class="n">attr2</span> <span class="c"># the original attribute did not change</span> +<span class="s">'something else'</span> +</pre></div> + +</div> +</div> +<div class="section" id="compatibility-notes"> +<h1><a class="toc-backref" href="#id15">Compatibility notes</a></h1> +<p>Version 3.2 is the first version of the <tt class="docutils literal">decorator</tt> module to officially +support Python 3.0. Actually, the module has supported Python 3.0 from +the beginning, via the <tt class="docutils literal">2to3</tt> conversion tool, but this step has +been now integrated in the build process, thanks to the <a class="reference external" href="http://packages.python.org/distribute/">distribute</a> +project, the Python 3-compatible replacement of easy_install. +The hard work (for me) has been converting the documentation and the +doctests. This has been possibly only now that <a class="reference external" href="http://docutils.sourceforge.net/">docutils</a> and <a class="reference external" href="http://pygments.org/">pygments</a> +have been ported to Python 3.</p> +<p>The <tt class="docutils literal">decorator</tt> module <em>per se</em> does not contain any change, apart +from the removal of the functions <tt class="docutils literal">get_info</tt> and <tt class="docutils literal">new_wrapper</tt>, +which have been deprecated for years. <tt class="docutils literal">get_info</tt> has been removed +since it was little used and since it had to be changed anyway to work +with Python 3.0; <tt class="docutils literal">new_wrapper</tt> has been removed since it was +useless: its major use case (converting signature changing decorators +to signature preserving decorators) has been subsumed by +<tt class="docutils literal">decorator_apply</tt> and the other use case can be managed with the +<tt class="docutils literal">FunctionMaker</tt>.</p> +<p>There are a few changes in the documentation: I removed the +<tt class="docutils literal">decorator_factory</tt> example, which was confusing some of my users, +and I removed the part about exotic signatures in the Python 3 +documentation, since Python 3 does not support them. +Notice that there is no support for Python 3 <a class="reference external" href="http://www.python.org/dev/peps/pep-3107/">function annotations</a> +since it seems premature at the moment, when most people are +still using Python 2.X.</p> +<p>Finally <tt class="docutils literal">decorator</tt> cannot be used as a class decorator and the +<a class="reference external" href="http://www.phyast.pitt.edu/~micheles/python/documentation.html#class-decorators-and-decorator-factories">functionality introduced in version 2.3</a> has been removed. That +means that in order to define decorator factories with classes you +need to define the <tt class="docutils literal">__call__</tt> method explicitly (no magic anymore). +All these changes should not cause any trouble, since they were +all rarely used features. Should you have any trouble, you can always +downgrade to the 2.3 version.</p> +<p>The examples shown here have been tested with Python 2.6. Python 2.4 +is also supported - of course the examples requiring the <tt class="docutils literal">with</tt> +statement will not work there. Python 2.5 works fine, but if you +run the examples here in the interactive interpreter +you will notice a few differences since +<tt class="docutils literal">getargspec</tt> returns an <tt class="docutils literal">ArgSpec</tt> namedtuple instead of a regular +tuple. That means that running the file +<tt class="docutils literal">documentation.py</tt> under Python 2.5 will a few errors, but +they are not serious.</p> +</div> +<div class="section" id="licence"> +<h1><a class="toc-backref" href="#id16">LICENCE</a></h1> +<p>Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met:</p> +<pre class="literal-block"> +Copyright (c) 2005, Michele Simionato +All rights reserved. + +Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +Redistributions in bytecode form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in +the documentation and/or other materials provided with the +distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS +OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR +TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE +USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH +DAMAGE. +</pre> +<p>If you use this software and you are happy with it, consider sending me a +note, just to gratify my ego. On the other hand, if you use this software and +you are unhappy with it, send me a patch!</p> +</div> +</div> +</body> +</html> |