- [feature] Added support for Beaker cache regions

  in templates.   Usage of regions should be considered
  as superseding the very obsolete idea of passing in
  backend options, timeouts, etc. within templates.
- rewrite the cache docs again which had a lot of misleading/inaccurate
info.

5 files changed, 250 insertions, 99 deletions

diff --git a/CHANGES b/CHANGES
index a718726..5f96eec 100644
--- a/CHANGES
+++ b/CHANGES
@@ -12,6 +12,11 @@
   core plugin is the mako.cache.CacheImpl
   class.
 
+- [feature] Added support for Beaker cache regions
+  in templates.   Usage of regions should be considered
+  as superseding the very obsolete idea of passing in 
+  backend options, timeouts, etc. within templates.
+
 - [feature] The 'put' method on Cache is now
   'set'.  'put' is there for backwards compatibility.
 
diff --git a/doc/build/caching.rst b/doc/build/caching.rst
index 4b391f9..f701513 100644
--- a/doc/build/caching.rst
+++ b/doc/build/caching.rst
@@ -20,7 +20,7 @@ method will return content directly from the cache. When the
 :class:`.Template` object itself falls out of scope, its corresponding
 cache is garbage collected along with the template.
 
-By default, caching requires that the ``beaker`` package be installed on the
+By default, caching requires that the `Beaker <http://beaker.readthedocs.org/>`_ package be installed on the
 system, however the mechanism of caching can be customized to use 
 any third party or user defined system - see :ref:`cache_plugins`.
 
@@ -29,7 +29,7 @@ its options can be used with the ``<%def>`` tag as well:
 
 .. sourcecode:: mako
 
-    <%def name="mycomp" cached="True" cache_timeout="30" cache_type="memory">
+    <%def name="mycomp" cached="True" cache_timeout="60">
         other text
     </%def>
 
@@ -37,94 +37,169 @@ its options can be used with the ``<%def>`` tag as well:
 
 .. sourcecode:: mako
 
-    <%block cached="True" cache_timeout="30" cache_type="memory">
+    <%block cached="True" cache_timeout="60">
         other text
     </%block>
 
 Cache arguments
 ================
 
-The various cache arguments are cascaded from their default
-values, to the arguments specified programmatically to the
-:class:`.Template` or its originating :class:`.TemplateLookup`, then to those
-defined in the ``<%page>`` tag of an individual template, and
-finally to an individual ``<%def>`` or ``<%block>`` tag within the template. This
-means you can define, for example, a cache type of ``dbm`` on your
-:class:`.TemplateLookup`, a cache timeout of 60 seconds in a particular
-template's ``<%page>`` tag, and within one of that template's
-``<%def>`` tags ``cache=True``, and that one particular def will
-then cache its data using a ``dbm`` cache and a data timeout of 60
-seconds.
-
-The caching options always available include:
-
-* ``cached="False|True"`` - turn caching on
+Mako has two cache arguments available on tags that are 
+available in all cases.   The rest of the arguments 
+available are specific to a backend.
+
+The two generic tags are:
+
+* ``cached="True"`` - enable caching for this ``<%page>``, 
+  ``<%def>``, or ``<%block>``.
 * ``cache_key`` - the "key" used to uniquely identify this content
-  in the cache.  The total namespace of keys within the cache is
-  local to the current template, and the default value of "key"
-  is the name of the def which is storing its data. It is an
-  evaluable tag, so you can put a Python expression to calculate
-  the value of the key on the fly. For example, heres a page
+  in the cache.   Usually, this key is chosen automatically
+  based on the name of the rendering callable (i.e. ``body``
+  when used in ``<%page>``, the name of the def when using ``<%def>``,
+  the explicit or internally-generated name when using ``<%block>``).
+  Using the ``cache_key`` parameter, the key can be overridden
+  using a fixed or programmatically generated value.
+
+  For example, here's a page
   that caches any page which inherits from it, based on the
   filename of the calling template:
  
-.. sourcecode:: mako
+  .. sourcecode:: mako
 
-    <%page cached="True" cache_key="${self.filename}"/>
+     <%page cached="True" cache_key="${self.filename}"/>
 
-    ${next.body()}
+     ${next.body()}
  
-    ## rest of template
+     ## rest of template
+
+On a :class:`.Template` or :class:`.TemplateLookup`, the 
+caching can be configured using these arguments:
+
+* ``cache_enabled`` - Setting this
+  to ``False`` will disable all caching functionality
+  when the template renders.  Defaults to ``True``.
+  e.g.::
+
+    lookup = TemplateLookup(
+                    directories='/path/to/templates', 
+                    cache_enabled = False
+                    )
+
+* ``cache_impl`` - The string name of the cache backend
+  to use.   This defaults to ``beaker``, which has historically
+  been the only cache backend supported by Mako.  
+  New in 0.6.0.
 
-As of 0.5.1, the ``<%page>``, ``<%def>``, and ``<%block>`` tags 
+  For example, here's how to use the upcoming
+  `dogpile.cache <http://dogpilecache.readthedocs.org>`_
+  backend::
+
+    lookup = TemplateLookup(
+                    directories='/path/to/templates', 
+                    cache_impl = 'dogpile.cache',
+                    cache_args = {'regions':my_dogpile_regions}
+                    )
+
+* ``cache_args`` - A dictionary of cache parameters that
+  will be consumed by the cache backend.   See 
+  :ref:`beaker_backend` for examples.  New in 0.6.0.
+
+Backend-Specific Cache Arguments
+--------------------------------
+
+The ``<%page>``, ``<%def>``, and ``<%block>`` tags 
 accept any named argument that starts with the prefix ``"cache_"``.
 Those arguments are then packaged up and passed along to the
 underlying caching implementation, minus the ``"cache_"`` prefix.
 
-On :class:`.Template` and :class:`.TemplateLookup`, these additional
-arguments are accepted:
-
-* ``cache_impl`` - Name of the caching implementation to use, defaults
-  to ``beaker``.  New in 0.5.1.
-* ``cache_args`` - dictionary of default arguments to send to the 
-  caching system.  The arguments specified on the ``<%page>``, ``<%def>``,
-  and ``<%block>`` tags will supersede what is specified in this dictionary.
-  The names in the dictionary should not have the ``cache_`` prefix.
-  New in 0.5.1.
-
-Prior to version 0.5.1, the following arguments were instead used.  Note
-these arguments remain available and are copied into the newer ``cache_args``
-dictionary when present on :class:`.Template` or :class:`.TemplateLookup`:
-
-* ``cache_dir`` - Equivalent to the ``dir`` argument in the ``cache_args``
-  dictionary.  See the section on Beaker cache options for a description
-  of this argument.
-* ``cache_url`` - Equivalent to the ``url`` argument in the ``cache_args``
-  dictionary.  See the section on Beaker cache options for a description
-  of this argument.
-* ``cache_type`` - Equivalent to the ``type`` argument in the ``cache_args``
-  dictionary.  See the section on Beaker cache options for a description
-  of this argument.
-* ``cache_timeout`` - Equivalent to the ``timeout`` argument in the ``cache_args``
-  dictionary.  See the section on Beaker cache options for a description
-  of this argument.
-
-Beaker Cache Options
----------------------
-
-When using the default caching implementation based on Beaker,
-the following options are also available
-on the ``<%page>``, ``<%def>``, and ``<%block>`` tags, as well
-as within the ``cache_args`` dictionary of :class:`.Template` and
-:class:`.TemplateLookup` without the ``"cache_"`` prefix:
+The actual arguments understood are determined by the backend.
+
+* :ref:`beaker_backend` - Includes arguments understood by
+  Beaker
+* :ref:`mako_plugin` - Includes arguments understood by 
+  dogpile.cache.
+
+.. _beaker_backend:
+
+Using the Beaker Cache Backend
+-------------------------------
+
+When using Beaker, new implementations will want to make usage
+of **cache regions** so that cache configurations can be maintained
+externally to templates.  These configurations live under 
+named "regions" that can be referred to within templates themselves.  
+Support for Beaker cache regions is new in Mako 0.6.0.
+
+For example, suppose we would like two regions.  One is a "short term"
+region that will store content in a memory-based dictionary,
+expiring after 60 seconds.   The other is a Memcached region,
+where values should expire in five minutes.   To configure
+our :class:`.TemplateLookup`, first we get a handle to a 
+:class:`beaker.cache.CacheManager`::
+
+    from beaker.cache import CacheManager
+
+    manager = CacheManager(cache_regions={
+        'short_term':{
+            'type': 'memory',
+            'expire': 60
+        },
+        'long_term':{
+            'type': 'ext:memcached',
+            'url': '127.0.0.1:11211',
+            'expire': 300
+        }
+    })
+
+    lookup = TemplateLookup(
+                    directories=['/path/to/templates'],
+                    module_directory='/path/to/modules',
+                    cache_impl = 'beaker',
+                    cache_args = {
+                        'manager':manager
+                    }
+            )
+
+Our templates can then opt to cache data in one of either region,
+using the ``cache_region`` argument.   Such as using ``short_term``
+at the ``<%page>`` level:
+
+.. sourcecode:: mako
+
+    <%page cached="True" cache_region="short_term">
+
+    ## ...
+
+Or, ``long_term`` at the ``<%block>`` level:
+
+.. sourcecode:: mako
+
+    <%block name="header" cached="True" cache_region="long_term">
+        other text
+    </%block>
+
+The Beaker backend also works without regions.   There are a
+variety of arguments that can be passed to the ``cache_args``
+dictionary, which are also allowable in templates via the 
+``<%page>``, ``<%block>``,
+and ``<%def>`` tags specific to those sections.   The values
+given override those specified at the  :class:`.TemplateLookup` 
+or :class:`.Template` level.  
+
+With the possible exception 
+of ``cache_timeout``, these arguments are probably better off
+staying at the template configuration level.  Each argument
+specified as ``cache_XYZ`` in a template tag is specified
+without the ``cache_`` prefix in the ``cache_args`` dictionary:
 
 * ``cache_timeout`` - number of seconds in which to invalidate the
   cached data.  After this timeout, the content is re-generated
   on the next call.  Available as ``timeout`` in the ``cache_args``
   dictionary.
 * ``cache_type`` - type of caching. ``memory``, ``file``, ``dbm``, or
-  ``memcached``.  Available as ``type`` in the ``cache_args`` 
-  dictionary.
+  ``ext:memcached`` (note that  the string ``memcached`` is
+  also accepted by the Mako plugin, though not by Beaker itself).  
+  Available as ``type`` in the ``cache_args`` dictionary.
 * ``cache_url`` - (only used for ``memcached`` but required) a single
   IP address or a semi-colon separated list of IP address of
   memcache servers to use.  Available as ``url`` in the ``cache_args``
@@ -136,9 +211,18 @@ as within the ``cache_args`` dictionary of :class:`.Template` and
   template modules are stored). If neither option is available
   an exception is thrown.  Available as ``dir`` in the
   ``cache_args`` dictionary.
- 
-Accessing the Cache
-===================
+
+Using the dogpile.cache Backend
+--------------------------------
+
+`dogpile.cache`_ is a new replacement for Beaker.   It provides
+a modernized, slimmed down interface and is generally easier to use
+than Beaker.   As of this writing it has not yet been released.  dogpile.cache
+includes its own Mako cache plugin - see :ref:`mako_plugin` in the
+dogpile.cache documentation.
+
+Programmatic Cache Access
+=========================
 
 The :class:`.Template`, as well as any template-derived :class:`.Namespace`, has
 an accessor called ``cache`` which returns the ``Cache`` object
@@ -150,13 +234,13 @@ values:
 .. sourcecode:: mako
 
     <%
-        local.cache.put("somekey", type="memory", "somevalue")
+        local.cache.set("somekey", type="memory", "somevalue")
     %>
  
 Above, the cache associated with the ``local`` namespace is
 accessed and a key is placed within a memory cache.
 
-More commonly the ``cache`` object is used to invalidate cached
+More commonly, the ``cache`` object is used to invalidate cached
 sections programmatically:
 
 .. sourcecode:: python
@@ -177,7 +261,7 @@ itself using the ``impl`` attribute::
 
     template.cache.impl.do_something_special()
 
-But note using implementation-specific methods will mean you can't 
+Note that using implementation-specific methods will mean you can't 
 swap in a different kind of :class:`.CacheImpl` implementation at a 
 later time.
 
@@ -186,7 +270,7 @@ later time.
 Cache Plugins
 ==============
 
-As of 0.5.1, the mechanism used by caching can be plugged in
+The mechanism used by caching can be plugged in
 using a :class:`.CacheImpl` subclass.    This class implements
 the rudimental methods Mako needs to implement the caching
 API.   Mako includes the :class:`.BeakerCacheImpl` class to 
@@ -208,14 +292,14 @@ An example plugin that implements a local dictionary cache::
             super(SimpleCacheImpl, self).__init__(cache)
             self._cache = {}
 
-        def get_and_replace(self, key, creation_function, **kw):
+        def get_or_create(self, key, creation_function, **kw):
             if key in self._cache:
                 return self._cache[key]
             else:
                 self._cache[key] = value = creation_function()
                 return value
 
-        def put(self, key, value, **kwargs):
+        def set(self, key, value, **kwargs):
             self._cache[key] = value
  
         def get(self, key, **kwargs):
@@ -243,19 +327,19 @@ Guidelines for writing cache plugins
   attribute, which is essentially the unique modulename of the template, is 
   a good value to use in order to represent a unique namespace of keys specific
   to the template.
-* Templates only use the :meth:`.CacheImpl.get_and_replace()` method.  
-  The :meth:`.CacheImpl.put`,
+* Templates only use the :meth:`.CacheImpl.get_or_create()` method 
+  in an implicit fashion.  The :meth:`.CacheImpl.set`,
   :meth:`.CacheImpl.get`, and :meth:`.CacheImpl.invalidate` methods are 
-  only used in response to end-user programmatic access to the corresponding
+  only used in response to direct programmatic access to the corresponding
   methods on the :class:`.Cache` object.
 * :class:`.CacheImpl` will be accessed in a multithreaded fashion if the
   :class:`.Template` itself is used multithreaded.  Care should be taken
   to ensure caching implementations are threadsafe.
 * A library like `Dogpile <http://pypi.python.org/pypi/Dogpile>`_, which
   is a minimal locking system derived from Beaker, can be used to help
-  implement the :meth:`.CacheImpl.get_and_replace` method in a threadsafe
+  implement the :meth:`.CacheImpl.get_or_create` method in a threadsafe
   way that can maximize effectiveness across multiple threads as well
-  as processes. :meth:`.CacheImpl.get_and_replace` is the
+  as processes. :meth:`.CacheImpl.get_or_create` is the
   key method used by templates.
 * All arguments passed to ``**kw`` come directly from the parameters
   inside the ``<%def>``, ``<%block>``, or ``<%page>`` tags directly,
@@ -268,7 +352,7 @@ Guidelines for writing cache plugins
 * The directory where :class:`.Template` places module files can
   be acquired using the accessor ``self.cache.template.module_directory``.
   This directory can be a good place to throw cache-related work
-  files, underneath a prefix like "_my_cache_work" so that name
+  files, underneath a prefix like ``_my_cache_work`` so that name
   conflicts with generated modules don't occur.
 
 API Reference
diff --git a/doc/build/conf.py b/doc/build/conf.py
index 43c29eb..508b316 100644
--- a/doc/build/conf.py
+++ b/doc/build/conf.py
@@ -30,7 +30,7 @@ import mako
 #extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode',
 #                'sphinx.ext.doctest', 'builder.builders']
 
-extensions = ['sphinx.ext.autodoc',
+extensions = ['sphinx.ext.autodoc','sphinx.ext.intersphinx',
                 'sphinx.ext.doctest', 'builder.builders']
 
 # Add any paths that contain templates here, relative to this directory.
@@ -280,3 +280,8 @@ epub_copyright = u'Mako authors'
 
 # Allow duplicate toc entries.
 #epub_tocdup = True
+
+intersphinx_mapping = {
+    'dogpile.cache':('http://dogpilecache.readthedocs.org/en/latest', None),
+    'beaker':('http://beaker.readthedocs.org/en/latest',None),
+}
diff --git a/mako/ext/beaker_cache.py b/mako/ext/beaker_cache.py
index 90cd09a..f0b50fa 100644
--- a/mako/ext/beaker_cache.py
+++ b/mako/ext/beaker_cache.py
@@ -24,7 +24,10 @@ class BeakerCacheImpl(CacheImpl):
                             "the Beaker package is required to use cache "
                             "functionality.")
 
-            _beaker_cache = beaker_cache.CacheManager()
+            if 'manager' in cache.template.cache_args:
+                _beaker_cache = cache.template.cache_args['manager']
+            else:
+                _beaker_cache = beaker_cache.CacheManager()
         super(BeakerCacheImpl, self).__init__(cache)
 
     def _get_cache(self, **kw):
@@ -34,11 +37,21 @@ class BeakerCacheImpl(CacheImpl):
         elif self.cache.template.module_directory:
             kw['data_dir'] = self.cache.template.module_directory
 
+        if 'manager' in kw:
+            kw.pop('manager')
+
         if kw.get('type') == 'memcached':
             kw['type'] = 'ext:memcached'
 
-        return _beaker_cache.get_cache(self.cache.id, **kw), \
-                    {'expiretime':expiretime, 'starttime':self.cache.starttime}
+        if 'region' in kw:
+            region = kw.pop('region')
+            cache = _beaker_cache.get_cache_region(self.cache.id, region, **kw)
+        else:
+            cache = _beaker_cache.get_cache(self.cache.id, **kw)
+        cache_args = {'starttime':self.cache.starttime}
+        if expiretime:
+            cache_args['expiretime'] = expiretime
+        return cache, cache_args
 
     def get_or_create(self, key, creation_function, **kw):
         cache, kw = self._get_cache(**kw)
diff --git a/test/test_cache.py b/test/test_cache.py
index 8240e7b..4ba5d1f 100644
--- a/test/test_cache.py
+++ b/test/test_cache.py
@@ -1,13 +1,14 @@
 from mako.template import Template
 from mako.lookup import TemplateLookup
 from mako import lookup
-import shutil, unittest, os
+import shutil, unittest, os, time
 from util import result_lines
 from test import TemplateTest, template_base, module_base
 from test import eq_
 
 try:
     import beaker
+    import beaker.cache
 except:
     from nose import SkipTest
     raise SkipTest("Beaker is required for these tests.")
@@ -15,33 +16,78 @@ except:
 from mako.cache import register_plugin, CacheImpl
 
 class MockCacheImpl(CacheImpl):
+    realcacheimpl = None
     def __init__(self, cache):
         self.cache = cache
-        self.realcacheimpl = cache._load_impl("beaker")
+        use_beaker= self.cache.template.cache_args.get('use_beaker', True)
+        if use_beaker:
+            self.realcacheimpl = cache._load_impl("beaker")
 
     def get_or_create(self, key, creation_function, **kw):
         self.key = key
         self.kwargs = kw.copy()
-        return self.realcacheimpl.get_or_create(key, creation_function, **kw)
+        if self.realcacheimpl:
+            return self.realcacheimpl.get_or_create(key, creation_function, **kw)
+        else:
+            return creation_function()
 
     def put(self, key, value, **kw):
         self.key = key
         self.kwargs = kw.copy()
-        self.realcacheimpl.put(key, value, **kw)
+        if self.realcacheimpl:
+            self.realcacheimpl.put(key, value, **kw)
  
     def get(self, key, **kw):
         self.key = key
         self.kwargs = kw.copy()
-        return self.realcacheimpl.get(key, **kw)
+        if self.realcacheimpl:
+            return self.realcacheimpl.get(key, **kw)
  
     def invalidate(self, key, **kw):
         self.key = key
         self.kwargs = kw.copy()
-        self.realcacheimpl.invalidate(key, **kw)
+        if self.realcacheimpl:
+            self.realcacheimpl.invalidate(key, **kw)
 
 
 register_plugin("mock", __name__, "MockCacheImpl")
 
+class BeakerCacheTest(TemplateTest):
+    def _regions(self):
+        return beaker.cache.CacheManager(
+            cache_regions = {
+                'short':{
+                    'expire':1,
+                    'type':'memory'
+                },
+                'long':{
+                    'expire':60,
+                    'type':'memory'
+                }
+            }
+        )
+
+    def test_region(self):
+        t = Template("""
+            <%block name="foo" cached="True" cache_region="short">
+                short term ${x}
+            </%block>
+            <%block name="bar" cached="True" cache_region="long">
+                long term ${x}
+            </%block>
+            <%block name="lala">
+                none ${x}
+            </%block>
+        """, cache_args={"manager":self._regions()})
+
+        r1 = result_lines(t.render(x=5))
+        time.sleep(2)
+        r2 = result_lines(t.render(x=6))
+        r3 = result_lines(t.render(x=7))
+        eq_(r1, ["short term 5", "long term 5", "none 5"])
+        eq_(r2, ["short term 6", "long term 5", "none 6"])
+        eq_(r3, ["short term 6", "long term 5", "none 7"])
+
 class CacheTest(TemplateTest):
     def _install_mock_cache(self, template):
         template.cache_impl = 'mock'
@@ -387,7 +433,6 @@ class CacheTest(TemplateTest):
         </%def>
         """)
  
-        import time
         x1 = t.render()
         time.sleep(3)
         x2 = t.render()
@@ -401,7 +446,6 @@ class CacheTest(TemplateTest):
         </%def>
         """)
  
-        import time
         x1 = t.render(x=1)
         time.sleep(3)
         x2 = t.render(x=2)
@@ -470,26 +514,26 @@ class CacheTest(TemplateTest):
                     cache_timeout="50" cache_foo="foob">
             </%def>
             ${foo()}
-        """)
+        """, cache_args={'use_beaker':False})
         m = self._install_mock_cache(t)
         t.render()
-        eq_(m.kwargs, {'region':'myregion', 'timeout':50, 'foo':'foob'})
+        eq_(m.kwargs, {'use_beaker':False,'region':'myregion', 'timeout':50, 'foo':'foob'})
 
     def test_custom_args_block(self):
         t = Template("""
             <%block name="foo" cached="True" cache_region="myregion" 
                     cache_timeout="50" cache_foo="foob">
             </%block>
-        """)
+        """, cache_args={'use_beaker':False})
         m = self._install_mock_cache(t)
         t.render()
-        eq_(m.kwargs, {'region':'myregion', 'timeout':50, 'foo':'foob'})
+        eq_(m.kwargs, {'use_beaker':False, 'region':'myregion', 'timeout':50, 'foo':'foob'})
 
     def test_custom_args_page(self):
         t = Template("""
             <%page cached="True" cache_region="myregion" 
                     cache_timeout="50" cache_foo="foob"/>
-        """)
+        """, cache_args={'use_beaker':False})
         m = self._install_mock_cache(t)
         t.render()
-        eq_(m.kwargs, {'region':'myregion', 'timeout':50, 'foo':'foob'})
\ No newline at end of file
+        eq_(m.kwargs, {'use_beaker':False, 'region':'myregion', 'timeout':50, 'foo':'foob'})
\ No newline at end of file