backport stable

14 files changed, 276 insertions, 204 deletions

diff --git a/ChangeLog b/ChangeLog
index f790bcb..37294e1 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,8 +1,24 @@
 ChangeLog for logilab.common
 ============================
 
---
-    * don't raise string exception in testlib (closes #35331)
+2010-08-26  --  0.51.0
+    * testlib: don't raise string exception (closes #35331)
+
+    * hg: new module regrouping some mercurial utility functions
+
+    * clcommands: refactored to get more object oriented api.
+
+    * optparser: module is now deprecated, use clcommands instead
+
+    * textutils: new split_url_or_path and text_to_dict functions
+
+    * logging_ext:
+      - init_log now accept optionaly any arbitrary handler
+      - threshold default to DEBUG if debug flag is true  and no threshold specified
+
+    * date: new ustrftime implementation working around datetime limitaion on dates < 1900
+
+
 
 2010-06-04  --  0.50.3
     * logging: added new optional kw argument to init_log rotating_parameters
@@ -10,8 +26,8 @@ ChangeLog for logilab.common
     * date: fix nb_open_days() codomain, positive natural numbers are expected
 
     * configuration:
-	 - skip option with no type, avoid pb with generated option such as long-help
-	- handle level on man page generation
+      - skip option with no type, avoid pb with generated option such as long-help
+      - handle level on man page generation
 
 
 
diff --git a/README b/README
index ca05a6d..ea480b2 100644
--- a/README
+++ b/README
@@ -4,26 +4,21 @@ Logilab's common library
 What's this ?
 -------------
 
-This package contains some modules used by differents Logilab's
-projects.
+This package contains some modules used by differents Logilab's projects.
 
 It is released under the GNU Public License.
 
-There is no documentation available yet but the source code should be
-clean and well documented.
+There is no documentation available yet but the source code should be clean and
+well documented.
 
 Designed to ease:
 
 * handling command line options and configuration files
 * writing interactive command line tools
-* manipulation files and character strings
-* interfacing to OmniORB
-* generating SQL queries
-* running unit tests
-* manipulating tree structures
+* manipulation of files and character strings
+* manipulation of common structures such as graph, tree, and pattern such as visitor
 * generating text and HTML reports
-* logging
-* parsing XML processing instructions
+* accessing some external libraries such as OmniORB_, Pyro_...
 * more...
 
 
@@ -42,152 +37,129 @@ For installation options, see ::
 Provided modules
 ----------------
 
-Here is a brief description of the available modules :
+Here is a brief description of the available modules.
 
-* adbh.py:
-  helper functions for using database advanced. Supported RDBMS
-  include PostgreSQL, MySQL and sqlite. See also db.py.
+Modules providing high-level features
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-* astutils:
-  Deprecated module. Use logilab.astng.
+* `cache`, a cache implementation with a least recently used algorithm.
 
-* bind.py :
-  Deprecated module.
-  Provides a way to optimize globals in certain functions by binding
-  their names to values provided in a dictionary.
+* `changelog`, a tiny library to manipulate our simplified ChangeLog file format.
 
-* cache.py :
-  A cache implementation with a least recently used algorithm.
+* `clcommands`, high-level classes to define command line programs handling
+  different subcommands. It is based on `configuration` to get easy command line
+  / configuration file handling.
 
-* clcommands.py:
-  helper functions for command line programs handling different
-  subcommands
+* `cli`, a base class for interactive programs using the command line.
 
-* cli.py :
-  Command line interface helper classes (for interactive programs
-  using the command line)
+* `configuration`, some classes to handle unified configuration from both
+  command line (using optparse) and configuration file (using ConfigParser).
 
-* compat.py:
-  Transparent compatibility layer between different python version
+* `dbf`, read Visual Fox Pro DBF files.
 
-* configuration.py :
-  Two mix-in classes to handle configuration from both command line
-  (using optik/optparse) and configuration file.
+* `proc`, interface to Linux /proc.
 
-* corbautils.py:
-  Useful functions for use with the OmniORB CORBA library.
+* `umessage`, unicode email support.
 
-* daemon.py :
-  A daemon mix-in class.
+* `ureports`, micro-reports, a way to create simple reports using python objects
+  without care of the final formatting. ReST and html formatters are provided.
 
-* date.py:
-  date manipulation helper functions
 
-* db.py :
-  A generic method to get a database connection. See also adbh.py.
+Modules providing low-level functions and structures
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-* debugger.py:
-  pdb customization
+* `compat`, provides a transparent compatibility layer between different python
+  versions.
 
-* decorators.py:
-  useful decorators (cached, timed...)
+* `date`, a set of date manipulation functions.
 
-* deprecation.py:
-  mark functions / classes as deprecated or moved
+* `daemon`, a daemon function and mix-in class to properly start an Unix daemon
+  process.
 
-* fileutils.py :
-  Some file / file path manipulation utilities.
+* `decorators`, function decorators such as cached, timed...
 
-* graph.py:
-  graph manipulations, dot file generation
+* `deprecation`, decorator, metaclass & all to mark functions / classes as
+  deprecated or moved
 
-* html.py :
-  Deprecated module
-  Return an html formatted traceback from python exception infos.
+* `fileutils`, some file / file path manipulation utilities.
 
-* interface.py
-  Bases class for interfaces.
+* `graph`, graph manipulations functions such as cycle detection, bases for dot
+  file generation.
 
-* logger.py :
-  Deprecated module : use logging from stdlib.
-  Define a logger interface and two concrete loggers : one which prints
-  everything on stdout, the other using syslog.
+* `modutils`, python module manipulation functions.
 
-* logging_ext.py:
-  extensions to stdlib's logging module
+* `shellutils`, some powerful shell like functions to replace shell scripts with
+  python scripts.
 
-* logservice.py:
-  Deprecated module. Use logging from stdlib.
+* `tasksqueue`, a prioritized tasks queue implementation.
 
-* modutils.py :
-  Module manipulation utilities.
+* `textutils`, some text manipulation functions (ansi colorization, line wrapping,
+  rest support...).
 
-* monclient.py:
-  Deprecated module
+* `tree`, base class to represent tree structure, and some others to make it
+  works with the visitor implementation (see below).
 
-* monserver.py:
-  Deprecated module
+* `visitor`, a generic visitor pattern implementation.
 
-* optik_ext :
-  Add an abstraction level to transparently import optik classes from
-  optparse (python >= 2.3) or the optik package. It also defines two
-  new option types (regexp, csv, color, date...)
 
-* optparser.py:
-  extend optparse's OptionParser to support commands
+Modules extending some standard modules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-* patricia.py :
-  A Python implementation of PATRICIA trie (Practical Algorithm to
-  Retrieve Information Coded in Alphanumeric).
+* `debugger`,  `pdb` customization.
 
-* pdf_ext.py:
-  pdf and fdf file manipulations, with pdftk.
+* `logging_ext`, extensions to `logging` module such as a colorized formatter
+  and an easier initialization function.
 
-* pytest.py:
-  unittest runner. See testlib
+* `optik_ext`, defines some new option types (regexp, csv, color, date, etc.)
+  for `optik` / `optparse`
 
-* shellutils.py:
-  Some utilities to replace shell scripts with python scripts.
+* `xmlrpcutils`, auth support for XML-RPC
 
-* sqlgen.py :
-  Helper class to generate SQL strings to use with python's DB-API.
 
-* table.py:
-  manage tabular data (supports column and row names, sorting, grouping...
+Modules extending some external modules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-* testlib.py :
-  Generic tests execution methods.
+* `corbautils`, useful functions for use with the OmniORB_ CORBA library.
 
-* textutils.py:
-  Some text manipulation utilities (ansi colorization, line wrapping,
-  rest support...)
+* `hg`, some Mercurial_ utility functions.
 
-* tree.py :
-  Base class to represent tree structure, and some others to make it
-  works with the visitor implementation (see below).
+* `pdf_ext`, pdf and fdf file manipulations, with pdftk.
+
+* `pyro_ext`, some Pyro_ utility functions.
+
+* `sphinx_ext`, Sphinx_ plugin defining a `autodocstring` directive.
+
+* `vcgutils` , utilities functions to generate file readable with Georg Sander's
+  vcg tool (Visualization of Compiler Graphs).
+
+
+To be deprecated modules
+~~~~~~~~~~~~~~~~~~~~~~~~
 
-* umessage.py:
-  unicode email support
+Those `logilab.common` modules will much probably be deprecated in future
+versions:
 
-* ureports:
-  Provides a way to create simple reports using python objects
-  without care of the final formatting. Some formatters text and html
-  are provided.
+* `testlib`: use `unittest2`_ instead
+* `pytest`: use `discover`_ instead
+* `interface`: use `zope.interface`_ if you really want this
+* `table`, `xmlutils`: is that used?
+* `sphinxutils`: we won't go that way imo (i == syt)
 
-* vcgutils.py :
-  utilities functions to generate file readable with Georg Sander's vcg
-  (Visualization of Compiler Graphs).
 
-* visitor.py :
-  A generic visitor pattern implementation.
+Deprecated modules
+~~~~~~~~~~~~~~~~~~
 
-* twisted_distutils.py
-  This module enables the installation of plugins.tml files using standard
-  distutils syntax. Note that you can use this to install files that
-  are not twisted plugins in any package directory of your application.
+Those `logilab.common` modules are only there for backward compatibility. They
+can go away at anytime.
+
+* `optparser`: use `clcommands` instead
+
+* `adbh`, `db`, `sqlgen`: see `logilab.database`_ instead
+
+* `contexts`: content move to `shellutils`
+
+* `html`: deprecated without replacement
 
-* xmlrpcutils.py:
-  Auth support for XML RPC
 
 Comments, support, bug reports
 ------------------------------
@@ -203,3 +175,13 @@ http://lists.logilab.org/mailman/listinfo/python-projects
 
 Archives are available at
 http://lists.logilab.org/pipermail/python-projects/
+
+
+.. _Pyro: http://pyro.sourceforge.net/
+.. _OmniORB: http://omniorb.sourceforge.net/
+.. _Mercurial: http://mercurial.selenic.com
+.. _Sphinx: http://sphinx.pocoo.org/
+.. _`logilab.database`: http://www.logilab.org/project/logilab-database/
+.. _`unittest2`: http://pypi.python.org/pypi/unittest2
+.. _`discover`: http://pypi.python.org/pypi/discover
+.. _`zope.interface`: http://pypi.python.org/pypi/zope.interface
diff --git a/__pkginfo__.py b/__pkginfo__.py
index 00248a2..4232a18 100644
--- a/__pkginfo__.py
+++ b/__pkginfo__.py
@@ -20,24 +20,22 @@ __docformat__ = "restructuredtext en"
 
 distname = 'logilab-common'
 modname = 'common'
+subpackage_of = 'logilab'
+subpackage_master = True
 
-numversion = (0, 50, 3)
+numversion = (0, 51, 0)
 version = '.'.join([str(num) for num in numversion])
 
 license = 'LGPL' # 2.1 or later
-
-author = "Logilab"
-author_email = "contact@logilab.fr"
-
 description = "collection of low-level Python packages and modules used by Logilab projects"
 web = "http://www.logilab.org/project/%s" % distname
 ftp = "ftp://ftp.logilab.org/pub/%s" % modname
 mailinglist = "mailto://python-projects@lists.logilab.org"
+author = "Logilab"
+author_email = "contact@logilab.fr"
 
-subpackage_of = 'logilab'
-subpackage_master = True
 
-scripts = ('bin/pytest',)
 from os.path import join
+scripts = [join('bin', 'pytest')]
 include_dirs = [join('test', 'data')]
 pyversions = ['2.4', '2.5', '2.6']
diff --git a/clcommands.py b/clcommands.py
index 1c312a2..02e7deb 100644
--- a/clcommands.py
+++ b/clcommands.py
@@ -196,8 +196,8 @@ class Command(Configuration):
         if rcfile:
             self.load_file_configuration(rcfile)
         args = self.load_command_line_configuration(args)
-        self.check_args(args)
         try:
+            self.check_args(args)
             self.run(args)
         except KeyboardInterrupt:
             print 'interrupted'
diff --git a/contexts.py b/contexts.py
index 83c9603..d78c327 100644
--- a/contexts.py
+++ b/contexts.py
@@ -1,54 +1,5 @@
-# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
-# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
-#
-# This file is part of logilab-common.
-#
-# logilab-common is free software: you can redistribute it and/or modify it under
-# the terms of the GNU Lesser General Public License as published by the Free
-# Software Foundation, either version 2.1 of the License, or (at your option) any
-# later version.
-#
-# logilab-common is distributed in the hope that it will be useful, but WITHOUT
-# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
-# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
-# details.
-#
-# You should have received a copy of the GNU Lesser General Public License along
-# with logilab-common.  If not, see <http://www.gnu.org/licenses/>.
-"""A few useful context managers"""
-
-__docformat__ = "restructuredtext en"
-
-import sys
-
-if sys.version_info < (2, 5):
-    raise ImportError("python >= 2.5 is required to import logilab.common.contexts")
-
-import os
-import tempfile
-import shutil
-
-class tempdir(object):
-
-    def __enter__(self):
-        self.path = tempfile.mkdtemp()
-        return self.path
-
-    def __exit__(self, exctype, value, traceback):
-        # rmtree in all cases
-        shutil.rmtree(self.path)
-        return traceback is None
-
-
-class pushd(object):
-    def __init__(self, directory):
-        self.directory = directory
-
-    def __enter__(self):
-        self.cwd = os.getcwd()
-        os.chdir(self.directory)
-        return self.directory
-
-    def __exit__(self, exctype, value, traceback):
-        os.chdir(self.cwd)
+from warnings import warn
+warn('logilab.common.contexts module is deprecated, use logilab.common.shellutils instead',
+     DeprecationWarning, stacklevel=1)
 
+from logilab.common.shellutils import tempfile, pushd
diff --git a/debian/changelog b/debian/changelog
index 34e5e18..fd1ba44 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+logilab-common (0.51.0-1) unstable; urgency=low
+
+  * new upstream release
+
+ -- Sylvain Thénault <sylvain.thenault@logilab.fr>  Thu, 26 Aug 2010 09:28:35 +0200
+
 logilab-common (0.50.3-1) unstable; urgency=low
 
   * new upstream release
diff --git a/hg.py b/hg.py
new file mode 100644
index 0000000..406c856
--- /dev/null
+++ b/hg.py
@@ -0,0 +1,98 @@
+# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
+# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
+#
+# This file is part of logilab-common.
+#
+# logilab-common is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Lesser General Public License as published by the Free
+# Software Foundation, either version 2.1 of the License, or (at your option) any
+# later version.
+#
+# logilab-common is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Lesser General Public License along
+# with logilab-common.  If not, see <http://www.gnu.org/licenses/>.
+"""mercurial utilities (mercurial should be installed)"""
+
+__docformat__ = "restructuredtext en"
+
+import os.path as osp
+
+try:
+    from mercurial.error import RepoError
+    from mercurial.__version__ import version as hg_version
+except ImportError:
+    from mercurial.repo import RepoError
+    from mercurial.version import get_version
+    hg_version = get_version()
+
+from mercurial.hg import repository as Repository
+from mercurial.ui import ui as Ui
+from mercurial.node import short
+try:
+    # mercurial >= 1.2 (?)
+    from mercurial.cmdutil import walkchangerevs
+except ImportError, ex:
+    from mercurial.commands import walkchangerevs
+try:
+    # mercurial >= 1.1 (.1?)
+    from mercurial.util import cachefunc
+except ImportError, ex:
+    def cachefunc(func):
+        return func
+try:
+    # mercurial >= 1.3.1
+    from mercurial import encoding
+    _encoding = encoding.encoding
+except ImportError:
+    try:
+        from mercurial.util import _encoding
+    except ImportError:
+        import locale
+        # stay compatible with mercurial 0.9.1 (etch debian release)
+        # (borrowed from mercurial.util 1.1.2)
+        try:
+            _encoding = os.environ.get("HGENCODING")
+            if sys.platform == 'darwin' and not _encoding:
+                # On darwin, getpreferredencoding ignores the locale environment and
+                # always returns mac-roman. We override this if the environment is
+                # not C (has been customized by the user).
+                locale.setlocale(locale.LC_CTYPE, '')
+                _encoding = locale.getlocale()[1]
+            if not _encoding:
+                _encoding = locale.getpreferredencoding() or 'ascii'
+        except locale.Error:
+            _encoding = 'ascii'
+try:
+    # demandimport causes problems when activated, ensure it isn't
+    # XXX put this in apycot where the pb has been noticed?
+    from mercurial import demandimport
+    demandimport.disable()
+except:
+    pass
+
+Ui.warn = lambda *args, **kwargs: 0 # make it quiet
+
+def find_repository(path):
+    """returns <path>'s mercurial repository
+
+    None if <path> is not under hg control
+    """
+    path = osp.realpath(osp.abspath(path))
+    while not osp.isdir(osp.join(path, ".hg")):
+        oldpath = path
+        path = osp.dirname(path)
+        if path == oldpath:
+            return None
+    return path
+
+
+def get_repository(path):
+    """Simple function that open a hg repository"""
+    repopath = find_repository(path)
+    if repopath is None:
+        raise RuntimeError('no repository found in %s' % osp.abspath(path))
+    return Repository(Ui(), path=repopath)
diff --git a/html.py b/html.py
index d57a53e..0cfc81a 100644
--- a/html.py
+++ b/html.py
@@ -15,14 +15,12 @@
 #
 # You should have received a copy of the GNU Lesser General Public License along
 # with logilab-common.  If not, see <http://www.gnu.org/licenses/>.
-"""render a tree in HTML.
-
-
-
-
-"""
+"""render a tree in HTML."""
 __docformat__ = "restructuredtext en"
 
+from warnings import warn
+warn('lgc.html module is deprecated', DeprecationWarning, stacklevel=2)
+
 
 def render_HTML_tree(tree, selected_node=None, render_node=None, caption=None):
     """
diff --git a/interface.py b/interface.py
index 4ce019e..0d5835d 100644
--- a/interface.py
+++ b/interface.py
@@ -23,10 +23,6 @@
   _ Attribute objects
 
   This module requires at least python 2.2
-
-
-
-
 """
 __docformat__ = "restructuredtext en"
 
diff --git a/optparser.py b/optparser.py
index 123e6a6..e6c4bb0 100644
--- a/optparser.py
+++ b/optparser.py
@@ -31,7 +31,9 @@ With mymod.build that defines two functions run and add_options
 """
 __docformat__ = "restructuredtext en"
 
-# XXX merge with optik_ext ? merge with clcommands ?
+from warnings import warn
+warn('lgc.optparser module is deprecated, use lgc.clcommands instead', DeprecationWarning,
+     stacklevel=2)
 
 import sys
 import optparse
diff --git a/shellutils.py b/shellutils.py
index 853e7f7..4c6e8a8 100644
--- a/shellutils.py
+++ b/shellutils.py
@@ -42,6 +42,31 @@ except ImportError:
         raise NoSuchProcess()
 
 
+class tempdir(object):
+
+    def __enter__(self):
+        self.path = tempfile.mkdtemp()
+        return self.path
+
+    def __exit__(self, exctype, value, traceback):
+        # rmtree in all cases
+        shutil.rmtree(self.path)
+        return traceback is None
+
+
+class pushd(object):
+    def __init__(self, directory):
+        self.directory = directory
+
+    def __enter__(self):
+        self.cwd = os.getcwd()
+        os.chdir(self.directory)
+        return self.directory
+
+    def __exit__(self, exctype, value, traceback):
+        os.chdir(self.cwd)
+
+
 def chown(path, login=None, group=None):
     """Same as `os.chown` function but accepting user login or group name as
     argument. If login or group is omitted, it's left unchanged.
diff --git a/table.py b/table.py
index 22c7ba9..8efe911 100644
--- a/table.py
+++ b/table.py
@@ -15,12 +15,7 @@
 #
 # You should have received a copy of the GNU Lesser General Public License along
 # with logilab-common.  If not, see <http://www.gnu.org/licenses/>.
-"""Table management module.
-
-
-
-
-"""
+"""Table management module."""
 __docformat__ = "restructuredtext en"
 
 from logilab.common.compat import enumerate, sum, set
diff --git a/textutils.py b/textutils.py
index 16926f0..db69d3b 100644
--- a/textutils.py
+++ b/textutils.py
@@ -45,6 +45,7 @@ __docformat__ = "restructuredtext en"
 
 import sys
 import re
+import os.path as osp
 from unicodedata import normalize as _uninormalize
 try:
     from os import linesep
@@ -251,6 +252,15 @@ def splitstrip(string, sep=','):
 get_csv = deprecated()(splitstrip)
 
 
+def split_url_or_path(url_or_path):
+    """return the latest component of a string containing either an url of the
+    form <scheme>://<path> or a local file system path
+    """
+    if '://' in url_or_path:
+        return url_or_path.rstrip('/').rsplit('/', 1)
+    return osp.split(url_or_path.rstrip(osp.sep))
+
+
 def text_to_dict(text):
     """parse multilines text containing simple 'key=value' lines and return a
     dict of {'key': 'value'}. When the same key is encountered multiple time,
diff --git a/xmlrpcutils.py b/xmlrpcutils.py
index 5e3df07..82a85e6 100644
--- a/xmlrpcutils.py
+++ b/xmlrpcutils.py
@@ -15,12 +15,7 @@
 #
 # You should have received a copy of the GNU Lesser General Public License along
 # with logilab-common.  If not, see <http://www.gnu.org/licenses/>.
-"""XML-RPC utilities.
-
-
-
-
-"""
+"""XML-RPC utilities."""
 __docformat__ = "restructuredtext en"
 
 import xmlrpclib