diff options
| author | Jelmer Vernooij <jelmer@samba.org> | 2010-09-03 23:28:11 +0200 |
|---|---|---|
| committer | Jelmer Vernooij <jelmer@samba.org> | 2010-09-03 23:28:11 +0200 |
| commit | ec87be037ad5ab9617dc33266656edfa137d0691 (patch) | |
| tree | 7bb2179437a2ba69c4b8ea2924c6cbcb7e2faef7 | |
| parent | 844a4f35d6ab0dc1542c40b03cef19a87b9dda08 (diff) | |
| download | python-fastimport-ec87be037ad5ab9617dc33266656edfa137d0691.tar.gz | |
Split python-fastimport into its own separate package.
106 files changed, 71 insertions, 14067 deletions
diff --git a/.bzrignore b/.bzrignore deleted file mode 100644 index 35d21d7..0000000 --- a/.bzrignore +++ /dev/null @@ -1,4 +0,0 @@ -build -# executables -exporters/svn-archive -exporters/svn-fast-export @@ -1,347 +0,0 @@ -============================ -bzr-fastimport Release Notes -============================ - -.. contents:: - -0.9 28-Feb-2010 -=============== - -New Features ------------- - -* The fast-import command now takes an optional but recommended - DESTINATION argument. A repository will be created at this - location and branches will be created within there. If the user - is running bzr 1.17 up to 2.0, format "2a" is used for these, - otherwise the default format is used. A format can be explicitly - given using the new --format option. (Ian Clatworthy) - -* Wrapper commands simplifying the generation of fast-import - files from other tools are now provided. The general usage is: - - bzr fast-export-from-xxx SOURCE project.fi - - Before starting an export, these commands make an effort to - check that dependent tools are installed. So far, wrapper - commands are available for cvs, darcs, git, hg (Mercurial), - mnt (Monotone), p4 (Perforce) and svn (Subversion). - (Ian Clatworthy, Matt McClure) - -* darcs-fast-export is now bundled. In fact, the project has - merged into this one for the purposes of ongoing bug fixing - and development. (Miklos Vajna) - -* fast-export now supports a --no-plain parameter which causes - richer metadata to be included in the output using the - recently agreed 'feature' extension capability. The features - used are called multiple-authors, commit-properties and - empty-directories. (Ian Clatworthy) - -* fast-import and fast-import-filter now support user mapping - via the new --user-map option. The argument is a file specifying - how user-ids should be mapped to preferred user-ids. - (Ian Clatworthy) - -* svn-fast-export now supports an address option (to control the - default email domain) and a first-rev option (to select revisions - since a given one). (Ted Gould) - -Improvements ------------- - -* Large repositories now compress better thanks to a change in - how file-ids are assigned. (Ian Clatworthy, John Arbash Meinel) - -* Memory usage is improved by flushing blobs to a disk cache - when appropriate. (John Arbash Meinel) - -* If a fast-import source ends in ".gz", it is assumed to be in - gzip format and the stream is implicitly uncompressed. This - means fast-import dump files generated by fast-export-from-xxx - can be stored gzipped to save space. (Ian Clatworthy) - -* The working tree is now implicitly updated for trunk. Other - branches still need to be explicitly created using 'bzr update'. - (Ian Clatworthy) - -* Directories that become empty following a delete or rename of - one of their children are now implicitly pruned. If required, - this will be made optional at a later date. - (Tom Widmer, Ian Clatworthy) - -* Blob tracking is now more intelligently done by an implicit - first pass to collect blob usage statistics. This eliminates - the need for an explicit 2-step process in all cases except - where stdin is used as the input source. (Ian Clatworthy) - -* Updated the bundled version of hg-fast-export to be the latest - code (25-May-2009) from http://repo.or.cz/w/fast-export.git. - (Ian Clatworthy) - -Bug Fixes ---------- - -* Fixed the format used to create branches in a shared repository. - It now selects the best branch format based on the repository - format, rather than assume the default branch format is the right - one. (Ian Clatworthy) - -* Fixed inventory delta generation when deleting directories. - (Previously the child paths included were relative to the - directory when they ought to be relative to the root.) - (Ian Clatworthy) - -* Gracefully handle email addresses with unicode characters. - (Ian Clatworthy) - -* Gracefully handle an empty input stream. (Gonéri Le Bouder) - -* Gracefully handle git submodules by ignoring them. - (Ian Clatworthy) - -* Get git-bzr working again. (Gonéri Le Bouder) - -Documentation -------------- - -* Improved documentation has been published in the Bazaar Data Migration - Guide: http://doc.bazaar-vcs.org/migration/en/data-migration/. - - -0.8 22-Jul-2009 -=============== - -Compatibility Breaks --------------------- - -* ``exporters/bzr-fast-export`` has been replaced with a - ``fast-export`` command. Some minor issues have been - fixed at the same time: the first commit now goes into - refs/heads/master (not refs/head/tmp); there's no - checkpoint at the top of the stream; parent commits are - now always given lower mark numbers than the commits they - are merged into. (Ian Clatworthy) - -* The ``fast-import`` command now uses a different mapping of - git reference names to bzr branch names. In summary: - - * ``refs/heads/foo`` is mapped to ``foo`` - * ``refs/remotes/origin/foo`` is mapped to ``foo.remote`` - * ``refs/tags/foo`` is mapped to ``foo.tag`` - * ``*/master`` is mapped to ``trunk``, ``trunk.remote``, etc. - * ``*/trunk`` is mapped to ``git-trunk``, ``git-trunk.remote``, etc. - - This new mapping has been introduced so that more natural - branch names are used and to enable round-tripping back to git. - (Ian Clatworthy) - -* The old ``fast-import-filter`` command is now called - ``fast-import-query``. ``fast-import-filter`` now - really filters the input to produce a fast-import stream - based on filtering criteria. See below. - (Ian Clatworthy) - -* The ``--inv-fulltext`` option is no longer supported. It was - only used in experimental mode for old versions of bzrlib so - it added more complexity than value. (Ian Clatworthy) - -New Features ------------- - -* Added ``fast-import-filter`` command for splitting out a - subdirectory or bunch of files into their own project. It can - also be used to create a new repository without any history - for nominated files and directories. This is useful for - removing information which is a security risk, huge binary - files like ISO images, etc. - (Ian Clatworthy) - -* Copying of files and symbolic links is now supported. - (Ian Clatworthy) - -* Initial cut at reset support. (Brian de Alwis, Ian Clatworthy) - -Improvements ------------- - -* If ``refs/heads/xxx`` and ``refs/remotes/origin/xxx`` are both - defined, the latter is now mapped to a bzr branch called - ``xxx.remote`` rather than ``remotes--origins--xxx``. - (Ian Clatworthy) - -* ``bzr fast-import-info`` now handles an unlimited # of parents for a - revision. The spec suggests the maximum ought to be 16 but the linux - kernel has revisions with more than that. - (Ian Clatworthy) - -* ``bzr fast-import-info`` now reports on things that may need caching, - i.e. merges, rename old paths and copy source paths. - (Ian Clatworthy) - -* Tag commands with a missing from clause now produce a warning but - are otherwise ignored. (Scott James Remnant, Ian Clatworthy) - -* The fastimport-id-map file can now have more revisions than the - repository. (Scott James Remnant) - -* Updated the bundled version of hg-fast-export to be the latest - code from http://repo.or.cz/w/fast-export.git. It should now - support recent Mercurial releases. - (Ian Clatworthy, #318903) - -Bug Fixes ---------- - -* Fixed a *bad* bug where filecopy commands were being parsed - as filerename commands. Repositories generated by previous - version of bzr-fast-import where the input stream contained - filecopy commands might be missing data (the copy source will - no longer be there if it was unchanged since the copy happened) - and ought to be regenerated. - (Ian Clatworthy) - -* Fixed how the per-file graph is generated. The per-file graph - may still be less than perfect in the case where a file is only - changed in a merge and not the parent, but in the vast majority - of cases now, ``bzr check`` should no longer report inconsistent - parents. (Ian Clatworthy) - -* Fix ``os`` import as needed on Windows. - (Ian Clatworthy, esskov, #264988) - -* Handle a directory turning into a file and then the children - of that directory being deleted. - (Ian Clatworthy, #309486) - -* Handle an empty email section. - (Ian Clatworthy) - -* Handle multiple merges within the one merge clause. That's illegal - according to the spec but git-fast-export does it. - (Ian Clatworthy, #259711) - -* Handle names and paths that aren't utf8 encoded. The spec recommends - utf8 encoding of these but git-fast-export doesn't always do that. - (Ian Clatworthy, #289088) - -* Ignore lightweight tags with no from clause rather than abort. - (It seems git-fast-export outputs these commands frequently now - while it didn't appear to in early versions.) - (Ian Clatworthy, edice, #259711) - -* Import into rich-root (and subtree) repositories without aborting. - (Ian Clatworthy, #300921) - -* Recursively delete children when a directory is deleted. - (Scott James Remnant) - -* The ``deleteall`` command now only tries to delete files in the - nominated branch, not all known files. As a consequence, - it should now be possible (if it wasn't before) to import - multiple Darcs branches (via darcs-fast-export) at once. - (Ian Clatworthy) - -Testing -------- - -* A large number of tests have been added. - (Ian Clatworthy) - -Internals ---------- - -* Refactored ``processors/generic_processor.py`` into a bunch of modules. - (Ian Clatworthy) - - -0.7 09-Feb-2009 -=============== - -Compatibility Breaks --------------------- - -* bzr-fast-export.py renamed to bzr-fast-export. - (Jelmer Vernooij) - -Improvements ------------- - -* Add support for the deleteall command. - (Miklos Vajna, #284941) - -Bug Fixes ---------- - -* bzr-fast-export now exports rm+mv correctly. - (Jonas) - -* Fix recursive rename handling in bzr-fast-export. - (Pieter de Bie, #287785) - -* hg-fast-export should use binary mode on Windows. - (Alexey Stukalov) - -* setup.py no longer assumes python2.4. - (schickb@gmail.com) - -* setup.py support fixed. - (Jelmer Vernooij) - -* Update the last-modified revision for a renamed file. - (John Arbash Meinel) - - -0.6 23-Jul-2008 -=============== - -Improvements ------------- - -* Added NEWS containing Release Notes. (Ian Clatworthy) - -* ``bzr help fastimport`` now provides help that is useful. - (Ian Clatworthy) - -* Numerous fixes to ``bzr-fast-export.py`` to improve round-tripping - with Git. Added ``--import-marks`` and ``--export-marks`` options - to ``fast-import`` as well. - (Pieter de Bie) - -* ``svn-fast-export.py`` now supports a regular-expression to specify - the branches to export. - (Mirko Friedenhagen) - -Bug Fixes ---------- - -* Support the new Repository API added in bzr.dev r3510. The old API - will be used for earlier versions of bzr including bzr 1.6beta2 and - earlier. (Ian Clatworthy) - -Compatibility Breaks --------------------- - -* The ``--inv-fulltext`` option is not yet supported when the new - Repository API is used to import revisions. The option can be - provided but it will be ignored. (Ian Clatworthy) - -API Breaks - -* The ``RevisionLoader`` class has been renamed to ``RevisionLoader1``. - The ``ExperimentalRevisionLoader`` class has been renamed to - ``ImportRevisionLoader1``. New classes called ``RevisionLoader2`` - and ``ImportRevisionLoader2`` are provided that use the new - Repository API. (Ian Clatworthy) - -Internals ---------- - -* Improved packaging by adding a setup.py file. (Ian Clatworthy) - - -0.5 02-Jun-2008 -=============== - -* Version suitable for Bazaar 1.5. - (Ian Clatworthy) diff --git a/README.txt b/README.txt deleted file mode 100644 index 0b85646..0000000 --- a/README.txt +++ /dev/null @@ -1,49 +0,0 @@ -bzr-fastimport: Backend for fast Bazaar data importers -====================================================== - -Dependencies ------------- - -Required and recommended packages are: - -* Python 2.4 or later - -* Bazaar 1.18 or later. - - -Installation ------------- - -The easiest way to install this plugin is to either copy or symlink the -directory into your ~/.bazaar/plugins directory. Be sure to rename the -directory to fastimport (instead of bzr-fastimport). - -See http://bazaar-vcs.org/UsingPlugins for other options such as -using the BZR_PLUGIN_PATH environment variable. - - -Testing -------- - -To test the plugin after installation: - - bzr selftest fastimport - - -Documentation -------------- - -To view the documentation after installation: - - bzr help fastimport - - -Licensing ---------- - -For copyright and licensing details of the exporters, see the relevant -files in the exporters/ directory. - -Otherwise this plugin is (C) Copyright Canonical Limited 2008 under the -GPL Version 2 or later. Please see the file COPYING.txt for the licence -details. diff --git a/__init__.py b/__init__.py deleted file mode 100644 index 94acacf..0000000 --- a/__init__.py +++ /dev/null @@ -1,961 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -r"""FastImport Plugin -================= - -The fastimport plugin provides stream-based importing and exporting of -data into and out of Bazaar. As well as enabling interchange between -multiple VCS tools, fastimport/export can be useful for complex branch -operations, e.g. partitioning off part of a code base in order to Open -Source it. - -The normal import recipe is:: - - bzr fast-export-from-xxx SOURCE project.fi - bzr fast-import project.fi project.bzr - -If fast-export-from-xxx doesn't exist yet for the tool you're importing -from, the alternative recipe is:: - - front-end > project.fi - bzr fast-import project.fi project.bzr - -In either case, if you wish to save disk space, project.fi can be -compressed to gzip format after it is generated like this:: - - (generate project.fi) - gzip project.fi - bzr fast-import project.fi.gz project.bzr - -The list of known front-ends and their status is documented on -http://bazaar-vcs.org/BzrFastImport/FrontEnds. The fast-export-from-xxx -commands provide simplified access to these so that the majority of users -can generate a fast-import dump file without needing to study up on all -the options - and the best combination of them to use - for the front-end -relevant to them. In some cases, a fast-export-from-xxx wrapper will require -that certain dependencies are installed so it checks for these before -starting. A wrapper may also provide a limited set of options. See the -online help for the individual commands for details:: - - bzr help fast-export-from-cvs - bzr help fast-export-from-darcs - bzr help fast-export-from-hg - bzr help fast-export-from-git - bzr help fast-export-from-mtn - bzr help fast-export-from-p4 - bzr help fast-export-from-svn - -Once a fast-import dump file is created, it can be imported into a -Bazaar repository using the fast-import command. If required, you can -manipulate the stream first using the fast-import-filter command. -This is useful for creating a repository with just part of a project -or for removing large old binaries (say) from history that are no longer -valuable to retain. For further details on importing, manipulating and -reporting on fast-import streams, see the online help for the commands:: - - bzr help fast-import - bzr help fast-import-filter - bzr help fast-import-info - bzr help fast-import-query - -Finally, you may wish to generate a fast-import dump file from a Bazaar -repository. The fast-export command is provided for that purpose. - -To report bugs or publish enhancements, visit the bzr-fastimport project -page on Launchpad, https://launchpad.net/bzr-fastimport. -""" - -version_info = (0, 9, 0, 'dev', 0) - -from bzrlib import bzrdir -from bzrlib.commands import Command, register_command -from bzrlib.option import Option, ListOption, RegistryOption - - -def test_suite(): - import tests - return tests.test_suite() - - -def _run(source, processor_factory, control, params, verbose, - user_map=None): - """Create and run a processor. - - :param source: a filename or '-' for standard input. If the - filename ends in .gz, it will be opened as a gzip file and - the stream will be implicitly uncompressed - :param processor_factory: a callable for creating a processor - :param control: the BzrDir of the destination or None if no - destination is expected - :param user_map: if not None, the file containing the user map. - """ - import parser - stream = _get_source_stream(source) - user_mapper = _get_user_mapper(user_map) - proc = processor_factory(control, params=params, verbose=verbose) - p = parser.ImportParser(stream, verbose=verbose, user_mapper=user_mapper) - return proc.process(p.iter_commands) - - -def _get_source_stream(source): - if source == '-': - import sys - stream = helpers.binary_stream(sys.stdin) - elif source.endswith('.gz'): - import gzip - stream = gzip.open(source, "rb") - else: - stream = open(source, "rb") - return stream - - -def _get_user_mapper(filename): - import user_mapper - if filename is None: - return None - f = open(filename) - lines = f.readlines() - f.close() - return user_mapper.UserMapper(lines) - - -class cmd_fast_import(Command): - """Backend for fast Bazaar data importers. - - This command reads a mixed command/data stream and creates - branches in a Bazaar repository accordingly. The preferred - recipe is:: - - bzr fast-import project.fi project.bzr - - Numerous commands are provided for generating a fast-import file - to use as input. These are named fast-export-from-xxx where xxx - is one of cvs, darcs, git, hg, mtn, p4 or svn. - To specify standard input as the input stream, use a - source name of '-' (instead of project.fi). If the source name - ends in '.gz', it is assumed to be compressed in gzip format. - - project.bzr will be created if it doesn't exist. If it exists - already, it should be empty or be an existing Bazaar repository - or branch. If not specified, the current directory is assumed. - - fast-import will intelligently select the format to use when - creating a repository or branch. If you are running Bazaar 1.17 - up to Bazaar 2.0, the default format for Bazaar 2.x ("2a") is used. - Otherwise, the current default format ("pack-0.92" for Bazaar 1.x) - is used. If you wish to specify a custom format, use the `--format` - option. - - .. note:: - - To maintain backwards compatibility, fast-import lets you - create the target repository or standalone branch yourself. - It is recommended though that you let fast-import create - these for you instead. - - :Branch mapping rules: - - Git reference names are mapped to Bazaar branch names as follows: - - * refs/heads/foo is mapped to foo - * refs/remotes/origin/foo is mapped to foo.remote - * refs/tags/foo is mapped to foo.tag - * */master is mapped to trunk, trunk.remote, etc. - * */trunk is mapped to git-trunk, git-trunk.remote, etc. - - :Branch creation rules: - - When a shared repository is created or found at the destination, - branches are created inside it. In the simple case of a single - branch (refs/heads/master) inside the input file, the branch is - project.bzr/trunk. - - When a standalone branch is found at the destination, the trunk - is imported there and warnings are output about any other branches - found in the input file. - - When a branch in a shared repository is found at the destination, - that branch is made the trunk and other branches, if any, are - created in sister directories. - - :Working tree updates: - - The working tree is generated for the trunk branch. If multiple - branches are created, a message is output on completion explaining - how to create the working trees for other branches. - - :Custom exporters: - - The fast-export-from-xxx commands typically call more advanced - xxx-fast-export scripts. You are welcome to use the advanced - scripts if you prefer. - - If you wish to write a custom exporter for your project, see - http://bazaar-vcs.org/BzrFastImport for the detailed protocol - specification. In many cases, exporters can be written quite - quickly using whatever scripting/programming language you like. - - :User mapping: - - Some source repositories store just the user name while Bazaar - prefers a full email address. You can adjust user-ids while - importing by using the --user-map option. The argument is a - text file with lines in the format:: - - old-id = new-id - - Blank lines and lines beginning with # are ignored. - If old-id has the special value '@', then users without an - email address will get one created by using the matching new-id - as the domain, unless a more explicit address is given for them. - For example, given the user-map of:: - - @ = example.com - bill = William Jones <bill@example.com> - - then user-ids are mapped as follows:: - - maria => maria <maria@example.com> - bill => William Jones <bill@example.com> - - .. note:: - - User mapping is supported by both the fast-import and - fast-import-filter commands. - - :Blob tracking: - - As some exporters (like git-fast-export) reuse blob data across - commits, fast-import makes two passes over the input file by - default. In the first pass, it collects data about what blobs are - used when, along with some other statistics (e.g. total number of - commits). In the second pass, it generates the repository and - branches. - - .. note:: - - The initial pass isn't done if the --info option is used - to explicitly pass in information about the input stream. - It also isn't done if the source is standard input. In the - latter case, memory consumption may be higher than otherwise - because some blobs may be kept in memory longer than necessary. - - :Restarting an import: - - At checkpoints and on completion, the commit-id -> revision-id - map is saved to a file called 'fastimport-id-map' in the control - directory for the repository (e.g. .bzr/repository). If the import - is interrupted or unexpectedly crashes, it can be started again - and this file will be used to skip over already loaded revisions. - As long as subsequent exports from the original source begin - with exactly the same revisions, you can use this feature to - maintain a mirror of a repository managed by a foreign tool. - If and when Bazaar is used to manage the repository, this file - can be safely deleted. - - :Examples: - - Import a Subversion repository into Bazaar:: - - bzr fast-export-from-svn /svn/repo/path project.fi - bzr fast-import project.fi project.bzr - - Import a CVS repository into Bazaar:: - - bzr fast-export-from-cvs /cvs/repo/path project.fi - bzr fast-import project.fi project.bzr - - Import a Git repository into Bazaar:: - - bzr fast-export-from-git /git/repo/path project.fi - bzr fast-import project.fi project.bzr - - Import a Mercurial repository into Bazaar:: - - bzr fast-export-from-hg /hg/repo/path project.fi - bzr fast-import project.fi project.bzr - - Import a Darcs repository into Bazaar:: - - bzr fast-export-from-darcs /darcs/repo/path project.fi - bzr fast-import project.fi project.bzr - """ - hidden = False - _see_also = ['fast-export', 'fast-import-filter', 'fast-import-info'] - takes_args = ['source', 'destination?'] - takes_options = ['verbose', - Option('user-map', type=str, - help="Path to file containing a map of user-ids.", - ), - Option('info', type=str, - help="Path to file containing caching hints.", - ), - Option('trees', - help="Update all working trees, not just trunk's.", - ), - Option('count', type=int, - help="Import this many revisions then exit.", - ), - Option('checkpoint', type=int, - help="Checkpoint automatically every N revisions." - " The default is 10000.", - ), - Option('autopack', type=int, - help="Pack every N checkpoints. The default is 4.", - ), - Option('inv-cache', type=int, - help="Number of inventories to cache.", - ), - RegistryOption.from_kwargs('mode', - 'The import algorithm to use.', - title='Import Algorithm', - default='Use the preferred algorithm (inventory deltas).', - classic="Use the original algorithm (mutable inventories).", - experimental="Enable experimental features.", - value_switches=True, enum_switch=False, - ), - Option('import-marks', type=str, - help="Import marks from file." - ), - Option('export-marks', type=str, - help="Export marks to file." - ), - RegistryOption('format', - help='Specify a format for the created repository. See' - ' "bzr help formats" for details.', - lazy_registry=('bzrlib.bzrdir', 'format_registry'), - converter=lambda name: bzrdir.format_registry.make_bzrdir(name), - value_switches=False, title='Repository format'), - ] - aliases = [] - def run(self, source, destination='.', verbose=False, info=None, - trees=False, count=-1, checkpoint=10000, autopack=4, inv_cache=-1, - mode=None, import_marks=None, export_marks=None, format=None, - user_map=None): - from bzrlib.errors import BzrCommandError, NotBranchError - from bzrlib.plugins.fastimport.processors import generic_processor - from bzrlib.plugins.fastimport.fastimport.helpers import ( - open_destination_directory, - ) - # If no format is given and the user is running a release - # leading up to 2.0, select 2a for them. Otherwise, use - # the default format. - if format is None: - import bzrlib - bzr_version = bzrlib.version_info[0:2] - if bzr_version in [(1,17), (1,18), (2,0)]: - format = bzrdir.format_registry.make_bzrdir('2a') - control = open_destination_directory(destination, format=format) - - # If an information file was given and the source isn't stdin, - # generate the information by reading the source file as a first pass - if info is None and source != '-': - info = self._generate_info(source) - - # Do the work - if mode is None: - mode = 'default' - params = { - 'info': info, - 'trees': trees, - 'count': count, - 'checkpoint': checkpoint, - 'autopack': autopack, - 'inv-cache': inv_cache, - 'mode': mode, - 'import-marks': import_marks, - 'export-marks': export_marks, - } - return _run(source, generic_processor.GenericProcessor, control, - params, verbose, user_map=user_map) - - def _generate_info(self, source): - from cStringIO import StringIO - import parser - from bzrlib.plugins.fastimport.processors import info_processor - stream = _get_source_stream(source) - output = StringIO() - try: - proc = info_processor.InfoProcessor(verbose=True, outf=output) - p = parser.ImportParser(stream) - return_code = proc.process(p.iter_commands) - lines = output.getvalue().splitlines() - finally: - output.close() - stream.seek(0) - return lines - - -class cmd_fast_import_filter(Command): - """Filter a fast-import stream to include/exclude files & directories. - - This command is useful for splitting a subdirectory or bunch of - files out from a project to create a new project complete with history - for just those files. It can also be used to create a new project - repository that removes all references to files that should not have - been committed, e.g. security-related information (like passwords), - commercially sensitive material, files with an incompatible license or - large binary files like CD images. - - To specify standard input as the input stream, use a source name - of '-'. If the source name ends in '.gz', it is assumed to be - compressed in gzip format. - - :File/directory filtering: - - This is supported by the -i and -x options. Excludes take precedence - over includes. - - When filtering out a subdirectory (or file), the new stream uses the - subdirectory (or subdirectory containing the file) as the root. As - fast-import doesn't know in advance whether a path is a file or - directory in the stream, you need to specify a trailing '/' on - directories passed to the `--includes option`. If multiple files or - directories are given, the new root is the deepest common directory. - - Note: If a path has been renamed, take care to specify the *original* - path name, not the final name that it ends up with. - - :User mapping: - - Some source repositories store just the user name while Bazaar - prefers a full email address. You can adjust user-ids - by using the --user-map option. The argument is a - text file with lines in the format:: - - old-id = new-id - - Blank lines and lines beginning with # are ignored. - If old-id has the special value '@', then users without an - email address will get one created by using the matching new-id - as the domain, unless a more explicit address is given for them. - For example, given the user-map of:: - - @ = example.com - bill = William Jones <bill@example.com> - - then user-ids are mapped as follows:: - - maria => maria <maria@example.com> - bill => William Jones <bill@example.com> - - .. note:: - - User mapping is supported by both the fast-import and - fast-import-filter commands. - - :Examples: - - Create a new project from a library (note the trailing / on the - directory name of the library):: - - front-end | bzr fast-import-filter -i lib/xxx/ > xxx.fi - bzr fast-import xxx.fi mylibrary.bzr - (lib/xxx/foo is now foo) - - Create a new repository without a sensitive file:: - - front-end | bzr fast-import-filter -x missile-codes.txt > clean.fi - bzr fast-import clean.fi clean.bzr - """ - hidden = False - _see_also = ['fast-import'] - takes_args = ['source'] - takes_options = ['verbose', - ListOption('include_paths', short_name='i', type=str, - help="Only include commits affecting these paths." - " Directories should have a trailing /." - ), - ListOption('exclude_paths', short_name='x', type=str, - help="Exclude these paths from commits." - ), - Option('user-map', type=str, - help="Path to file containing a map of user-ids.", - ), - ] - aliases = [] - encoding_type = 'exact' - def run(self, source, verbose=False, include_paths=None, - exclude_paths=None, user_map=None): - from bzrlib.plugins.fastimport.processors import filter_processor - params = { - 'include_paths': include_paths, - 'exclude_paths': exclude_paths, - } - return _run(source, filter_processor.FilterProcessor, None, params, - verbose, user_map=user_map) - - -class cmd_fast_import_info(Command): - """Output information about a fast-import stream. - - This command reads a fast-import stream and outputs - statistics and interesting properties about what it finds. - When run in verbose mode, the information is output as a - configuration file that can be passed to fast-import to - assist it in intelligently caching objects. - - To specify standard input as the input stream, use a source name - of '-'. If the source name ends in '.gz', it is assumed to be - compressed in gzip format. - - :Examples: - - Display statistics about the import stream produced by front-end:: - - front-end | bzr fast-import-info - - - Create a hints file for running fast-import on a large repository:: - - front-end | bzr fast-import-info -v - > front-end.cfg - """ - hidden = False - _see_also = ['fast-import'] - takes_args = ['source'] - takes_options = ['verbose'] - aliases = [] - def run(self, source, verbose=False): - from bzrlib.plugins.fastimport.processors import info_processor - return _run(source, info_processor.InfoProcessor, None, {}, verbose) - - -class cmd_fast_import_query(Command): - """Query a fast-import stream displaying selected commands. - - To specify standard input as the input stream, use a source name - of '-'. If the source name ends in '.gz', it is assumed to be - compressed in gzip format. - - To specify a commit to display, give its mark using the - --commit-mark option. The commit will be displayed with - file-commands included but with inline blobs hidden. - - To specify the commands to display, use the -C option one or - more times. To specify just some fields for a command, use the - syntax:: - - command=field1,... - - By default, the nominated fields for the nominated commands - are displayed tab separated. To see the information in - a name:value format, use verbose mode. - - Note: Binary fields (e.g. data for blobs) are masked out - so it is generally safe to view the output in a terminal. - - :Examples: - - Show the commit with mark 429:: - - bzr fast-import-query xxx.fi -m429 - - Show all the fields of the reset and tag commands:: - - bzr fast-import-query xxx.fi -Creset -Ctag - - Show the mark and merge fields of the commit commands:: - - bzr fast-import-query xxx.fi -Ccommit=mark,merge - """ - hidden = True - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source'] - takes_options = ['verbose', - Option('commit-mark', short_name='m', type=str, - help="Mark of the commit to display." - ), - ListOption('commands', short_name='C', type=str, - help="Display fields for these commands." - ), - ] - aliases = [] - def run(self, source, verbose=False, commands=None, commit_mark=None): - from bzrlib.plugins.fastimport.processors import query_processor - from bzrlib.plugins.fastimport import helpers - params = helpers.defines_to_dict(commands) or {} - if commit_mark: - params['commit-mark'] = commit_mark - return _run(source, query_processor.QueryProcessor, None, params, - verbose) - - -class cmd_fast_export(Command): - """Generate a fast-import stream from a Bazaar branch. - - This program generates a stream from a Bazaar branch in fast-import - format used by tools such as bzr fast-import, git-fast-import and - hg-fast-import. - - If no destination is given or the destination is '-', standard output - is used. Otherwise, the destination is the name of a file. If the - destination ends in '.gz', the output will be compressed into gzip - format. - - :Round-tripping: - - Recent versions of the fast-import specification support features - that allow effective round-tripping of many Bazaar branches. As - such, fast-exporting a branch and fast-importing the data produced - will create a new repository with equivalent history, i.e. - "bzr log -v -p --include-merges --forward" on the old branch and - new branch should produce similar, if not identical, results. - - .. note:: - - Be aware that the new repository may appear to have similar history - but internally it is quite different with new revision-ids and - file-ids assigned. As a consequence, the ability to easily merge - with branches based on the old repository is lost. Depending on your - reasons for producing a new repository, this may or may not be an - issue. - - :Interoperability: - - fast-export can use the following "extended features" to - produce a richer data stream: - - * *multiple-authors* - if a commit has multiple authors (as commonly - occurs in pair-programming), all authors will be included in the - output, not just the first author - - * *commit-properties* - custom metadata per commit that Bazaar stores - in revision properties (e.g. branch-nick and bugs fixed by this - change) will be included in the output. - - * *empty-directories* - directories, even the empty ones, will be - included in the output. - - To disable these features and produce output acceptable to git 1.6, - use the --plain option. To enable these features, use --no-plain. - Currently, --plain is the default but that will change in the near - future once the feature names and definitions are formally agreed - to by the broader fast-import developer community. - - :Examples: - - To produce data destined for import into Bazaar:: - - bzr fast-export --no-plain my-bzr-branch my.fi.gz - - To produce data destined for Git 1.6:: - - bzr fast-export --plain my-bzr-branch my.fi - - To import several unmerged but related branches into the same repository, - use the --{export,import}-marks options, and specify a name for the git - branch like this:: - - bzr fast-export --export-marks=marks.bzr project.dev | - GIT_DIR=project/.git git-fast-import --export-marks=marks.git - - bzr fast-export --import-marks=marks.bzr -b other project.other | - GIT_DIR=project/.git git-fast-import --import-marks=marks.git - - If you get a "Missing space after source" error from git-fast-import, - see the top of the commands.py module for a work-around. - """ - hidden = False - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source', 'destination?'] - takes_options = ['verbose', 'revision', - Option('git-branch', short_name='b', type=str, - argname='FILE', - help='Name of the git branch to create (default=master).' - ), - Option('checkpoint', type=int, argname='N', - help="Checkpoint every N revisions (default=10000)." - ), - Option('marks', type=str, argname='FILE', - help="Import marks from and export marks to file." - ), - Option('import-marks', type=str, argname='FILE', - help="Import marks from file." - ), - Option('export-marks', type=str, argname='FILE', - help="Export marks to file." - ), - Option('plain', - help="Exclude metadata to maximise interoperability." - ), - ] - aliases = [] - encoding_type = 'exact' - def run(self, source, destination=None, verbose=False, - git_branch="master", checkpoint=10000, marks=None, - import_marks=None, export_marks=None, revision=None, - plain=True): - from bzrlib.plugins.fastimport import bzr_exporter - - if marks: - import_marks = export_marks = marks - exporter = bzr_exporter.BzrFastExporter(source, - destination=destination, - git_branch=git_branch, checkpoint=checkpoint, - import_marks_file=import_marks, export_marks_file=export_marks, - revision=revision, verbose=verbose, plain_format=plain) - return exporter.run() - - -class cmd_fast_export_from_cvs(Command): - """Generate a fast-import file from a CVS repository. - - Destination is a dump file, typically named xxx.fi where xxx is - the name of the project. If '-' is given, standard output is used. - - cvs2svn 2.3 or later must be installed as its cvs2bzr script is used - under the covers to do the export. - - The source must be the path on your filesystem to the part of the - repository you wish to convert. i.e. either that path or a parent - directory must contain a CVSROOT subdirectory. The path may point to - either the top of a repository or to a path within it. In the latter - case, only that project within the repository will be converted. - - .. note:: - Remote access to the repository is not sufficient - the path - must point into a copy of the repository itself. See - http://cvs2svn.tigris.org/faq.html#repoaccess for instructions - on how to clone a remote CVS repository locally. - - By default, the trunk, branches and tags are all exported. If you - only want the trunk, use the `--trunk-only` option. - - By default, filenames, log messages and author names are expected - to be encoded in ascii. Use the `--encoding` option to specify an - alternative. If multiple encodings are used, specify the option - multiple times. For a list of valid encoding names, see - http://docs.python.org/lib/standard-encodings.html. - - Windows users need to install GNU sort and use the `--sort` - option to specify its location. GNU sort can be downloaded from - http://unxutils.sourceforge.net/. - """ - hidden = False - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source', 'destination'] - takes_options = ['verbose', - Option('trunk-only', - help="Export just the trunk, ignoring tags and branches." - ), - ListOption('encoding', type=str, argname='CODEC', - help="Encoding used for filenames, commit messages " - "and author names if not ascii." - ), - Option('sort', type=str, argname='PATH', - help="GNU sort program location if not on the path." - ), - ] - aliases = [] - encoding_type = 'exact' - def run(self, source, destination, verbose=False, trunk_only=False, - encoding=None, sort=None): - from bzrlib.plugins.fastimport.exporters import fast_export_from - custom = [] - if trunk_only: - custom.append("--trunk-only") - if encoding: - for enc in encoding: - custom.extend(['--encoding', enc]) - if sort: - custom.extend(['--sort', sort]) - fast_export_from(source, destination, 'cvs', verbose, custom) - - -class cmd_fast_export_from_darcs(Command): - """Generate a fast-import file from a Darcs repository. - - Destination is a dump file, typically named xxx.fi where xxx is - the name of the project. If '-' is given, standard output is used. - - Darcs 2.2 or later must be installed as various subcommands are - used to access the source repository. The source may be a network - URL but using a local URL is recommended for performance reasons. - """ - hidden = False - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source', 'destination'] - takes_options = ['verbose', - Option('encoding', type=str, argname='CODEC', - help="Encoding used for commit messages if not utf-8." - ), - ] - aliases = [] - encoding_type = 'exact' - def run(self, source, destination, verbose=False, encoding=None): - from bzrlib.plugins.fastimport.exporters import fast_export_from - custom = None - if encoding is not None: - custom = ['--encoding', encoding] - fast_export_from(source, destination, 'darcs', verbose, custom) - - -class cmd_fast_export_from_hg(Command): - """Generate a fast-import file from a Mercurial repository. - - Destination is a dump file, typically named xxx.fi where xxx is - the name of the project. If '-' is given, standard output is used. - - Mercurial 1.2 or later must be installed as its libraries are used - to access the source repository. Given the APIs currently used, - the source repository must be a local file, not a network URL. - """ - hidden = False - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source', 'destination'] - takes_options = ['verbose'] - aliases = [] - encoding_type = 'exact' - def run(self, source, destination, verbose=False): - from bzrlib.plugins.fastimport.exporters import fast_export_from - fast_export_from(source, destination, 'hg', verbose) - - -class cmd_fast_export_from_git(Command): - """Generate a fast-import file from a Git repository. - - Destination is a dump file, typically named xxx.fi where xxx is - the name of the project. If '-' is given, standard output is used. - - Git 1.6 or later must be installed as the git fast-export - subcommand is used under the covers to generate the stream. - The source must be a local directory. - - .. note:: - - Earlier versions of Git may also work fine but are - likely to receive less active support if problems arise. - """ - hidden = False - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source', 'destination'] - takes_options = ['verbose'] - aliases = [] - encoding_type = 'exact' - def run(self, source, destination, verbose=False): - from bzrlib.plugins.fastimport.exporters import fast_export_from - fast_export_from(source, destination, 'git', verbose) - - -class cmd_fast_export_from_mtn(Command): - """Generate a fast-import file from a Monotone repository. - - Destination is a dump file, typically named xxx.fi where xxx is - the name of the project. If '-' is given, standard output is used. - - Monotone 0.43 or later must be installed as the mtn git_export - subcommand is used under the covers to generate the stream. - The source must be a local directory. - """ - hidden = False - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source', 'destination'] - takes_options = ['verbose'] - aliases = [] - encoding_type = 'exact' - def run(self, source, destination, verbose=False): - from bzrlib.plugins.fastimport.exporters import fast_export_from - fast_export_from(source, destination, 'mtn', verbose) - - -class cmd_fast_export_from_p4(Command): - """Generate a fast-import file from a Perforce repository. - - Source is a Perforce depot path, e.g., //depot/project - - Destination is a dump file, typically named xxx.fi where xxx is - the name of the project. If '-' is given, standard output is used. - - bzrp4 must be installed as its p4_fast_export.py module is used under - the covers to do the export. bzrp4 can be downloaded from - https://launchpad.net/bzrp4/. - - The P4PORT environment variable must be set, and you must be logged - into the Perforce server. - - By default, only the HEAD changelist is exported. To export all - changelists, append '@all' to the source. To export a revision range, - append a comma-delimited pair of changelist numbers to the source, - e.g., '100,200'. - """ - hidden = False - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source', 'destination'] - takes_options = [] - aliases = [] - encoding_type = 'exact' - def run(self, source, destination, verbose=False): - from bzrlib.plugins.fastimport.exporters import fast_export_from - custom = [] - fast_export_from(source, destination, 'p4', verbose, custom) - - -class cmd_fast_export_from_svn(Command): - """Generate a fast-import file from a Subversion repository. - - Destination is a dump file, typically named xxx.fi where xxx is - the name of the project. If '-' is given, standard output is used. - - Python-Subversion (Python bindings to the Subversion APIs) - 1.4 or later must be installed as this library is used to - access the source repository. The source may be a network URL - but using a local URL is recommended for performance reasons. - """ - hidden = False - _see_also = ['fast-import', 'fast-import-filter'] - takes_args = ['source', 'destination'] - takes_options = ['verbose', - Option('trunk-path', type=str, argname="STR", - help="Path in repo to /trunk.\n" - "May be `regex:/cvs/(trunk)/proj1/(.*)` in " - "which case the first group is used as the " - "branch name and the second group is used " - "to match files.", - ), - Option('branches-path', type=str, argname="STR", - help="Path in repo to /branches." - ), - Option('tags-path', type=str, argname="STR", - help="Path in repo to /tags." - ), - ] - aliases = [] - encoding_type = 'exact' - def run(self, source, destination, verbose=False, trunk_path=None, - branches_path=None, tags_path=None): - from bzrlib.plugins.fastimport.exporters import fast_export_from - custom = [] - if trunk_path is not None: - custom.extend(['--trunk-path', trunk_path]) - if branches_path is not None: - custom.extend(['--branches-path', branches_path]) - if tags_path is not None: - custom.extend(['--tags-path', tags_path]) - fast_export_from(source, destination, 'svn', verbose, custom) - - -register_command(cmd_fast_import) -register_command(cmd_fast_import_filter) -register_command(cmd_fast_import_info) -register_command(cmd_fast_import_query) -register_command(cmd_fast_export) -register_command(cmd_fast_export_from_cvs) -register_command(cmd_fast_export_from_darcs) -register_command(cmd_fast_export_from_hg) -register_command(cmd_fast_export_from_git) -register_command(cmd_fast_export_from_mtn) -register_command(cmd_fast_export_from_p4) -register_command(cmd_fast_export_from_svn) diff --git a/branch_mapper.py b/branch_mapper.py deleted file mode 100644 index acc37c9..0000000 --- a/branch_mapper.py +++ /dev/null @@ -1,58 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""An object that maps git ref names to bzr branch names. Note that it is not -used to map git ref names to bzr tag names.""" - - -import re - - -class BranchMapper(object): - _GIT_TRUNK_RE = re.compile('(?:git-)*trunk') - - def git_to_bzr(self, ref_name): - """Map a git reference name to a Bazaar branch name. - """ - parts = ref_name.split('/') - if parts[0] == 'refs': - parts.pop(0) - category = parts.pop(0) - if category == 'heads': - git_name = '/'.join(parts) - bazaar_name = self._git_to_bzr_name(git_name) - else: - if category == 'remotes' and parts[0] == 'origin': - parts.pop(0) - git_name = '/'.join(parts) - if category.endswith('s'): - category = category[:-1] - name_no_ext = self._git_to_bzr_name(git_name) - bazaar_name = "%s.%s" % (name_no_ext, category) - return bazaar_name - - def _git_to_bzr_name(self, git_name): - # Make a simple name more bzr-like, by mapping git 'master' to bzr 'trunk'. - # To avoid collision, map git 'trunk' to bzr 'git-trunk'. Likewise - # 'git-trunk' to 'git-git-trunk' and so on, such that the mapping is - # one-to-one in both directions. - if git_name == 'master': - bazaar_name = 'trunk' - elif self._GIT_TRUNK_RE.match(git_name): - bazaar_name = 'git-%s' % (git_name,) - else: - bazaar_name = git_name - return bazaar_name diff --git a/branch_updater.py b/branch_updater.py deleted file mode 100644 index 6ec7154..0000000 --- a/branch_updater.py +++ /dev/null @@ -1,179 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""An object that updates a bunch of branches based on data imported.""" - -from operator import itemgetter - -from bzrlib import bzrdir, errors, osutils, transport -from bzrlib.trace import error, note - -from bzrlib.plugins.fastimport.fastimport.helpers import ( - single_plural, - ) -from bzrlib.plugins.fastimport.helpers import ( - best_format_for_objects_in_a_repository, - ) - - -class BranchUpdater(object): - - def __init__(self, repo, branch, cache_mgr, heads_by_ref, last_ref, tags): - """Create an object responsible for updating branches. - - :param heads_by_ref: a dictionary where - names are git-style references like refs/heads/master; - values are one item lists of commits marks. - """ - self.repo = repo - self.branch = branch - self.cache_mgr = cache_mgr - self.heads_by_ref = heads_by_ref - self.last_ref = last_ref - self.tags = tags - self._branch_format = \ - best_format_for_objects_in_a_repository(repo) - - def update(self): - """Update the Bazaar branches and tips matching the heads. - - If the repository is shared, this routine creates branches - as required. If it isn't, warnings are produced about the - lost of information. - - :return: updated, lost_heads where - updated = the list of branches updated ('trunk' is first) - lost_heads = a list of (bazaar-name,revision) for branches that - would have been created had the repository been shared - """ - updated = [] - branch_tips, lost_heads = self._get_matching_branches() - for br, tip in branch_tips: - if self._update_branch(br, tip): - updated.append(br) - return updated, lost_heads - - def _get_matching_branches(self): - """Get the Bazaar branches. - - :return: branch_tips, lost_heads where - branch_tips = a list of (branch,tip) tuples for branches. The - first tip is the 'trunk'. - lost_heads = a list of (bazaar-name,revision) for branches that - would have been created had the repository been shared and - everything succeeded - """ - branch_tips = [] - lost_heads = [] - ref_names = self.heads_by_ref.keys() - if self.branch is not None: - trunk = self.select_trunk(ref_names) - default_tip = self.heads_by_ref[trunk][0] - branch_tips.append((self.branch, default_tip)) - ref_names.remove(trunk) - - # Convert the reference names into Bazaar speak. If we haven't - # already put the 'trunk' first, do it now. - git_to_bzr_map = {} - for ref_name in ref_names: - git_to_bzr_map[ref_name] = self.cache_mgr.branch_mapper.git_to_bzr(ref_name) - if ref_names and self.branch is None: - trunk = self.select_trunk(ref_names) - git_bzr_items = [(trunk, git_to_bzr_map[trunk])] - del git_to_bzr_map[trunk] - else: - git_bzr_items = [] - git_bzr_items.extend(sorted(git_to_bzr_map.items(), key=itemgetter(1))) - - # Policy for locating branches - def dir_under_current(name, ref_name): - # Using the Bazaar name, get a directory under the current one - repo_base = self.repo.bzrdir.transport.base - return osutils.pathjoin(repo_base, "..", name) - def dir_sister_branch(name, ref_name): - # Using the Bazaar name, get a sister directory to the branch - return osutils.pathjoin(self.branch.base, "..", name) - if self.branch is not None: - dir_policy = dir_sister_branch - else: - dir_policy = dir_under_current - - # Create/track missing branches - shared_repo = self.repo.is_shared() - for ref_name, name in git_bzr_items: - tip = self.heads_by_ref[ref_name][0] - if shared_repo: - location = dir_policy(name, ref_name) - try: - br = self.make_branch(location) - branch_tips.append((br,tip)) - continue - except errors.BzrError, ex: - error("ERROR: failed to create branch %s: %s", - location, ex) - lost_head = self.cache_mgr.revision_ids[tip] - lost_info = (name, lost_head) - lost_heads.append(lost_info) - return branch_tips, lost_heads - - def select_trunk(self, ref_names): - """Given a set of ref names, choose one as the trunk.""" - for candidate in ['refs/heads/master']: - if candidate in ref_names: - return candidate - # Use the last reference in the import stream - return self.last_ref - - def make_branch(self, location): - """Make a branch in the repository if not already there.""" - to_transport = transport.get_transport(location) - to_transport.create_prefix() - try: - return bzrdir.BzrDir.open(location).open_branch() - except errors.NotBranchError, ex: - return bzrdir.BzrDir.create_branch_convenience(location, - format=self._branch_format, - possible_transports=[to_transport]) - - def _update_branch(self, br, last_mark): - """Update a branch with last revision and tag information. - - :return: whether the branch was changed or not - """ - last_rev_id = self.cache_mgr.revision_ids[last_mark] - revs = list(self.repo.iter_reverse_revision_history(last_rev_id)) - revno = len(revs) - existing_revno, existing_last_rev_id = br.last_revision_info() - changed = False - if revno != existing_revno or last_rev_id != existing_last_rev_id: - br.set_last_revision_info(revno, last_rev_id) - changed = True - # apply tags known in this branch - my_tags = {} - if self.tags: - ancestry = self.repo.get_ancestry(last_rev_id) - for tag,rev in self.tags.items(): - if rev in ancestry: - my_tags[tag] = rev - if my_tags: - br.tags._set_tag_dict(my_tags) - changed = True - if changed: - tagno = len(my_tags) - note("\t branch %s now has %d %s and %d %s", br.nick, - revno, single_plural(revno, "revision", "revisions"), - tagno, single_plural(tagno, "tag", "tags")) - return changed diff --git a/bzr_commit_handler.py b/bzr_commit_handler.py deleted file mode 100644 index bd206bf..0000000 --- a/bzr_commit_handler.py +++ /dev/null @@ -1,865 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""CommitHandlers that build and save revisions & their inventories.""" - - -from bzrlib import ( - errors, - generate_ids, - inventory, - osutils, - revision, - serializer, - ) -from bzrlib.plugins.fastimport.fastimport import ( - commands, - helpers, - processor, - ) - - -_serializer_handles_escaping = hasattr(serializer.Serializer, - 'squashes_xml_invalid_characters') - - -def copy_inventory(inv): - # This currently breaks revision-id matching - #if hasattr(inv, "_get_mutable_inventory"): - # # TODO: Make this a public API on inventory - # return inv._get_mutable_inventory() - - # TODO: Shallow copy - deep inventory copying is expensive - return inv.copy() - - -class GenericCommitHandler(processor.CommitHandler): - """Base class for Bazaar CommitHandlers.""" - - def __init__(self, command, cache_mgr, rev_store, verbose=False, - prune_empty_dirs=True): - super(GenericCommitHandler, self).__init__(command) - self.cache_mgr = cache_mgr - self.rev_store = rev_store - self.verbose = verbose - self.branch_ref = command.ref - self.prune_empty_dirs = prune_empty_dirs - # This tracks path->file-id for things we're creating this commit. - # If the same path is created multiple times, we need to warn the - # user and add it just once. - # If a path is added then renamed or copied, we need to handle that. - self._new_file_ids = {} - # This tracks path->file-id for things we're modifying this commit. - # If a path is modified then renamed or copied, we need the make - # sure we grab the new content. - self._modified_file_ids = {} - # This tracks the paths for things we're deleting this commit. - # If the same path is added or the destination of a rename say, - # then a fresh file-id is required. - self._paths_deleted_this_commit = set() - - def pre_process_files(self): - """Prepare for committing.""" - self.revision_id = self.gen_revision_id() - # cache of texts for this commit, indexed by file-id - self.data_for_commit = {} - #if self.rev_store.expects_rich_root(): - self.data_for_commit[inventory.ROOT_ID] = [] - - # Track the heads and get the real parent list - parents = self.cache_mgr.track_heads(self.command) - - # Convert the parent commit-ids to bzr revision-ids - if parents: - self.parents = [self.cache_mgr.revision_ids[p] - for p in parents] - else: - self.parents = [] - self.debug("%s id: %s, parents: %s", self.command.id, - self.revision_id, str(self.parents)) - - # Tell the RevisionStore we're starting a new commit - self.revision = self.build_revision() - self.parent_invs = [self.get_inventory(p) for p in self.parents] - self.rev_store.start_new_revision(self.revision, self.parents, - self.parent_invs) - - # cache of per-file parents for this commit, indexed by file-id - self.per_file_parents_for_commit = {} - if self.rev_store.expects_rich_root(): - self.per_file_parents_for_commit[inventory.ROOT_ID] = () - - # Keep the basis inventory. This needs to be treated as read-only. - if len(self.parents) == 0: - self.basis_inventory = self._init_inventory() - else: - self.basis_inventory = self.get_inventory(self.parents[0]) - if hasattr(self.basis_inventory, "root_id"): - self.inventory_root_id = self.basis_inventory.root_id - else: - self.inventory_root_id = self.basis_inventory.root.file_id - - # directory-path -> inventory-entry for current inventory - self.directory_entries = {} - - def _init_inventory(self): - return self.rev_store.init_inventory(self.revision_id) - - def get_inventory(self, revision_id): - """Get the inventory for a revision id.""" - try: - inv = self.cache_mgr.inventories[revision_id] - except KeyError: - if self.verbose: - self.mutter("get_inventory cache miss for %s", revision_id) - # Not cached so reconstruct from the RevisionStore - inv = self.rev_store.get_inventory(revision_id) - self.cache_mgr.inventories[revision_id] = inv - return inv - - def _get_data(self, file_id): - """Get the data bytes for a file-id.""" - return self.data_for_commit[file_id] - - def _get_lines(self, file_id): - """Get the lines for a file-id.""" - return osutils.split_lines(self._get_data(file_id)) - - def _get_per_file_parents(self, file_id): - """Get the lines for a file-id.""" - return self.per_file_parents_for_commit[file_id] - - def _get_inventories(self, revision_ids): - """Get the inventories for revision-ids. - - This is a callback used by the RepositoryStore to - speed up inventory reconstruction. - """ - present = [] - inventories = [] - # If an inventory is in the cache, we assume it was - # successfully loaded into the revision store - for revision_id in revision_ids: - try: - inv = self.cache_mgr.inventories[revision_id] - present.append(revision_id) - except KeyError: - if self.verbose: - self.note("get_inventories cache miss for %s", revision_id) - # Not cached so reconstruct from the revision store - try: - inv = self.get_inventory(revision_id) - present.append(revision_id) - except: - inv = self._init_inventory() - self.cache_mgr.inventories[revision_id] = inv - inventories.append(inv) - return present, inventories - - def bzr_file_id_and_new(self, path): - """Get a Bazaar file identifier and new flag for a path. - - :return: file_id, is_new where - is_new = True if the file_id is newly created - """ - if path not in self._paths_deleted_this_commit: - # Try file-ids renamed in this commit - id = self._modified_file_ids.get(path) - if id is not None: - return id, False - - # Try the basis inventory - id = self.basis_inventory.path2id(path) - if id is not None: - return id, False - - # Try the other inventories - if len(self.parents) > 1: - for inv in self.parent_invs[1:]: - id = self.basis_inventory.path2id(path) - if id is not None: - return id, False - - # Doesn't exist yet so create it - dirname, basename = osutils.split(path) - id = generate_ids.gen_file_id(basename) - self.debug("Generated new file id %s for '%s' in revision-id '%s'", - id, path, self.revision_id) - self._new_file_ids[path] = id - return id, True - - def bzr_file_id(self, path): - """Get a Bazaar file identifier for a path.""" - return self.bzr_file_id_and_new(path)[0] - - def _format_name_email(self, name, email): - """Format name & email as a string.""" - if email: - return "%s <%s>" % (name, email) - else: - return name - - def gen_revision_id(self): - """Generate a revision id. - - Subclasses may override this to produce deterministic ids say. - """ - committer = self.command.committer - # Perhaps 'who' being the person running the import is ok? If so, - # it might be a bit quicker and give slightly better compression? - who = self._format_name_email(committer[0], committer[1]) - timestamp = committer[2] - return generate_ids.gen_revision_id(who, timestamp) - - def build_revision(self): - rev_props = self._legal_revision_properties(self.command.properties) - if 'branch-nick' not in rev_props: - rev_props['branch-nick'] = self.cache_mgr.branch_mapper.git_to_bzr( - self.branch_ref) - self._save_author_info(rev_props) - committer = self.command.committer - who = self._format_name_email(committer[0], committer[1]) - message = self.command.message - if not _serializer_handles_escaping: - # We need to assume the bad ol' days - message = helpers.escape_commit_message(message) - return revision.Revision( - timestamp=committer[2], - timezone=committer[3], - committer=who, - message=message, - revision_id=self.revision_id, - properties=rev_props, - parent_ids=self.parents) - - def _legal_revision_properties(self, props): - """Clean-up any revision properties we can't handle.""" - # For now, we just check for None because that's not allowed in 2.0rc1 - result = {} - if props is not None: - for name, value in props.items(): - if value is None: - self.warning( - "converting None to empty string for property %s" - % (name,)) - result[name] = '' - else: - result[name] = value - return result - - def _save_author_info(self, rev_props): - author = self.command.author - if author is None: - return - if self.command.more_authors: - authors = [author] + self.command.more_authors - author_ids = [self._format_name_email(a[0], a[1]) for a in authors] - elif author != self.command.committer: - author_ids = [self._format_name_email(author[0], author[1])] - else: - return - # If we reach here, there are authors worth storing - rev_props['authors'] = "\n".join(author_ids) - - def _modify_item(self, path, kind, is_executable, data, inv): - """Add to or change an item in the inventory.""" - # If we've already added this, warn the user that we're ignoring it. - # In the future, it might be nice to double check that the new data - # is the same as the old but, frankly, exporters should be fixed - # not to produce bad data streams in the first place ... - existing = self._new_file_ids.get(path) - if existing: - # We don't warn about directories because it's fine for them - # to be created already by a previous rename - if kind != 'directory': - self.warning("%s already added in this commit - ignoring" % - (path,)) - return - - # Create the new InventoryEntry - basename, parent_id = self._ensure_directory(path, inv) - file_id = self.bzr_file_id(path) - ie = inventory.make_entry(kind, basename, parent_id, file_id) - ie.revision = self.revision_id - if kind == 'file': - ie.executable = is_executable - # lines = osutils.split_lines(data) - ie.text_sha1 = osutils.sha_string(data) - ie.text_size = len(data) - self.data_for_commit[file_id] = data - elif kind == 'directory': - self.directory_entries[path] = ie - # There are no lines stored for a directory so - # make sure the cache used by get_lines knows that - self.data_for_commit[file_id] = '' - elif kind == 'symlink': - ie.symlink_target = data.encode('utf8') - # There are no lines stored for a symlink so - # make sure the cache used by get_lines knows that - self.data_for_commit[file_id] = '' - else: - self.warning("Cannot import items of kind '%s' yet - ignoring '%s'" - % (kind, path)) - return - # Record it - if file_id in inv: - old_ie = inv[file_id] - if old_ie.kind == 'directory': - self.record_delete(path, old_ie) - self.record_changed(path, ie, parent_id) - else: - try: - self.record_new(path, ie) - except: - print "failed to add path '%s' with entry '%s' in command %s" \ - % (path, ie, self.command.id) - print "parent's children are:\n%r\n" % (ie.parent_id.children,) - raise - - def _ensure_directory(self, path, inv): - """Ensure that the containing directory exists for 'path'""" - dirname, basename = osutils.split(path) - if dirname == '': - # the root node doesn't get updated - return basename, self.inventory_root_id - try: - ie = self._get_directory_entry(inv, dirname) - except KeyError: - # We will create this entry, since it doesn't exist - pass - else: - return basename, ie.file_id - - # No directory existed, we will just create one, first, make sure - # the parent exists - dir_basename, parent_id = self._ensure_directory(dirname, inv) - dir_file_id = self.bzr_file_id(dirname) - ie = inventory.entry_factory['directory'](dir_file_id, - dir_basename, parent_id) - ie.revision = self.revision_id - self.directory_entries[dirname] = ie - # There are no lines stored for a directory so - # make sure the cache used by get_lines knows that - self.data_for_commit[dir_file_id] = '' - - # It's possible that a file or symlink with that file-id - # already exists. If it does, we need to delete it. - if dir_file_id in inv: - self.record_delete(dirname, ie) - self.record_new(dirname, ie) - return basename, ie.file_id - - def _get_directory_entry(self, inv, dirname): - """Get the inventory entry for a directory. - - Raises KeyError if dirname is not a directory in inv. - """ - result = self.directory_entries.get(dirname) - if result is None: - if dirname in self._paths_deleted_this_commit: - raise KeyError - try: - file_id = inv.path2id(dirname) - except errors.NoSuchId: - # In a CHKInventory, this is raised if there's no root yet - raise KeyError - if file_id is None: - raise KeyError - result = inv[file_id] - # dirname must be a directory for us to return it - if result.kind == 'directory': - self.directory_entries[dirname] = result - else: - raise KeyError - return result - - def _delete_item(self, path, inv): - newly_added = self._new_file_ids.get(path) - if newly_added: - # We've only just added this path earlier in this commit. - file_id = newly_added - # note: delta entries look like (old, new, file-id, ie) - ie = self._delta_entries_by_fileid[file_id][3] - else: - file_id = inv.path2id(path) - if file_id is None: - self.mutter("ignoring delete of %s as not in inventory", path) - return - try: - ie = inv[file_id] - except errors.NoSuchId: - self.mutter("ignoring delete of %s as not in inventory", path) - return - self.record_delete(path, ie) - - def _copy_item(self, src_path, dest_path, inv): - newly_changed = self._new_file_ids.get(src_path) or \ - self._modified_file_ids.get(src_path) - if newly_changed: - # We've only just added/changed this path earlier in this commit. - file_id = newly_changed - # note: delta entries look like (old, new, file-id, ie) - ie = self._delta_entries_by_fileid[file_id][3] - else: - file_id = inv.path2id(src_path) - if file_id is None: - self.warning("ignoring copy of %s to %s - source does not exist", - src_path, dest_path) - return - ie = inv[file_id] - kind = ie.kind - if kind == 'file': - if newly_changed: - content = self.data_for_commit[file_id] - else: - content = self.rev_store.get_file_text(self.parents[0], file_id) - self._modify_item(dest_path, kind, ie.executable, content, inv) - elif kind == 'symlink': - self._modify_item(dest_path, kind, False, ie.symlink_target, inv) - else: - self.warning("ignoring copy of %s %s - feature not yet supported", - kind, path) - - def _rename_item(self, old_path, new_path, inv): - existing = self._new_file_ids.get(old_path) or \ - self._modified_file_ids.get(old_path) - if existing: - # We've only just added/modified this path earlier in this commit. - # Change the add/modify of old_path to an add of new_path - self._rename_pending_change(old_path, new_path, existing) - return - - file_id = inv.path2id(old_path) - if file_id is None: - self.warning( - "ignoring rename of %s to %s - old path does not exist" % - (old_path, new_path)) - return - ie = inv[file_id] - rev_id = ie.revision - new_file_id = inv.path2id(new_path) - if new_file_id is not None: - self.record_delete(new_path, inv[new_file_id]) - self.record_rename(old_path, new_path, file_id, ie) - - # The revision-id for this entry will be/has been updated and - # that means the loader then needs to know what the "new" text is. - # We therefore must go back to the revision store to get it. - lines = self.rev_store.get_file_lines(rev_id, file_id) - self.data_for_commit[file_id] = ''.join(lines) - - def _delete_all_items(self, inv): - for name, root_item in inv.root.children.iteritems(): - inv.remove_recursive_id(root_item.file_id) - - def _warn_unless_in_merges(self, fileid, path): - if len(self.parents) <= 1: - return - for parent in self.parents[1:]: - if fileid in self.get_inventory(parent): - return - self.warning("ignoring delete of %s as not in parent inventories", path) - - -class InventoryCommitHandler(GenericCommitHandler): - """A CommitHandler that builds and saves Inventory objects.""" - - def pre_process_files(self): - super(InventoryCommitHandler, self).pre_process_files() - - # Seed the inventory from the previous one. Note that - # the parent class version of pre_process_files() has - # already set the right basis_inventory for this branch - # but we need to copy it in order to mutate it safely - # without corrupting the cached inventory value. - if len(self.parents) == 0: - self.inventory = self.basis_inventory - else: - self.inventory = copy_inventory(self.basis_inventory) - self.inventory_root = self.inventory.root - - # directory-path -> inventory-entry for current inventory - self.directory_entries = dict(self.inventory.directories()) - - # Initialise the inventory revision info as required - if self.rev_store.expects_rich_root(): - self.inventory.revision_id = self.revision_id - else: - # In this revision store, root entries have no knit or weave. - # When serializing out to disk and back in, root.revision is - # always the new revision_id. - self.inventory.root.revision = self.revision_id - - def post_process_files(self): - """Save the revision.""" - self.cache_mgr.inventories[self.revision_id] = self.inventory - self.rev_store.load(self.revision, self.inventory, None, - lambda file_id: self._get_data(file_id), - lambda file_id: self._get_per_file_parents(file_id), - lambda revision_ids: self._get_inventories(revision_ids)) - - def record_new(self, path, ie): - try: - # If this is a merge, the file was most likely added already. - # The per-file parent(s) must therefore be calculated and - # we can't assume there are none. - per_file_parents, ie.revision = \ - self.rev_store.get_parents_and_revision_for_entry(ie) - self.per_file_parents_for_commit[ie.file_id] = per_file_parents - self.inventory.add(ie) - except errors.DuplicateFileId: - # Directory already exists as a file or symlink - del self.inventory[ie.file_id] - # Try again - self.inventory.add(ie) - - def record_changed(self, path, ie, parent_id): - # HACK: no API for this (del+add does more than it needs to) - per_file_parents, ie.revision = \ - self.rev_store.get_parents_and_revision_for_entry(ie) - self.per_file_parents_for_commit[ie.file_id] = per_file_parents - self.inventory._byid[ie.file_id] = ie - parent_ie = self.inventory._byid[parent_id] - parent_ie.children[ie.name] = ie - - def record_delete(self, path, ie): - self.inventory.remove_recursive_id(ie.file_id) - - def record_rename(self, old_path, new_path, file_id, ie): - # For a rename, the revision-id is always the new one so - # no need to change/set it here - ie.revision = self.revision_id - per_file_parents, _ = \ - self.rev_store.get_parents_and_revision_for_entry(ie) - self.per_file_parents_for_commit[file_id] = per_file_parents - new_basename, new_parent_id = self._ensure_directory(new_path, - self.inventory) - self.inventory.rename(file_id, new_parent_id, new_basename) - - def modify_handler(self, filecmd): - if filecmd.dataref is not None: - data = self.cache_mgr.fetch_blob(filecmd.dataref) - else: - data = filecmd.data - self.debug("modifying %s", filecmd.path) - self._modify_item(filecmd.path, filecmd.kind, - filecmd.is_executable, data, self.inventory) - - def delete_handler(self, filecmd): - self.debug("deleting %s", filecmd.path) - self._delete_item(filecmd.path, self.inventory) - - def copy_handler(self, filecmd): - src_path = filecmd.src_path - dest_path = filecmd.dest_path - self.debug("copying %s to %s", src_path, dest_path) - self._copy_item(src_path, dest_path, self.inventory) - - def rename_handler(self, filecmd): - old_path = filecmd.old_path - new_path = filecmd.new_path - self.debug("renaming %s to %s", old_path, new_path) - self._rename_item(old_path, new_path, self.inventory) - - def deleteall_handler(self, filecmd): - self.debug("deleting all files (and also all directories)") - self._delete_all_items(self.inventory) - - -class InventoryDeltaCommitHandler(GenericCommitHandler): - """A CommitHandler that builds Inventories by applying a delta.""" - - def pre_process_files(self): - super(InventoryDeltaCommitHandler, self).pre_process_files() - self._dirs_that_might_become_empty = set() - - # A given file-id can only appear once so we accumulate - # the entries in a dict then build the actual delta at the end - self._delta_entries_by_fileid = {} - if len(self.parents) == 0 or not self.rev_store.expects_rich_root(): - if self.parents: - old_path = '' - else: - old_path = None - # Need to explicitly add the root entry for the first revision - # and for non rich-root inventories - root_id = inventory.ROOT_ID - root_ie = inventory.InventoryDirectory(root_id, u'', None) - root_ie.revision = self.revision_id - self._add_entry((old_path, '', root_id, root_ie)) - - def post_process_files(self): - """Save the revision.""" - delta = self._get_final_delta() - inv = self.rev_store.load_using_delta(self.revision, - self.basis_inventory, delta, None, - self._get_data, - self._get_per_file_parents, - self._get_inventories) - self.cache_mgr.inventories[self.revision_id] = inv - #print "committed %s" % self.revision_id - - def _get_final_delta(self): - """Generate the final delta. - - Smart post-processing of changes, e.g. pruning of directories - that would become empty, goes here. - """ - delta = list(self._delta_entries_by_fileid.values()) - if self.prune_empty_dirs and self._dirs_that_might_become_empty: - candidates = self._dirs_that_might_become_empty - while candidates: - never_born = set() - parent_dirs_that_might_become_empty = set() - for path, file_id in self._empty_after_delta(delta, candidates): - newly_added = self._new_file_ids.get(path) - if newly_added: - never_born.add(newly_added) - else: - delta.append((path, None, file_id, None)) - parent_dir = osutils.dirname(path) - if parent_dir: - parent_dirs_that_might_become_empty.add(parent_dir) - candidates = parent_dirs_that_might_become_empty - # Clean up entries that got deleted before they were ever added - if never_born: - delta = [de for de in delta if de[2] not in never_born] - return delta - - def _empty_after_delta(self, delta, candidates): - #self.mutter("delta so far is:\n%s" % "\n".join([str(de) for de in delta])) - #self.mutter("candidates for deletion are:\n%s" % "\n".join([c for c in candidates])) - new_inv = self._get_proposed_inventory(delta) - result = [] - for dir in candidates: - file_id = new_inv.path2id(dir) - if file_id is None: - continue - ie = new_inv[file_id] - if ie.kind != 'directory': - continue - if len(ie.children) == 0: - result.append((dir, file_id)) - if self.verbose: - self.note("pruning empty directory %s" % (dir,)) - return result - - def _get_proposed_inventory(self, delta): - if len(self.parents): - # new_inv = self.basis_inventory._get_mutable_inventory() - # Note that this will create unreferenced chk pages if we end up - # deleting entries, because this 'test' inventory won't end up - # used. However, it is cheaper than having to create a full copy of - # the inventory for every commit. - new_inv = self.basis_inventory.create_by_apply_delta(delta, - 'not-a-valid-revision-id:') - else: - new_inv = inventory.Inventory(revision_id=self.revision_id) - # This is set in the delta so remove it to prevent a duplicate - del new_inv[inventory.ROOT_ID] - try: - new_inv.apply_delta(delta) - except errors.InconsistentDelta: - self.mutter("INCONSISTENT DELTA IS:\n%s" % "\n".join([str(de) for de in delta])) - raise - return new_inv - - def _add_entry(self, entry): - # We need to combine the data if multiple entries have the same file-id. - # For example, a rename followed by a modification looks like: - # - # (x, y, f, e) & (y, y, f, g) => (x, y, f, g) - # - # Likewise, a modification followed by a rename looks like: - # - # (x, x, f, e) & (x, y, f, g) => (x, y, f, g) - # - # Here's a rename followed by a delete and a modification followed by - # a delete: - # - # (x, y, f, e) & (y, None, f, None) => (x, None, f, None) - # (x, x, f, e) & (x, None, f, None) => (x, None, f, None) - # - # In summary, we use the original old-path, new new-path and new ie - # when combining entries. - old_path = entry[0] - new_path = entry[1] - file_id = entry[2] - ie = entry[3] - existing = self._delta_entries_by_fileid.get(file_id, None) - if existing is not None: - old_path = existing[0] - entry = (old_path, new_path, file_id, ie) - if new_path is None and old_path is None: - # This is a delete cancelling a previous add - del self._delta_entries_by_fileid[file_id] - parent_dir = osutils.dirname(existing[1]) - self.mutter("cancelling add of %s with parent %s" % (existing[1], parent_dir)) - if parent_dir: - self._dirs_that_might_become_empty.add(parent_dir) - return - else: - self._delta_entries_by_fileid[file_id] = entry - - # Collect parent directories that might become empty - if new_path is None: - # delete - parent_dir = osutils.dirname(old_path) - # note: no need to check the root - if parent_dir: - self._dirs_that_might_become_empty.add(parent_dir) - elif old_path is not None and old_path != new_path: - # rename - old_parent_dir = osutils.dirname(old_path) - new_parent_dir = osutils.dirname(new_path) - if old_parent_dir and old_parent_dir != new_parent_dir: - self._dirs_that_might_become_empty.add(old_parent_dir) - - # Calculate the per-file parents, if not already done - if file_id in self.per_file_parents_for_commit: - return - if old_path is None: - # add - # If this is a merge, the file was most likely added already. - # The per-file parent(s) must therefore be calculated and - # we can't assume there are none. - per_file_parents, ie.revision = \ - self.rev_store.get_parents_and_revision_for_entry(ie) - self.per_file_parents_for_commit[file_id] = per_file_parents - elif new_path is None: - # delete - pass - elif old_path != new_path: - # rename - per_file_parents, _ = \ - self.rev_store.get_parents_and_revision_for_entry(ie) - self.per_file_parents_for_commit[file_id] = per_file_parents - else: - # modify - per_file_parents, ie.revision = \ - self.rev_store.get_parents_and_revision_for_entry(ie) - self.per_file_parents_for_commit[file_id] = per_file_parents - - def record_new(self, path, ie): - self._add_entry((None, path, ie.file_id, ie)) - - def record_changed(self, path, ie, parent_id=None): - self._add_entry((path, path, ie.file_id, ie)) - self._modified_file_ids[path] = ie.file_id - - def record_delete(self, path, ie): - self._add_entry((path, None, ie.file_id, None)) - self._paths_deleted_this_commit.add(path) - if ie.kind == 'directory': - try: - del self.directory_entries[path] - except KeyError: - pass - for child_relpath, entry in \ - self.basis_inventory.iter_entries_by_dir(from_dir=ie): - child_path = osutils.pathjoin(path, child_relpath) - self._add_entry((child_path, None, entry.file_id, None)) - self._paths_deleted_this_commit.add(child_path) - if entry.kind == 'directory': - try: - del self.directory_entries[child_path] - except KeyError: - pass - - def record_rename(self, old_path, new_path, file_id, old_ie): - new_ie = old_ie.copy() - new_basename, new_parent_id = self._ensure_directory(new_path, - self.basis_inventory) - new_ie.name = new_basename - new_ie.parent_id = new_parent_id - new_ie.revision = self.revision_id - self._add_entry((old_path, new_path, file_id, new_ie)) - self._modified_file_ids[new_path] = file_id - self._paths_deleted_this_commit.discard(new_path) - if new_ie.kind == 'directory': - self.directory_entries[new_path] = new_ie - - def _rename_pending_change(self, old_path, new_path, file_id): - """Instead of adding/modifying old-path, add new-path instead.""" - # note: delta entries look like (old, new, file-id, ie) - old_ie = self._delta_entries_by_fileid[file_id][3] - - # Delete the old path. Note that this might trigger implicit - # deletion of newly created parents that could now become empty. - self.record_delete(old_path, old_ie) - - # Update the dictionaries used for tracking new file-ids - if old_path in self._new_file_ids: - del self._new_file_ids[old_path] - else: - del self._modified_file_ids[old_path] - self._new_file_ids[new_path] = file_id - - # Create the new InventoryEntry - kind = old_ie.kind - basename, parent_id = self._ensure_directory(new_path, - self.basis_inventory) - ie = inventory.make_entry(kind, basename, parent_id, file_id) - ie.revision = self.revision_id - if kind == 'file': - ie.executable = old_ie.executable - ie.text_sha1 = old_ie.text_sha1 - ie.text_size = old_ie.text_size - elif kind == 'symlink': - ie.symlink_target = old_ie.symlink_target - - # Record it - self.record_new(new_path, ie) - - def modify_handler(self, filecmd): - if filecmd.dataref is not None: - if filecmd.kind == commands.DIRECTORY_KIND: - data = None - elif filecmd.kind == commands.TREE_REFERENCE_KIND: - data = filecmd.dataref - else: - data = self.cache_mgr.fetch_blob(filecmd.dataref) - else: - data = filecmd.data - self.debug("modifying %s", filecmd.path) - self._modify_item(filecmd.path, filecmd.kind, - filecmd.is_executable, data, self.basis_inventory) - - def delete_handler(self, filecmd): - self.debug("deleting %s", filecmd.path) - self._delete_item(filecmd.path, self.basis_inventory) - - def copy_handler(self, filecmd): - src_path = filecmd.src_path - dest_path = filecmd.dest_path - self.debug("copying %s to %s", src_path, dest_path) - self._copy_item(src_path, dest_path, self.basis_inventory) - - def rename_handler(self, filecmd): - old_path = filecmd.old_path - new_path = filecmd.new_path - self.debug("renaming %s to %s", old_path, new_path) - self._rename_item(old_path, new_path, self.basis_inventory) - - def deleteall_handler(self, filecmd): - self.debug("deleting all files (and also all directories)") - # I'm not 100% sure this will work in the delta case. - # But clearing out the basis inventory so that everything - # is added sounds ok in theory ... - # We grab a copy as the basis is likely to be cached and - # we don't want to destroy the cached version - self.basis_inventory = copy_inventory(self.basis_inventory) - self._delete_all_items(self.basis_inventory) diff --git a/bzr_exporter.py b/bzr_exporter.py deleted file mode 100755 index 8cff2ab..0000000 --- a/bzr_exporter.py +++ /dev/null @@ -1,503 +0,0 @@ -# -*- coding: utf-8 -*- - -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA -# -# Original Copyright (c) 2008 Adeodato Simó -# Original License: MIT (See exporters/bzr-fast-export.LICENSE) -# -# vim: fileencoding=utf-8 - -"""Core engine for the fast-export command.""" - -# TODO: if a new_git_branch below gets merged repeatedly, the tip of the branch -# is not updated (because the parent of commit is already merged, so we don't -# set new_git_branch to the previously used name) - -from email.Utils import parseaddr -import sys, time - -import bzrlib.branch -import bzrlib.revision -from bzrlib import ( - builtins, - errors as bazErrors, - osutils, - progress, - trace, - ) - -from bzrlib.plugins.fastimport import commands, helpers, marks_file - - -class BzrFastExporter(object): - - def __init__(self, source, destination, git_branch=None, checkpoint=-1, - import_marks_file=None, export_marks_file=None, revision=None, - verbose=False, plain_format=False): - """Export branch data in fast import format. - - :param plain_format: if True, 'classic' fast-import format is - used without any extended features; if False, the generated - data is richer and includes information like multiple - authors, revision properties, etc. - """ - self.source = source - if destination is None or destination == '-': - self.outf = helpers.binary_stream(sys.stdout) - elif destination.endswith('gz'): - import gzip - self.outf = gzip.open(destination, 'wb') - else: - self.outf = open(destination, 'wb') - self.git_branch = git_branch - self.checkpoint = checkpoint - self.import_marks_file = import_marks_file - self.export_marks_file = export_marks_file - self.revision = revision - self.excluded_revisions = set() - self.plain_format = plain_format - self._multi_author_api_available = hasattr(bzrlib.revision.Revision, - 'get_apparent_authors') - self.properties_to_exclude = ['authors', 'author'] - - # Progress reporting stuff - self.verbose = verbose - if verbose: - self.progress_every = 100 - else: - self.progress_every = 1000 - self._start_time = time.time() - self._commit_total = 0 - - # Load the marks and initialise things accordingly - self.revid_to_mark = {} - self.branch_names = {} - if self.import_marks_file: - marks_info = marks_file.import_marks(self.import_marks_file) - if marks_info is not None: - self.revid_to_mark = dict((r, m) for m, r in - marks_info[0].items()) - self.branch_names = marks_info[1] - - def interesting_history(self): - if self.revision: - rev1, rev2 = builtins._get_revision_range(self.revision, - self.branch, "fast-export") - start_rev_id = rev1.rev_id - end_rev_id = rev2.rev_id - else: - start_rev_id = None - end_rev_id = None - self.note("Calculating the revisions to include ...") - view_revisions = reversed([rev_id for rev_id, _, _, _ in - self.branch.iter_merge_sorted_revisions(end_rev_id, start_rev_id)]) - # If a starting point was given, we need to later check that we don't - # start emitting revisions from before that point. Collect the - # revisions to exclude now ... - if start_rev_id is not None: - self.note("Calculating the revisions to exclude ...") - self.excluded_revisions = set([rev_id for rev_id, _, _, _ in - self.branch.iter_merge_sorted_revisions(start_rev_id)]) - return list(view_revisions) - - def run(self): - # Open the source - self.branch = bzrlib.branch.Branch.open_containing(self.source)[0] - - # Export the data - self.branch.repository.lock_read() - try: - interesting = self.interesting_history() - self._commit_total = len(interesting) - self.note("Starting export of %d revisions ..." % - self._commit_total) - if not self.plain_format: - self.emit_features() - for revid in interesting: - self.emit_commit(revid, self.git_branch) - if self.branch.supports_tags(): - self.emit_tags() - finally: - self.branch.repository.unlock() - - # Save the marks if requested - self._save_marks() - self.dump_stats() - - def note(self, msg, *args): - """Output a note but timestamp it.""" - msg = "%s %s" % (self._time_of_day(), msg) - trace.note(msg, *args) - - def warning(self, msg, *args): - """Output a warning but timestamp it.""" - msg = "%s WARNING: %s" % (self._time_of_day(), msg) - trace.warning(msg, *args) - - def _time_of_day(self): - """Time of day as a string.""" - # Note: this is a separate method so tests can patch in a fixed value - return time.strftime("%H:%M:%S") - - def report_progress(self, commit_count, details=''): - if commit_count and commit_count % self.progress_every == 0: - if self._commit_total: - counts = "%d/%d" % (commit_count, self._commit_total) - else: - counts = "%d" % (commit_count,) - minutes = (time.time() - self._start_time) / 60 - rate = commit_count * 1.0 / minutes - if rate > 10: - rate_str = "at %.0f/minute " % rate - else: - rate_str = "at %.1f/minute " % rate - self.note("%s commits exported %s%s" % (counts, rate_str, details)) - - def dump_stats(self): - time_required = progress.str_tdelta(time.time() - self._start_time) - rc = len(self.revid_to_mark) - self.note("Exported %d %s in %s", - rc, helpers.single_plural(rc, "revision", "revisions"), - time_required) - - def print_cmd(self, cmd): - self.outf.write("%r\n" % cmd) - - def _save_marks(self): - if self.export_marks_file: - revision_ids = dict((m, r) for r, m in self.revid_to_mark.items()) - marks_file.export_marks(self.export_marks_file, revision_ids, - self.branch_names) - - def is_empty_dir(self, tree, path): - path_id = tree.path2id(path) - if path_id is None: - self.warning("Skipping empty_dir detection - no file_id for %s" % - (path,)) - return False - - # Continue if path is not a directory - if tree.kind(path_id) != 'directory': - return False - - # Use treewalk to find the contents of our directory - contents = list(tree.walkdirs(prefix=path))[0] - if len(contents[1]) == 0: - return True - else: - return False - - def emit_features(self): - for feature in sorted(commands.FEATURE_NAMES): - self.print_cmd(commands.FeatureCommand(feature)) - - def emit_commit(self, revid, git_branch): - if revid in self.revid_to_mark or revid in self.excluded_revisions: - return - - # Get the Revision object - try: - revobj = self.branch.repository.get_revision(revid) - except bazErrors.NoSuchRevision: - # This is a ghost revision. Mark it as not found and next! - self.revid_to_mark[revid] = -1 - return - - # Get the primary parent - # TODO: Consider the excluded revisions when deciding the parents. - # Currently, a commit with parents that are excluded ought to be - # triggering the git_branch calculation below (and it is not). - # IGC 20090824 - ncommits = len(self.revid_to_mark) - nparents = len(revobj.parent_ids) - if nparents == 0: - if ncommits: - # This is a parentless commit but it's not the first one - # output. We need to create a new temporary branch for it - # otherwise git-fast-import will assume the previous commit - # was this one's parent - git_branch = self._next_tmp_branch_name() - parent = bzrlib.revision.NULL_REVISION - else: - parent = revobj.parent_ids[0] - - # Print the commit - git_ref = 'refs/heads/%s' % (git_branch,) - mark = ncommits + 1 - self.revid_to_mark[revid] = mark - file_cmds = self._get_filecommands(parent, revid) - self.print_cmd(self._get_commit_command(git_ref, mark, revobj, - file_cmds)) - - # Report progress and checkpoint if it's time for that - self.report_progress(ncommits) - if (self.checkpoint > 0 and ncommits - and ncommits % self.checkpoint == 0): - self.note("Exported %i commits - adding checkpoint to output" - % ncommits) - self._save_marks() - self.print_cmd(commands.CheckpointCommand()) - - def _get_name_email(self, user): - if user.find('<') == -1: - # If the email isn't inside <>, we need to use it as the name - # in order for things to round-trip correctly. - # (note: parseaddr('a@b.com') => name:'', email: 'a@b.com') - name = user - email = '' - else: - name, email = parseaddr(user) - return name, email - - def _get_commit_command(self, git_ref, mark, revobj, file_cmds): - # Get the committer and author info - committer = revobj.committer - name, email = self._get_name_email(committer) - committer_info = (name, email, revobj.timestamp, revobj.timezone) - if self._multi_author_api_available: - more_authors = revobj.get_apparent_authors() - author = more_authors.pop(0) - else: - more_authors = [] - author = revobj.get_apparent_author() - if more_authors: - name, email = self._get_name_email(author) - author_info = (name, email, revobj.timestamp, revobj.timezone) - more_author_info = [] - for a in more_authors: - name, email = self._get_name_email(a) - more_author_info.append( - (name, email, revobj.timestamp, revobj.timezone)) - elif author != committer: - name, email = self._get_name_email(author) - author_info = (name, email, revobj.timestamp, revobj.timezone) - more_author_info = None - else: - author_info = None - more_author_info = None - - # Get the parents in terms of marks - non_ghost_parents = [] - for p in revobj.parent_ids: - if p in self.excluded_revisions: - continue - try: - parent_mark = self.revid_to_mark[p] - non_ghost_parents.append(":%s" % parent_mark) - except KeyError: - # ghost - ignore - continue - if non_ghost_parents: - from_ = non_ghost_parents[0] - merges = non_ghost_parents[1:] - else: - from_ = None - merges = None - - # Filter the revision properties. Some metadata (like the - # author information) is already exposed in other ways so - # don't repeat it here. - if self.plain_format: - properties = None - else: - properties = revobj.properties - for prop in self.properties_to_exclude: - try: - del properties[prop] - except KeyError: - pass - - # Build and return the result - return commands.CommitCommand(git_ref, mark, author_info, - committer_info, revobj.message, from_, merges, iter(file_cmds), - more_authors=more_author_info, properties=properties) - - def _get_revision_trees(self, parent, revision_id): - try: - tree_old = self.branch.repository.revision_tree(parent) - except bazErrors.UnexpectedInventoryFormat: - self.warning("Parent is malformed - diffing against previous parent") - # We can't find the old parent. Let's diff against his parent - pp = self.branch.repository.get_revision(parent) - tree_old = self.branch.repository.revision_tree(pp.parent_ids[0]) - tree_new = None - try: - tree_new = self.branch.repository.revision_tree(revision_id) - except bazErrors.UnexpectedInventoryFormat: - # We can't really do anything anymore - self.warning("Revision %s is malformed - skipping" % revision_id) - return tree_old, tree_new - - def _get_filecommands(self, parent, revision_id): - """Get the list of FileCommands for the changes between two revisions.""" - tree_old, tree_new = self._get_revision_trees(parent, revision_id) - if not(tree_old and tree_new): - # Something is wrong with this revision - ignore the filecommands - return [] - - changes = tree_new.changes_from(tree_old) - - # Make "modified" have 3-tuples, as added does - my_modified = [ x[0:3] for x in changes.modified ] - - # The potential interaction between renames and deletes is messy. - # Handle it here ... - file_cmds, rd_modifies, renamed = self._process_renames_and_deletes( - changes.renamed, changes.removed, revision_id, tree_old) - - # Map kind changes to a delete followed by an add - for path, id_, kind1, kind2 in changes.kind_changed: - path = self._adjust_path_for_renames(path, renamed, revision_id) - # IGC: I don't understand why a delete is needed here. - # In fact, it seems harmful? If you uncomment this line, - # please file a bug explaining why you needed to. - #file_cmds.append(commands.FileDeleteCommand(path)) - my_modified.append((path, id_, kind2)) - - # Record modifications - for path, id_, kind in changes.added + my_modified + rd_modifies: - if kind == 'file': - text = tree_new.get_file_text(id_) - file_cmds.append(commands.FileModifyCommand(path, 'file', - tree_new.is_executable(id_), None, text)) - elif kind == 'symlink': - file_cmds.append(commands.FileModifyCommand(path, 'symlink', - False, None, tree_new.get_symlink_target(id_))) - elif kind == 'directory': - if not self.plain_format: - file_cmds.append(commands.FileModifyCommand(path, 'directory', - False, None, None)) - else: - self.warning("cannot export '%s' of kind %s yet - ignoring" % - (path, kind)) - return file_cmds - - def _process_renames_and_deletes(self, renames, deletes, - revision_id, tree_old): - file_cmds = [] - modifies = [] - renamed = [] - - # See https://bugs.edge.launchpad.net/bzr-fastimport/+bug/268933. - # In a nutshell, there are several nasty cases: - # - # 1) bzr rm a; bzr mv b a; bzr commit - # 2) bzr mv x/y z; bzr rm x; commmit - # - # The first must come out with the delete first like this: - # - # D a - # R b a - # - # The second case must come out with the rename first like this: - # - # R x/y z - # D x - # - # So outputting all deletes first or all renames first won't work. - # Instead, we need to make multiple passes over the various lists to - # get the ordering right. - - must_be_renamed = {} - old_to_new = {} - deleted_paths = set([p for p, _, _ in deletes]) - for (oldpath, newpath, id_, kind, - text_modified, meta_modified) in renames: - emit = kind != 'directory' or not self.plain_format - if newpath in deleted_paths: - if emit: - file_cmds.append(commands.FileDeleteCommand(newpath)) - deleted_paths.remove(newpath) - if (self.is_empty_dir(tree_old, oldpath)): - self.note("Skipping empty dir %s in rev %s" % (oldpath, - revision_id)) - continue - #oldpath = self._adjust_path_for_renames(oldpath, renamed, - # revision_id) - renamed.append([oldpath, newpath]) - old_to_new[oldpath] = newpath - if emit: - file_cmds.append(commands.FileRenameCommand(oldpath, newpath)) - if text_modified or meta_modified: - modifies.append((newpath, id_, kind)) - - # Renaming a directory implies all children must be renamed. - # Note: changes_from() doesn't handle this - if kind == 'directory': - for p, e in tree_old.inventory.iter_entries_by_dir(from_dir=id_): - if e.kind == 'directory' and self.plain_format: - continue - old_child_path = osutils.pathjoin(oldpath, p) - new_child_path = osutils.pathjoin(newpath, p) - must_be_renamed[old_child_path] = new_child_path - - # Add children not already renamed - if must_be_renamed: - renamed_already = set(old_to_new.keys()) - still_to_be_renamed = set(must_be_renamed.keys()) - renamed_already - for old_child_path in sorted(still_to_be_renamed): - new_child_path = must_be_renamed[old_child_path] - if self.verbose: - self.note("implicitly renaming %s => %s" % (old_child_path, - new_child_path)) - file_cmds.append(commands.FileRenameCommand(old_child_path, - new_child_path)) - - # Record remaining deletes - for path, id_, kind in deletes: - if path not in deleted_paths: - continue - if kind == 'directory' and self.plain_format: - continue - #path = self._adjust_path_for_renames(path, renamed, revision_id) - file_cmds.append(commands.FileDeleteCommand(path)) - return file_cmds, modifies, renamed - - def _adjust_path_for_renames(self, path, renamed, revision_id): - # If a previous rename is found, we should adjust the path - for old, new in renamed: - if path == old: - self.note("Changing path %s given rename to %s in revision %s" - % (path, new, revision_id)) - path = new - elif path.startswith(old + '/'): - self.note( - "Adjusting path %s given rename of %s to %s in revision %s" - % (path, old, new, revision_id)) - path = path.replace(old + "/", new + "/") - return path - - def emit_tags(self): - for tag, revid in self.branch.tags.get_tag_dict().items(): - try: - mark = self.revid_to_mark[revid] - except KeyError: - self.warning('not creating tag %r pointing to non-existent ' - 'revision %s' % (tag, revid)) - else: - git_ref = 'refs/tags/%s' % tag - self.print_cmd(commands.ResetCommand(git_ref, ":" + str(mark))) - - def _next_tmp_branch_name(self): - """Return a unique branch name. The name will start with "tmp".""" - prefix = 'tmp' - if prefix not in self.branch_names: - self.branch_names[prefix] = 0 - else: - self.branch_names[prefix] += 1 - prefix = '%s.%d' % (prefix, self.branch_names[prefix]) - return prefix diff --git a/cache_manager.py b/cache_manager.py deleted file mode 100644 index 5a31a00..0000000 --- a/cache_manager.py +++ /dev/null @@ -1,318 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""A manager of caches.""" - -import atexit -import os -import shutil -import tempfile -import weakref - -from bzrlib import lru_cache, trace -from bzrlib.plugins.fastimport import branch_mapper, helpers - - -class _Cleanup(object): - """This class makes sure we clean up when CacheManager goes away. - - We use a helper class to ensure that we are never in a refcycle. - """ - - def __init__(self, disk_blobs): - self.disk_blobs = disk_blobs - self.tempdir = None - self.small_blobs = None - - def __del__(self): - self.finalize() - - def finalize(self): - if self.disk_blobs is not None: - for info in self.disk_blobs.itervalues(): - if info[-1] is not None: - os.unlink(info[-1]) - self.disk_blobs = None - if self.small_blobs is not None: - self.small_blobs.close() - self.small_blobs = None - if self.tempdir is not None: - shutil.rmtree(self.tempdir) - - -class _Cleanup(object): - """This class makes sure we clean up when CacheManager goes away. - - We use a helper class to ensure that we are never in a refcycle. - """ - - def __init__(self, disk_blobs): - self.disk_blobs = disk_blobs - self.tempdir = None - self.small_blobs = None - - def __del__(self): - self.finalize() - - def finalize(self): - if self.disk_blobs is not None: - for info in self.disk_blobs.itervalues(): - if info[-1] is not None: - os.unlink(info[-1]) - self.disk_blobs = None - if self.small_blobs is not None: - self.small_blobs.close() - self.small_blobs = None - if self.tempdir is not None: - shutils.rmtree(self.tempdir) - - -class CacheManager(object): - - _small_blob_threshold = 25*1024 - _sticky_cache_size = 300*1024*1024 - _sticky_flushed_size = 100*1024*1024 - - def __init__(self, info=None, verbose=False, inventory_cache_size=10): - """Create a manager of caches. - - :param info: a ConfigObj holding the output from - the --info processor, or None if no hints are available - """ - self.verbose = verbose - - # dataref -> data. datref is either :mark or the sha-1. - # Sticky blobs are referenced more than once, and are saved until their - # refcount goes to 0 - self._blobs = {} - self._sticky_blobs = {} - self._sticky_memory_bytes = 0 - # if we overflow our memory cache, then we will dump large blobs to - # disk in this directory - self._tempdir = None - # id => (offset, n_bytes, fname) - # if fname is None, then the content is stored in the small file - self._disk_blobs = {} - self._cleanup = _Cleanup(self._disk_blobs) - - # revision-id -> Inventory cache - # these are large and we probably don't need too many as - # most parents are recent in history - self.inventories = lru_cache.LRUCache(inventory_cache_size) - - # import commmit-ids -> revision-id lookup table - # we need to keep all of these but they are small - self.revision_ids = {} - - # (path, branch_ref) -> file-ids - as generated. - # (Use store_file_id/fetch_fileid methods rather than direct access.) - - # Head tracking: last ref, last id per ref & map of commit ids to ref*s* - self.last_ref = None - self.last_ids = {} - self.heads = {} - - # Work out the blobs to make sticky - None means all - self._blob_ref_counts = {} - if info is not None: - try: - blobs_by_counts = info['Blob reference counts'] - # The parser hands values back as lists, already parsed - for count, blob_list in blobs_by_counts.items(): - n = int(count) - for b in blob_list: - self._blob_ref_counts[b] = n - except KeyError: - # info not in file - possible when no blobs used - pass - - # BranchMapper has no state (for now?), but we keep it around rather - # than reinstantiate on every usage - self.branch_mapper = branch_mapper.BranchMapper() - - def dump_stats(self, note=trace.note): - """Dump some statistics about what we cached.""" - # TODO: add in inventory stastistics - note("Cache statistics:") - self._show_stats_for(self._sticky_blobs, "sticky blobs", note=note) - self._show_stats_for(self.revision_ids, "revision-ids", note=note) - # These aren't interesting so omit from the output, at least for now - #self._show_stats_for(self._blobs, "other blobs", note=note) - #self._show_stats_for(self.last_ids, "last-ids", note=note) - #self._show_stats_for(self.heads, "heads", note=note) - - def _show_stats_for(self, dict, label, note=trace.note, tuple_key=False): - """Dump statistics about a given dictionary. - - By the key and value need to support len(). - """ - count = len(dict) - if tuple_key: - size = sum(map(len, (''.join(k) for k in dict.keys()))) - else: - size = sum(map(len, dict.keys())) - size += sum(map(len, dict.values())) - size = size * 1.0 / 1024 - unit = 'K' - if size > 1024: - size = size / 1024 - unit = 'M' - if size > 1024: - size = size / 1024 - unit = 'G' - note(" %-12s: %8.1f %s (%d %s)" % (label, size, unit, count, - helpers.single_plural(count, "item", "items"))) - - def clear_all(self): - """Free up any memory used by the caches.""" - self._blobs.clear() - self._sticky_blobs.clear() - self.revision_ids.clear() - self.last_ids.clear() - self.heads.clear() - self.inventories.clear() - - def _flush_blobs_to_disk(self): - blobs = self._sticky_blobs.keys() - sticky_blobs = self._sticky_blobs - total_blobs = len(sticky_blobs) - blobs.sort(key=lambda k:len(sticky_blobs[k])) - if self._tempdir is None: - tempdir = tempfile.mkdtemp(prefix='bzr_fastimport_blobs-') - self._tempdir = tempdir - self._cleanup.tempdir = self._tempdir - self._cleanup.small_blobs = tempfile.TemporaryFile( - prefix='small-blobs-', dir=self._tempdir) - small_blob_ref = weakref.ref(self._cleanup.small_blobs) - # Even though we add it to _Cleanup it seems that the object can be - # destroyed 'too late' for cleanup to actually occur. Probably a - # combination of bzr's "die directly, don't clean up" and how - # exceptions close the running stack. - def exit_cleanup(): - small_blob = small_blob_ref() - if small_blob is not None: - small_blob.close() - shutil.rmtree(tempdir, ignore_errors=True) - atexit.register(exit_cleanup) - count = 0 - bytes = 0 - n_small_bytes = 0 - while self._sticky_memory_bytes > self._sticky_flushed_size: - id = blobs.pop() - blob = self._sticky_blobs.pop(id) - n_bytes = len(blob) - self._sticky_memory_bytes -= n_bytes - if n_bytes < self._small_blob_threshold: - f = self._cleanup.small_blobs - f.seek(0, os.SEEK_END) - self._disk_blobs[id] = (f.tell(), n_bytes, None) - f.write(blob) - n_small_bytes += n_bytes - else: - fd, name = tempfile.mkstemp(prefix='blob-', dir=self._tempdir) - os.write(fd, blob) - os.close(fd) - self._disk_blobs[id] = (0, n_bytes, name) - bytes += n_bytes - del blob - count += 1 - trace.note('flushed %d/%d blobs w/ %.1fMB (%.1fMB small) to disk' - % (count, total_blobs, bytes / 1024. / 1024, - n_small_bytes / 1024. / 1024)) - - - def store_blob(self, id, data): - """Store a blob of data.""" - # Note: If we're not reference counting, everything has to be sticky - if not self._blob_ref_counts or id in self._blob_ref_counts: - self._sticky_blobs[id] = data - self._sticky_memory_bytes += len(data) - if self._sticky_memory_bytes > self._sticky_cache_size: - self._flush_blobs_to_disk() - elif data == '': - # Empty data is always sticky - self._sticky_blobs[id] = data - else: - self._blobs[id] = data - - def _decref(self, id, cache, fn): - if not self._blob_ref_counts: - return False - count = self._blob_ref_counts.get(id, None) - if count is not None: - count -= 1 - if count <= 0: - del cache[id] - if fn is not None: - os.unlink(fn) - del self._blob_ref_counts[id] - return True - else: - self._blob_ref_counts[id] = count - return False - - def fetch_blob(self, id): - """Fetch a blob of data.""" - if id in self._blobs: - return self._blobs.pop(id) - if id in self._disk_blobs: - (offset, n_bytes, fn) = self._disk_blobs[id] - if fn is None: - f = self._cleanup.small_blobs - f.seek(offset) - content = f.read(n_bytes) - else: - fp = open(fn, 'rb') - try: - content = fp.read() - finally: - fp.close() - self._decref(id, self._disk_blobs, fn) - return content - content = self._sticky_blobs[id] - if self._decref(id, self._sticky_blobs, None): - self._sticky_memory_bytes -= len(content) - return content - - def track_heads(self, cmd): - """Track the repository heads given a CommitCommand. - - :param cmd: the CommitCommand - :return: the list of parents in terms of commit-ids - """ - # Get the true set of parents - if cmd.from_ is not None: - parents = [cmd.from_] - else: - last_id = self.last_ids.get(cmd.ref) - if last_id is not None: - parents = [last_id] - else: - parents = [] - parents.extend(cmd.merges) - - # Track the heads - self.track_heads_for_ref(cmd.ref, cmd.id, parents) - return parents - - def track_heads_for_ref(self, cmd_ref, cmd_id, parents=None): - if parents is not None: - for parent in parents: - if parent in self.heads: - del self.heads[parent] - self.heads.setdefault(cmd_id, set()).add(cmd_ref) - self.last_ids[cmd_ref] = cmd_id - self.last_ref = cmd_ref diff --git a/doc/notes.txt b/doc/notes.txt deleted file mode 100644 index e081e08..0000000 --- a/doc/notes.txt +++ /dev/null @@ -1,93 +0,0 @@ -======================= -Notes on bzr-fastimport -======================= - -..contents:: - -Features -======== - -fast-import ------------ - -Things that ought to work: - -* add & deletes of files and symlinks - -* automatic creation of directories (but not deletion) - -* executable permission - -* branches created based on where the import is run: - - * import into a shared repository outside a branch - branches - are created as subdirectories of the current directory - - * import into a branch inside a shared repository - current - branch becomes the trunk and other branches are created - as sister directories - - * import into a standalone tree - warnings are given - for branches (heads) found but not imported - -* merge tracking - -Things that probably work (more testing needed): - -* separate author to committer - -* lightweight tags - - -Known Limitations -================= - -Parsing -------- - -Things not supported yet: - -* renaming a path that contains a space in the old name - -* copying a path that contains a space in the source name - -* delimited data sections (all data must be length prefixed currently) - -* rfc2822 dates. - -fast-import ------------ - -Things not supported yet: - -* deterministic revision-ids as an option - -* 'reset' handling - -* 'filedeleteall' handling - -Things not recorded in Bazaar: - -* tagger and message for (non-lightweight) tags - -* copy semantics - - -Custom Enhancements -=================== - -General -------- - -The date format is auto-detected. - -Parsing -------- - -These enhancements over the specification are provided in order -to read data produced by some verisons of git-fast-export: - -* A person's name may be empty - -* Long file modes with an extra leading 0, i.e. 0000644, - 0000755 and 0120000 are legal. diff --git a/explorer/logos/cvs.png b/explorer/logos/cvs.png Binary files differdeleted file mode 100644 index e279bdf..0000000 --- a/explorer/logos/cvs.png +++ /dev/null diff --git a/explorer/logos/darcs.png b/explorer/logos/darcs.png Binary files differdeleted file mode 100644 index ca9365f..0000000 --- a/explorer/logos/darcs.png +++ /dev/null diff --git a/explorer/logos/git.png b/explorer/logos/git.png Binary files differdeleted file mode 100644 index aae35a7..0000000 --- a/explorer/logos/git.png +++ /dev/null diff --git a/explorer/logos/mercurial.png b/explorer/logos/mercurial.png Binary files differdeleted file mode 100644 index 60effbc..0000000 --- a/explorer/logos/mercurial.png +++ /dev/null diff --git a/explorer/logos/monotone.png b/explorer/logos/monotone.png Binary files differdeleted file mode 100644 index 16f1908..0000000 --- a/explorer/logos/monotone.png +++ /dev/null diff --git a/explorer/logos/perforce.png b/explorer/logos/perforce.png Binary files differdeleted file mode 100644 index e62897c..0000000 --- a/explorer/logos/perforce.png +++ /dev/null diff --git a/explorer/logos/subversion.png b/explorer/logos/subversion.png Binary files differdeleted file mode 100644 index d28702a..0000000 --- a/explorer/logos/subversion.png +++ /dev/null diff --git a/explorer/tools.xml b/explorer/tools.xml deleted file mode 100644 index 2386737..0000000 --- a/explorer/tools.xml +++ /dev/null @@ -1,20 +0,0 @@ -<folder title="Tools"> - <folder title="Migration Tools"> - <folder title="Export From" icon="actions/edit-redo"> - <tool action="qrun fast-export" icon="logos/bazaar" title="Bazaar" type="bzr" /> - <tool action="qrun fast-export-from-cvs" icon="logos/cvs" title="CVS" type="bzr" /> - <tool action="qrun fast-export-from-darcs" icon="logos/darcs" title="Darcs" type="bzr" /> - <tool action="qrun fast-export-from-git" icon="logos/git" title="Git" type="bzr" /> - <tool action="qrun fast-export-from-hg" icon="logos/mercurial" title="Mercurial" type="bzr" /> - <tool action="qrun fast-export-from-mtn" icon="logos/monotone" title="Monotone" type="bzr" /> - <tool action="qrun fast-export-from-p4" icon="logos/perforce" title="Perforce" type="bzr" /> - <tool action="qrun fast-export-from-svn" icon="logos/subversion" title="Subversion" type="bzr" /> - </folder> - <folder title="Import From" icon="actions/go-jump"> - <tool action="qrun fast-import" icon="mimetypes/text-x-generic-template" title="Fast Import Stream" type="bzr" /> - </folder> - <separator/> - <tool action="qrun fast-import-filter" icon="actions/media-playback-pause" title="Fast Import Filter" type="bzr" /> - </folder> -</folder> - diff --git a/exporters/Makefile b/exporters/Makefile deleted file mode 100644 index 2b71211..0000000 --- a/exporters/Makefile +++ /dev/null @@ -1,15 +0,0 @@ -SVN ?= /usr -CFLAGS += -I${SVN}/include/subversion-1 -pipe -O2 -std=c99 -CFLAGS += `pkg-config --cflags apr-1` -LDFLAGS += -L${SVN}/lib -lsvn_fs-1 -lsvn_repos-1 -LDFLAGS += `pkg-config --libs apr-1` - -all: svn-fast-export svn-archive - -svn-fast-export: svn-fast-export.c -svn-archive: svn-archive.c - -.PHONY: clean - -clean: - rm -rf svn-fast-export svn-archive diff --git a/exporters/__init__.py b/exporters/__init__.py deleted file mode 100644 index e9bf1f6..0000000 --- a/exporters/__init__.py +++ /dev/null @@ -1,327 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Simplified and unified access to the various xxx-fast-export tools.""" - - -import gzip, os, subprocess, sys - -from bzrlib import errors -from bzrlib.trace import note, warning - - -class MissingDependency(Exception): - - def __init__(self, tool, minimum_version, missing): - self.tool = tool - self.minimum_version = minimum_version - self.missing = missing - - def get_message(self): - return "%s missing. Please install %s %s or later and try again." % \ - (self.missing, self.tool, self.minimum_version) - - -class _Exporter(object): - - def check_install(self, tool_name, minimum_version, required_commands=None, - required_libraries=None): - """Check dependencies are correctly installed. - - :param tool_name: name of the tool - :param minimum_version: minimum version required - :param required_commands: list of commands that must be on the path - :param required_libraries: list of Python libraries that must be - available - :raises MissingDependency: if a required dependency is not found - """ - self.tool_name = tool_name - self.minimum_version = minimum_version - if required_commands: - for cmd in required_commands: - self._check_cmd_available(cmd) - if required_libraries: - for lib in required_libraries: - self._check_lib_available(lib) - - def _check_cmd_available(self, cmd): - try: - if isinstance(cmd, str): - args = [cmd] - else: - args = cmd - retcode = subprocess.call(args, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - except OSError: - raise MissingDependency(self.tool_name, self.minimum_version, cmd) - - def _check_lib_available(self, lib): - try: - __import__(lib) - except ImportError: - raise MissingDependency(self.tool_name, self.minimum_version, lib) - - def generate(self, source, destination, verbose=False, custom=None): - """Generate a fast import stream. - - :param source: the source filename or URL - :param destination: filename or '-' for standard output - :param verbose: if True, output additional diagnostics - :param custom: a list of custom options to be added to the - command line of the underlying scripts used. If an option - and its argument are to be separated by a space, pass them - as consecutive items. - """ - raise NotImplementedError(self.generate) - - def get_output_info(self, dest): - """Get the output streams/filenames given a destination filename. - - :return: outf, basename, marks where - outf is a file-like object for storing the output, - basename is the name without the .fi and .gz prefixes - marks is the name of the marks file to use, if any - """ - if dest == '-': - return sys.stdout, None, None - else: - #if dest.endswith('.gz'): - # outf = gzip.open(dest, 'wb') - # base = dest[:-3] - #else: - outf = open(dest, 'w') - base = dest - if base.endswith(".fi"): - base = dest[:-3] - marks = "%s.marks" % (base,) - return outf, base, marks - - def execute(self, args, outf, cwd=None): - """Execute a command, capture the output and close files. - - :param args: list of arguments making up the command - :param outf: a file-like object for storing the output, - :param cwd: current working directory to use - :return: the return code - """ - if cwd is not None: - note("Executing %s in directory %s ..." % (" ".join(args), cwd)) - else: - note("Executing %s ..." % (" ".join(args),)) - try: - p = subprocess.Popen(args, stdout=outf, cwd=cwd) - p.wait() - finally: - if outf != sys.stdout: - outf.close() - return p.returncode - - def report_results(self, retcode, destination): - """Report whether the export succeeded or otherwise.""" - if retcode == 0: - note("Export to %s completed successfully." % (destination,)) - else: - warning("Export to %s exited with error code %d." - % (destination, retcode)) - - def execute_exporter_script(self, args, outf): - """Execute an exporter script, capturing the output. - - The script must be a Python script under the exporters directory. - - :param args: list of arguments making up the script, the first of - which is the script name relative to the exporters directory. - :param outf: a file-like object for storing the output, - :return: the return code - """ - # Note: currently assume Python is on the path. We could work around - # this later (for Windows users say) by packaging the scripts as Python - # modules and calling their internals directly. - exporters_dir = os.path.dirname(__file__) - script_abspath = os.path.join(exporters_dir, args[0]) - actual_args = ['python', script_abspath] + args[1:] - return self.execute(actual_args, outf) - - -class CvsExporter(_Exporter): - - def __init__(self): - self.check_install('cvs2svn', '2.30', ['cvs2bzr']) - self.check_install('CVS', '1.11', ['cvs']) - - def generate(self, source, destination, verbose=False, custom=None): - """Generate a fast import stream. See _Exporter.generate() for details.""" - # TODO: pass a custom cvs2bzr-default.options file as soon as - # cvs2bzr handles --options along with others. - args = ["cvs2bzr", "--dumpfile", destination] - outf, base, marks = self.get_output_info(destination) - # Marks aren't supported by cvs2bzr so no need to set that option - if custom: - args.extend(custom) - args.append(source) - retcode = self.execute(args, outf) - self.report_results(retcode, destination) - - -class DarcsExporter(_Exporter): - - def __init__(self): - self.check_install('Darcs', '2.2', [('darcs', '--version')]) - - def generate(self, source, destination, verbose=False, custom=None): - """Generate a fast import stream. See _Exporter.generate() for details.""" - args = ["darcs/darcs-fast-export"] - outf, base, marks = self.get_output_info(destination) - if marks: - args.append('--export-marks=%s' % marks) - if custom: - args.extend(custom) - args.append(source) - retcode = self.execute_exporter_script(args, outf) - self.report_results(retcode, destination) - - -class MercurialExporter(_Exporter): - - def __init__(self): - self.check_install('Mercurial', '1.2', None, ['mercurial']) - - def generate(self, source, destination, verbose=False, custom=None): - """Generate a fast import stream. See _Exporter.generate() for details.""" - # XXX: Should we add --force here? - args = ["hg-fast-export.py", "-r", source, "-s"] - outf, base, marks = self.get_output_info(destination) - if base: - args.append('--marks=%s.marks' % (base,)) - args.append('--mapping=%s.mapping' % (base,)) - args.append('--heads=%s.heads' % (base,)) - args.append('--status=%s.status' % (base,)) - if custom: - args.extend(custom) - retcode = self.execute_exporter_script(args, outf) - self.report_results(retcode, destination) - - -class GitExporter(_Exporter): - - def __init__(self): - self.cmd_name = "git" - if sys.platform == 'win32': - self.cmd_name = "git.cmd" - self.check_install('Git', '1.6', [self.cmd_name]) - - def generate(self, source, destination, verbose=False, custom=None): - """Generate a fast import stream. See _Exporter.generate() for details.""" - args = [self.cmd_name, "fast-export", "--all", "--signed-tags=warn"] - outf, base, marks = self.get_output_info(destination) - if marks: - marks = os.path.abspath(marks) - # Note: we don't pass import-marks because that creates - # a stream of incremental changes, not the full thing. - # We may support incremental output later ... - #if os.path.exists(marks): - # args.append('--import-marks=%s' % marks) - args.append('--export-marks=%s' % marks) - if custom: - args.extend(custom) - retcode = self.execute(args, outf, cwd=source) - self.report_results(retcode, destination) - - -class MonotoneExporter(_Exporter): - - def __init__(self): - self.check_install('Monotone', '0.43', ['mtn']) - - def generate(self, source, destination, verbose=False, custom=None): - """Generate a fast import stream. See _Exporter.generate() for details.""" - args = ["mtn", "git_export"] - outf, base, marks = self.get_output_info(destination) - if marks: - marks = os.path.abspath(marks) - if os.path.exists(marks): - args.append('--import-marks=%s' % marks) - args.append('--export-marks=%s' % marks) - if custom: - args.extend(custom) - retcode = self.execute(args, outf, cwd=source) - self.report_results(retcode, destination) - - -class PerforceExporter(_Exporter): - - def __init__(self): - self.check_install('p4', '2009.1', ['p4']) - self.check_install('Perforce Python API', '2009.1', None, ['P4']) - self.check_install('bzrp4', '', None, ['bzrlib.plugins.bzrp4']) - - def generate(self, source, destination, verbose=False, custom=None): - """Generate a fast import stream. See _Exporter.generate() for details.""" - from bzrlib.plugins.bzrp4 import p4_fast_export - outf, base, marks = self.get_output_info(destination) - # Marks aren't supported by p4_fast_export so no need to set that - # option - original_stdout = sys.stdout - sys.stdout = outf - try: - retcode = p4_fast_export.main([source]) - finally: - sys.stdout = original_stdout - self.report_results(retcode, destination) - - -class SubversionExporter(_Exporter): - - def __init__(self): - self.check_install('Python Subversion', '1.4', None, - ['svn.fs', 'svn.core', 'svn.repos']) - - def generate(self, source, destination, verbose=False, custom=None): - """Generate a fast import stream. See _Exporter.generate() for details.""" - args = ["svn-fast-export.py"] - outf, base, marks = self.get_output_info(destination) - # Marks aren't supported by svn-fast-export so no need to set that option - if custom: - args.extend(custom) - args.append(source) - retcode = self.execute_exporter_script(args, outf) - self.report_results(retcode, destination) - - -def fast_export_from(source, destination, tool, verbose=False, custom=None): - # Get the exporter - if tool == 'cvs': - factory = CvsExporter - elif tool == 'darcs': - factory = DarcsExporter - elif tool == 'hg': - factory = MercurialExporter - elif tool == 'git': - factory = GitExporter - elif tool == 'mtn': - factory = MonotoneExporter - elif tool == 'p4': - factory = PerforceExporter - elif tool == 'svn': - factory = SubversionExporter - try: - exporter = factory() - except MissingDependency, ex: - raise errors.BzrError(ex.get_message()) - - # Do the export - exporter.generate(source, destination, verbose=verbose, - custom=custom) diff --git a/exporters/bzr-fast-export.LICENSE b/exporters/bzr-fast-export.LICENSE deleted file mode 100644 index 26e5cc9..0000000 --- a/exporters/bzr-fast-export.LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -bzr-fast-export is Copyright (c) 2008 Adeodato Simó, and licensed under -the terms of the MIT license: - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - "Software"), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY - CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, - TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE - SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/exporters/bzr-fast-export.README b/exporters/bzr-fast-export.README deleted file mode 100644 index 93b0b38..0000000 --- a/exporters/bzr-fast-export.README +++ /dev/null @@ -1,7 +0,0 @@ -bzr-fast-export is no longer provided as a separate script. -Instead a fast-export command is now available. For help, run ... - - bzr help fast-export - -Ian C. -18-Feb-2009 diff --git a/exporters/darcs/.gitignore b/exporters/darcs/.gitignore deleted file mode 100644 index d26377c..0000000 --- a/exporters/darcs/.gitignore +++ /dev/null @@ -1,3 +0,0 @@ -Changelog -HEADER.html -.htaccess diff --git a/exporters/darcs/Makefile b/exporters/darcs/Makefile deleted file mode 100644 index 0c81c68..0000000 --- a/exporters/darcs/Makefile +++ /dev/null @@ -1,55 +0,0 @@ -VERSION = 0.9 -DATE := $(shell date +%Y-%m-%d) - -INSTALL = /usr/bin/install -c -DESTDIR = -prefix = /usr -bindir = $(prefix)/bin -mandir = $(prefix)/share/man/man1 - -MAN_TXT = $(wildcard *.txt) -MAN_HTML=$(patsubst %.txt,%.html,$(MAN_TXT)) -MAN=$(patsubst %.txt,%.1,$(MAN_TXT)) - -PROGRAMS = darcs-fast-export darcs-fast-import d2x x2d git-darcs - -all: man - -install: all - $(INSTALL) -d $(DESTDIR)$(bindir) - $(INSTALL) -d $(DESTDIR)$(mandir) - $(INSTALL) -m755 $(PROGRAMS) $(DESTDIR)$(bindir) - $(INSTALL) -m644 *.1 $(DESTDIR)$(mandir) - -doc: HEADER.html Changelog html - -HEADER.html: README Makefile - asciidoc -a toc -a numbered -a sectids -o HEADER.html README - -Changelog: .git/refs/heads/master - git log >Changelog - -%.html: %.txt - asciidoc $^ - -%.1: %.txt asciidoc.conf - a2x --asciidoc-opts="-f asciidoc.conf" \ - -a dfe_version=$(VERSION) -a dfe_date=$(DATE) -f manpage $< - -man: $(MAN) - -html: $(MAN_HTML) - -dist: - git archive --format=tar --prefix=darcs-fast-export-$(VERSION)/ $(VERSION) > darcs-fast-export-$(VERSION).tar - mkdir -p darcs-fast-export-$(VERSION) - git log > darcs-fast-export-$(VERSION)/Changelog - tar rf darcs-fast-export-$(VERSION).tar darcs-fast-export-$(VERSION)/Changelog - rm -rf darcs-fast-export-$(VERSION) - gzip -f -9 darcs-fast-export-$(VERSION).tar - -release: - git tag -l |grep -q $(VERSION) || dg tag $(VERSION) - $(MAKE) dist - gpg --comment "See http://vmiklos.hu/gpg/ for info" \ - -ba darcs-fast-export-$(VERSION).tar.gz diff --git a/exporters/darcs/NEWS b/exporters/darcs/NEWS deleted file mode 100644 index 1a0daa5..0000000 --- a/exporters/darcs/NEWS +++ /dev/null @@ -1,26 +0,0 @@ -VERSION DESCRIPTION ------------------------------------------------------------------------------ -0.9 - fix handling of accents in tag names - - warning fixes for Python-2.6 - - git-darcs: the add subcommand can now remember d-f-e - options - - git-darcs: new list, find-darcs and find-git subcommands -0.8 - revert the "not exporting unchanged files multiple - times" optimization, it causes corrupted results in some - cases. -0.7 - new darcs-fast-export option: --progress - - massive speedup in darcs-fast-export due to not - exporting unchanged files multiple times and reading - patches directly -0.6 - add a new darcs-fast-import script, allowing two-way sync - - new darcs-fast-export option: --git-branch - - add a new git-darcs script, making two-way sync easy -0.5 - new --help, --encoding, --authors-file, --working and - --logfile options - - add "hashed" support (see darcs init -h) - - add incremental conversion support for darcs1 as well - - add d2x wrapper script -0.4 - add incremental conversion support -0.3 - add darcs2 support -0.2 - add bzr and hg support -0.1 - initial short and fast version, supporting darcs1->git diff --git a/exporters/darcs/README b/exporters/darcs/README deleted file mode 100644 index 3fc9449..0000000 --- a/exporters/darcs/README +++ /dev/null @@ -1,187 +0,0 @@ -= darcs backend for fast data importers -Miklos Vajna <vmiklos-at-frugalware-dot-org> - -== Purpose and Features - -darcs-fast-export is a tool to dump a http://darcs.net/[darcs] -repository in a format understood by "fast-importers" such as -http://git.or.cz/[git] -http://www.kernel.org/pub/software/scm/git/docs/git-fast-import.html[fast-import]. -It exhibits the following _features:_ - -Fast:: - darcs-fast-export provides a fast darcs backend for fast-import. - See link:t/bench-results/[here] for exact details. - -Correct:: - darcs-fast-export produces correct results in any extreme cases. - It has been tested with a collection of large darcs repos (called - http://code.haskell.org/darcs/big-zoo/[big-zoo]). And several testcases - under the `t/` directory. - -Independent:: - Ideally it should work with any fast importer, but actually it has been - tested with git fast-import, bzr fast-import and hg fastimport. (These - are the three fast-import implementations available ATM.) - -Formats:: - It supports the 'darcs-2', 'hashed', and 'old-fashioned-inventory' darcs - repository formats. - -Incremental conversions:: - It supports the usual `--export-marks` / `--import-marks` switches to - allow incremental conversion. - -Wrapper scripts:: - A wrapper script called `d2x` is available if you find typing - `--export-marks` / `--import-marks` all the time boring. A similar one - is also provided for the other direction, called `x2d`. Finally, if you - want to work on darcs repos with git, you can use the `git-darcs` - wrapper. - -Author mappings:: - Supports `--authors-file` option like Git's SVN adaptor, for DARCS - repositories that originated in CVS or SVN. - -Import script:: - The pair of `darcs-fast-export`, `darcs-fast-import` is also - included in this repo. It has been tested with the fast-expoters of Git, - Hg, Bzr and - of course - Darcs itself. - -Two-way sync:: - Using `darcs-fast-export` / `darcs-fast-import`, it is possible to - convert a darcs repo to an other VCS, work there, then convert your work - back to Darcs (or vica versa). This has been tested with "darcs -> git; - hack hack; git -> darcs". - -== Usage - -See the manpages: - -* link:darcs-fast-export.html[darcs-fast-export] -* link:darcs-fast-import.html[darcs-fast-import] -* link:d2x.html[d2x] -* link:x2d.html[x2d] -* link:git-darcs.html[git-darcs] - -=== Example - -Assuming that `test/` is a darcs repo, you could do this: ----- -$ mkdir test.git -$ cd test.git -$ git --bare init -$ cd .. -$ darcs-fast-export test |(cd test.git; git fast-import) ----- - -For more examples (especially for bzr and hg), see the `t/` directory. - -== Download - -Using git: ----- -$ git clone git://vmiklos.hu/darcs-fast-export ----- - -== Status - -In general, darcs-fast-export should work fine. darcs-fast-import has -known problems with tags - other than that it should be okay. git-darcs -should work properly as long as you are not paying too much attention to -the imported tags (newly created tags won't be pushed back). - -darcs-fast-export has been tested with the following versions: - -Darcs version (see http://bugs.darcs.net/issue844[this bug] on why do -you need such a new version): ----- -$ darcs --version -2.2.0 (release) ----- - -Git version: ----- -$ git --version -git version 1.6.0.2 ----- - -Bzr versions: ----- -$ bzr version -Bazaar (bzr) 1.12 -$ (cd ~/bzr/fastimport; bzr log --limit 1|grep revno) -revno: 181 ----- - -Yes, you need the fastiport plugin from BZR, the last hg release series -supported by fastimport-0.6 is hg-1.0.x. - -Mercurial (Hg) version: ----- -$ hg version -Mercurial Distributed SCM (version 1.3) ----- - -Strictly speaking this document is a wrong place to talk about -configuring hg fastimport. However... you will need something like: - ----- -$ hg clone http://vc.gerg.ca/hg/pyfastimport -$ hg clone http://vc.gerg.ca/hg/hg-fastimport -$ sudo ln -s /path/to/pyfastimport/fastimport /usr/lib/python2.6/site-packages/fastimport -$ sudo ln -s /path/to/hg-fastimport/hgfastimport /usr/lib/python2.6/site-packages/hgfastimport -echo -e "[extensions]\nfastimport = /usr/lib/python2.6/site-packages/hgfastimport" > ~/.hgrc ----- - -and once you installed the plugin correctly, you should have something like: - ----- -$ ls /usr/lib/python*/site-packages/hgext/fastimport/__init__.py -/usr/lib/python2.5/site-packages/hgext/fastimport/__init__.py ----- - -== Additional resources - -You can reach the Changelog link:Changelog[here], and a gitweb interface -http://vmiklos.hu/gitweb/?p=darcs-fast-export.git[here]. - -The fast-import stream format documentation is -http://git.kernel.org/?p=git/git.git;a=blob;f=fast-import.c;hb=HEAD[here] -if you're interested. - -== Alternatives - -- http://repo.or.cz/w/darcs2git.git[darcs2git] tries to find conflict - resolutions (to map them to merge commits), but it's rather slow - because of this. It does not support the darcs2 format and/or - incremental conversions, either. darcs-fast-export may support mapping - to merge commits later, but not before - http://bugs.darcs.net/issue1261[this issue] is addressed. - -- http://progetti.arstecnica.it/tailor[tailor] is an any2any VCS - converter, but it produces corrupted results when converting the - big-zoo - see http://progetti.arstecnica.it/tailor/ticket/171[this - ticket]. - -- http://git.sanityinc.com/?p=darcs-to-git.git[darcs-to-git] is similar - to darcs2git, but it fails for the testcases found in the testsuite of - darcs-fast-export. - -- http://github.com/freshtonic/undarcs/tree/master[undarcs] claims to be - fast, but its own README says it produces incorrect results. When I - tried, it did not handle the darcs2 format, binary files and incremental - support. - -== Thanks - -- Jason Dagit for helping me with darcs2 issues -- Shawn O. Pearce and Johannes Schindelin for writing `git-fast-import` - / `git-fast-export` -- Ian Clatworthy for writing bzr fast-import -- Paul Crowley for writing hg fast-import -- Matthias Andree for assorted improvements, among them the --help, - --encoding and --authors-file features (using Python's optparse), support - for hashed repositories, `_darcs/format` interpretation, and mangling - whitespace in tags to cope with repos imported into DARCS from CVS. -- Pieter de Bie for writing git-bzr, which was the base of git-darcs diff --git a/exporters/darcs/TODO b/exporters/darcs/TODO deleted file mode 100644 index c6892c8..0000000 --- a/exporters/darcs/TODO +++ /dev/null @@ -1,8 +0,0 @@ -more intelligent tests, such as detect if the hg fastimport extension is -not enabled, etc. - -parse the patches manually so we can avoid re-adding existing files manually. - -avoid darcs apply. - -import: handle evil merges (git-subtree), maybe using git log --first-parent diff --git a/exporters/darcs/asciidoc.conf b/exporters/darcs/asciidoc.conf deleted file mode 100644 index cb31717..0000000 --- a/exporters/darcs/asciidoc.conf +++ /dev/null @@ -1,21 +0,0 @@ -ifdef::doctype-manpage[] -ifdef::backend-docbook[] -[header] -template::[header-declarations] -<refentry> - <refentryinfo> - <date>{dfe_date}</date> - </refentryinfo> - <refmeta> - <refentrytitle>{mantitle}</refentrytitle> - <manvolnum>{manvolnum}</manvolnum> - <refmiscinfo class="source">darcs-fast-export</refmiscinfo> - <refmiscinfo class="version">{dfe_version}</refmiscinfo> - <refmiscinfo class="manual">darcs-fast-export manual</refmiscinfo> - </refmeta> - <refnamediv> - <refname>{manname}</refname> - <refpurpose>{manpurpose}</refpurpose> - </refnamediv> -endif::backend-docbook[] -endif::doctype-manpage[] diff --git a/exporters/darcs/d2x b/exporters/darcs/d2x deleted file mode 100755 index 959cc00..0000000 --- a/exporters/darcs/d2x +++ /dev/null @@ -1,114 +0,0 @@ -#!/bin/sh -# -# d2x - convert darcs repos to git, bzr or hg using fast-import -# -# Copyright (c) 2008 by Miklos Vajna <vmiklos@frugalware.org> -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, -# USA. -# - -usage() -{ - echo "Usage: d2x -f format darcsrepo" -} - -die() -{ - echo "$@" - usage - exit 1 -} - -check_up_to_date() -{ - upstreamnum=$(cd $origin; darcs show repo|grep 'Num Patches'|sed 's/.*: //') - if [ "$upstreamnum" = "$(eval $*)" ]; then - echo "No remote changes to pull!" - exit 0 - fi -} - -case $1 in - -h|--help) - usage - exit 0 - ;; - -f) - format="$2" - shift 2 - ;; -esac - -[ -n "$format" ] || die "Target format is not given!" - -case $format in - git|bzr|hg) - ;; - *) - die "The requested target format is not yet supported!" - ;; -esac - -origin="$1" -shift 1 - -[ -d "$origin" ] || die "Source repo does not exist!" - -# convert to abspath -cd $origin -origin=$(pwd) - -dmark="$origin.$format/darcs/dfe-marks" -fmark="$origin.$format/darcs/ffi-marks" - -mkdir -p $origin.$format/darcs -cd $origin.$format - -common_opts="--working $origin.$format/darcs/repo --logfile $origin.$format/darcs/log $origin" -if [ ! -f $dmark ]; then - case $format in - git) - git --bare init - darcs-fast-export $* --export-marks=$dmark $common_opts | \ - git fast-import --export-marks=$fmark - ;; - bzr) - bzr init-repo . - darcs-fast-export $* --export-marks=$dmark $common_opts | \ - bzr fast-import --export-marks=$fmark - - ;; - hg) - hg init - darcs-fast-export $* $origin | \ - hg fastimport - - esac -else - case $format in - git) - check_up_to_date "git rev-list HEAD |wc -l" - darcs-fast-export $* --export-marks=$dmark --import-marks=$dmark $common_opts | \ - git fast-import --export-marks=$fmark --import-marks=$fmark - ;; - bzr) - check_up_to_date "cd master; bzr revno" - darcs-fast-export $* --export-marks=$dmark --import-marks=$dmark $common_opts | \ - bzr fast-import --export-marks=$fmark --import-marks=$fmark - - ;; - hg) - die "Incremental conversion to hg is not yet supported by hg fastimport." - ;; - esac -fi diff --git a/exporters/darcs/d2x.txt b/exporters/darcs/d2x.txt deleted file mode 100644 index 41732fd..0000000 --- a/exporters/darcs/d2x.txt +++ /dev/null @@ -1,27 +0,0 @@ -= d2x(1) - -== NAME - -d2x - convert darcs repos to git, bzr or hg using fast-import - -== SYNOPSIS - -d2x -f <format> <darcsrepo> [<darcs-fast-export options>] - -== DESCRIPTION - -d2x is a wrapper script that just automates doing an initial or -continuing an incremental conversion. All it does is initializing the -target repo, starting darcs-fast-export and the relevant importer with -the proper switches and pipe the exporter's output to the importer's -standard input. - -== OPTIONS - ---help:: - Display usage. - --f <format>:: - Specify the format of the target repo. Currently supported targets are - git, bzr and hg. Incremental conversion is supported in case of git and - bzr. diff --git a/exporters/darcs/darcs-fast-export b/exporters/darcs/darcs-fast-export deleted file mode 100755 index fa850de..0000000 --- a/exporters/darcs/darcs-fast-export +++ /dev/null @@ -1,380 +0,0 @@ -#!/usr/bin/env python - -""" - - darcs-fast-export - darcs backend for fast data importers - - Copyright (c) 2008, 2009 Miklos Vajna <vmiklos@frugalware.org> - Copyright (c) 2008 Matthias Andree <matthias.andree@gmx.de> - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2, or (at your option) - any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -""" - -import xml.dom.minidom -import xml.parsers.expat -import os -import sys -import gzip -import time -import calendar -import shutil -import subprocess -import optparse -import re -import urllib -import urllib2 -import StringIO - -sys = reload(sys) -sys.setdefaultencoding("utf-8") - -class Handler: - def __init__(self): - self.hashes = [] - self.authormap = {} - self.export_marks = [] - self.import_marks = [] - - def get_patchname(self, patch): - ret = [] - s = "" - if patch.attributes['inverted'].value == 'True': - s = "UNDO: " - cs = patch.getElementsByTagName("name")[0].childNodes - if cs.length > 0: - ret.append(s + cs[0].data) - lines = patch.getElementsByTagName("comment") - if lines: - for i in lines[0].childNodes[0].data.split('\n'): - if not i.startswith("Ignore-this: "): - ret.append(i) - return "\n".join(ret).encode('utf-8') - - def get_author(self, patch): - """darcs allows any freeform string, but fast-import has a more - strict format, so fix up broken author names here.""" - - author = patch.attributes['author'].value - if author in self.authormap: - author = self.authormap[author] - if not len(author): - author = "darcs-fast-export <darcs-fast-export>" - # add missing name - elif not ">" in author: - author = "%s <%s>" % (author.split('@')[0], author) - # avoid double quoting - elif author[0] == '"' and author[-1] == '"': - author = author[1:-1] - # name after email - elif author[-1] != '>': - author = author[author.index('>')+2:] + ' ' + author[:author.index('>')+1] - return author.encode('utf-8') - - def get_date(self, patch): - try: - date = time.strptime(patch, "%Y%m%d%H%M%S") - except ValueError: - date = time.strptime(patch[:19] + patch[-5:], '%a %b %d %H:%M:%S %Y') - return calendar.timegm(date) - - def progress(self, s): - print "progress [%s] %s" % (time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()), s) - sys.stdout.flush() - - def log(self, s): - self.logsock.write("[%s] %s" % (time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()), s)) - self.logsock.flush() - - def parse_inventory(self, sock=None): - prev = None - nextprev = False - buf = [] - if not sock: - sock = self.open(os.path.join(self.origin, "_darcs", "hashed_inventory")) - for i in sock.readlines(): - if i.startswith("hash"): - buf.insert(0, i[6:-1]) - if i.startswith("Starting with inventory:"): - nextprev = True - elif nextprev: - prev = i[:-1] - nextprev = False - sock.close() - for i in buf: - self.hashes.insert(0, i) - if prev: - sock = self.gzip_open(os.path.join(self.origin, "_darcs", "inventories", prev)) - self.parse_inventory(sock) - - # this is like gzip.open but supports urls as well - def gzip_open(self, path): - if os.path.exists(path): - return gzip.open(path) - buf = urllib.urlopen(path).read() - sock = StringIO.StringIO(buf) - return gzip.GzipFile(fileobj=sock) - - # this is like os.path.exists but supports urls as well - def path_exists(self, path): - if os.path.exists(path): - return True - else: - try: - urllib2.urlopen(urllib2.Request(path)) - return True - except urllib2.HTTPError, e: - return False - - # this is like open, but supports urls as well - def open(self, path): - if os.path.exists(path): - return open(path) - else: - return urllib.urlopen(path) - - def handle_opts(self): - # Option Parser - usage="%prog [options] darcsrepo" - opp = optparse.OptionParser(usage=usage) - opp.add_option("--import-marks", metavar="IFILE", - help="read state for incremental imports from IFILE") - opp.add_option("--export-marks", metavar="OFILE", - help="write state for incremental imports from OFILE") - opp.add_option("--encoding", - help="encoding of log [default: %default], if unspecified and input isn't utf-8, guess") - opp.add_option("--authors-file", metavar="F", - help="read author transformations in old=new format from F") - opp.add_option("--working", metavar="W", - help="working directory which is removed at the end of non-incremental conversions") - opp.add_option("--logfile", metavar="L", - help="log file which contains the output of external programs invoked during the conversion") - opp.add_option("--git-branch", metavar="B", - help="git branch [default: refs/heads/master]") - opp.add_option("--progress", metavar="P", - help="insert progress statements after every n commit [default: 100]") - (self.options, self.args) = opp.parse_args() - if len(self.args) < 1: - opp.error("darcsrepo required") - - # read author mapping file in gitauthors format, - # i. e. in=out (one per # line) - if self.options.authors_file: - sock = open(self.options.authors_file) - self.authormap = dict([i.strip().split('=',1) for i in sock]) - sock.close() - - if "://" not in self.args[0]: - self.origin = os.path.abspath(self.args[0]) - else: - self.origin = self.args[0].strip('/') - if self.options.working: - self.working = os.path.abspath(self.options.working) - else: - if "://" not in self.origin: - self.working = "%s.darcs" % self.origin - else: - self.working = "%s.darcs" % os.path.split(self.origin)[-1] - if self.options.logfile: - logfile = os.path.abspath(self.options.logfile) - else: - if "://" not in self.origin: - logfile = "%s.log" % self.origin - else: - logfile = "%s.log" % os.path.split(self.origin)[-1] - self.logsock = open(logfile, "a") - if self.options.git_branch: - self.git_branch = self.options.git_branch - else: - self.git_branch = "refs/heads/master" - - if self.options.progress: - self.prognum = int(self.options.progress) - else: - self.prognum = 100 - - def handle_import_marks(self): - if self.options.import_marks: - sock = open(self.options.import_marks) - for i in sock.readlines(): - line = i.strip() - if not len(line): - continue - self.import_marks.append(line.split(' ')[1]) - self.export_marks.append(line) - sock.close() - - def get_patches(self): - self.progress("getting list of patches") - if not len(self.import_marks): - sock = os.popen("darcs changes --xml --reverse --repo %s" % self.origin) - else: - sock = os.popen("darcs changes --xml --reverse --repo %s --from-match 'hash %s'" % (self.origin, self.import_marks[-1])) - buf = sock.read() - sock.close() - # this is hackish. we need to escape some bad chars, otherwise the xml - # will not be valid - buf = buf.replace('\x1b', '^[') - if self.options.encoding: - xmldoc = xml.dom.minidom.parseString(unicode(buf, self.options.encoding).encode('utf-8')) - else: - try: - xmldoc = xml.dom.minidom.parseString(buf) - except xml.parsers.expat.ExpatError: - try: - import chardet - except ImportError: - sys.exit("Error, encoding is not utf-8. Please " + - "either specify it with the --encoding " + - "option or install chardet.") - self.progress("encoding is not utf8, guessing charset") - encoding = chardet.detect(buf)['encoding'] - self.progress("detected encoding is %s" % encoding) - xmldoc = xml.dom.minidom.parseString(unicode(buf, encoding).encode('utf-8')) - sys.stdout.flush() - return xmldoc.getElementsByTagName('patch') - - def setup_workdir(self): - darcs2 = False - self.oldfashionedpatch = True - self.cwd = os.getcwd() - if self.path_exists(os.path.join(self.origin, "_darcs", "format")): - sock = self.open(os.path.join(self.origin, "_darcs", "format")) - format = [x.strip() for x in sock] - sock.close() - darcs2 = 'darcs-2' in format - self.oldfashionedpatch = not 'hashed' in format - if not self.oldfashionedpatch: - self.progress("parsing the inventory") - if "://" not in self.origin: - os.chdir(self.origin) - self.parse_inventory() - if not self.options.import_marks or not os.path.exists(self.working): - # init the tmp darcs repo - os.mkdir(self.working) - os.chdir(self.working) - if darcs2: - os.system("darcs init --darcs-2") - else: - os.system("darcs init --old-fashioned-inventory") - else: - os.chdir(self.working) - if self.options.import_marks: - sock = os.popen("darcs pull -a --match 'hash %s' %s" % (self.import_marks[-1], self.origin)) - self.log("Building/updating working directory:\n%s" % sock.read()) - sock.close() - - def export_patches(self): - patches = self.get_patches() - # this is the number of the NEXT patch - count = 1 - if len(self.import_marks): - patches = patches[1:] - count = len(self.import_marks) + 1 - if len(self.export_marks): - # this is the mark number of the NEXT patch - markcount = int(self.export_marks[-1].split(' ')[0][1:]) + 1 - else: - markcount = count - # this may be huge and we need it many times - patchnum = len(patches) - - if not len(self.import_marks): - self.progress("starting export, repo has %d patches" % patchnum) - else: - self.progress("continuing export, %d patches to convert" % patchnum) - paths = [] - for i in patches: - # apply the patch - hash = i.attributes['hash'].value - buf = ["\nNew patches:\n"] - if self.oldfashionedpatch: - sock = self.gzip_open(os.path.join(self.origin, "_darcs", "patches", hash)) - else: - sock = self.gzip_open(os.path.join(self.origin, "_darcs", "patches", self.hashes[count-1])) - buf.append(sock.read()) - sock.close() - sock = os.popen("darcs changes --context") - buf.append(sock.read()) - sock.close() - sock = subprocess.Popen(["darcs", "apply", "--allow-conflicts"], stdin=subprocess.PIPE, stdout=subprocess.PIPE) - sock.stdin.write("".join(buf)) - sock.stdin.close() - self.log("Applying %s:\n%s" % (hash, sock.stdout.read())) - sock.stdout.close() - message = self.get_patchname(i) - # export the commit - print "commit %s" % self.git_branch - print "mark :%s" % markcount - if self.options.export_marks: - self.export_marks.append(":%s %s" % (markcount, hash)) - date = self.get_date(i.attributes['date'].value) - print "committer %s %s +0000" % (self.get_author(i), date) - print "data %d\n%s" % (len(message), message) - if markcount > 1: - print "from :%s" % (markcount-1) - # export the files - for j in paths: - print "D %s" % j - paths = [] - for (root, dirs, files) in os.walk ("."): - for f in files: - j = os.path.normpath(os.path.join(root, f)) - if j.startswith("_darcs") or "-darcs-backup" in j: - continue - paths.append(j) - sock = open(j) - buf = sock.read() - sock.close() - # darcs does not track the executable bit :/ - print "M 644 inline %s" % j - print "data %s\n%s" % (len(buf), buf) - if message[:4] == "TAG ": - tag = re.sub('[^\xe9-\xf8\w.\-]+', '_', message[4:].strip().split('\n')[0]).strip('_') - print "tag %s" % tag - print "from :%s" % markcount - print "tagger %s %s +0000" % (self.get_author(i), date) - print "data %d\n%s" % (len(message), message) - if count % self.prognum == 0: - self.progress("%d/%d patches" % (count, patchnum)) - count += 1 - markcount += 1 - - os.chdir(self.cwd) - - if not self.options.export_marks: - shutil.rmtree(self.working) - self.logsock.close() - - def handle_export_marks(self): - if self.options.export_marks: - self.progress("writing export marks") - sock = open(self.options.export_marks, 'w') - sock.write("\n".join(self.export_marks)) - sock.write("\n") - sock.close() - - self.progress("finished") - - def handle(self): - self.handle_opts() - self.handle_import_marks() - self.setup_workdir() - self.export_patches() - self.handle_export_marks() - -if __name__ == "__main__": - h = Handler() - h.handle() diff --git a/exporters/darcs/darcs-fast-export.txt b/exporters/darcs/darcs-fast-export.txt deleted file mode 100644 index d404ecf..0000000 --- a/exporters/darcs/darcs-fast-export.txt +++ /dev/null @@ -1,68 +0,0 @@ -= darcs-fast-export(1) - -== NAME - -darcs-fast-export - darcs frontend to git fast-import - -== SYNOPSIS - -darcs-fast-export [<options>] <darcsrepo> - -== DESCRIPTION - -darcs-fast-export expects one argument, the path to the source darcs -repository. It will print the git fast-import format on standard output -(stdout). - -The script can produce the fast-import stream format from the darcs -repository. It supports incremental conversion as well, via the ---import-marks / --export-marks switches. - -Optionally the darcsrepo string may be a HTTP repository, in that case -only the patches are downloaded, not the pristine, speeding up a -one-time import. - -== OPTIONS - --h, --help:: - Display usage. - ---import-marks=<file>:: - Import marks from <file>. This is read at the beginning of the - conversion at once. Use it if you want to continue an incremental - conversion. - ---export-marks=<file>:: - Export marks to <file> at the end of the conversion. It can be the - same as the one for --import-marks as it is written only once at the - end. Use it if you want to be able to incrementally update the target - repository later. - ---encoding=<encoding>:: - The encoding of the author names and commit messages in the repository. - The default is utf-8. If it is not the default, it will be guessed. - Given that it takes some time, you can explicitly specify it as an - option to make the conversion faster. Content in the output will encoded - as utf-8 and will be written that way to the target repository, unless - the importer re-encodes it again to some other character set. - ---working=<directory>:: - The conversion is done by applying the patches one by one and recording - the state of the working directory. You can specify the path of this - directory using this option. - ---logfile=<logfile>:: - The output of external commands are redirected to a log file. You can - specify the path of that file with this parameter. - ---git-branch=<branch>:: - There is only one branch in one darcs repository, but the fast-import - stream format allows multiple branches, thus the exporter has to name - darcs's branch. The default value is 'refs/heads/master'. - ---progress=<n>:: - Insert progress statements after every <n> patches, to be shown by the - fast importer during import. The default value is '100'. - ---authors-file=<file>:: - Read author transformations in old=new format from <file>. diff --git a/exporters/darcs/darcs-fast-import b/exporters/darcs/darcs-fast-import deleted file mode 100755 index c50ab90..0000000 --- a/exporters/darcs/darcs-fast-import +++ /dev/null @@ -1,366 +0,0 @@ -#!/usr/bin/env python - -""" - - darcs-fast-export - darcs backend for fast data exporters - - Copyright (c) 2008, 2009, 2010 Miklos Vajna <vmiklos@frugalware.org> - Copyright (c) 2008 Matthias Andree <matthias.andree@gmx.de> - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2, or (at your option) - any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -""" - -import sys -import os -import re -import time -import shutil -import optparse -import subprocess - -class Handler: - def __init__(self): - self.marks = {} - self.files = [] - self.prevfiles = None - self.ch = None - self.line = None - self.unread_line = False - self.eof = False - self.debug = False - self.export_marks = [] - self.import_marks = [] - - def read_next_line(self): - if self.unread_line: - self.unread_line = False - return - self.line = "" - if self.eof: - return - if self.ch: - self.line += self.ch - self.ch = None - buf = sys.stdin.readline() - if not len(buf): - self.eof = True - else: - self.line += buf - if self.debug: - print "read_next_line: '%s'" % self.line - - def read(self, length): - buf = "" - if self.ch: - buf += self.ch - self.ch = None - buf += sys.stdin.read(length) - if self.debug: - print "read: '%s'" % buf - return buf - - def skip_optional_lf(self): - self.ch = self.read(1) - if self.ch == "\n": - self.ch = None - - def bug(self, s): - raise Exception(s) - - def get_date(self, ts, tz): - # first fix the case when tz is higher than +1200, as - # darcs won't accept it - if int(tz[:3]) > 12: - ts = str(int(ts) + 60*60*24) - tz = str(int(tz[:3])-24) + tz[3:] - # int(ts) is seconds since epoch. Since we're trying to - # capture both the absolute time of the commit and the - # localtime in the timezone of the committer, we need to turn - # the (seconds-since-epoch, committer-timezone-offset) pair - # that we get from the git-fast-export stream format into a - # localized-time-plus-timezone-marker string that darcs will - # accept. Therefore, we parse the timezone-offset (which - # looks like +0500 or +0000 or -0730 or something) and add it - # to seconds-since-epoch before calling gmtime(). - mo = re.search(r'^([\+\-])(\d\d)(\d\d)$', tz) - offset = 60*60*int(mo.group(2)) + 60*int(mo.group(3)) - if mo.group(1) == "-": - offset = -offset - offset_time = int(ts) + offset - s = time.strftime("%a %b %d %H:%M:%S %Y", time.gmtime(offset_time)) - items = s.split(' ') - return " ".join(items[:-1]) + " " + tz + " " + items[-1] - - def invoke_darcs(self, cmdline): - if os.system("darcs %s" % cmdline) != 0: - self.bug("darcs failed") - - def invoke_add(self, path): - self.invoke_darcs("add --boring --case-ok %s" % path) - - def handle_mark(self): - if self.line.startswith("mark :"): - self.mark_num = int(self.line[6:-1]) - self.read_next_line() - - def handle_data(self): - if not self.line.startswith("data "): - self.bug("Expected 'data n' command, found: '%s'" % self.line[:-1]) - length = int(self.line[5:-1]) - self.buf = self.read(length) - self.skip_optional_lf() - - def handle_blob(self): - self.read_next_line() - self.handle_mark() - self.handle_data() - self.marks[self.mark_num] = self.buf - - def handle_ident(self, s): - items = s.split(' ') - self.ident = " ".join(items[:-2]) - self.date = self.get_date(items[-2], items[-1]) - - def handle_msg(self): - items = self.buf.split('\n') - self.short = items[0] - self.long = "\n".join(items[1:]) - - def handle_tag(self): - version = self.line[:-1].split(' ')[1] - self.read_next_line() - if self.line.startswith("from "): - self.read_next_line() - if self.line.startswith("tagger "): - self.handle_ident(self.line[7:-1]) - self.read_next_line() - self.handle_data() - self.skip_optional_lf() - sock = subprocess.Popen(["darcs", "tag", "--pipe"], stdin=subprocess.PIPE, stdout=subprocess.PIPE) - buf = [self.date, self.ident, version] - sock.stdin.write("\n".join(buf)) - sock.stdin.close() - self.log("Tagging %s:\n%s" % (version, sock.stdout.read())) - sock.stdout.close() - - def handle_commit(self): - if not self.prevfiles and self.options.import_marks: - # first commit in an incremental continued - # import - for (root, dirs, files) in os.walk("."): - for i in files: - path = os.path.normpath(os.path.join(root, i)) - if path.startswith("_darcs") or "-darcs-backup" in path: - continue - self.files.append(path) - self.prevfiles = self.files[:] - adds = [] - symlinks = [] - - self.read_next_line() - self.handle_mark() - if self.line.startswith("author "): - self.handle_ident(self.line[7:-1]) - self.read_next_line() - if self.line.startswith("committer "): - self.handle_ident(self.line[10:-1]) - self.read_next_line() - self.handle_data() - self.skip_optional_lf() - self.handle_msg() - self.read_next_line() - if self.line.startswith("from "): - self.read_next_line() - while self.line.startswith("merge "): - self.read_next_line() - change = False - while len(self.line) > 0: - if self.line.startswith("deleteall"): - path = self.line[2:-1] - for path in self.files: - os.unlink(path) - self.files = [] - change = True - elif self.line.startswith("D "): - path = self.line[2:-1] - if os.path.exists(path): - os.unlink(path) - if path in self.files: - self.files.remove(path) - change = True - elif self.line.startswith("R "): - self.invoke_darcs("mv %s" % self.line[2:]) - change = True - elif self.line.startswith("C "): - src, dest = self.line[:-1].split(' ')[1:] - shutil.copy(src.strip('"'), dest.strip('"')) - self.invoke_add(dest) - change = True - elif self.line.startswith("M "): - items = self.line.split(' ') - path = items[3][:-1] - dir = os.path.split(path)[0] - if len(dir) and not os.path.exists(dir): - os.makedirs(dir) - if items[1] == "120000": - if not self.options.symhack: - print "Adding symbolic links (symlinks) is not supported by Darcs." - sys.exit(2) - idx = int(items[2][1:]) # TODO: handle inline symlinks - symlinks.append((self.marks[idx], path)) - self.read_next_line() - continue - sock = open(path, "w") - if items[2] != "inline": - idx = int(items[2][1:]) - sock.write(self.marks[idx]) - else: - self.read_next_line() - self.handle_data() - sock.write(self.buf) - sock.close() - if path not in self.prevfiles: - adds.append(path) - if path not in self.files: - self.files.append(path) - change = True - else: - self.unread_line = True - break - self.read_next_line() - if not len(self.line): - break - - if not change: - # darcs does not support empty commits - return - for i in adds: - self.invoke_add(i) - sock = subprocess.Popen(["darcs", "record", "--ignore-times", "-a", "--pipe"], stdin=subprocess.PIPE, stdout=subprocess.PIPE) - buf = [self.date, self.ident, self.short, self.long] - sock.stdin.write("\n".join(buf)) - sock.stdin.close() - self.log("Recording :%s:\n%s" % (self.mark_num, sock.stdout.read())) - sock.stdout.close() - - for src, path in symlinks: - # symlink does not do what we want if path is - # already there - if os.path.exists(path): - # rmtree() does not work on symlinks - if os.path.islink(path): - os.remove(path) - else: - shutil.rmtree(path) - os.symlink(src, path) - if self.options.export_marks: - # yeah, an xml parser would be better, but - # should we mess with encodings just because of - # this? i hope not - sock = os.popen("darcs changes --last=1 --xml", "r") - buf = sock.read() - sock.close() - hash = buf.split('\n')[1].split("'")[-2] - self.export_marks.append(":%s %s" % (self.mark_num, hash)) - - def handle_progress(self, s): - print "import progress [%s] %s" % (time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()), s.strip()) - sys.stdout.flush() - - def handle_opts(self): - # Option Parser - usage="%prog [options]" - opp = optparse.OptionParser(usage=usage) - opp.set_defaults(symhack=False) - opp.add_option("--import-marks", metavar="IFILE", - help="read state for incremental imports from IFILE") - opp.add_option("--export-marks", metavar="OFILE", - help="write state for incremental imports to OFILE") - opp.add_option("--logfile", metavar="L", - help="log file which contains the output of external programs invoked during the conversion") - opp.add_option("--symhack", action="store_true", dest="symhack", - help="Do not error out when a symlink would be created, just create it in the workdir") - opp.add_option("--progress", metavar="P", - help="insert progress statements after every n commit [default: 100]") - (self.options, args) = opp.parse_args() - - if self.options.logfile: - logfile = self.options.logfile - else: - logfile = "_darcs/import.log" - self.logsock = open(os.path.abspath(logfile), "a") - - if self.options.progress: - self.prognum = int(self.options.progress) - else: - self.prognum = 0 - - def log(self, s): - self.logsock.write("[%s] %s" % (time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()), s)) - self.logsock.flush() - - def handle_export_marks(self): - if self.options.export_marks: - sock = open(self.options.export_marks, 'w') - sock.write("\n".join(self.export_marks)) - sock.write("\n") - sock.close() - - def handle_import_marks(self): - if self.options.import_marks: - sock = open(self.options.import_marks) - for i in sock.readlines(): - line = i.strip() - if not len(line): - continue - self.import_marks.append(line.split(' ')[1]) - self.export_marks.append(line) - sock.close() - - def handle(self): - self.handle_opts() - self.handle_import_marks() - - commitcount = 0 - while not self.eof: - self.read_next_line() - if not len(self.line[:-1]): - pass - elif self.line.startswith("blob"): - self.handle_blob() - elif self.line.startswith("commit"): - self.handle_commit() - commitcount += 1 - if self.prognum != 0 and commitcount % self.prognum == 0: - self.handle_progress("%d patches" % commitcount) - elif self.line.startswith("tag"): - self.handle_tag() - elif self.line.startswith("reset"): - self.read_next_line() - if not self.line.startswith("from "): - self.unread_line = True - elif self.line.startswith("checkpoint"): - pass - elif self.line.startswith("progress"): - self.handle_progress(self.line[9:]) - else: - self.bug("'%s': invalid command" % self.line[:-1]) - - self.handle_export_marks() - -if __name__ == "__main__": - h = Handler() - h.handle() diff --git a/exporters/darcs/darcs-fast-import.txt b/exporters/darcs/darcs-fast-import.txt deleted file mode 100644 index a7f2a12..0000000 --- a/exporters/darcs/darcs-fast-import.txt +++ /dev/null @@ -1,57 +0,0 @@ -= darcs-fast-import(1) - -== NAME - -darcs-fast-import - darcs backend to the 'fast-import stream' format - -== SYNOPSIS - -darcs-fast-import [<options>] - -== DESCRIPTION - -darcs-fast-import can produce a darcs repository from a fast-import -stream, read from the standard input. It supports incremental conversion -as well, via the --import-marks / --export-marks switches. - -== OPTIONS - --h, --help:: - Display usage. - ---import-marks:: - Import marks from a given file. This is read at the beginning of the - conversion at once. Use it if you want to continue an incremental - conversion. - ---export-marks:: - Export marks to a given file at the end of the conversion. It can be the - same as the one for --import-marks as it is written only once at the - end. Use it if you want to be able to incrementally update the target - repository later. - ---logfile:: - The output of external commands are redirected to a log file. You can - specify the path of that file with this parameter. - ---symhack:: - Enable hack for symbolic links. darcs add does not handle them - but in case they are just added, we can create them in the working - directory. This can be handy in case for example the symbolic link is in - a subdirectory of the project and you don't even care about that - subdirectory. So the hack can be useful, but be extremely careful when - you use it. - ---progress=<n>:: - Insert progress statements after every <n> created patches. The - default is not to print anything as progress info is usually provided by - the exporter. Use this option in case the exporter does not have such a - switch but you still want to get some feedback. - -== EXIT CODES - -The exit code is: - -* 0 on success -* 1 on unhandled exception -* 2 in case the stream would try to let the importer create a symlink diff --git a/exporters/darcs/git-darcs b/exporters/darcs/git-darcs deleted file mode 100755 index 18455a2..0000000 --- a/exporters/darcs/git-darcs +++ /dev/null @@ -1,281 +0,0 @@ -#!/bin/bash -# -# git-darcs - bidirectional operation between a darcs repo and git -# -# Copyright (c) 2008, 2010 by Miklos Vajna <vmiklos@frugalware.org> -# -# Based on git-bzr, which is -# -# Copyright (c) 2008 Pieter de Bie <pdebie@ai.rug.nl> -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, -# USA. -# - -add() -{ - name="$1" - shift - location="$1" - shift - if ! [ -n "$name" -a -n "$location" ]; then - echo "Usage: git darcs add name location [darcs-fast-export options]" - return 1 - fi - if git remote show |grep -q $name; then - echo "There is already a remote with that name" - return 1 - fi - if [ -n "$(git config git-darcs.$name.location)" ]; then - echo "There is already a darcs repo with that name" - return 1 - fi - repo=$location/_darcs - if [ ! -d $repo ] && ! wget --quiet --spider $repo; then - echo "Remote is not a darcs repository" - return 1 - fi - git config git-darcs.$name.location $location - echo "Darcs repo $name added. You can fetch it with 'git darcs fetch $name'" - if ! [ -z "$*" ]; then - git config git-darcs.$name.darcs-fast-export-options "$*" - echo "darcs-fast-export will get options: $*" - fi -} - -get_location() -{ - l=$(git config git-darcs.$remote.location) - if [ -z "$l" ]; then - echo "Cannot find darcs remote with name '$remote'." >&2 - return 1 - fi - echo $l -} - -fetch() -{ - remote="$1" - shift - if ! [ -n "$remote" -a -z "$*" ]; then - echo "Usage: git darcs fetch reponame" - return 1 - fi - location=$(get_location $remote) || return $? - git_map=$git_dir/darcs-git/$remote-git-map - darcs_map=$git_dir/darcs-git/$remote-darcs-map - common_opts="--working $git_dir/darcs-git/repo --logfile $git_dir/darcs-git/fetch.log --git-branch=refs/remotes/darcs/$remote" - dfe_opts=$(git config git-darcs.$remote.darcs-fast-export-options) - pre_fetch="$(git config git-darcs.$remote.pre-fetch)" - if [ -n "$pre_fetch" ]; then - $pre_fetch - fi - if [ ! -f $git_map -a ! -f $darcs_map ]; then - echo "There doesn't seem to be an existing refmap." - echo "Doing an initial import" - mkdir -p $git_dir/darcs-git - darcs-fast-export --export-marks=$darcs_map $common_opts $dfe_opts $location | \ - git fast-import --export-marks=$git_map - elif [ -f $git_map -a -f $darcs_map ]; then - echo "Updating remote $remote" - old_rev=$(git rev-parse refs/remotes/darcs/$remote) - darcs-fast-export --import-marks=$darcs_map --export-marks=$darcs_map $common_opts $dfe_opts $location | \ - git fast-import --quiet --import-marks=$git_map --export-marks=$git_map - new_rev=$(git rev-parse refs/remotes/darcs/$remote) - if [ "$old_rev" != "$new_rev" ]; then - echo "Fetched the following updates:" - git shortlog $old_rev..$new_rev - else - echo "Nothing fetched." - return 0 - fi - else - echo "One of the mapfiles is missing! Something went wrong!" - return 1 - fi - post_fetch="$(git config git-darcs.$remote.post-fetch)" - if [ -n "$post_fetch" ]; then - $post_fetch - fi -} - -pull() -{ - remote="$1" - shift - if ! [ -n "$remote" -a -z "$*" ]; then - echo "Usage: git darcs pull reponame" - return 1 - fi - fetch $remote || return $? - # see if we need to merge or rebase - branch=$(git symbolic-ref HEAD|sed 's|.*/||') - if [ "$(git config branch.$branch.rebase)" = "true" ]; then - git rebase refs/remotes/darcs/$remote - else - git merge refs/remotes/darcs/$remote - fi -} - -push() -{ - remote="$1" - shift - if ! [ -n "$remote" -a -z "$*" ]; then - echo "Usage: git darcs push reponame" - return 1 - fi - location=$(get_location $remote) || return $? - if [ -n "$(git rev-list --left-right HEAD...refs/remotes/darcs/$remote | sed -n '/^>/ p')" ]; then - echo "HEAD is not a strict child of $remote, cannot push. Merge first" - return 1 - fi - if [ -z "$(git rev-list --left-right HEAD...refs/remotes/darcs/$remote | sed -n '/^</ p')" ]; then - echo "Nothing to push. Commit something first" - return 1 - fi - git_map=$git_dir/darcs-git/$remote-git-map - darcs_map=$git_dir/darcs-git/$remote-darcs-map - if [ ! -f $git_map -o ! -f $darcs_map ]; then - echo "We do not have refmapping yet. Then how can I push?" - return 1 - fi - pre_push="$(git config git-darcs.$remote.pre-push)" - if [ -n "$pre_push" ]; then - $pre_push - fi - echo "Pushing the following updates:" - git shortlog refs/remotes/darcs/$remote.. - git fast-export --import-marks=$git_map --export-marks=$git_map HEAD | \ - (cd $location; darcs-fast-import --import-marks=$darcs_map --export-marks=$darcs_map \ - --logfile $git_dir/darcs-git/push.log) - if [ $? == 0 ]; then - git update-ref refs/remotes/darcs/$remote HEAD - post_push="$(git config git-darcs.$remote.post-push)" - if [ -n "$post_push" ]; then - $post_push - fi - fi -} - -# List the darcs remotes -list() -{ - if [ -z "$*" ] - then - git config -l | sed -n -e '/git-darcs\..*/ {s/git-darcs\.//; s/\.location=.*//p}' - return 0 - elif [ "$#" -eq 1 ] - then - case $1 in - -v|--verbose) - git config -l | sed -n -e '/git-darcs\..*/ {s/git-darcs\.//; s/\.location=/\t/p}' - return 0 - ;; - esac - fi - echo "Usage: git darcs list [-v|--verbose]" - return 1 -} - -# Find the darcs commit(s) supporting a git SHA1 prefix -find_darcs() -{ - sha1="$1" - shift - if [ -z "$sha1" -o -n "$*" ] - then - echo "Usage: git darcs find-darcs <sha1-prefix>" - return 1 - fi - for remote in $(git for-each-ref --format='%(refname)' refs/remotes/darcs) - do - remote=`basename $remote` - git_map=$git_dir/darcs-git/$remote-git-map - darcs_map=$git_dir/darcs-git/$remote-darcs-map - if [ ! -f $git_map -o ! -f $darcs_map ] - then - echo "Missing mappings for remote $remote" - return 1 - fi - for row in `sed -n -e "/:.* $sha1.*/ s/[^ ]*/&/p" $git_map` - do - sed -n -e "/$row / {s/[^ ]*//; s/.*/$remote\t&/p}" $darcs_map - done - done -} - -# Find the git commit(s) supporting a darcs patch prefix -find_git() -{ - patch="$1" - shift - if [ -z "$patch" -o -n "$*" ] - then - echo "Usage: git darcs find-git <patch-prefix>" - return 1 - fi - for remote in $(git for-each-ref --format='%(refname)' refs/remotes/darcs) - do - remote=`basename $remote` - git_map=$git_dir/darcs-git/$remote-git-map - darcs_map=$git_dir/darcs-git/$remote-darcs-map - if [ ! -f $git_map -o ! -f $darcs_map ] - then - echo "Missing mappings for remote $remote" - return 1 - fi - for row in `sed -n -e "/:.* $patch.*/ s/[^ ]*/&/p" $darcs_map` - do - sed -n -e "/$row / {s/[^ ]* \(.*\)/$remote\t\1/p}" $git_map - done - done -} - -git rev-parse 2> /dev/null -if [ $? != 0 ]; then - echo "Must be inside a git repository to work" - exit 1 -fi - -git_dir=$(git rev-parse --git-dir) -# make it absolute -cd $git_dir -git_dir=$(pwd) -cd - >/dev/null -command="$1" -shift - -case $command in - add|push|fetch|pull|list) - ;; - find-darcs) - command=find_darcs - ;; - find-git) - command=find_git - ;; - *) - echo "Usage: git darcs [COMMAND] [OPTIONS]" - echo "Commands: add, push, fetch, pull, list, find-darcs, find-git" - exit 1 - ;; -esac - - -up=$(git rev-parse --show-cdup) -[ -z "$up" ] && up="." -cd $up -$command "$@" diff --git a/exporters/darcs/git-darcs.txt b/exporters/darcs/git-darcs.txt deleted file mode 100644 index 8bf5b33..0000000 --- a/exporters/darcs/git-darcs.txt +++ /dev/null @@ -1,92 +0,0 @@ -= git-darcs(1) - -== NAME - -git-darcs - a bidirectional git - darcs gateway - -== SYNOPSIS - -git-darcs <command> <options> - -== DESCRIPTION - -git darcs can convert a darcs repo to a git one, can update such an -existing git repo later, and finally can push back your changes from the -git repo to the darcs one. - -A typical workflow is: - ----- -$ mkdir git-repo -$ cd git-repo -$ git init -$ git darcs add upstream /path/to/darcs-repo -$ git darcs pull upstream - -... hack, hack, hack ... - -$ git darcs push upstream ----- - -== GLOBAL OPTIONS - --h, --help:: - Display usage. - -== COMMANDS - -The supported commands are the followings: - -add:: - This can register a new darcs repo in the git one, so that you - can fetch from it. The syntax is `add nick path [dfe-options]`. - Add any options you want to be passed to darcs-fast-export, - like --encoding=utf-8, or --authors-file AUTHORMAP. Remember - that if AUTHORMAP is not absolute, it will be interpreted - relative to the git repository's root directory. - -push:: - Transfers your changes created in the current branch back the - darcs one. The syntax is `push nick`. - -fetch:: - Downloads changes from the darcs repo and updates the - `darcs/<nick>` branch. None of your local branches are updated. - -pull:: - Calls `fetch` then `git merge` or `git rebase` based on the - `branch.<branchname>.rebase` configuration setting, where `<branchname>` - is the current branch. The default is - just like with `git pull` - is - to `git merge`. - -list:: - List the name [and location] of each registered darcs repo. - The syntax is `list [-v|--verbose]`. - -find-darcs:: - Searches for darcs patches matching a SHA1 prefix. - The syntax is `find-darcs <sha1-prefix>`. - -find-git:: - Searches for git commits matching a darcs patch prefix. - The syntax is `find-git <patch-prefix>`. - -== HOOKS - -It's possible to automatically run before and after the fetch and the -push subcommand. For example if you want to automatically run `darcs -pull -a` before a `git darcs fetch upstream`: - ----- -git config git-darcs.upstream.pre-fetch "darcs pull -a --repodir=/path/to/darcs-repo" ----- - -Or in case you want to automatically `darcs send` all patches after a -`git darcs push upstream`: - ----- -git config git-darcs.upstream.post-push "darcs send -a --repodir=/path/to/darcs-repo" ----- - -== SEE-ALSO -*git*(1), *darcs*(1) diff --git a/exporters/darcs/t/Makefile b/exporters/darcs/t/Makefile deleted file mode 100644 index de8a7ab..0000000 --- a/exporters/darcs/t/Makefile +++ /dev/null @@ -1,9 +0,0 @@ -T = $(wildcard test*.sh) - -all: $(T) - @echo "passed $$(echo $(T)|wc -w) tests." - -$(T): - @echo "*** $@ ***"; sh $@ - -.PHONY: $(T) diff --git a/exporters/darcs/t/bench-results/Makefile b/exporters/darcs/t/bench-results/Makefile deleted file mode 100644 index 0157f69..0000000 --- a/exporters/darcs/t/bench-results/Makefile +++ /dev/null @@ -1,5 +0,0 @@ -bench-results.png: bench-results.gnu bench-results.dat - gnuplot bench-results.gnu - -bench-results.dat: bench-results.py $(wildcard ../darcs-benchmark/big-zoo/*.log) - python bench-results.py > bench-results.dat diff --git a/exporters/darcs/t/bench-results/bench-results.gnu b/exporters/darcs/t/bench-results/bench-results.gnu deleted file mode 100644 index f4e8917..0000000 --- a/exporters/darcs/t/bench-results/bench-results.gnu +++ /dev/null @@ -1,6 +0,0 @@ -set terminal png -set output 'bench-results.png' -unset key -set xlabel "number of patches" -set ylabel "elapsed time in hours" -plot 'bench-results.dat' with linespoints diff --git a/exporters/darcs/t/bench-results/bench-results.py b/exporters/darcs/t/bench-results/bench-results.py deleted file mode 100644 index fbb834b..0000000 --- a/exporters/darcs/t/bench-results/bench-results.py +++ /dev/null @@ -1,23 +0,0 @@ -from glob import glob -import re - -def cmp_data(a, b): - return cmp(a[0], b[0]) - -logs = glob("../darcs-benchmark/big-zoo/*.log") - -data = [] - -for i in logs: - sock = open(i) - for j in sock.readlines(): - if "Num Patches:" in j: - patches = int(j.split(": ")[1].strip()) - elif j.startswith("real"): - l = re.sub("real\t([0-9]+)m([0-9.]+)s\n", r"\1 \2", j).split(" ") - secs = int(l[0])*60 + float(l[1]) - hours = secs / 3600 - data.append([patches, hours]) -data.sort(cmp=cmp_data) -for i in data: - print "%s %s" % (i[0], i[1]) diff --git a/exporters/darcs/t/bench-tailor.sh b/exporters/darcs/t/bench-tailor.sh deleted file mode 100644 index 7567f7b..0000000 --- a/exporters/darcs/t/bench-tailor.sh +++ /dev/null @@ -1,59 +0,0 @@ -#!/bin/sh - -create_config() -{ - cd $1 - mypath=$(pwd) - cd - >/dev/null - myname=$(basename $mypath) - - cat > config << EOF -[DEFAULT] -encoding-errors-policy = replace - -[$myname] -source = darcs:$myname -target = git:$myname - -[darcs:$myname] -subdir = darcs -repository = $mypath - -[git:$myname] -subdir = git -repository = $mypath.git -EOF -} - -PATH=$HOME/darcs/tailor:$PATH -if [ ! -d darcs-benchmark ]; then - darcs get http://code.haskell.org/darcs/darcs-benchmark - cd darcs-benchmark -else - cd darcs-benchmark - darcs pull -a -fi -sh initialise.sh -cd big-zoo -if [ -n "$1" ]; then - targets=$1 -else - targets=*_play.tar.gz -fi -for i in $targets -do - echo "benchmarking $i" - rm -rf _playground - tar xf $i - cd _playground - log="../$i.tailor-$(tailor --version).log" - create_config sandbox - sh -c 'time tailor --configfile config' 2>&1 |tee $log - if diff --exclude _darcs --exclude .git -Nur sandbox git >/dev/null; then - echo "ok, the result is correct" >> $log - else - echo "ouch, the result is corrupted" >> $log - exit 1 - fi - cd .. -done diff --git a/exporters/darcs/t/bench.sh b/exporters/darcs/t/bench.sh deleted file mode 100644 index a4b3d0d..0000000 --- a/exporters/darcs/t/bench.sh +++ /dev/null @@ -1,38 +0,0 @@ -#!/bin/sh - -# this is a test as well, but it would take a lot of time, so don't -# prefix it with 'test'. - -. lib.sh - -if [ ! -d darcs-benchmark ]; then - darcs get http://code.haskell.org/darcs/darcs-benchmark - cd darcs-benchmark -else - cd darcs-benchmark - darcs pull -a -fi -sh initialise.sh -cd big-zoo -if [ -n "$1" ]; then - targets=$1 -else - targets=*_play.tar.gz -fi -for i in $targets -do - echo "benchmarking $i" - rm -rf _playground - tar xf $i - cd _playground - log="../$i.d-f-e-$(git describe).log" - sh -c 'time d2x -f git sandbox' 2>&1 |tee $log - darcs show repo --repodir sandbox |egrep -v 'Root|Cache|Default' >> $log - if diff_git sandbox >/dev/null; then - echo "ok, the result is correct" >> $log - else - echo "ouch, the result is corrupted" >> $log - exit 1 - fi - cd .. -done diff --git a/exporters/darcs/t/data/hungarian.gif b/exporters/darcs/t/data/hungarian.gif Binary files differdeleted file mode 100644 index 41a36fe..0000000 --- a/exporters/darcs/t/data/hungarian.gif +++ /dev/null diff --git a/exporters/darcs/t/lib-httpd.sh b/exporters/darcs/t/lib-httpd.sh deleted file mode 100644 index fad953e..0000000 --- a/exporters/darcs/t/lib-httpd.sh +++ /dev/null @@ -1,67 +0,0 @@ -#!/bin/sh -# -# This is based on git's t/lib-httpd.sh, which is -# Copyright (c) 2008 Clemens Buchacher <drizzd@aon.at> -# - -if test -n "$DFE_TEST_SKIP_HTTPD" -then - echo "skipping test (undef DFE_TEST_SKIP_HTTPD to enable)" - exit -fi - -LIB_HTTPD_PATH=${LIB_HTTPD_PATH-'/usr/sbin/httpd'} -LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'8111'} - -HTTPD_ROOT_PATH="$PWD"/httpd -HTTPD_DOCUMENT_ROOT_PATH=$HTTPD_ROOT_PATH/www - -if ! test -x "$LIB_HTTPD_PATH" -then - echo "skipping test, no web server found at '$LIB_HTTPD_PATH'" - exit -fi - -HTTPD_VERSION=`$LIB_HTTPD_PATH -v | \ - sed -n 's/^Server version: Apache\/\([0-9]*\)\..*$/\1/p; q'` - -if test -n "$HTTPD_VERSION" -then - if test -z "$LIB_HTTPD_MODULE_PATH" - then - if ! test $HTTPD_VERSION -ge 2 - then - echo "skipping test, at least Apache version 2 is required" - exit - fi - - LIB_HTTPD_MODULE_PATH='/usr/lib/apache' - fi -else - error "Could not identify web server at '$LIB_HTTPD_PATH'" -fi - -HTTPD_PARA="-d $HTTPD_ROOT_PATH -f $HTTPD_ROOT_PATH/apache.conf" - -prepare_httpd() { - mkdir -p $HTTPD_DOCUMENT_ROOT_PATH - - ln -s $LIB_HTTPD_MODULE_PATH $HTTPD_ROOT_PATH/modules - - echo "PidFile httpd.pid" > $HTTPD_ROOT_PATH/apache.conf - echo "DocumentRoot www" >> $HTTPD_ROOT_PATH/apache.conf - echo "ErrorLog error.log" >> $HTTPD_ROOT_PATH/apache.conf - - HTTPD_URL=http://127.0.0.1:$LIB_HTTPD_PORT -} - -start_httpd() { - prepare_httpd - - "$LIB_HTTPD_PATH" $HTTPD_PARA \ - -c "Listen 127.0.0.1:$LIB_HTTPD_PORT" -k start -} - -stop_httpd() { - "$LIB_HTTPD_PATH" $HTTPD_PARA -k stop -} diff --git a/exporters/darcs/t/lib.sh b/exporters/darcs/t/lib.sh deleted file mode 100644 index 095f9ef..0000000 --- a/exporters/darcs/t/lib.sh +++ /dev/null @@ -1,337 +0,0 @@ -export DARCS_EMAIL="user@example.com" -export GIT_PAGER=cat -export PATH="$(pwd)/..:$PATH" -pypath="/$(python -c 'from distutils import sysconfig; print sysconfig.get_python_lib()[1:]')/" - -_drrec() -{ - darcs rec --ignore-times "$@" -} - -_drrec_multiline() -{ - echo -e "`LANG= LC_ALL= date +"%a %b %d %H:%M:%S %Z %Y"` -$DARCS_EMAIL -$@" | darcs rec --ignore-times -a --pipe . -} - -_drrecamend() -{ - echo y |darcs amend-rec --ignore-times -a -} - -create_darcs() -{ - rm -rf $1 - mkdir -p $1 - cd $1 - darcs init $2 - echo A > file - darcs add file - _drrec -a -m A - cd .. - rm -rf $1.tmp - darcs get $1 $1.tmp - cd $1 - echo B > file - _drrec -a -m B - cd ../$1.tmp - echo C > file - _drrec -a -m C - cd ../$1 - darcs pull -a ../$1.tmp - echo D > file - _drrec_multiline "first line -second line -third line" - darcs tag 1.0 - echo e > file - _drrec -a -m e - echo f > file - _drrec --author="éáõû <$DARCS_EMAIL>" -a -m f - echo g > file - _drrec --author="" -a -m g - cp ../data/hungarian.gif . - darcs add hungarian.gif - _drrec -a -m "add a binary file" - rm file - echo test > file2 - darcs add file2 - _drrec -a -m "replace file with file2" - touch file3 - darcs add file3 - _drrec -a -m "add empty file" - rm file3 - _drrec -a -m "remove file" - mkdir dir dir2 - darcs add dir - darcs add dir2 - _drrec -a -m "add empty dirs" - darcs mv dir dir-p - darcs mv dir2 dir2-p - _drrec -a -m "rename empty dirs" - echo a > a - echo b > b - darcs add a b - _drrec -a -m "add a b" - rm b - _drrec -a -m "remove and rename" - darcs mv a b - _drrecamend - echo c > c - darcs add c - # empty commit message - _drrec -a -m "" - cd .. -} - -create_bzr() -{ - rm -rf $1 - mkdir -p $1 - cd $1 - bzr init $2 - echo A > file - bzr add file - bzr commit -m A - cd .. - rm -rf $1.tmp - bzr branch $1 $1.tmp - cd $1 - echo B > file - bzr commit -m B - cd ../$1.tmp - echo C > file - bzr commit -m C - cd ../$1 - bzr merge ../$1.tmp - echo D > file - bzr resolve file - echo "first line -second line -third line" | bzr commit -F /dev/stdin - bzr tag 1.0 - echo e > file - bzr commit -m e - #echo f > file - #bzr commit --author="éáõû <$DARCS_EMAIL>" -m f - #echo g > file - #_drrec --author="" -a -m g - cp ../data/hungarian.gif . - bzr add hungarian.gif - bzr commit -m "add a binary file" - rm file - echo test > file2 - bzr add file2 - bzr commit -m "replace file with file2" - touch file3 - bzr add file3 - bzr commit -m "add empty file" - rm file3 - bzr commit -m "remove file" - cd .. -} - -create_hg() -{ - rm -rf $1 - mkdir -p $1 - cd $1 - hg init $2 - echo A > file - hg add file - hg commit -m A - cd .. - rm -rf $1.tmp - hg clone $1 $1.tmp - cd $1 - echo B > file - hg commit -m B - cd ../$1.tmp - echo C > file - hg commit -m C - cd ../$1 - hg pull ../$1.tmp - hg merge - echo D > file - hg resolve -m file - echo "first line -second line -third line" | hg commit -l /dev/stdin - hg tag 1.0 - echo e > file - hg commit -m e - #echo f > file - #bzr commit --author="éáõû <$DARCS_EMAIL>" -m f - #echo g > file - #_drrec --author="" -a -m g - cp ../data/hungarian.gif . - hg add hungarian.gif - hg commit -m "add a binary file" - hg rm file - echo test > file2 - hg add file2 - hg commit -m "replace file with file2" - touch file3 - hg add file3 - hg commit -m "add empty file" - hg rm file3 - hg commit -m "remove file" - mkdir subdir - echo test > subdir/file - hg add subdir/file - hg commit -m "add subdir file" - echo test2 > subdir/file - hg commit -m "commit with weird date" -d "Fri Apr 03 12:38:26 2009 +1300" - cd .. -} -create_git() -{ - rm -rf $1 - mkdir -p $1 - cd $1 - git init $2 - git commit --allow-empty -m 'root commit' - echo A > file - git add file - git commit -a -m A - echo B > file - git commit -a -m B - git checkout -b tmp HEAD~1 - echo C > file - git commit -a -m C - git checkout master - git merge tmp - echo D > file - echo "first line -second line -third line" | git commit -a -F - - git branch -d tmp - git tag 1.0 - echo e > file - git commit -a -m e - echo f > file - git config i18n.commitencoding ISO-8859-2 - git commit --author="éáõû <$DARCS_EMAIL>" -a -m f - cp ../data/hungarian.gif . - git add hungarian.gif - git commit -a -m "add a binary file" - rm file - echo test > file2 - git add file2 - git commit -a -m "replace file with file2" - touch file3 - git add file3 - git commit -a -m "add empty file" - rm file3 - git commit -a -m "remove file" - # now add back 'file' with its old conents, so the mark gets - # reused - echo f > file - git add file - git commit -a -m "file: other -> f" - # this is a boring file for Darcs - touch foo.pyc - git add foo.pyc - git commit -a -m "boring file" - # replace an uppercase file to a lowercase one - echo SPAM > SPAM - git add SPAM - git commit -a -m SPAM - rm SPAM - echo spam > spam - git add spam - git commit -a -m "SPAM -> spam" - cd .. -} - -diff_git() -{ - rm -rf $1.git.nonbare - git clone -q $1.git $1.git.nonbare - diff --exclude _darcs --exclude .git --exclude '*-darcs-backup*' -Nur $1.git.nonbare $1 - return $? -} - -diff_importgit() -{ - test -z "`(cd $1.darcs; darcs diff)`" && - diff --exclude _darcs --exclude .git --exclude '*-darcs-backup*' -Nur $1 $1.darcs - return $? -} - -diff_importhg() -{ - test -z "`(cd $1.darcs; darcs diff)`" && - diff --exclude _darcs --exclude .hg --exclude '*-darcs-backup*' --exclude 'hg-export.*' \ - --exclude '.hgtags' --exclude '*.orig' -Nur $1 $1.darcs - return $? -} - -diff_importdarcs() -{ - test -z "`(cd $1.darcs; darcs diff)`" && - diff --exclude _darcs --exclude '*-darcs-backup*' -Nur $1 $2 - return $? -} - -diff_importbzr() -{ - test -z "`(cd $1.darcs; darcs diff)`" && - diff --exclude _darcs --exclude .bzr --exclude '*-darcs-backup*' -Nur $1 $1.darcs - return $? -} - -diff_bzr() -{ - cd $1.bzr/trunk - bzr update - cd - >/dev/null - diff --exclude _darcs --exclude .bzr --exclude '*-darcs-backup*' -Nur $1.bzr/trunk $1 - return $? -} - -diff_hg() -{ - hg -R $1.hg update - diff --exclude _darcs --exclude .hg --exclude '*-darcs-backup*' -Nur $1.hg $1 - return $? -} - -die() -{ - echo "fatal: $@" - exit 1 -} - -upd_file_darcs() -{ - cd $1 - echo $3 > $2 - _drrec -a -m "updated '$2' to '$3'" - cd .. -} - -upd_file_git() -{ - cd $1 - echo $3 > $2 - git commit -a -m "updated '$2' to '$3'" - cd .. -} - -upd_file_bzr() -{ - cd $1 - echo $3 > $2 - bzr commit -m "updated '$2' to '$3'" - cd .. -} - -upd_file_hg() -{ - cd $1 - echo $3 > $2 - hg commit -m "updated '$2' to '$3'" - cd .. -} diff --git a/exporters/darcs/t/test-bzr.sh b/exporters/darcs/t/test-bzr.sh deleted file mode 100644 index 479f259..0000000 --- a/exporters/darcs/t/test-bzr.sh +++ /dev/null @@ -1,16 +0,0 @@ -. ./lib.sh - -create_darcs test --old-fashioned-inventory - -rm -rf test.darcs test.bzr -mkdir test.bzr -cd test.bzr -bzr init-repo . -cd .. -if [ "$1" != "--stdout" ]; then - darcs-fast-export test |(cd test.bzr; bzr fast-import -) - diff_bzr test - exit $? -else - darcs-fast-export test -fi diff --git a/exporters/darcs/t/test-git-d2x.sh b/exporters/darcs/t/test-git-d2x.sh deleted file mode 100644 index 364edec..0000000 --- a/exporters/darcs/t/test-git-d2x.sh +++ /dev/null @@ -1,19 +0,0 @@ -. ./lib.sh - -create_darcs test --old-fashioned-inventory - -rm -rf test.git -if [ "$1" != "--stdout" ]; then - d2x -f git test - diff_git test || die "initial conversion differs" - upd_file_darcs test file2 upd_contents - d2x -f git test - diff_git test || die "update differs" - upd_file_darcs test hungarian.gif "binary to text" - d2x -f git test - diff_git test || die "update2 differs" - d2x -f git test - diff_git test || die "update3 (noop) differs" -else - darcs-fast-export test -fi diff --git a/exporters/darcs/t/test-git-incremental.sh b/exporters/darcs/t/test-git-incremental.sh deleted file mode 100644 index 1c62b9a..0000000 --- a/exporters/darcs/t/test-git-incremental.sh +++ /dev/null @@ -1,24 +0,0 @@ -. ./lib.sh - -create_darcs test --old-fashioned-inventory - -rm -rf test.darcs test.git -mkdir test.git -cd test.git -git --bare init -cd .. -if [ "$1" != "--stdout" ]; then - dmark="$(pwd)/test.dfe-marks" - gmark="$(pwd)/test.gfi-marks" - rm -f $mark $gmark - darcs-fast-export --export-marks=$dmark test |(cd test.git; git fast-import --export-marks=$gmark) - diff_git test || die "initial conversion differs" - upd_file_darcs test file2 upd_contents - darcs-fast-export --export-marks=$dmark --import-marks=$dmark test |(cd test.git; git fast-import --export-marks=$gmark --import-marks=$gmark) - diff_git test || die "update differs" - upd_file_darcs test hungarian.gif "binary to text" - darcs-fast-export --export-marks=$dmark --import-marks=$dmark test |(cd test.git; git fast-import --export-marks=$gmark --import-marks=$gmark) - diff_git test || die "update2 differs" -else - darcs-fast-export test -fi diff --git a/exporters/darcs/t/test-git-progress.sh b/exporters/darcs/t/test-git-progress.sh deleted file mode 100644 index 6586e80..0000000 --- a/exporters/darcs/t/test-git-progress.sh +++ /dev/null @@ -1,18 +0,0 @@ -. ./lib.sh - -create_darcs test --old-fashioned-inventory - -rm -rf test.darcs test.git -mkdir test.git -cd test.git -git --bare init -cd .. -if [ "$1" != "--stdout" ]; then - darcs-fast-export --progres 2 test |(cd test.git; git fast-import) - if [ $? = 0 ]; then - diff_git test - exit $? - fi -else - darcs-fast-export test -fi diff --git a/exporters/darcs/t/test-git.sh b/exporters/darcs/t/test-git.sh deleted file mode 100644 index de504ee..0000000 --- a/exporters/darcs/t/test-git.sh +++ /dev/null @@ -1,18 +0,0 @@ -. ./lib.sh - -create_darcs test --old-fashioned-inventory - -rm -rf test.darcs test.git -mkdir test.git -cd test.git -git --bare init -cd .. -if [ "$1" != "--stdout" ]; then - darcs-fast-export test |(cd test.git; git fast-import) - if [ $? = 0 ]; then - diff_git test - exit $? - fi -else - darcs-fast-export test -fi diff --git a/exporters/darcs/t/test-hg-d2x.sh b/exporters/darcs/t/test-hg-d2x.sh deleted file mode 100644 index bc83385..0000000 --- a/exporters/darcs/t/test-hg-d2x.sh +++ /dev/null @@ -1,12 +0,0 @@ -. ./lib.sh - -create_darcs test --old-fashioned-inventory - -rm -rf test.hg -if [ "$1" != "--stdout" ]; then - d2x -f hg test - diff_hg test - exit $? -else - darcs-fast-export test -fi diff --git a/exporters/darcs/t/test-hg.sh b/exporters/darcs/t/test-hg.sh deleted file mode 100644 index 95bfc4b..0000000 --- a/exporters/darcs/t/test-hg.sh +++ /dev/null @@ -1,16 +0,0 @@ -. ./lib.sh - -create_darcs test --old-fashioned-inventory - -rm -rf test.darcs test.hg -mkdir test.hg -cd test.hg -hg init -cd .. -if [ "$1" != "--stdout" ]; then - darcs-fast-export test |(cd test.hg; hg fastimport -) - diff_hg test - exit $? -else - darcs-fast-export test -fi diff --git a/exporters/darcs/t/test2-bzr-d2x.sh b/exporters/darcs/t/test2-bzr-d2x.sh deleted file mode 100644 index 13812eb..0000000 --- a/exporters/darcs/t/test2-bzr-d2x.sh +++ /dev/null @@ -1,19 +0,0 @@ -. ./lib.sh - -create_darcs test2 --darcs-2 - -rm -rf test2.bzr -if [ "$1" != "--stdout" ]; then - d2x -f bzr test2 - diff_bzr test2 || die "initial conversion differs" - upd_file_darcs test2 file2 upd_contents - d2x -f bzr test2 - diff_bzr test2 || die "update differs" - upd_file_darcs test2 hungarian.gif "binary to text" - d2x -f bzr test2 - diff_bzr test2 || die "update2 differs" - d2x -f bzr test2 - diff_bzr test2 || die "update3 (noop) differs" -else - darcs-fast-export test2 -fi diff --git a/exporters/darcs/t/test2-bzr-incremental.sh b/exporters/darcs/t/test2-bzr-incremental.sh deleted file mode 100644 index d464559..0000000 --- a/exporters/darcs/t/test2-bzr-incremental.sh +++ /dev/null @@ -1,21 +0,0 @@ -. ./lib.sh - -create_darcs test2 --darcs-2 - -rm -rf test2.darcs test2.bzr -mkdir test2.bzr -cd test2.bzr -bzr init-repo . -cd .. -if [ "$1" != "--stdout" ]; then - dmark="$(pwd)/test2.dfe-marks" - bmark="$(pwd)/test2.bfi-marks" - rm -f $mark $gmark - darcs-fast-export --export-marks=$dmark test2 |(cd test2.bzr; bzr fast-import --export-marks=$bmark -) - diff_bzr test2 || die "initial conversion differs" - upd_file_darcs test2 file2 upd_contents - darcs-fast-export --export-marks=$dmark --import-marks=$dmark test2 |(cd test2.bzr; bzr fast-import --export-marks=$bmark --import-marks=$bmark -) - diff_bzr test2 || die "update differs" -else - darcs-fast-export test2 -fi diff --git a/exporters/darcs/t/test2-git-funny-tagname.sh b/exporters/darcs/t/test2-git-funny-tagname.sh deleted file mode 100644 index 03eca66..0000000 --- a/exporters/darcs/t/test2-git-funny-tagname.sh +++ /dev/null @@ -1,25 +0,0 @@ -. ./lib.sh - -create_darcs test2 --darcs-2 -cd test2 -darcs tag "this :just (won't work; die)" -darcs tag "accent-tag-éáőű" -cd .. - -rm -rf test2.darcs test2.git -mkdir test2.git -cd test2.git -git --bare init -cd .. -if [ "$1" != "--stdout" ]; then - darcs-fast-export test2 |(cd test2.git; git fast-import) - ret=$? - if [ $ret = 0 ]; then - diff_git test2 - exit $? - else - exit $ret - fi -else - darcs-fast-export test2 -fi diff --git a/exporters/darcs/t/test2-git-http.sh b/exporters/darcs/t/test2-git-http.sh deleted file mode 100644 index 02549e4..0000000 --- a/exporters/darcs/t/test2-git-http.sh +++ /dev/null @@ -1,22 +0,0 @@ -. ./lib.sh -. ./lib-httpd.sh - -rm -rf test2.darcs test2.git httpd -create_darcs test2 --darcs-2 -mkdir -p $HTTPD_DOCUMENT_ROOT_PATH -mv -v test2 $HTTPD_DOCUMENT_ROOT_PATH -ln -s $HTTPD_DOCUMENT_ROOT_PATH/test2 . - -mkdir test2.git -cd test2.git -git --bare init -cd .. -start_httpd -darcs-fast-export $HTTPD_URL/test2 |(cd test2.git; git fast-import) -ret=$? -stop_httpd -if [ $ret != 0 ]; then - exit $ret -fi -diff_git test2 -exit $? diff --git a/exporters/darcs/t/test2-git-incremental-specworkdir.sh b/exporters/darcs/t/test2-git-incremental-specworkdir.sh deleted file mode 100644 index 83731f2..0000000 --- a/exporters/darcs/t/test2-git-incremental-specworkdir.sh +++ /dev/null @@ -1,22 +0,0 @@ -. ./lib.sh - -create_darcs test2 --darcs-2 - -rm -rf test2.darcs test2.git -mkdir test2.git -cd test2.git -git --bare init -mkdir darcs -cd .. -if [ "$1" != "--stdout" ]; then - dmark="$(pwd)/test2.git/darcs/test2.dfe-marks" - gmark="$(pwd)/test2.git/darcs/test2.gfi-marks" - rm -f $mark $gmark - darcs-fast-export --export-marks=$dmark test2 --working test2.git/darcs/repo |(cd test2.git; git fast-import --export-marks=$gmark) - diff_git test2 || die "initial conversion differs" - upd_file_darcs test2 file2 upd_contents - darcs-fast-export --export-marks=$dmark --import-marks=$dmark test2 --working test2.git/darcs/repo |(cd test2.git; git fast-import --export-marks=$gmark --import-marks=$gmark) - diff_git test2 || die "update differs" -else - darcs-fast-export test2 -fi diff --git a/exporters/darcs/t/test2-git-incremental.sh b/exporters/darcs/t/test2-git-incremental.sh deleted file mode 100644 index 41a3937..0000000 --- a/exporters/darcs/t/test2-git-incremental.sh +++ /dev/null @@ -1,21 +0,0 @@ -. ./lib.sh - -create_darcs test2 --darcs-2 - -rm -rf test2.darcs test2.git -mkdir test2.git -cd test2.git -git --bare init -cd .. -if [ "$1" != "--stdout" ]; then - dmark="$(pwd)/test2.dfe-marks" - gmark="$(pwd)/test2.gfi-marks" - rm -f $mark $gmark - darcs-fast-export --export-marks=$dmark test2 |(cd test2.git; git fast-import --export-marks=$gmark) - diff_git test2 || die "initial conversion differs" - upd_file_darcs test2 file2 upd_contents - darcs-fast-export --export-marks=$dmark --import-marks=$dmark test2 |(cd test2.git; git fast-import --export-marks=$gmark --import-marks=$gmark) - diff_git test2 || die "update differs" -else - darcs-fast-export test2 -fi diff --git a/exporters/darcs/t/test2-git.sh b/exporters/darcs/t/test2-git.sh deleted file mode 100644 index a8fc005..0000000 --- a/exporters/darcs/t/test2-git.sh +++ /dev/null @@ -1,18 +0,0 @@ -. ./lib.sh - -create_darcs test2 --darcs-2 - -rm -rf test2.darcs test2.git -mkdir test2.git -cd test2.git -git --bare init -cd .. -if [ "$1" != "--stdout" ]; then - darcs-fast-export test2 |(cd test2.git; git fast-import) - if [ $? = 0 ]; then - diff_git test2 - exit $? - fi -else - darcs-fast-export test2 -fi diff --git a/exporters/darcs/t/testimport-bzr-x2d.sh b/exporters/darcs/t/testimport-bzr-x2d.sh deleted file mode 100644 index ebe014b..0000000 --- a/exporters/darcs/t/testimport-bzr-x2d.sh +++ /dev/null @@ -1,15 +0,0 @@ -. ./lib.sh - -create_bzr test - -rm -rf test.darcs -x2d -f bzr test -diff_importbzr test || die "initial conversion differs" -upd_file_bzr test file2 upd_contents -x2d -f bzr test -diff_importbzr test || die "update differs" -upd_file_bzr test hungarian.gif "binary to text" -x2d -f bzr test -diff_importbzr test || die "update2 differs" -x2d -f bzr test -diff_importbzr test || die "update3 (noop) differs" diff --git a/exporters/darcs/t/testimport-bzr.sh b/exporters/darcs/t/testimport-bzr.sh deleted file mode 100644 index 358c5e5..0000000 --- a/exporters/darcs/t/testimport-bzr.sh +++ /dev/null @@ -1,15 +0,0 @@ -. ./lib.sh - -create_bzr test - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -(cd test; bzr fast-export .) | (cd test.darcs; darcs-fast-import) -if [ $? != 0 ]; then - exit 1 -fi -diff_importbzr test -exit $? diff --git a/exporters/darcs/t/testimport-copy.sh b/exporters/darcs/t/testimport-copy.sh deleted file mode 100644 index 109d87e..0000000 --- a/exporters/darcs/t/testimport-copy.sh +++ /dev/null @@ -1,26 +0,0 @@ -. ./lib.sh - -rm -rf test -mkdir test -cd test -git init -echo a > file -git add file -git commit -m a1 -cp file file2 -git add file2 -git commit -m b -cd .. - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -(cd test; git fast-export -C -C HEAD) > out -cat out | (cd test.darcs; darcs-fast-import) -if [ $? != 0 ]; then - exit 1 -fi -diff_importgit test -exit $? diff --git a/exporters/darcs/t/testimport-darcs.sh b/exporters/darcs/t/testimport-darcs.sh deleted file mode 100644 index 8b6d603..0000000 --- a/exporters/darcs/t/testimport-darcs.sh +++ /dev/null @@ -1,17 +0,0 @@ -. ./lib.sh - -create_darcs test2 --darcs-2 - -rm -rf test2.importdarcs test2.darcs -mkdir test2.importdarcs -cd test2.importdarcs -darcs init -cd .. - -darcs-fast-export test2 | (cd test2.importdarcs; darcs-fast-import) - -if [ $? != 0 ]; then - exit 1 -fi -diff_importdarcs test2 test2.importdarcs -exit $? diff --git a/exporters/darcs/t/testimport-deleteall.sh b/exporters/darcs/t/testimport-deleteall.sh deleted file mode 100644 index 11c5a83..0000000 --- a/exporters/darcs/t/testimport-deleteall.sh +++ /dev/null @@ -1,31 +0,0 @@ -. ./lib.sh - -rm -rf test -mkdir test -cd test -git init -echo a > file -git add file -echo A > file2 -git add file2 -git commit -m a12 -git rm file* -echo b>file3 -git add file3 -git commit -m b -cd .. - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -(cd test; git fast-export --progress=2 HEAD) > out -sed -i '/^D file$/d' out -sed -i 's/^D file2$/deleteall/' out -cat out | (cd test.darcs; darcs-fast-import) -if [ $? != 0 ]; then - exit 1 -fi -diff_importgit test -exit $? diff --git a/exporters/darcs/t/testimport-git-incremental.sh b/exporters/darcs/t/testimport-git-incremental.sh deleted file mode 100644 index 6c92880..0000000 --- a/exporters/darcs/t/testimport-git-incremental.sh +++ /dev/null @@ -1,16 +0,0 @@ -. ./lib.sh - -create_git test - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -gmark="$(pwd)/test.gfe-marks" -dmark="$(pwd)/test.dfi-marks" -(cd test; git fast-export --export-marks=$gmark HEAD) | (cd test.darcs; darcs-fast-import --export-marks=$dmark) -diff_importgit test || die "initial conversion differs" -upd_file_git test file2 upd_contents -(cd test; git fast-export --export-marks=$gmark --import-marks=$gmark HEAD) | (cd test.darcs; darcs-fast-import --export-marks=$dmark --import-marks=$dmark) -diff_importgit test || die "update differs" diff --git a/exporters/darcs/t/testimport-git-twoway-gd.sh b/exporters/darcs/t/testimport-git-twoway-gd.sh deleted file mode 100644 index 0e0c981..0000000 --- a/exporters/darcs/t/testimport-git-twoway-gd.sh +++ /dev/null @@ -1,34 +0,0 @@ -. ./lib.sh - -create_darcs test - -rm -rf test.git -mkdir test.git -cd test.git -git init -git darcs add upstream ../test -git darcs pull upstream -cd .. -diff_git test || die "initial fetch differs" -upd_file_darcs test file2 upd_contents -cd test.git -git darcs pull upstream -cd .. -diff_git test || die "fetch #1 differs" -upd_file_git test.git file2 upd_contents2 -cd test.git -git darcs push upstream -cd .. -diff_git test || die "push #1 difers" -upd_file_darcs test file2 upd_contents3 -upd_file_darcs test file2 upd_contents32 -cd test.git -git darcs pull upstream -cd .. -diff_git test || die "fetch #2 (multiple commits) differs" -upd_file_git test.git file2 upd_contents4 -upd_file_git test.git file2 upd_contents42 -cd test.git -git darcs push upstream -cd .. -diff_git test || die "push #2 (multiple commits) differs" diff --git a/exporters/darcs/t/testimport-git-twoway.sh b/exporters/darcs/t/testimport-git-twoway.sh deleted file mode 100644 index f9b515a..0000000 --- a/exporters/darcs/t/testimport-git-twoway.sh +++ /dev/null @@ -1,30 +0,0 @@ -. ./lib.sh - -create_git test - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -gmark="$(pwd)/test.gmarks" -dmark="$(pwd)/test.dmarks" - -(cd test; git fast-export --export-marks=$gmark HEAD) | (cd test.darcs; darcs-fast-import --export-marks=$dmark) -diff_importgit test || die "initial conversion differs" -upd_file_git test file2 upd_contents -(cd test; git fast-export --export-marks=$gmark --import-marks=$gmark HEAD) | (cd test.darcs; darcs-fast-import --export-marks=$dmark --import-marks=$dmark) -diff_importgit test || die "git -> darcs update #1 differs" -upd_file_darcs test.darcs file2 upd_contents2 -darcs-fast-export --export-marks=$dmark --import-marks=$dmark --working test/.git/darcs test.darcs | (cd test; git fast-import --export-marks=$gmark --import-marks=$gmark) -(cd test; git checkout -f) -diff_importgit test || die "darcs -> git update #2 differs" -upd_file_git test file2 upd_contents3 -upd_file_git test file2 upd_contents32 -(cd test; git fast-export --export-marks=$gmark --import-marks=$gmark HEAD) | (cd test.darcs; darcs-fast-import --export-marks=$dmark --import-marks=$dmark) -diff_importgit test || die "git -> darcs update #3 differs" -upd_file_darcs test.darcs file2 upd_contents4 -upd_file_darcs test.darcs file2 upd_contents42 -darcs-fast-export --export-marks=$dmark --import-marks=$dmark --working test/.git/darcs test.darcs | (cd test; git fast-import --export-marks=$gmark --import-marks=$gmark) -(cd test; git checkout -f) -diff_importgit test || die "darcs -> git update #4 differs" diff --git a/exporters/darcs/t/testimport-git-x2d.sh b/exporters/darcs/t/testimport-git-x2d.sh deleted file mode 100644 index f3f02a7..0000000 --- a/exporters/darcs/t/testimport-git-x2d.sh +++ /dev/null @@ -1,15 +0,0 @@ -. ./lib.sh - -create_git test - -rm -rf test.darcs -x2d -f git test -diff_importgit test || die "initial conversion differs" -upd_file_git test file2 upd_contents -x2d -f git test -diff_importgit test || die "update differs" -upd_file_git test hungarian.gif "binary to text" -x2d -f git test -diff_importgit test || die "update2 differs" -x2d -f git test -diff_importgit test || die "update3 (noop) differs" diff --git a/exporters/darcs/t/testimport-git.sh b/exporters/darcs/t/testimport-git.sh deleted file mode 100644 index 2e64e62..0000000 --- a/exporters/darcs/t/testimport-git.sh +++ /dev/null @@ -1,15 +0,0 @@ -. ./lib.sh - -create_git test - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -(cd test; git fast-export --progress=2 HEAD) | (cd test.darcs; darcs-fast-import) -if [ $? != 0 ]; then - exit 1 -fi -diff_importgit test -exit $? diff --git a/exporters/darcs/t/testimport-gitsymlink.sh b/exporters/darcs/t/testimport-gitsymlink.sh deleted file mode 100644 index 100c583..0000000 --- a/exporters/darcs/t/testimport-gitsymlink.sh +++ /dev/null @@ -1,45 +0,0 @@ -. ./lib.sh - -create_git test -cd test -# add two dirs with the some contents, then remove the second -# and make it a symlink to the first -mkdir dira -echo blabla > dira/file -echo blablabla > dira/file2 -mkdir dirb -touch dirb/file -touch dirb/file2 -git add dira dirb -git commit -a -m "add dira/dirb" -rm -rf dirb -ln -s dira dirb -git add dirb -git commit -a -m "change a dir to a symlink" -cd .. - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -(cd test; git fast-export --progress=2 HEAD) | (cd test.darcs; darcs-fast-import) -# we *do* want this to fail, but with error code 2. that means that we -# detected that symlinks are not supported and the user does not get a -# meaningless exception -if [ $? != 2 ]; then - exit 1 -fi - -# now try with the symhack option -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -(cd test; git fast-export --progress=2 HEAD) | (cd test.darcs; darcs-fast-import --symhack) -if [ $? != 0 ]; then - exit 1 -fi -diff_importgit test -exit $? diff --git a/exporters/darcs/t/testimport-hg-x2d.sh b/exporters/darcs/t/testimport-hg-x2d.sh deleted file mode 100644 index a1d7d62..0000000 --- a/exporters/darcs/t/testimport-hg-x2d.sh +++ /dev/null @@ -1,15 +0,0 @@ -. ./lib.sh - -create_hg test - -rm -rf test.darcs -x2d -f hg test -diff_importhg test || die "initial conversion differs" -upd_file_hg test file2 upd_contents -x2d -f hg test -diff_importhg test || die "update differs" -upd_file_hg test hungarian.gif "binary to text" -x2d -f hg test -diff_importhg test || die "update2 differs" -x2d -f hg test -diff_importhg test || die "update3 (noop) differs" diff --git a/exporters/darcs/t/testimport-hg.sh b/exporters/darcs/t/testimport-hg.sh deleted file mode 100644 index 7f6d215..0000000 --- a/exporters/darcs/t/testimport-hg.sh +++ /dev/null @@ -1,15 +0,0 @@ -. ./lib.sh - -create_hg test - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -(cd test; $pypath/bzrlib/plugins/fastimport/exporters/hg-fast-export.py -r .) | (cd test.darcs; darcs-fast-import) -if [ $? != 0 ]; then - exit 1 -fi -diff_importhg test -exit $? diff --git a/exporters/darcs/t/testimport-rename.sh b/exporters/darcs/t/testimport-rename.sh deleted file mode 100644 index c6fa29f..0000000 --- a/exporters/darcs/t/testimport-rename.sh +++ /dev/null @@ -1,25 +0,0 @@ -. ./lib.sh - -rm -rf test -mkdir test -cd test -git init -echo a > file -git add file -git commit -m a1 -git mv file file2 -git commit -m b -cd .. - -rm -rf test.darcs -mkdir test.darcs -cd test.darcs -darcs init -cd .. -(cd test; git fast-export -M HEAD) > out -cat out | (cd test.darcs; darcs-fast-import) -if [ $? != 0 ]; then - exit 1 -fi -diff_importgit test -exit $? diff --git a/exporters/darcs/x2d b/exporters/darcs/x2d deleted file mode 100755 index 398103d..0000000 --- a/exporters/darcs/x2d +++ /dev/null @@ -1,127 +0,0 @@ -#!/bin/sh -# -# x2d - convert git, bzr or hg repos to darcs using fast-export -# -# Copyright (c) 2008 by Miklos Vajna <vmiklos@frugalware.org> -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, -# USA. -# - -usage() -{ - echo "Usage: x2d -f format repo" -} - -die() -{ - echo "$@" - usage - exit 1 -} - -check_up_to_date() -{ - upstreamnum=$(darcs show repo|grep 'Num Patches'|sed 's/.*: //') - if [ "$upstreamnum" = "$(cd $origin; eval $*)" ]; then - echo "No remote changes to pull!" - exit 0 - fi -} - -case $1 in - -h|--help) - usage - exit 0 - ;; - -f) - format="$2" - shift 2 - ;; -esac - -[ -n "$format" ] || die "Source format is not given!" - -case $format in - git|bzr|hg) - ;; - *) - die "The requested source format is not yet supported!" - ;; -esac - -common_opts="" -while [ -n "$2" ] -do - common_opts="$common_opts $1" - shift 1 -done -origin="$1" -shift 1 - -[ -d "$origin" ] || die "Source repo does not exist!" - -# convert to abspath -cd $origin -origin=$(pwd) - -dmark="$origin.darcs/_darcs/fast-import/dfe-marks" -fmark="$origin.darcs/_darcs/fast-import/ffi-marks" - -mkdir -p $origin.darcs -cd $origin.darcs - -common_opts="$common_opts --logfile $origin.darcs/_darcs/fast-import/log" -pypath="/$(python -c 'from distutils import sysconfig; print sysconfig.get_python_lib()[1:]')/" - -if [ ! -f $dmark ]; then - darcs init - mkdir -p _darcs/fast-import - case $format in - git) - (cd $origin; git fast-export --export-marks=$fmark HEAD) | \ - darcs-fast-import --export-marks=$dmark $common_opts - ;; - bzr) - (cd $origin; bzr fast-export \ - --export-marks=$fmark . ) | darcs-fast-import --export-marks=$dmark $common_opts - ;; - hg) - (cd $origin; $pypath/bzrlib/plugins/fastimport/exporters/hg-fast-export.py -r . ) | \ - darcs-fast-import --export-marks=$dmark $common_opts - esac -else - case $format in - git) - check_up_to_date "git rev-list HEAD |wc -l" - (cd $origin; git fast-export --export-marks=$fmark --import-marks=$fmark HEAD) | \ - darcs-fast-import --export-marks=$dmark --import-marks=$dmark $common_opts - ;; - bzr) - # bzr revno is not good here, because at merges - # it produces less revision than the number we - # have in darcs - check_up_to_date "bzr log --include-merges |grep -c revno:" - (cd $origin; bzr fast-export \ - --export-marks=$fmark --import-marks=$fmark . ) | \ - darcs-fast-import --export-marks=$dmark --import-marks=$dmark $common_opts - ;; - hg) - check_up_to_date 'echo $(($(hg tip --template "{rev}")+1))' - (cd $origin; $pypath/bzrlib/plugins/fastimport/exporters/hg-fast-export.py -r . ) | \ - darcs-fast-import --export-marks=$dmark --import-marks=$dmark $common_opts - ;; - esac -fi diff --git a/exporters/darcs/x2d.txt b/exporters/darcs/x2d.txt deleted file mode 100644 index 25ed6bb..0000000 --- a/exporters/darcs/x2d.txt +++ /dev/null @@ -1,28 +0,0 @@ -= x2d(1) - -== NAME - -x2d - convert git, bzr or hg repos to a darcs one using fast-export - -== SYNOPSIS - -x2d -f <format> [<importoptions>] <otherrepo> - -== DESCRIPTION - -x2d is a wrapper script that just automates doing an initial or -continuing an incremental conversion. All it does is initializing the -target darcs repo, starting darcs-fast-import and the relevant exporter -with the proper switches and pipe the importer's output to the -importer's standard input. - -== OPTIONS - ---help:: - Display usage. - --f <format>:: - Specify the format of the source repo. Currently supported sources are - git, bzr and hg. Incremental conversion is supported for all of them. - -The rest of the options is directly passed to darcs-fast-import. diff --git a/exporters/hg-fast-export.README b/exporters/hg-fast-export.README deleted file mode 100644 index a5999de..0000000 --- a/exporters/hg-fast-export.README +++ /dev/null @@ -1,54 +0,0 @@ -hg-fast-import.py - mercurial to bzr converter using bzr fast-import - -Legal -===== - -Most hg-* scripts are licensed under the MIT license[0] and were written -by Rocco Rutte <pdmef@gmx.net> with hints and help from the git list and -#mercurial on freenode. hg-fast-export[1] was integrated into -bzr-fastimport by Ian Clatworthy with permission from Rocco. - -The current maintainer is Frej Drejhammar <frej.drejhammar@gmail.com>. - -Usage -===== - -Using hg-fast-export is quite simple for a mercurial repository <repo>: - - bzr init-repo foo.bzr - cd foo.bzr - hg-fast-import.py -r <repo> | bzr fast-import - - -Notes/Limitations -================= - -hg-fast-import supports multiple branches but only named branches with exactly -one head each. Otherwise commits to the tip of these heads within branch -will get flattened into merge commits. - -The way the hg API and remote access protocol is designed it is not -possible to use hg-fast-export on remote repositories -(http/ssh). First clone the repository, then convert it. - -Design -====== - -hg-fast-import.py was designed in a way that doesn't require a 2-pass mechanism -or any prior repository analysis: if just outputs what it finds. -This also implies that it heavily relies on strictly -linear ordering of changesets from hg, i.e. its append-only storage -model so that changesets hg-fast-import already saw never get modified. - -Todo -==== - -Test incremental imports, particularly handling of branches and tags. - -For one-time conversions, everything should be fine. - -Footnotes -========= - -[0] http://www.opensource.org/licenses/mit-license.php - -[1] http://repo.or.cz/w/fast-export.git diff --git a/exporters/hg-fast-export.py b/exporters/hg-fast-export.py deleted file mode 100755 index 45c9ab4..0000000 --- a/exporters/hg-fast-export.py +++ /dev/null @@ -1,442 +0,0 @@ -#!/usr/bin/env python - -# Copyright (c) 2007, 2008 Rocco Rutte <pdmef@gmx.net> and others. -# License: MIT <http://www.opensource.org/licenses/mit-license.php> - -from mercurial import repo,hg,cmdutil,util,ui,revlog,node -from hg2git import setup_repo,fixup_user,get_branch,get_changeset -from hg2git import load_cache,save_cache,get_git_sha1,set_default_branch,set_origin_name -from tempfile import mkstemp -from optparse import OptionParser -import re -import sys -import os - -# silly regex to catch Signed-off-by lines in log message -sob_re=re.compile('^Signed-[Oo]ff-[Bb]y: (.+)$') -# insert 'checkpoint' command after this many commits or none at all if 0 -cfg_checkpoint_count=0 -# write some progress message every this many file contents written -cfg_export_boundary=1000 - -def gitmode(flags): - return 'l' in flags and '120000' or 'x' in flags and '100755' or '100644' - -def wr(msg=''): - if msg == None: - msg = '' - print msg - #map(lambda x: sys.stderr.write('\t[%s]\n' % x),msg.split('\n')) - -def checkpoint(count): - count=count+1 - if cfg_checkpoint_count>0 and count%cfg_checkpoint_count==0: - sys.stderr.write("Checkpoint after %d commits\n" % count) - wr('checkpoint') - wr() - return count - -def get_parent_mark(parent,marks): - """Get the mark for some parent. - If we saw it in the current session, return :%d syntax and - otherwise the SHA1 from the cache.""" - return marks.get(str(parent),':%d' % (parent+1)) - -def file_mismatch(f1,f2): - """See if two revisions of a file are not equal.""" - return node.hex(f1)!=node.hex(f2) - -def split_dict(dleft,dright,l=[],c=[],r=[],match=file_mismatch): - """Loop over our repository and find all changed and missing files.""" - for left in dleft.keys(): - right=dright.get(left,None) - if right==None: - # we have the file but our parent hasn't: add to left set - l.append(left) - elif match(dleft[left],right): - # we have it but checksums mismatch: add to center set - c.append(left) - for right in dright.keys(): - left=dleft.get(right,None) - if left==None: - # if parent has file but we don't: add to right set - r.append(right) - # change is already handled when comparing child against parent - return l,c,r - -def get_filechanges(repo,revision,parents,mleft): - """Given some repository and revision, find all changed/deleted files.""" - l,c,r=[],[],[] - for p in parents: - if p<0: continue - mright=repo.changectx(p).manifest() - l,c,r=split_dict(mleft,mright,l,c,r) - l.sort() - c.sort() - r.sort() - return l,c,r - -def get_author(logmessage,committer,authors): - """As git distincts between author and committer of a patch, try to - extract author by detecting Signed-off-by lines. - - This walks from the end of the log message towards the top skipping - empty lines. Upon the first non-empty line, it walks all Signed-off-by - lines upwards to find the first one. For that (if found), it extracts - authorship information the usual way (authors table, cleaning, etc.) - - If no Signed-off-by line is found, this defaults to the committer. - - This may sound stupid (and it somehow is), but in log messages we - accidentially may have lines in the middle starting with - "Signed-off-by: foo" and thus matching our detection regex. Prevent - that.""" - - loglines=logmessage.split('\n') - i=len(loglines) - # from tail walk to top skipping empty lines - while i>=0: - i-=1 - if len(loglines[i].strip())==0: continue - break - if i>=0: - # walk further upwards to find first sob line, store in 'first' - first=None - while i>=0: - m=sob_re.match(loglines[i]) - if m==None: break - first=m - i-=1 - # if the last non-empty line matches our Signed-Off-by regex: extract username - if first!=None: - r=fixup_user(first.group(1),authors) - return r - return committer - -def export_file_contents(ctx,manifest,files): - count=0 - max=len(files) - for file in files: - # Skip .hgtags files. They only get us in trouble. - if file == ".hgtags": - sys.stderr.write('Skip %s\n' % (file)) - continue - d=ctx.filectx(file).data() - wr('M %s inline %s' % (gitmode(manifest.flags(file)),file)) - wr('data %d' % len(d)) # had some trouble with size() - wr(d) - count+=1 - if count%cfg_export_boundary==0: - sys.stderr.write('Exported %d/%d files\n' % (count,max)) - if max>cfg_export_boundary: - sys.stderr.write('Exported %d/%d files\n' % (count,max)) - -def is_merge(parents): - c=0 - for parent in parents: - if parent>=0: - c+=1 - return c>1 - -def sanitize_name(name,what="branch"): - """Sanitize input roughly according to git-check-ref-format(1)""" - - def dot(name): - if name[0] == '.': return '_'+name[1:] - return name - - n=name - p=re.compile('([[ ~^:?*]|\.\.)') - n=p.sub('_', n) - if n[-1] == '/': n=n[:-1]+'_' - n='/'.join(map(dot,n.split('/'))) - p=re.compile('_+') - n=p.sub('_', n) - - if n!=name: - sys.stderr.write('Warning: sanitized %s [%s] to [%s]\n' % (what,name,n)) - return n - -def export_commit(ui,repo,revision,marks,mapping,heads,last,max,count,authors,sob,brmap): - def get_branchname(name): - if brmap.has_key(name): - return brmap[name] - n=sanitize_name(name) - brmap[name]=n - return n - - (revnode,_,user,(time,timezone),files,desc,branch,_)=get_changeset(ui,repo,revision,authors) - parents=repo.changelog.parentrevs(revision) - - branch=get_branchname(branch) - - wr('commit refs/heads/%s' % branch) - wr('mark :%d' % (revision+1)) - if sob: - wr('author %s %d %s' % (get_author(desc,user,authors),time,timezone)) - wr('committer %s %d %s' % (user,time,timezone)) - wr('data %d' % (len(desc)+1)) # wtf? - wr(desc) - wr() - - pidx1, pidx2 = 0, 1 - if parents[1] > 0: - if parents[0] <= 0 or \ - repo.changelog.node(parents[0]) < repo.changelog.node(parents[1]): - pidx1, pidx2 = 1, 0 - - full_rev=False - if revision==0: full_rev=True - - src=heads.get(branch,'') - link='' - if src!='': - # if we have a cached head, this is an incremental import: initialize it - # and kill reference so we won't init it again - wr('from %s' % src) - heads[branch]='' - sys.stderr.write('%s: Initializing to parent [%s]\n' % - (branch,src)) - link=src # avoid making a merge commit for incremental import - elif link=='' and not heads.has_key(branch) and revision>0: - if parents[0]>=0: - # newly created branch with parent: connect to parent - tmp=get_parent_mark(parents[0],marks) - wr('from %s' % tmp) - sys.stderr.write('%s: Link new branch to parent [%s]\n' % - (branch,tmp)) - link=tmp # avoid making a merge commit for branch fork - else: - # newly created branch without parent: feed full revision - full_rev=True - elif last.get(branch,revision) != parents[pidx1] and parents[pidx1] > 0 and revision > 0: - pm=get_parent_mark(parents[pidx1],marks) - sys.stderr.write('%s: Placing commit [r%d] in branch [%s] on top of [r%d]\n' % - (branch,revision,branch,parents[pidx1])); - wr('from %s' % pm) - - if parents[pidx2] > 0: - pm=get_parent_mark(parents[pidx2],marks) - sys.stderr.write('%s: Merging with parent [%s] from [r%d]\n' % - (branch,pm,parents[pidx2])) - wr('merge %s' % pm) - - last[branch]=revision - heads[branch]='' - # we need this later to write out tags - marks[str(revision)]=':%d'%(revision+1) - - ctx=repo.changectx(str(revision)) - man=ctx.manifest() - added,changed,removed,type=[],[],[],'' - - if full_rev: - # first revision: feed in full manifest - added=man.keys() - added.sort() - type='full' - elif is_merge(parents): - # later merge revision: feed in changed manifest - # for many files comparing checksums is expensive so only do it for - # merges where we really need it due to hg's revlog logic - added,changed,removed=get_filechanges(repo,revision,parents,man) - type='thorough delta' - else: - # later non-merge revision: feed in changed manifest - # if we have exactly one parent, just take the changes from the - # manifest without expensively comparing checksums - f=repo.status(repo.lookup(parents[0]),revnode)[:3] - added,changed,removed=f[1],f[0],f[2] - type='simple delta' - - sys.stderr.write('%s: Exporting %s revision %d/%d with %d/%d/%d added/changed/removed files\n' % - (branch,type,revision+1,max,len(added),len(changed),len(removed))) - - map(lambda r: wr('D %s' % r),removed) - export_file_contents(ctx,man,added) - export_file_contents(ctx,man,changed) - wr() - - return checkpoint(count) - -def export_tags(ui,repo,marks_cache,mapping_cache,count,authors): - l=repo.tagslist() - for tag,node in l: - tag=sanitize_name(tag,"tag") - # ignore latest revision - if tag=='tip': continue - # ignore tags to nodes that are missing (ie, 'in the future') - if node.encode('hex_codec') not in mapping_cache: - sys.stderr.write('Tag %s refers to unseen node %s\n' % (tag, node.encode('hex_codec'))) - continue - - rev=int(mapping_cache[node.encode('hex_codec')]) - - ref=marks_cache.get(str(rev),':%d' % (rev)) - if ref==None: - sys.stderr.write('Failed to find reference for creating tag' - ' %s at r%d\n' % (tag,rev)) - continue - sys.stderr.write('Exporting tag [%s] at [hg r%d] [git %s]\n' % (tag,rev,ref)) - wr('reset refs/tags/%s' % tag) - wr('from %s' % ref) - wr() - count=checkpoint(count) - return count - -def load_authors(filename): - cache={} - if not os.path.exists(filename): - return cache - f=open(filename,'r') - l=0 - lre=re.compile('^([^=]+)[ ]*=[ ]*(.+)$') - for line in f.readlines(): - l+=1 - m=lre.match(line) - if m==None: - sys.stderr.write('Invalid file format in [%s], line %d\n' % (filename,l)) - continue - # put key:value in cache, key without ^: - cache[m.group(1).strip()]=m.group(2).strip() - f.close() - sys.stderr.write('Loaded %d authors\n' % l) - return cache - -def verify_heads(ui,repo,cache,force): - branches=repo.branchtags() - l=[(-repo.changelog.rev(n), n, t) for t, n in branches.items()] - l.sort() - - # get list of hg's branches to verify, don't take all git has - for _,_,b in l: - b=get_branch(b) - sha1=get_git_sha1(b) - c=cache.get(b) - if sha1!=None and c!=None: - sys.stderr.write('Verifying branch [%s]\n' % b) - if sha1!=c: - sys.stderr.write('Error: Branch [%s] modified outside hg-fast-export:' - '\n%s (repo) != %s (cache)\n' % (b,sha1,c)) - if not force: return False - - # verify that branch has exactly one head - t={} - for h in repo.heads(): - (_,_,_,_,_,_,branch,_)=get_changeset(ui,repo,h) - if t.get(branch,False): - sys.stderr.write('Error: repository has at least one unnamed head: hg r%s\n' % - repo.changelog.rev(h)) - if not force: return False - t[branch]=True - - return True - -def mangle_mark(mark): - return str(int(mark)-1) - -def hg2git(repourl,m,marksfile,mappingfile,headsfile,tipfile,authors={},sob=False,force=False): - _max=int(m) - - try: - import msvcrt - msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY) - except ImportError: - pass - - marks_cache=load_cache(marksfile,mangle_mark) - mapping_cache=load_cache(mappingfile) - heads_cache=load_cache(headsfile) - state_cache=load_cache(tipfile) - - ui,repo=setup_repo(repourl) - - if not verify_heads(ui,repo,heads_cache,force): - return 1 - - try: - tip=repo.changelog.count() - except AttributeError: - tip=len(repo) - - min=int(state_cache.get('tip',0)) - max=_max - if _max<0 or max>tip: - max=tip - - for rev in range(0,max): - (revnode,_,_,_,_,_,_,_)=get_changeset(ui,repo,rev,authors) - mapping_cache[revnode.encode('hex_codec')] = str(rev) - - - c=0 - last={} - brmap={} - for rev in range(min,max): - c=export_commit(ui,repo,rev,marks_cache,mapping_cache,heads_cache,last,max,c,authors,sob,brmap) - - state_cache['tip']=max - state_cache['repo']=repourl - save_cache(tipfile,state_cache) - save_cache(mappingfile,mapping_cache) - - c=export_tags(ui,repo,marks_cache,mapping_cache,c,authors) - - sys.stderr.write('Issued %d commands\n' % c) - - return 0 - -if __name__=='__main__': - def bail(parser,opt): - sys.stderr.write('Error: No %s option given\n' % opt) - parser.print_help() - sys.exit(2) - - parser=OptionParser() - - parser.add_option("-m","--max",type="int",dest="max", - help="Maximum hg revision to import") - parser.add_option("--mapping",dest="mappingfile", - help="File to read last run's hg-to-git SHA1 mapping") - parser.add_option("--marks",dest="marksfile", - help="File to read git-fast-import's marks from") - parser.add_option("--heads",dest="headsfile", - help="File to read last run's git heads from") - parser.add_option("--status",dest="statusfile", - help="File to read status from") - parser.add_option("-r","--repo",dest="repourl", - help="URL of repo to import") - parser.add_option("-s",action="store_true",dest="sob", - default=False,help="Enable parsing Signed-off-by lines") - parser.add_option("-A","--authors",dest="authorfile", - help="Read authormap from AUTHORFILE") - parser.add_option("-f","--force",action="store_true",dest="force", - default=False,help="Ignore validation errors by force") - parser.add_option("-M","--default-branch",dest="default_branch", - help="Set the default branch") - parser.add_option("-o","--origin",dest="origin_name", - help="use <name> as namespace to track upstream") - - (options,args)=parser.parse_args() - - m=-1 - if options.max!=None: m=options.max - - if options.marksfile==None: options.marksfile = 'hg-export.marks' - if options.mappingfile==None: options.mappingfile = 'hg-export.mapping' - if options.headsfile==None: options.headsfile = 'hg-export.heads' - if options.statusfile==None: options.statusfile = 'hg-export.status' - if options.repourl==None: bail(parser,'--repo') - - a={} - if options.authorfile!=None: - a=load_authors(options.authorfile) - - if options.default_branch!=None: - set_default_branch(options.default_branch) - - if options.origin_name!=None: - set_origin_name(options.origin_name) - - sys.exit(hg2git(options.repourl,m,options.marksfile,options.mappingfile,options.headsfile, - options.statusfile,authors=a,sob=options.sob,force=options.force)) diff --git a/exporters/hg-fast-export.sh b/exporters/hg-fast-export.sh deleted file mode 100755 index a2ef9ea..0000000 --- a/exporters/hg-fast-export.sh +++ /dev/null @@ -1,100 +0,0 @@ -#!/bin/sh - -# Copyright (c) 2007, 2008 Rocco Rutte <pdmef@gmx.net> and others. -# License: MIT <http://www.opensource.org/licenses/mit-license.php> - -ROOT="`dirname $0`" -REPO="" -PFX="hg2git" -SFX_MAPPING="mapping" -SFX_MARKS="marks" -SFX_HEADS="heads" -SFX_STATE="state" -QUIET="" -PYTHON=${PYTHON:-python} - -USAGE="[--quiet] [-r <repo>] [-m <max>] [-s] [-A <file>] [-M <name>] [-o <name>]" -LONG_USAGE="Import hg repository <repo> up to either tip or <max> -If <repo> is omitted, use last hg repository as obtained from state file, -GIT_DIR/$PFX-$SFX_STATE by default. - -Note: The argument order matters. - -Options: - -m Maximum revision to import - --quiet Passed to git-fast-import(1) - -s Enable parsing Signed-off-by lines - -A Read author map from file - (Same as in git-svnimport(1) and git-cvsimport(1)) - -r Mercurial repository to import - -M Set the default branch name (default to 'master') - -o Use <name> as branch namespace to track upstream (eg 'origin') -" - -. "$(git --exec-path)/git-sh-setup" -cd_to_toplevel - -while case "$#" in 0) break ;; esac -do - case "$1" in - -r|--r|--re|--rep|--repo) - shift - REPO="$1" - ;; - --q|--qu|--qui|--quie|--quiet) - QUIET="--quiet" - ;; - -*) - # pass any other options down to hg2git.py - break - ;; - *) - break - ;; - esac - shift -done - -# for convenience: get default repo from state file -if [ x"$REPO" = x -a -f "$GIT_DIR/$PFX-$SFX_STATE" ] ; then - REPO="`egrep '^:repo ' "$GIT_DIR/$PFX-$SFX_STATE" | cut -d ' ' -f 2`" - echo "Using last hg repository \"$REPO\"" -fi - -# make sure we have a marks cache -if [ ! -f "$GIT_DIR/$PFX-$SFX_MARKS" ] ; then - touch "$GIT_DIR/$PFX-$SFX_MARKS" -fi - -GIT_DIR="$GIT_DIR" $PYTHON "$ROOT/hg-fast-export.py" \ - --repo "$REPO" \ - --marks "$GIT_DIR/$PFX-$SFX_MARKS" \ - --mapping "$GIT_DIR/$PFX-$SFX_MAPPING" \ - --heads "$GIT_DIR/$PFX-$SFX_HEADS" \ - --status "$GIT_DIR/$PFX-$SFX_STATE" \ - "$@" \ -| git fast-import $QUIET --export-marks="$GIT_DIR/$PFX-$SFX_MARKS.tmp" - -# move recent marks cache out of the way... -if [ -f "$GIT_DIR/$PFX-$SFX_MARKS" ] ; then - mv "$GIT_DIR/$PFX-$SFX_MARKS" "$GIT_DIR/$PFX-$SFX_MARKS.old" -else - touch "$GIT_DIR/$PFX-$SFX_MARKS.old" -fi - -# ...to create a new merged one -cat "$GIT_DIR/$PFX-$SFX_MARKS.old" "$GIT_DIR/$PFX-$SFX_MARKS.tmp" \ -| uniq > "$GIT_DIR/$PFX-$SFX_MARKS" - -# cleanup -rm -rf "$GIT_DIR/$PFX-$SFX_MARKS.old" "$GIT_DIR/$PFX-$SFX_MARKS.tmp" - -# save SHA1s of current heads for incremental imports -# and connectivity (plus sanity checking) -for head in `git branch | sed 's#^..##'` ; do - id="`git rev-parse $head`" - echo ":$head $id" -done > "$GIT_DIR/$PFX-$SFX_HEADS" - -# check diff with color: -# ( for i in `find . -type f | grep -v '\.git'` ; do diff -u $i $REPO/$i ; done | cdiff ) | less -r diff --git a/exporters/hg2git.py b/exporters/hg2git.py deleted file mode 100755 index 3c0d1e3..0000000 --- a/exporters/hg2git.py +++ /dev/null @@ -1,113 +0,0 @@ -#!/usr/bin/env python - -# Copyright (c) 2007, 2008 Rocco Rutte <pdmef@gmx.net> and others. -# License: MIT <http://www.opensource.org/licenses/mit-license.php> - -from mercurial import repo,hg,cmdutil,util,ui,revlog,node -import re -import os -import sys - -# default git branch name -cfg_master='master' -# default origin name -origin_name='' -# silly regex to see if user field has email address -user_re=re.compile('([^<]+) (<[^>]+>)$') -# silly regex to clean out user names -user_clean_re=re.compile('^["]([^"]+)["]$') - -def set_default_branch(name): - global cfg_master - cfg_master = name - -def set_origin_name(name): - global origin_name - origin_name = name - -def setup_repo(url): - try: - myui=ui.ui(interactive=False) - except TypeError: - myui=ui.ui() - myui.setconfig('ui', 'interactive', 'off') - return myui,hg.repository(myui,url) - -def fixup_user(user,authors): - if authors!=None: - # if we have an authors table, try to get mapping - # by defaulting to the current value of 'user' - user=authors.get(user,user) - name,mail,m='','',user_re.match(user) - if m==None: - # if we don't have 'Name <mail>' syntax, use 'user - # <devnull@localhost>' if use contains no at and - # 'user <user>' otherwise - name=user - if '@' not in user: - mail='<devnull@localhost>' - else: - mail='<%s>' % user - else: - # if we have 'Name <mail>' syntax, everything is fine :) - name,mail=m.group(1),m.group(2) - - # remove any silly quoting from username - m2=user_clean_re.match(name) - if m2!=None: - name=m2.group(1) - return '%s %s' % (name,mail) - -def get_branch(name): - # 'HEAD' is the result of a bug in mutt's cvs->hg conversion, - # other CVS imports may need it, too - if name=='HEAD' or name=='default' or name=='': - name=cfg_master - if origin_name: - return origin_name + '/' + name - return name - -def get_changeset(ui,repo,revision,authors={}): - node=repo.lookup(revision) - (manifest,user,(time,timezone),files,desc,extra)=repo.changelog.read(node) - tz="%+03d%02d" % (-timezone / 3600, ((-timezone % 3600) / 60)) - branch=get_branch(extra.get('branch','master')) - return (node,manifest,fixup_user(user,authors),(time,tz),files,desc,branch,extra) - -def mangle_key(key): - return key - -def load_cache(filename,get_key=mangle_key): - cache={} - if not os.path.exists(filename): - return cache - f=open(filename,'r') - l=0 - for line in f.readlines(): - l+=1 - fields=line.split(' ') - if fields==None or not len(fields)==2 or fields[0][0]!=':': - sys.stderr.write('Invalid file format in [%s], line %d\n' % (filename,l)) - continue - # put key:value in cache, key without ^: - cache[get_key(fields[0][1:])]=fields[1].split('\n')[0] - f.close() - return cache - -def save_cache(filename,cache): - f=open(filename,'w+') - map(lambda x: f.write(':%s %s\n' % (str(x),str(cache.get(x)))),cache.keys()) - f.close() - -def get_git_sha1(name,type='heads'): - try: - # use git-rev-parse to support packed refs - cmd="GIT_DIR='%s' git rev-parse --verify refs/%s/%s 2>/dev/null" % (os.getenv('GIT_DIR','/dev/null'),type,name) - p=os.popen(cmd) - l=p.readline() - p.close() - if l == None or len(l) == 0: - return None - return l[0:40] - except IOError: - return None diff --git a/exporters/svn-archive.c b/exporters/svn-archive.c deleted file mode 100644 index 6632d19..0000000 --- a/exporters/svn-archive.c +++ /dev/null @@ -1,240 +0,0 @@ -/* - * svn-archive.c - * ---------- - * Walk through a given revision of a local Subversion repository and export - * all of the contents as a tarfile. - * - * Author: Chris Lee <clee@kde.org> - * License: MIT <http://www.opensource.org/licenses/mit-license.php> - */ - -#define _XOPEN_SOURCE -#include <unistd.h> -#include <string.h> -#include <stdio.h> -#include <time.h> - -#ifndef PATH_MAX -#define PATH_MAX 4096 -#endif - -#include <apr_general.h> -#include <apr_strings.h> -#include <apr_getopt.h> -#include <apr_lib.h> - -#include <svn_types.h> -#include <svn_pools.h> -#include <svn_repos.h> -#include <svn_fs.h> - -#undef SVN_ERR -#define SVN_ERR(expr) SVN_INT_ERR(expr) -#define apr_sane_push(arr, contents) *(char **)apr_array_push(arr) = contents - -#define TRUNK "/trunk" - -static time_t archive_time; - -time_t get_epoch(char *svn_date) -{ - struct tm tm = {0}; - char *date = malloc(strlen(svn_date) * sizeof(char *)); - strncpy(date, svn_date, strlen(svn_date) - 8); - strptime(date, "%Y-%m-%dT%H:%M:%S", &tm); - free(date); - return mktime(&tm); -} - -int tar_header(apr_pool_t *pool, char *path, char *node, size_t f_size) -{ - char buf[512]; - unsigned int i, checksum; - svn_boolean_t is_dir; - - memset(buf, 0, sizeof(buf)); - - if ((strlen(path) == 0) && (strlen(node) == 0)) { - return 0; - } - - if (strlen(node) == 0) { - is_dir = 1; - } else { - is_dir = 0; - } - - if (strlen(path) == 0) { - strncpy(buf, apr_psprintf(pool, "%s", node), 99); - } else if (strlen(path) + strlen(node) < 100) { - strncpy(buf, apr_psprintf(pool, "%s/%s", path+1, node), 99); - } else { - fprintf(stderr, "really long file path...\n"); - strncpy(&buf[0], node, 99); - strncpy(&buf[345], path+1, 154); - } - - strncpy(&buf[100], apr_psprintf(pool, "%07o", (is_dir ? 0755 : 0644)), 7); - strncpy(&buf[108], apr_psprintf(pool, "%07o", 1000), 7); - strncpy(&buf[116], apr_psprintf(pool, "%07o", 1000), 7); - strncpy(&buf[124], apr_psprintf(pool, "%011lo", f_size), 11); - strncpy(&buf[136], apr_psprintf(pool, "%011lo", archive_time), 11); - strncpy(&buf[156], (is_dir ? "5" : "0"), 1); - strncpy(&buf[257], "ustar ", 8); - strncpy(&buf[265], "clee", 31); - strncpy(&buf[297], "clee", 31); - // strncpy(&buf[329], apr_psprintf(pool, "%07o", 0), 7); - // strncpy(&buf[337], apr_psprintf(pool, "%07o", 0), 7); - - strncpy(&buf[148], " ", 8); - checksum = 0; - for (i = 0; i < sizeof(buf); i++) { - checksum += buf[i]; - } - strncpy(&buf[148], apr_psprintf(pool, "%07o", checksum & 0x1fffff), 7); - - fwrite(buf, sizeof(char), sizeof(buf), stdout); - - return 0; -} - -int tar_footer() -{ - char block[1024]; - memset(block, 0, sizeof(block)); - fwrite(block, sizeof(char), sizeof(block), stdout); -} - -int dump_blob(svn_fs_root_t *root, char *prefix, char *path, char *node, apr_pool_t *pool) -{ - char *full_path, buf[512]; - apr_size_t len; - svn_stream_t *stream; - svn_filesize_t stream_length; - - full_path = apr_psprintf(pool, "%s%s/%s", prefix, path, node); - - SVN_ERR(svn_fs_file_length(&stream_length, root, full_path, pool)); - SVN_ERR(svn_fs_file_contents(&stream, root, full_path, pool)); - - tar_header(pool, path, node, stream_length); - - do { - len = sizeof(buf); - memset(buf, '\0', sizeof(buf)); - SVN_ERR(svn_stream_read(stream, buf, &len)); - fwrite(buf, sizeof(char), sizeof(buf), stdout); - } while (len == sizeof(buf)); - - return 0; -} - -int dump_tree(svn_fs_root_t *root, char *prefix, char *path, apr_pool_t *pool) -{ - const void *key; - void *val; - char *node, *subpath, *full_path; - - apr_pool_t *subpool; - apr_hash_t *dir_entries; - apr_hash_index_t *i; - - svn_boolean_t is_dir; - - tar_header(pool, path, "", 0); - - SVN_ERR(svn_fs_dir_entries(&dir_entries, root, apr_psprintf(pool, "%s/%s", prefix, path), pool)); - - subpool = svn_pool_create(pool); - - for (i = apr_hash_first(pool, dir_entries); i; i = apr_hash_next(i)) { - svn_pool_clear(subpool); - apr_hash_this(i, &key, NULL, &val); - node = (char *)key; - - subpath = apr_psprintf(subpool, "%s/%s", path, node); - full_path = apr_psprintf(subpool, "%s%s", prefix, subpath); - - svn_fs_is_dir(&is_dir, root, full_path, subpool); - - if (is_dir) { - dump_tree(root, prefix, subpath, subpool); - } else { - dump_blob(root, prefix, path, node, subpool); - } - } - - svn_pool_destroy(subpool); - - return 0; -} - -int crawl_filesystem(char *repos_path, char *root_path, apr_pool_t *pool) -{ - char *path; - - apr_hash_t *props; - apr_hash_index_t *i; - - svn_repos_t *repos; - svn_fs_t *fs; - svn_string_t *svndate; - svn_revnum_t youngest_rev, export_rev; - svn_fs_root_t *fs_root; - - SVN_ERR(svn_fs_initialize(pool)); - SVN_ERR(svn_repos_open(&repos, repos_path, pool)); - if ((fs = svn_repos_fs(repos)) == NULL) - return -1; - SVN_ERR(svn_fs_youngest_rev(&youngest_rev, fs, pool)); - - export_rev = youngest_rev; - - SVN_ERR(svn_fs_revision_root(&fs_root, fs, export_rev, pool)); - SVN_ERR(svn_fs_revision_proplist(&props, fs, export_rev, pool)); - - svndate = apr_hash_get(props, "svn:date", APR_HASH_KEY_STRING); - archive_time = get_epoch((char *)svndate->data); - - fprintf(stderr, "Exporting archive of r%ld... \n", export_rev); - - dump_tree(fs_root, root_path, "", pool); - - tar_footer(); - - fprintf(stderr, "done!\n"); - - return 0; -} - -int main(int argc, char *argv[]) -{ - apr_pool_t *pool; - apr_getopt_t *options; - - apr_getopt_option_t long_options[] = { - { "help", 'h', 0 }, - { "prefix", 'p', 0 }, - { "basename", 'b', 0 }, - { "revision", 'r', 0 }, - { NULL, 0, 0 } - }; - - if (argc < 2) { - fprintf(stderr, "usage: %s REPOS_PATH [prefix]\n", argv[0]); - return -1; - } - - if (apr_initialize() != APR_SUCCESS) { - fprintf(stderr, "You lose at apr_initialize().\n"); - return -1; - } - - pool = svn_pool_create(NULL); - - crawl_filesystem(argv[1], (argc == 3 ? argv[2] : TRUNK), pool); - - apr_terminate(); - - return 0; -} diff --git a/exporters/svn-fast-export.README b/exporters/svn-fast-export.README deleted file mode 100644 index e08277e..0000000 --- a/exporters/svn-fast-export.README +++ /dev/null @@ -1,12 +0,0 @@ -To compile svn-fast-export.c, use make. You'll need to install -some packages first using the package manager on your OS: - -* libsvn-dev - the Subversion libraries -* libapr1-dev - the Apache Portable Runtime libraries - -Note: If someone with good knowledge of the Subversion -Python bindings could rewrite svn-fast-export.py so that -https://bugs.launchpad.net/bzr-fastimport/+bug/273361 -went away, then there would be much rejoicing throughout -the land and the need for svn-fast-export.c would largely -disappear. diff --git a/exporters/svn-fast-export.c b/exporters/svn-fast-export.c deleted file mode 100644 index 34e7fc6..0000000 --- a/exporters/svn-fast-export.c +++ /dev/null @@ -1,187 +0,0 @@ -/* - * svn-fast-export.c - * ---------- - * Walk through each revision of a local Subversion repository and export it - * in a stream that git-fast-import can consume. - * - * Author: Chris Lee <clee@kde.org> - * License: MIT <http://www.opensource.org/licenses/mit-license.php> - */ - -#define _XOPEN_SOURCE -#include <unistd.h> -#include <string.h> -#include <stdio.h> -#include <time.h> - -#ifndef PATH_MAX -#define PATH_MAX 4096 -#endif - -#include <apr_lib.h> -#include <apr_getopt.h> -#include <apr_general.h> - -#include <svn_fs.h> -#include <svn_repos.h> -#include <svn_pools.h> -#include <svn_types.h> - -#undef SVN_ERR -#define SVN_ERR(expr) SVN_INT_ERR(expr) -#define apr_sane_push(arr, contents) *(char **)apr_array_push(arr) = contents - -#define TRUNK "/trunk/" - -time_t get_epoch(char *svn_date) -{ - struct tm tm = {0}; - char *date = malloc(strlen(svn_date) * sizeof(char *)); - strncpy(date, svn_date, strlen(svn_date) - 8); - strptime(date, "%Y-%m-%dT%H:%M:%S", &tm); - free(date); - return mktime(&tm); -} - -int dump_blob(svn_fs_root_t *root, char *full_path, apr_pool_t *pool) -{ - apr_size_t len; - svn_stream_t *stream, *outstream; - svn_filesize_t stream_length; - - SVN_ERR(svn_fs_file_length(&stream_length, root, full_path, pool)); - SVN_ERR(svn_fs_file_contents(&stream, root, full_path, pool)); - - fprintf(stdout, "data %lu\n", stream_length); - fflush(stdout); - - SVN_ERR(svn_stream_for_stdout(&outstream, pool)); - SVN_ERR(svn_stream_copy(stream, outstream, pool)); - - fprintf(stdout, "\n"); - fflush(stdout); - - return 0; -} - -int export_revision(svn_revnum_t rev, svn_fs_t *fs, apr_pool_t *pool) -{ - unsigned int mark; - const void *key; - void *val; - char *path, *file_change; - apr_pool_t *revpool; - apr_hash_t *changes, *props; - apr_hash_index_t *i; - apr_array_header_t *file_changes; - svn_string_t *author, *committer, *svndate, *svnlog; - svn_boolean_t is_dir; - svn_fs_root_t *fs_root; - svn_fs_path_change_t *change; - - fprintf(stderr, "Exporting revision %ld... ", rev); - - SVN_ERR(svn_fs_revision_root(&fs_root, fs, rev, pool)); - SVN_ERR(svn_fs_paths_changed(&changes, fs_root, pool)); - SVN_ERR(svn_fs_revision_proplist(&props, fs, rev, pool)); - - revpool = svn_pool_create(pool); - - file_changes = apr_array_make(pool, apr_hash_count(changes), sizeof(char *)); - mark = 1; - for (i = apr_hash_first(pool, changes); i; i = apr_hash_next(i)) { - svn_pool_clear(revpool); - apr_hash_this(i, &key, NULL, &val); - path = (char *)key; - change = (svn_fs_path_change_t *)val; - - SVN_ERR(svn_fs_is_dir(&is_dir, fs_root, path, revpool)); - - if (is_dir || strncmp(TRUNK, path, strlen(TRUNK))) { - continue; - } - - if (change->change_kind == svn_fs_path_change_delete) { - apr_sane_push(file_changes, (char *)svn_string_createf(pool, "D %s", path + strlen(TRUNK))->data); - } else { - apr_sane_push(file_changes, (char *)svn_string_createf(pool, "M 644 :%u %s", mark, path + strlen(TRUNK))->data); - fprintf(stdout, "blob\nmark :%u\n", mark++); - dump_blob(fs_root, (char *)path, revpool); - } - } - - if (file_changes->nelts == 0) { - fprintf(stderr, "skipping.\n"); - svn_pool_destroy(revpool); - return 0; - } - - author = apr_hash_get(props, "svn:author", APR_HASH_KEY_STRING); - if (svn_string_isempty(author)) - author = svn_string_create("nobody", pool); - svndate = apr_hash_get(props, "svn:date", APR_HASH_KEY_STRING); - svnlog = apr_hash_get(props, "svn:log", APR_HASH_KEY_STRING); - - fprintf(stdout, "commit refs/heads/master\n"); - fprintf(stdout, "committer %s <%s@localhost> %ld -0000\n", author->data, author->data, get_epoch((char *)svndate->data)); - fprintf(stdout, "data %d\n", svnlog->len); - fputs(svnlog->data, stdout); - fprintf(stdout, "\n"); - fputs(apr_array_pstrcat(pool, file_changes, '\n'), stdout); - fprintf(stdout, "\n\n"); - fflush(stdout); - - svn_pool_destroy(revpool); - - fprintf(stderr, "done!\n"); - - return 0; -} - -int crawl_revisions(char *repos_path) -{ - apr_pool_t *pool, *subpool; - svn_fs_t *fs; - svn_repos_t *repos; - svn_revnum_t youngest_rev, min_rev, max_rev, rev; - - pool = svn_pool_create(NULL); - - SVN_ERR(svn_fs_initialize(pool)); - SVN_ERR(svn_repos_open(&repos, repos_path, pool)); - if ((fs = svn_repos_fs(repos)) == NULL) - return -1; - SVN_ERR(svn_fs_youngest_rev(&youngest_rev, fs, pool)); - - min_rev = 1; - max_rev = youngest_rev; - - subpool = svn_pool_create(pool); - for (rev = min_rev; rev <= max_rev; rev++) { - svn_pool_clear(subpool); - export_revision(rev, fs, subpool); - } - - svn_pool_destroy(pool); - - return 0; -} - -int main(int argc, char *argv[]) -{ - if (argc != 2) { - fprintf(stderr, "usage: %s REPOS_PATH\n", argv[0]); - return -1; - } - - if (apr_initialize() != APR_SUCCESS) { - fprintf(stderr, "You lose at apr_initialize().\n"); - return -1; - } - - crawl_revisions(argv[1]); - - apr_terminate(); - - return 0; -} diff --git a/exporters/svn-fast-export.py b/exporters/svn-fast-export.py deleted file mode 100755 index fd88094..0000000 --- a/exporters/svn-fast-export.py +++ /dev/null @@ -1,225 +0,0 @@ -#!/usr/bin/python -# -# svn-fast-export.py -# ---------- -# Walk through each revision of a local Subversion repository and export it -# in a stream that git-fast-import can consume. -# -# Author: Chris Lee <clee@kde.org> -# License: MIT <http://www.opensource.org/licenses/mit-license.php> - -trunk_path = '/trunk/' -branches_path = '/branches/' -tags_path = '/tags/' -address = 'localhost' - -first_rev = 1 -final_rev = 0 - -import gc, sys, os.path -from optparse import OptionParser -from time import sleep, mktime, localtime, strftime, strptime -from svn.fs import svn_fs_dir_entries, svn_fs_file_length, svn_fs_file_contents, svn_fs_is_dir, svn_fs_revision_root, svn_fs_youngest_rev, svn_fs_revision_proplist, svn_fs_revision_prop, svn_fs_paths_changed -from svn.core import svn_pool_create, svn_pool_clear, svn_pool_destroy, svn_stream_read, svn_stream_for_stdout, svn_stream_copy, svn_stream_close, run_app -from svn.repos import svn_repos_open, svn_repos_fs - -ct_short = ['M', 'A', 'D', 'R', 'X'] - -def dump_file_blob(root, full_path, pool): - stream_length = svn_fs_file_length(root, full_path, pool) - stream = svn_fs_file_contents(root, full_path, pool) - sys.stdout.write("data %s\n" % stream_length) - sys.stdout.flush() - ostream = svn_stream_for_stdout(pool) - svn_stream_copy(stream, ostream, pool) - svn_stream_close(ostream) - sys.stdout.write("\n") - - -class Matcher(object): - - branch = None - - def __init__(self, trunk_path): - self.trunk_path = trunk_path - - def branchname(self): - return self.branch - - def __str__(self): - return super(Matcher, self).__str__() + ":" + self.trunk_path - - @staticmethod - def getMatcher(trunk_path): - if trunk_path.startswith("regex:"): - return RegexStringMatcher(trunk_path) - else: - return StaticStringMatcher(trunk_path) - -class StaticStringMatcher(Matcher): - - branch = "master" - - def matches(self, path): - return path.startswith(trunk_path) - - def replace(self, path): - return path.replace(self.trunk_path, '') - -class RegexStringMatcher(Matcher): - - def __init__(self, trunk_path): - super(RegexStringMatcher, self).__init__(trunk_path) - import re - self.matcher = re.compile(self.trunk_path[len("regex:"):]) - - def matches(self, path): - match = self.matcher.match(path) - if match: - self.branch = match.group(1) - return True - else: - return False - - def replace(self, path): - return self.matcher.sub("\g<2>", path) - -MATCHER = None - -def export_revision(rev, repo, fs, pool): - sys.stderr.write("Exporting revision %s... " % rev) - - revpool = svn_pool_create(pool) - svn_pool_clear(revpool) - - # Open a root object representing the youngest (HEAD) revision. - root = svn_fs_revision_root(fs, rev, revpool) - - # And the list of what changed in this revision. - changes = svn_fs_paths_changed(root, revpool) - - i = 1 - marks = {} - file_changes = [] - - for path, change_type in changes.iteritems(): - c_t = ct_short[change_type.change_kind] - if svn_fs_is_dir(root, path, revpool): - continue - if not MATCHER.matches(path): - # We don't handle branches. Or tags. Yet. - pass - else: - if c_t == 'D': - file_changes.append("D %s" % MATCHER.replace(path)) - else: - marks[i] = MATCHER.replace(path) - file_changes.append("M 644 :%s %s" % (i, marks[i])) - sys.stdout.write("blob\nmark :%s\n" % i) - dump_file_blob(root, path, revpool) - i += 1 - - # Get the commit author and message - props = svn_fs_revision_proplist(fs, rev, revpool) - - # Do the recursive crawl. - if props.has_key('svn:author'): - author = "%s <%s@%s>" % (props['svn:author'], props['svn:author'], address) - else: - author = 'nobody <nobody@users.sourceforge.net>' - - if len(file_changes) == 0: - svn_pool_destroy(revpool) - sys.stderr.write("skipping.\n") - return - - svndate = props['svn:date'][0:-8] - commit_time = mktime(strptime(svndate, '%Y-%m-%dT%H:%M:%S')) - sys.stdout.write("commit refs/heads/%s\n" % MATCHER.branchname()) - sys.stdout.write("committer %s %s -0000\n" % (author, int(commit_time))) - sys.stdout.write("data %s\n" % len(props['svn:log'])) - sys.stdout.write(props['svn:log']) - sys.stdout.write("\n") - sys.stdout.write('\n'.join(file_changes)) - sys.stdout.write("\n\n") - - svn_pool_destroy(revpool) - - sys.stderr.write("done!\n") - - #if rev % 1000 == 0: - # sys.stderr.write("gc: %s objects\n" % len(gc.get_objects())) - # sleep(5) - - -def crawl_revisions(pool, repos_path): - """Open the repository at REPOS_PATH, and recursively crawl all its - revisions.""" - global final_rev - - # Open the repository at REPOS_PATH, and get a reference to its - # versioning filesystem. - repos_obj = svn_repos_open(repos_path, pool) - fs_obj = svn_repos_fs(repos_obj) - - # Query the current youngest revision. - youngest_rev = svn_fs_youngest_rev(fs_obj, pool) - - - if final_rev == 0: - final_rev = youngest_rev - for rev in xrange(first_rev, final_rev + 1): - export_revision(rev, repos_obj, fs_obj, pool) - - -if __name__ == '__main__': - usage = '%prog [options] REPOS_PATH' - parser = OptionParser() - parser.set_usage(usage) - parser.add_option('-f', '--final-rev', help='Final revision to import', - dest='final_rev', metavar='FINAL_REV', type='int') - parser.add_option('-r', '--first-rev', help='First revision to import', - dest='first_rev', metavar='FIRST_REV', type='int') - parser.add_option('-t', '--trunk-path', help="Path in repo to /trunk, may be `regex:/cvs/(trunk)/proj1/(.*)`\nFirst group is used as branchname, second to match files", - dest='trunk_path', metavar='TRUNK_PATH') - parser.add_option('-b', '--branches-path', help='Path in repo to /branches', - dest='branches_path', metavar='BRANCHES_PATH') - parser.add_option('-T', '--tags-path', help='Path in repo to /tags', - dest='tags_path', metavar='TAGS_PATH') - parser.add_option('-a', '--address', help='Domain to put on users for their mail address', - dest='address', metavar='hostname', type='string') - (options, args) = parser.parse_args() - - if options.trunk_path != None: - trunk_path = options.trunk_path - if options.branches_path != None: - branches_path = options.branches_path - if options.tags_path != None: - tags_path = options.tags_path - if options.final_rev != None: - final_rev = options.final_rev - if options.first_rev != None: - first_rev = options.first_rev - if options.address != None: - address = options.address - - MATCHER = Matcher.getMatcher(trunk_path) - sys.stderr.write("%s\n" % MATCHER) - if len(args) != 1: - parser.print_help() - sys.exit(2) - - # Canonicalize (enough for Subversion, at least) the repository path. - repos_path = os.path.normpath(args[0]) - if repos_path == '.': - repos_path = '' - - try: - import msvcrt - msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY) - except ImportError: - pass - - # Call the app-wrapper, which takes care of APR initialization/shutdown - # and the creation and cleanup of our top-level memory pool. - run_app(crawl_revisions, repos_path) diff --git a/fastimport/dates.py b/fastimport/dates.py index 510ab85..ba484ef 100644 --- a/fastimport/dates.py +++ b/fastimport/dates.py @@ -25,7 +25,7 @@ Each routine returns timestamp,timezone where import time -from bzrlib.plugins.fastimport.fastimport import errors +from fastimport import errors def parse_raw(s, lineno=0): diff --git a/fastimport/errors.py b/fastimport/errors.py index 9a71d77..f2d5d88 100644 --- a/fastimport/errors.py +++ b/fastimport/errors.py @@ -16,18 +16,73 @@ """Exception classes for fastimport""" -from bzrlib import errors as bzr_errors - - # Prefix to messages to show location information _LOCATION_FMT = "line %(lineno)d: " +# ImportError is heavily based on BzrError -class ImportError(bzr_errors.BzrError): +class ImportError(StandardError): """The base exception class for all import processing exceptions.""" _fmt = "Unknown Import Error" + def __init__(self, msg=None, **kwds): + StandardError.__init__(self) + if msg is not None: + self._preformatted_string = msg + else: + self._preformatted_string = None + for key, value in kwds.items(): + setattr(self, key, value) + + def _format(self): + s = getattr(self, '_preformatted_string', None) + if s is not None: + # contains a preformatted message + return s + try: + fmt = self._fmt + if fmt: + d = dict(self.__dict__) + s = fmt % d + # __str__() should always return a 'str' object + # never a 'unicode' object. + return s + except (AttributeError, TypeError, NameError, ValueError, KeyError), e: + return 'Unprintable exception %s: dict=%r, fmt=%r, error=%r' \ + % (self.__class__.__name__, + self.__dict__, + getattr(self, '_fmt', None), + e) + + def __unicode__(self): + u = self._format() + if isinstance(u, str): + # Try decoding the str using the default encoding. + u = unicode(u) + elif not isinstance(u, unicode): + # Try to make a unicode object from it, because __unicode__ must + # return a unicode object. + u = unicode(u) + return u + + def __str__(self): + s = self._format() + if isinstance(s, unicode): + s = s.encode('utf8') + else: + # __str__ must return a str. + s = str(s) + return s + + def __repr__(self): + return '%s(%s)' % (self.__class__.__name__, str(self)) + + def __eq__(self, other): + if self.__class__ is not other.__class__: + return NotImplemented + return self.__dict__ == other.__dict__ + class ParsingError(ImportError): """The base exception class for all import processing exceptions.""" diff --git a/fastimport/processor.py b/fastimport/processor.py index 74f7183..7811a20 100644 --- a/fastimport/processor.py +++ b/fastimport/processor.py @@ -35,7 +35,7 @@ import errors class ImportProcessor(object): """Base class for import processors. - + Subclasses should override the pre_*, post_* and *_handler methods as appropriate. """ diff --git a/fastimport/tests/test_commands.py b/fastimport/tests/test_commands.py index 6efa4ce..ab44856 100644 --- a/fastimport/tests/test_commands.py +++ b/fastimport/tests/test_commands.py @@ -18,7 +18,7 @@ from testtools import TestCase -from bzrlib.plugins.fastimport.fastimport import ( +from fastimport import ( commands, ) diff --git a/fastimport/tests/test_errors.py b/fastimport/tests/test_errors.py index e3b807c..2b6b69d 100644 --- a/fastimport/tests/test_errors.py +++ b/fastimport/tests/test_errors.py @@ -18,7 +18,7 @@ from testtools import TestCase -from bzrlib.plugins.fastimport.fastimport import ( +from fastimport import ( errors, ) diff --git a/helpers.py b/helpers.py deleted file mode 100644 index 76d5838..0000000 --- a/helpers.py +++ /dev/null @@ -1,146 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Miscellaneous useful stuff.""" - -from bzrlib.plugins.fastimport.fastimport.helpers import ( - common_path, - ) - - -def common_directory(paths): - """Find the deepest common directory of a list of paths. - - :return: if no paths are provided, None is returned; - if there is no common directory, '' is returned; - otherwise the common directory with a trailing / is returned. - """ - from bzrlib import osutils - def get_dir_with_slash(path): - if path == '' or path.endswith('/'): - return path - else: - dirname, basename = osutils.split(path) - if dirname == '': - return dirname - else: - return dirname + '/' - - if not paths: - return None - elif len(paths) == 1: - return get_dir_with_slash(paths[0]) - else: - common = common_path(paths[0], paths[1]) - for path in paths[2:]: - common = common_path(common, path) - return get_dir_with_slash(common) - - -def escape_commit_message(message): - """Replace xml-incompatible control characters.""" - # This really ought to be provided by bzrlib. - # Code copied from bzrlib.commit. - - # Python strings can include characters that can't be - # represented in well-formed XML; escape characters that - # aren't listed in the XML specification - # (http://www.w3.org/TR/REC-xml/#NT-Char). - import re - message, _ = re.subn( - u'[^\x09\x0A\x0D\u0020-\uD7FF\uE000-\uFFFD]+', - lambda match: match.group(0).encode('unicode_escape'), - message) - return message - - -def best_format_for_objects_in_a_repository(repo): - """Find the high-level format for branches and trees given a repository. - - When creating branches and working trees within a repository, Bazaar - defaults to using the default format which may not be the best choice. - This routine does a reverse lookup of the high-level format registry - to find the high-level format that a shared repository was most likely - created via. - - :return: the BzrDirFormat or None if no matches were found. - """ - # Based on code from bzrlib/info.py ... - from bzrlib import bzrdir - repo_format = repo._format - candidates = [] - non_aliases = set(bzrdir.format_registry.keys()) - non_aliases.difference_update(bzrdir.format_registry.aliases()) - for key in non_aliases: - format = bzrdir.format_registry.make_bzrdir(key) - # LocalGitBzrDirFormat has no repository_format - if hasattr(format, "repository_format"): - if format.repository_format == repo_format: - candidates.append((key, format)) - if len(candidates): - # Assume the first one. Is there any reason not to do that? - name, format = candidates[0] - return format - else: - return None - - -def open_destination_directory(location, format=None, verbose=True): - """Open a destination directory and return the BzrDir. - - If destination has a control directory, it will be returned. - Otherwise, the destination should be empty or non-existent and - a shared repository will be created there. - - :param location: the destination directory - :param format: the format to use or None for the default - :param verbose: display the format used if a repository is created. - :return: BzrDir for the destination - """ - import os - from bzrlib import bzrdir, errors, trace, transport - try: - control, relpath = bzrdir.BzrDir.open_containing(location) - # XXX: Check the relpath is None here? - return control - except errors.NotBranchError: - pass - - # If the directory exists, check it is empty. Otherwise create it. - if os.path.exists(location): - contents = os.listdir(location) - if contents: - errors.BzrCommandError("Destination must have a .bzr directory, " - " not yet exist or be empty - files found in %s" % (location,)) - else: - try: - os.mkdir(location) - except IOError, ex: - errors.BzrCommandError("Unable to create %s: %s" % - (location, ex)) - - # Create a repository for the nominated format. - trace.note("Creating destination repository ...") - if format is None: - format = bzrdir.format_registry.make_bzrdir('default') - to_transport = transport.get_transport(location) - to_transport.ensure_base() - control = format.initialize_on_transport(to_transport) - repo = control.create_repository(shared=True) - if verbose: - from bzrlib.info import show_bzrdir_info - show_bzrdir_info(repo.bzrdir, verbose=0) - return control diff --git a/marks_file.py b/marks_file.py deleted file mode 100644 index 96e3cab..0000000 --- a/marks_file.py +++ /dev/null @@ -1,95 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Routines for reading/writing a marks file.""" - - -import re -from bzrlib.trace import warning - - -def import_marks(filename): - """Read the mapping of marks to revision-ids from a file. - - :param filename: the file to read from - :return: None if an error is encountered or (revision_ids, branch_names) - where - * revision_ids is a dictionary with marks as keys and revision-ids - as values - * branch_names is a dictionary mapping branch names to some magic # - """ - # Check that the file is readable and in the right format - try: - f = file(filename) - except IOError: - warning("Could not import marks file %s - not importing marks", - filename) - return None - firstline = f.readline() - match = re.match(r'^format=(\d+)$', firstline) - if not match: - warning("%r doesn't look like a marks file - not importing marks", - filename) - return None - elif match.group(1) != '1': - warning('format version in marks file %s not supported - not importing' - 'marks', filename) - return None - - # Read the branch info - branch_names = {} - for string in f.readline().rstrip('\n').split('\0'): - if not string: - continue - name, integer = string.rsplit('.', 1) - branch_names[name] = int(integer) - - # Read the revision info - revision_ids = {} - for line in f: - line = line.rstrip('\n') - mark, revid = line.split(' ', 1) - revision_ids[mark] = revid - f.close() - return (revision_ids, branch_names) - - -def export_marks(filename, revision_ids, branch_names=None): - """Save marks to a file. - - :param filename: filename to save data to - :param revision_ids: dictionary mapping marks -> bzr revision-ids - :param branch_names: dictionary mapping branch names to some magic # - """ - try: - f = file(filename, 'w') - except IOError: - warning("Could not open export-marks file %s - not exporting marks", - filename) - return - f.write('format=1\n') - - # Write the branch names line - if branch_names: - branch_names = [ '%s.%d' % x for x in branch_names.iteritems() ] - f.write('\0'.join(branch_names) + '\n') - else: - f.write('\0tmp.0\n') - - # Write the revision info - for mark, revid in revision_ids.iteritems(): - f.write('%s %s\n' % (mark, revid)) - f.close() diff --git a/processors/__init__.py b/processors/__init__.py deleted file mode 100644 index 8a16d9c..0000000 --- a/processors/__init__.py +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Actual import processors.""" diff --git a/processors/filter_processor.py b/processors/filter_processor.py deleted file mode 100644 index ebec5af..0000000 --- a/processors/filter_processor.py +++ /dev/null @@ -1,300 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Import processor that filters the input (and doesn't import).""" - - -from bzrlib import osutils -from bzrlib.trace import ( - warning, - ) -from bzrlib.plugins.fastimport.fastimport import ( - commands, - processor, - ) -from bzrlib.plugins.fastimport import ( - helpers, - ) - - -class FilterProcessor(processor.ImportProcessor): - """An import processor that filters the input to include/exclude objects. - - No changes to the current repository are made. - - Here are the supported parameters: - - * include_paths - a list of paths that commits must change in order to - be kept in the output stream - - * exclude_paths - a list of paths that should not appear in the output - stream - """ - - known_params = [ - 'include_paths', - 'exclude_paths', - ] - - def pre_process(self): - self.includes = self.params.get('include_paths') - self.excludes = self.params.get('exclude_paths') - # What's the new root, if any - self.new_root = helpers.common_directory(self.includes) - # Buffer of blobs until we know we need them: mark -> cmd - self.blobs = {} - # These are the commits we've output so far - self.interesting_commits = set() - # Map of commit-id to list of parents - self.parents = {} - - def pre_handler(self, cmd): - self.command = cmd - # Should this command be included in the output or not? - self.keep = False - # Blobs to dump into the output before dumping the command itself - self.referenced_blobs = [] - - def post_handler(self, cmd): - if not self.keep: - return - # print referenced blobs and the command - for blob_id in self.referenced_blobs: - self._print_command(self.blobs[blob_id]) - self._print_command(self.command) - - def progress_handler(self, cmd): - """Process a ProgressCommand.""" - # These always pass through - self.keep = True - - def blob_handler(self, cmd): - """Process a BlobCommand.""" - # These never pass through directly. We buffer them and only - # output them if referenced by an interesting command. - self.blobs[cmd.id] = cmd - self.keep = False - - def checkpoint_handler(self, cmd): - """Process a CheckpointCommand.""" - # These always pass through - self.keep = True - - def commit_handler(self, cmd): - """Process a CommitCommand.""" - # These pass through if they meet the filtering conditions - interesting_filecmds = self._filter_filecommands(cmd.file_iter) - if interesting_filecmds: - # If all we have is a single deleteall, skip this commit - if len(interesting_filecmds) == 1 and isinstance( - interesting_filecmds[0], commands.FileDeleteAllCommand): - pass - else: - # Remember just the interesting file commands - self.keep = True - cmd.file_iter = iter(interesting_filecmds) - - # Record the referenced blobs - for fc in interesting_filecmds: - if isinstance(fc, commands.FileModifyCommand): - if (fc.dataref is not None and - fc.kind != 'directory'): - self.referenced_blobs.append(fc.dataref) - - # Update from and merges to refer to commits in the output - cmd.from_ = self._find_interesting_from(cmd.from_) - cmd.merges = self._find_interesting_merges(cmd.merges) - self.interesting_commits.add(cmd.id) - - # Keep track of the parents - if cmd.from_ and cmd.merges: - parents = [cmd.from_] + cmd.merges - elif cmd.from_: - parents = [cmd.from_] - else: - parents = None - self.parents[":" + cmd.mark] = parents - - def reset_handler(self, cmd): - """Process a ResetCommand.""" - if cmd.from_ is None: - # We pass through resets that init a branch because we have to - # assume the branch might be interesting. - self.keep = True - else: - # Keep resets if they indirectly reference something we kept - cmd.from_ = self._find_interesting_from(cmd.from_) - self.keep = cmd.from_ is not None - - def tag_handler(self, cmd): - """Process a TagCommand.""" - # Keep tags if they indirectly reference something we kept - cmd.from_ = self._find_interesting_from(cmd.from_) - self.keep = cmd.from_ is not None - - def feature_handler(self, cmd): - """Process a FeatureCommand.""" - feature = cmd.feature_name - if feature not in commands.FEATURE_NAMES: - self.warning("feature %s is not supported - parsing may fail" - % (feature,)) - # These always pass through - self.keep = True - - def _print_command(self, cmd): - """Wrapper to avoid adding unnecessary blank lines.""" - text = repr(cmd) - self.outf.write(text) - if not text.endswith("\n"): - self.outf.write("\n") - - def _filter_filecommands(self, filecmd_iter): - """Return the filecommands filtered by includes & excludes. - - :return: a list of FileCommand objects - """ - if self.includes is None and self.excludes is None: - return list(filecmd_iter()) - - # Do the filtering, adjusting for the new_root - result = [] - for fc in filecmd_iter(): - if (isinstance(fc, commands.FileModifyCommand) or - isinstance(fc, commands.FileDeleteCommand)): - if self._path_to_be_kept(fc.path): - fc.path = self._adjust_for_new_root(fc.path) - else: - continue - elif isinstance(fc, commands.FileDeleteAllCommand): - pass - elif isinstance(fc, commands.FileRenameCommand): - fc = self._convert_rename(fc) - elif isinstance(fc, commands.FileCopyCommand): - fc = self._convert_copy(fc) - else: - warning("cannot handle FileCommands of class %s - ignoring", - fc.__class__) - continue - if fc is not None: - result.append(fc) - return result - - def _path_to_be_kept(self, path): - """Does the given path pass the filtering criteria?""" - if self.excludes and (path in self.excludes - or osutils.is_inside_any(self.excludes, path)): - return False - if self.includes: - return (path in self.includes - or osutils.is_inside_any(self.includes, path)) - return True - - def _adjust_for_new_root(self, path): - """Adjust a path given the new root directory of the output.""" - if self.new_root is None: - return path - elif path.startswith(self.new_root): - return path[len(self.new_root):] - else: - return path - - def _find_interesting_parent(self, commit_ref): - while True: - if commit_ref in self.interesting_commits: - return commit_ref - parents = self.parents.get(commit_ref) - if not parents: - return None - commit_ref = parents[0] - - def _find_interesting_from(self, commit_ref): - if commit_ref is None: - return None - return self._find_interesting_parent(commit_ref) - - def _find_interesting_merges(self, commit_refs): - if commit_refs is None: - return None - merges = [] - for commit_ref in commit_refs: - parent = self._find_interesting_parent(commit_ref) - if parent is not None: - merges.append(parent) - if merges: - return merges - else: - return None - - def _convert_rename(self, fc): - """Convert a FileRenameCommand into a new FileCommand. - - :return: None if the rename is being ignored, otherwise a - new FileCommand based on the whether the old and new paths - are inside or outside of the interesting locations. - """ - old = fc.old_path - new = fc.new_path - keep_old = self._path_to_be_kept(old) - keep_new = self._path_to_be_kept(new) - if keep_old and keep_new: - fc.old_path = self._adjust_for_new_root(old) - fc.new_path = self._adjust_for_new_root(new) - return fc - elif keep_old: - # The file has been renamed to a non-interesting location. - # Delete it! - old = self._adjust_for_new_root(old) - return commands.FileDeleteCommand(old) - elif keep_new: - # The file has been renamed into an interesting location - # We really ought to add it but we don't currently buffer - # the contents of all previous files and probably never want - # to. Maybe fast-import-info needs to be extended to - # remember all renames and a config file can be passed - # into here ala fast-import? - warning("cannot turn rename of %s into an add of %s yet" % - (old, new)) - return None - - def _convert_copy(self, fc): - """Convert a FileCopyCommand into a new FileCommand. - - :return: None if the copy is being ignored, otherwise a - new FileCommand based on the whether the source and destination - paths are inside or outside of the interesting locations. - """ - src = fc.src_path - dest = fc.dest_path - keep_src = self._path_to_be_kept(src) - keep_dest = self._path_to_be_kept(dest) - if keep_src and keep_dest: - fc.src_path = self._adjust_for_new_root(src) - fc.dest_path = self._adjust_for_new_root(dest) - return fc - elif keep_src: - # The file has been copied to a non-interesting location. - # Ignore it! - return None - elif keep_dest: - # The file has been copied into an interesting location - # We really ought to add it but we don't currently buffer - # the contents of all previous files and probably never want - # to. Maybe fast-import-info needs to be extended to - # remember all copies and a config file can be passed - # into here ala fast-import? - warning("cannot turn copy of %s into an add of %s yet" % - (src, dest)) - return None diff --git a/processors/generic_processor.py b/processors/generic_processor.py deleted file mode 100644 index e9006c1..0000000 --- a/processors/generic_processor.py +++ /dev/null @@ -1,569 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Import processor that supports all Bazaar repository formats.""" - - -import time -from bzrlib import ( - delta, - errors, - osutils, - progress, - ) -from bzrlib.repofmt import pack_repo -from bzrlib.trace import note -try: - import bzrlib.util.configobj.configobj as configobj -except ImportError: - import configobj -from bzrlib.plugins.fastimport import ( - branch_updater, - bzr_commit_handler, - cache_manager, - marks_file, - revision_store, - ) -from bzrlib.plugins.fastimport.fastimport import ( - commands, - errors as plugin_errors, - helpers, - idmapfile, - processor, - ) - - -# How many commits before automatically reporting progress -_DEFAULT_AUTO_PROGRESS = 1000 - -# How many commits before automatically checkpointing -_DEFAULT_AUTO_CHECKPOINT = 10000 - -# How many checkpoints before automatically packing -_DEFAULT_AUTO_PACK = 4 - -# How many inventories to cache -_DEFAULT_INV_CACHE_SIZE = 1 -_DEFAULT_CHK_INV_CACHE_SIZE = 1 - - -class GenericProcessor(processor.ImportProcessor): - """An import processor that handles basic imports. - - Current features supported: - - * blobs are cached in memory - * files and symlinks commits are supported - * checkpoints automatically happen at a configurable frequency - over and above the stream requested checkpoints - * timestamped progress reporting, both automatic and stream requested - * some basic statistics are dumped on completion. - - At checkpoints and on completion, the commit-id -> revision-id map is - saved to a file called 'fastimport-id-map'. If the import crashes - or is interrupted, it can be started again and this file will be - used to skip over already loaded revisions. The format of each line - is "commit-id revision-id" so commit-ids cannot include spaces. - - Here are the supported parameters: - - * info - name of a hints file holding the analysis generated - by running the fast-import-info processor in verbose mode. When - importing large repositories, this parameter is needed so - that the importer knows what blobs to intelligently cache. - - * trees - update the working trees before completing. - By default, the importer updates the repository - and branches and the user needs to run 'bzr update' for the - branches of interest afterwards. - - * count - only import this many commits then exit. If not set - or negative, all commits are imported. - - * checkpoint - automatically checkpoint every n commits over and - above any checkpoints contained in the import stream. - The default is 10000. - - * autopack - pack every n checkpoints. The default is 4. - - * inv-cache - number of inventories to cache. - If not set, the default is 1. - - * mode - import algorithm to use: default, experimental or classic. - - * import-marks - name of file to read to load mark information from - - * export-marks - name of file to write to save mark information to - """ - - known_params = [ - 'info', - 'trees', - 'count', - 'checkpoint', - 'autopack', - 'inv-cache', - 'mode', - 'import-marks', - 'export-marks', - ] - - def __init__(self, bzrdir, params=None, verbose=False, outf=None, - prune_empty_dirs=True): - processor.ImportProcessor.__init__(self, bzrdir, params, verbose) - self.prune_empty_dirs = prune_empty_dirs - - def pre_process(self): - self._start_time = time.time() - self._load_info_and_params() - if self.total_commits: - self.note("Starting import of %d commits ..." % - (self.total_commits,)) - else: - self.note("Starting import ...") - self.cache_mgr = cache_manager.CacheManager(self.info, self.verbose, - self.inventory_cache_size) - - if self.params.get("import-marks") is not None: - mark_info = marks_file.import_marks(self.params.get("import-marks")) - if mark_info is not None: - self.cache_mgr.revision_ids = mark_info[0] - self.skip_total = False - self.first_incremental_commit = True - else: - self.first_incremental_commit = False - self.skip_total = self._init_id_map() - if self.skip_total: - self.note("Found %d commits already loaded - " - "skipping over these ...", self.skip_total) - self._revision_count = 0 - - # mapping of tag name to revision_id - self.tags = {} - - # Create the revision store to use for committing, if any - self.rev_store = self._revision_store_factory() - - # Disable autopacking if the repo format supports it. - # THIS IS A HACK - there is no sanctioned way of doing this yet. - if isinstance(self.repo, pack_repo.KnitPackRepository): - self._original_max_pack_count = \ - self.repo._pack_collection._max_pack_count - def _max_pack_count_for_import(total_revisions): - return total_revisions + 1 - self.repo._pack_collection._max_pack_count = \ - _max_pack_count_for_import - else: - self._original_max_pack_count = None - - # Make groupcompress use the fast algorithm during importing. - # We want to repack at the end anyhow when more information - # is available to do a better job of saving space. - try: - from bzrlib import groupcompress - groupcompress._FAST = True - except ImportError: - pass - - # Create a write group. This is committed at the end of the import. - # Checkpointing closes the current one and starts a new one. - self.repo.start_write_group() - - def _load_info_and_params(self): - self._mode = bool(self.params.get('mode', 'default')) - self._experimental = self._mode == 'experimental' - - # This is currently hard-coded but might be configurable via - # parameters one day if that's needed - repo_transport = self.repo.control_files._transport - self.id_map_path = repo_transport.local_abspath("fastimport-id-map") - - # Load the info file, if any - info_path = self.params.get('info') - if info_path is not None: - self.info = configobj.ConfigObj(info_path) - else: - self.info = None - - # Decide which CommitHandler to use - self.supports_chk = getattr(self.repo._format, 'supports_chks', False) - if self.supports_chk and self._mode == 'classic': - note("Cannot use classic algorithm on CHK repositories" - " - using default one instead") - self._mode = 'default' - if self._mode == 'classic': - self.commit_handler_factory = \ - bzr_commit_handler.InventoryCommitHandler - else: - self.commit_handler_factory = \ - bzr_commit_handler.InventoryDeltaCommitHandler - - # Decide how often to automatically report progress - # (not a parameter yet) - self.progress_every = _DEFAULT_AUTO_PROGRESS - if self.verbose: - self.progress_every = self.progress_every / 10 - - # Decide how often (# of commits) to automatically checkpoint - self.checkpoint_every = int(self.params.get('checkpoint', - _DEFAULT_AUTO_CHECKPOINT)) - - # Decide how often (# of checkpoints) to automatically pack - self.checkpoint_count = 0 - self.autopack_every = int(self.params.get('autopack', - _DEFAULT_AUTO_PACK)) - - # Decide how big to make the inventory cache - cache_size = int(self.params.get('inv-cache', -1)) - if cache_size == -1: - if self.supports_chk: - cache_size = _DEFAULT_CHK_INV_CACHE_SIZE - else: - cache_size = _DEFAULT_INV_CACHE_SIZE - self.inventory_cache_size = cache_size - - # Find the maximum number of commits to import (None means all) - # and prepare progress reporting. Just in case the info file - # has an outdated count of commits, we store the max counts - # at which we need to terminate separately to the total used - # for progress tracking. - try: - self.max_commits = int(self.params['count']) - if self.max_commits < 0: - self.max_commits = None - except KeyError: - self.max_commits = None - if self.info is not None: - self.total_commits = int(self.info['Command counts']['commit']) - if (self.max_commits is not None and - self.total_commits > self.max_commits): - self.total_commits = self.max_commits - else: - self.total_commits = self.max_commits - - def _revision_store_factory(self): - """Make a RevisionStore based on what the repository supports.""" - new_repo_api = hasattr(self.repo, 'revisions') - if new_repo_api: - return revision_store.RevisionStore2(self.repo) - elif not self._experimental: - return revision_store.RevisionStore1(self.repo) - else: - def fulltext_when(count): - total = self.total_commits - if total is not None and count == total: - fulltext = True - else: - # Create an inventory fulltext every 200 revisions - fulltext = count % 200 == 0 - if fulltext: - self.note("%d commits - storing inventory as full-text", - count) - return fulltext - - return revision_store.ImportRevisionStore1( - self.repo, self.inventory_cache_size, - fulltext_when=fulltext_when) - - def _process(self, command_iter): - # if anything goes wrong, abort the write group if any - try: - processor.ImportProcessor._process(self, command_iter) - except: - if self.repo is not None and self.repo.is_in_write_group(): - self.repo.abort_write_group() - raise - - def post_process(self): - # Commit the current write group and checkpoint the id map - self.repo.commit_write_group() - self._save_id_map() - - if self.params.get("export-marks") is not None: - marks_file.export_marks(self.params.get("export-marks"), - self.cache_mgr.revision_ids) - - if self.cache_mgr.last_ref == None: - """Nothing to refresh""" - return - - # Update the branches - self.note("Updating branch information ...") - updater = branch_updater.BranchUpdater(self.repo, self.branch, - self.cache_mgr, helpers.invert_dictset(self.cache_mgr.heads), - self.cache_mgr.last_ref, self.tags) - branches_updated, branches_lost = updater.update() - self._branch_count = len(branches_updated) - - # Tell the user about branches that were not created - if branches_lost: - if not self.repo.is_shared(): - self.warning("Cannot import multiple branches into " - "a standalone branch") - self.warning("Not creating branches for these head revisions:") - for lost_info in branches_lost: - head_revision = lost_info[1] - branch_name = lost_info[0] - self.note("\t %s = %s", head_revision, branch_name) - - # Update the working trees as requested - self._tree_count = 0 - remind_about_update = True - if self._branch_count == 0: - self.note("no branches to update") - self.note("no working trees to update") - remind_about_update = False - elif self.params.get('trees', False): - trees = self._get_working_trees(branches_updated) - if trees: - self._update_working_trees(trees) - remind_about_update = False - else: - self.warning("No working trees available to update") - else: - # Update just the trunk. (This is always the first branch - # returned by the branch updater.) - trunk_branch = branches_updated[0] - trees = self._get_working_trees([trunk_branch]) - if trees: - self._update_working_trees(trees) - remind_about_update = self._branch_count > 1 - - # Dump the cache stats now because we clear it before the final pack - if self.verbose: - self.cache_mgr.dump_stats() - if self._original_max_pack_count: - # We earlier disabled autopacking, creating one pack every - # checkpoint instead. We now pack the repository to optimise - # how data is stored. - self.cache_mgr.clear_all() - self._pack_repository() - - # Finish up by dumping stats & telling the user what to do next. - self.dump_stats() - if remind_about_update: - # This message is explicitly not timestamped. - note("To refresh the working tree for other branches, " - "use 'bzr update' inside that branch.") - - def _update_working_trees(self, trees): - if self.verbose: - reporter = delta._ChangeReporter() - else: - reporter = None - for wt in trees: - self.note("Updating the working tree for %s ...", wt.basedir) - wt.update(reporter) - self._tree_count += 1 - - def _pack_repository(self, final=True): - # Before packing, free whatever memory we can and ensure - # that groupcompress is configured to optimise disk space - import gc - if final: - try: - from bzrlib import groupcompress - except ImportError: - pass - else: - groupcompress._FAST = False - gc.collect() - self.note("Packing repository ...") - self.repo.pack() - - # To be conservative, packing puts the old packs and - # indices in obsolete_packs. We err on the side of - # optimism and clear out that directory to save space. - self.note("Removing obsolete packs ...") - # TODO: Use a public API for this once one exists - repo_transport = self.repo._pack_collection.transport - repo_transport.clone('obsolete_packs').delete_multi( - repo_transport.list_dir('obsolete_packs')) - - # If we're not done, free whatever memory we can - if not final: - gc.collect() - - def _get_working_trees(self, branches): - """Get the working trees for branches in the repository.""" - result = [] - wt_expected = self.repo.make_working_trees() - for br in branches: - if br is None: - continue - elif br == self.branch: - if self.working_tree: - result.append(self.working_tree) - elif wt_expected: - try: - result.append(br.bzrdir.open_workingtree()) - except errors.NoWorkingTree: - self.warning("No working tree for branch %s", br) - return result - - def dump_stats(self): - time_required = progress.str_tdelta(time.time() - self._start_time) - rc = self._revision_count - self.skip_total - bc = self._branch_count - wtc = self._tree_count - self.note("Imported %d %s, updating %d %s and %d %s in %s", - rc, helpers.single_plural(rc, "revision", "revisions"), - bc, helpers.single_plural(bc, "branch", "branches"), - wtc, helpers.single_plural(wtc, "tree", "trees"), - time_required) - - def _init_id_map(self): - """Load the id-map and check it matches the repository. - - :return: the number of entries in the map - """ - # Currently, we just check the size. In the future, we might - # decide to be more paranoid and check that the revision-ids - # are identical as well. - self.cache_mgr.revision_ids, known = idmapfile.load_id_map( - self.id_map_path) - existing_count = len(self.repo.all_revision_ids()) - if existing_count < known: - raise plugin_errors.BadRepositorySize(known, existing_count) - return known - - def _save_id_map(self): - """Save the id-map.""" - # Save the whole lot every time. If this proves a problem, we can - # change to 'append just the new ones' at a later time. - idmapfile.save_id_map(self.id_map_path, self.cache_mgr.revision_ids) - - def blob_handler(self, cmd): - """Process a BlobCommand.""" - if cmd.mark is not None: - dataref = cmd.id - else: - dataref = osutils.sha_strings(cmd.data) - self.cache_mgr.store_blob(dataref, cmd.data) - - def checkpoint_handler(self, cmd): - """Process a CheckpointCommand.""" - # Commit the current write group and start a new one - self.repo.commit_write_group() - self._save_id_map() - # track the number of automatic checkpoints done - if cmd is None: - self.checkpoint_count += 1 - if self.checkpoint_count % self.autopack_every == 0: - self._pack_repository(final=False) - self.repo.start_write_group() - - def commit_handler(self, cmd): - """Process a CommitCommand.""" - if self.skip_total and self._revision_count < self.skip_total: - self.cache_mgr.track_heads(cmd) - # Check that we really do know about this commit-id - if not self.cache_mgr.revision_ids.has_key(cmd.id): - raise plugin_errors.BadRestart(cmd.id) - # Consume the file commands and free any non-sticky blobs - for fc in cmd.file_iter(): - pass - self.cache_mgr._blobs = {} - self._revision_count += 1 - if cmd.ref.startswith('refs/tags/'): - tag_name = cmd.ref[len('refs/tags/'):] - self._set_tag(tag_name, cmd.id) - return - if self.first_incremental_commit: - self.first_incremental_commit = None - parents = self.cache_mgr.track_heads(cmd) - - # 'Commit' the revision and report progress - handler = self.commit_handler_factory(cmd, self.cache_mgr, - self.rev_store, verbose=self.verbose, - prune_empty_dirs=self.prune_empty_dirs) - try: - handler.process() - except: - print "ABORT: exception occurred processing commit %s" % (cmd.id) - raise - self.cache_mgr.revision_ids[cmd.id] = handler.revision_id - self._revision_count += 1 - self.report_progress("(%s)" % cmd.id) - - if cmd.ref.startswith('refs/tags/'): - tag_name = cmd.ref[len('refs/tags/'):] - self._set_tag(tag_name, cmd.id) - - # Check if we should finish up or automatically checkpoint - if (self.max_commits is not None and - self._revision_count >= self.max_commits): - self.note("Stopping after reaching requested count of commits") - self.finished = True - elif self._revision_count % self.checkpoint_every == 0: - self.note("%d commits - automatic checkpoint triggered", - self._revision_count) - self.checkpoint_handler(None) - - def report_progress(self, details=''): - if self._revision_count % self.progress_every == 0: - if self.total_commits is not None: - counts = "%d/%d" % (self._revision_count, self.total_commits) - else: - counts = "%d" % (self._revision_count,) - minutes = (time.time() - self._start_time) / 60 - revisions_added = self._revision_count - self.skip_total - rate = revisions_added * 1.0 / minutes - if rate > 10: - rate_str = "at %.0f/minute " % rate - else: - rate_str = "at %.1f/minute " % rate - self.note("%s commits processed %s%s" % (counts, rate_str, details)) - - def progress_handler(self, cmd): - """Process a ProgressCommand.""" - # Most progress messages embedded in streams are annoying. - # Ignore them unless in verbose mode. - if self.verbose: - self.note("progress %s" % (cmd.message,)) - - def reset_handler(self, cmd): - """Process a ResetCommand.""" - if cmd.ref.startswith('refs/tags/'): - tag_name = cmd.ref[len('refs/tags/'):] - if cmd.from_ is not None: - self._set_tag(tag_name, cmd.from_) - elif self.verbose: - self.warning("ignoring reset refs/tags/%s - no from clause" - % tag_name) - return - - if cmd.from_ is not None: - self.cache_mgr.track_heads_for_ref(cmd.ref, cmd.from_) - - def tag_handler(self, cmd): - """Process a TagCommand.""" - if cmd.from_ is not None: - self._set_tag(cmd.id, cmd.from_) - else: - self.warning("ignoring tag %s - no from clause" % cmd.id) - - def _set_tag(self, name, from_): - """Define a tag given a name and import 'from' reference.""" - bzr_tag_name = name.decode('utf-8', 'replace') - bzr_rev_id = self.cache_mgr.revision_ids[from_] - self.tags[bzr_tag_name] = bzr_rev_id - - def feature_handler(self, cmd): - """Process a FeatureCommand.""" - feature = cmd.feature_name - if feature not in commands.FEATURE_NAMES: - raise plugin_errors.UnknownFeature(feature) diff --git a/processors/info_processor.py b/processors/info_processor.py deleted file mode 100644 index eb22b00..0000000 --- a/processors/info_processor.py +++ /dev/null @@ -1,282 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Import processor that dump stats about the input (and doesn't import).""" - - -from bzrlib.trace import ( - note, - warning, - ) -from bzrlib.plugins.fastimport import ( - cache_manager, - commands, - helpers, - processor, - ) - - -class InfoProcessor(processor.ImportProcessor): - """An import processor that dumps statistics about the input. - - No changes to the current repository are made. - - As well as providing useful information about an import - stream before importing it, this processor is useful for - benchmarking the speed at which data can be extracted from - the source. - """ - - def __init__(self, target=None, params=None, verbose=0, outf=None): - # Allow creation without a target - processor.ImportProcessor.__init__(self, target, params, verbose, - outf=outf) - - def pre_process(self): - self.note("Collecting statistics ...") - # Init statistics - self.cmd_counts = {} - for cmd in commands.COMMAND_NAMES: - self.cmd_counts[cmd] = 0 - self.file_cmd_counts = {} - for fc in commands.FILE_COMMAND_NAMES: - self.file_cmd_counts[fc] = 0 - self.parent_counts = {} - self.max_parent_count = 0 - self.committers = set() - self.separate_authors_found = False - self.symlinks_found = False - self.executables_found = False - self.sha_blob_references = False - self.lightweight_tags = 0 - # Blob usage tracking - self.blobs = {} - for usage in ['new', 'used', 'unknown', 'unmarked']: - self.blobs[usage] = set() - self.blob_ref_counts = {} - # Head tracking - delegate to the cache manager - self.cache_mgr = cache_manager.CacheManager(inventory_cache_size=0) - # Stuff to cache: a map from mark to # of times that mark is merged - self.merges = {} - # Stuff to cache: these are maps from mark to sets - self.rename_old_paths = {} - self.copy_source_paths = {} - - def post_process(self): - # Dump statistics - cmd_names = commands.COMMAND_NAMES - fc_names = commands.FILE_COMMAND_NAMES - self._dump_stats_group("Command counts", - [(c, self.cmd_counts[c]) for c in cmd_names], str) - self._dump_stats_group("File command counts", - [(c, self.file_cmd_counts[c]) for c in fc_names], str) - - # Commit stats - if self.cmd_counts['commit']: - p_items = [] - for i in xrange(0, self.max_parent_count + 1): - if i in self.parent_counts: - count = self.parent_counts[i] - p_items.append(("parents-%d" % i, count)) - merges_count = len(self.merges.keys()) - p_items.append(('total revisions merged', merges_count)) - flags = { - 'separate authors found': self.separate_authors_found, - 'executables': self.executables_found, - 'symlinks': self.symlinks_found, - 'blobs referenced by SHA': self.sha_blob_references, - } - self._dump_stats_group("Parent counts", p_items, str) - self._dump_stats_group("Commit analysis", flags.iteritems(), _found) - heads = helpers.invert_dictset(self.cache_mgr.heads) - self._dump_stats_group("Head analysis", heads.iteritems(), None, - _iterable_as_config_list) - # note("\t%d\t%s" % (len(self.committers), 'unique committers')) - self._dump_stats_group("Merges", self.merges.iteritems(), None) - # We only show the rename old path and copy source paths when -vv - # (verbose=2) is specified. The output here for mysql's data can't - # be parsed currently so this bit of code needs more work anyhow .. - if self.verbose >= 2: - self._dump_stats_group("Rename old paths", - self.rename_old_paths.iteritems(), len, - _iterable_as_config_list) - self._dump_stats_group("Copy source paths", - self.copy_source_paths.iteritems(), len, - _iterable_as_config_list) - - # Blob stats - if self.cmd_counts['blob']: - # In verbose mode, don't list every blob used - if self.verbose: - del self.blobs['used'] - self._dump_stats_group("Blob usage tracking", - self.blobs.iteritems(), len, _iterable_as_config_list) - if self.blob_ref_counts: - blobs_by_count = helpers.invert_dict(self.blob_ref_counts) - blob_items = blobs_by_count.items() - blob_items.sort() - self._dump_stats_group("Blob reference counts", - blob_items, len, _iterable_as_config_list) - - # Other stats - if self.cmd_counts['reset']: - reset_stats = { - 'lightweight tags': self.lightweight_tags, - } - self._dump_stats_group("Reset analysis", reset_stats.iteritems()) - - def _dump_stats_group(self, title, items, normal_formatter=None, - verbose_formatter=None): - """Dump a statistics group. - - In verbose mode, do so as a config file so - that other processors can load the information if they want to. - :param normal_formatter: the callable to apply to the value - before displaying it in normal mode - :param verbose_formatter: the callable to apply to the value - before displaying it in verbose mode - """ - if self.verbose: - self.outf.write("[%s]\n" % (title,)) - for name, value in items: - if verbose_formatter is not None: - value = verbose_formatter(value) - if type(name) == str: - name = name.replace(' ', '-') - self.outf.write("%s = %s\n" % (name, value)) - self.outf.write("\n") - else: - self.outf.write("%s:\n" % (title,)) - for name, value in items: - if normal_formatter is not None: - value = normal_formatter(value) - self.outf.write("\t%s\t%s\n" % (value, name)) - - def progress_handler(self, cmd): - """Process a ProgressCommand.""" - self.cmd_counts[cmd.name] += 1 - - def blob_handler(self, cmd): - """Process a BlobCommand.""" - self.cmd_counts[cmd.name] += 1 - if cmd.mark is None: - self.blobs['unmarked'].add(cmd.id) - else: - self.blobs['new'].add(cmd.id) - # Marks can be re-used so remove it from used if already there. - # Note: we definitely do NOT want to remove it from multi if - # it's already in that set. - try: - self.blobs['used'].remove(cmd.id) - except KeyError: - pass - - def checkpoint_handler(self, cmd): - """Process a CheckpointCommand.""" - self.cmd_counts[cmd.name] += 1 - - def commit_handler(self, cmd): - """Process a CommitCommand.""" - self.cmd_counts[cmd.name] += 1 - self.committers.add(cmd.committer) - if cmd.author is not None: - self.separate_authors_found = True - for fc in cmd.file_iter(): - self.file_cmd_counts[fc.name] += 1 - if isinstance(fc, commands.FileModifyCommand): - if fc.is_executable: - self.executables_found = True - if fc.kind == commands.SYMLINK_KIND: - self.symlinks_found = True - if fc.dataref is not None: - if fc.dataref[0] == ':': - self._track_blob(fc.dataref) - else: - self.sha_blob_references = True - elif isinstance(fc, commands.FileRenameCommand): - self.rename_old_paths.setdefault(cmd.id, set()).add(fc.old_path) - elif isinstance(fc, commands.FileCopyCommand): - self.copy_source_paths.setdefault(cmd.id, set()).add(fc.src_path) - - # Track the heads - parents = self.cache_mgr.track_heads(cmd) - - # Track the parent counts - parent_count = len(parents) - if self.parent_counts.has_key(parent_count): - self.parent_counts[parent_count] += 1 - else: - self.parent_counts[parent_count] = 1 - if parent_count > self.max_parent_count: - self.max_parent_count = parent_count - - # Remember the merges - if cmd.merges: - #self.merges.setdefault(cmd.ref, set()).update(cmd.merges) - for merge in cmd.merges: - if merge in self.merges: - self.merges[merge] += 1 - else: - self.merges[merge] = 1 - - def reset_handler(self, cmd): - """Process a ResetCommand.""" - self.cmd_counts[cmd.name] += 1 - if cmd.ref.startswith('refs/tags/'): - self.lightweight_tags += 1 - else: - if cmd.from_ is not None: - self.cache_mgr.track_heads_for_ref(cmd.ref, cmd.from_) - - def tag_handler(self, cmd): - """Process a TagCommand.""" - self.cmd_counts[cmd.name] += 1 - - def feature_handler(self, cmd): - """Process a FeatureCommand.""" - self.cmd_counts[cmd.name] += 1 - feature = cmd.feature_name - if feature not in commands.FEATURE_NAMES: - self.warning("feature %s is not supported - parsing may fail" - % (feature,)) - - def _track_blob(self, mark): - if mark in self.blob_ref_counts: - self.blob_ref_counts[mark] += 1 - pass - elif mark in self.blobs['used']: - self.blob_ref_counts[mark] = 2 - self.blobs['used'].remove(mark) - elif mark in self.blobs['new']: - self.blobs['used'].add(mark) - self.blobs['new'].remove(mark) - else: - self.blobs['unknown'].add(mark) - -def _found(b): - """Format a found boolean as a string.""" - return ['no', 'found'][b] - -def _iterable_as_config_list(s): - """Format an iterable as a sequence of comma-separated strings. - - To match what ConfigObj expects, a single item list has a trailing comma. - """ - items = sorted(s) - if len(items) == 1: - return "%s," % (items[0],) - else: - return ", ".join(items) diff --git a/processors/query_processor.py b/processors/query_processor.py deleted file mode 100644 index 5d33a5b..0000000 --- a/processors/query_processor.py +++ /dev/null @@ -1,97 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Import processor that queries the input (and doesn't import).""" - - -from bzrlib.plugins.fastimport import ( - commands, - processor, - ) - - -class QueryProcessor(processor.ImportProcessor): - """An import processor that queries the input. - - No changes to the current repository are made. - """ - - known_params = commands.COMMAND_NAMES + commands.FILE_COMMAND_NAMES + \ - ['commit-mark'] - - def __init__(self, target=None, params=None, verbose=False): - # Allow creation without a target - processor.ImportProcessor.__init__(self, target, params, verbose) - self.parsed_params = {} - self.interesting_commit = None - self._finished = False - if params: - if 'commit-mark' in params: - self.interesting_commit = params['commit-mark'] - del params['commit-mark'] - for name, value in params.iteritems(): - if value == 1: - # All fields - fields = None - else: - fields = value.split(',') - self.parsed_params[name] = fields - - def pre_handler(self, cmd): - """Hook for logic before each handler starts.""" - if self._finished: - return - if self.interesting_commit and cmd.name == 'commit': - if cmd.mark == self.interesting_commit: - print cmd.to_string() - self._finished = True - return - if self.parsed_params.has_key(cmd.name): - fields = self.parsed_params[cmd.name] - str = cmd.dump_str(fields, self.parsed_params, self.verbose) - print "%s" % (str,) - - def progress_handler(self, cmd): - """Process a ProgressCommand.""" - pass - - def blob_handler(self, cmd): - """Process a BlobCommand.""" - pass - - def checkpoint_handler(self, cmd): - """Process a CheckpointCommand.""" - pass - - def commit_handler(self, cmd): - """Process a CommitCommand.""" - for fc in cmd.file_iter(): - pass - - def reset_handler(self, cmd): - """Process a ResetCommand.""" - pass - - def tag_handler(self, cmd): - """Process a TagCommand.""" - pass - - def feature_handler(self, cmd): - """Process a FeatureCommand.""" - feature = cmd.feature_name - if feature not in commands.FEATURE_NAMES: - self.warning("feature %s is not supported - parsing may fail" - % (feature,)) diff --git a/revision_store.py b/revision_store.py deleted file mode 100644 index 399dabe..0000000 --- a/revision_store.py +++ /dev/null @@ -1,735 +0,0 @@ -# Copyright (C) 2008, 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""An abstraction of a repository providing just the bits importing needs.""" - -import cStringIO - -from bzrlib import ( - errors, - graph as _mod_graph, - inventory, - knit, - lru_cache, - osutils, - revision as _mod_revision, - trace, - ) - - -class _TreeShim(object): - """Fake a Tree implementation. - - This implements just enough of the tree api to make commit builder happy. - """ - - def __init__(self, repo, basis_inv, inv_delta, content_provider): - self._repo = repo - self._content_provider = content_provider - self._basis_inv = basis_inv - self._inv_delta = inv_delta - self._new_info_by_id = dict([(file_id, (new_path, ie)) - for _, new_path, file_id, ie in inv_delta]) - - def id2path(self, file_id): - if file_id in self._new_info_by_id: - new_path = self._new_info_by_id[file_id][0] - if new_path is None: - raise errors.NoSuchId(self, file_id) - return new_path - return self._basis_inv.id2path(file_id) - - def path2id(self, path): - # CommitBuilder currently only requires access to the root id. We don't - # build a map of renamed files, etc. One possibility if we ever *do* - # need more than just root, is to defer to basis_inv.path2id() and then - # check if the file_id is in our _new_info_by_id dict. And in that - # case, return _new_info_by_id[file_id][0] - if path != '': - raise NotImplementedError(_TreeShim.path2id) - # TODO: Handle root renames? - return self._basis_inv.root.file_id - - def get_file_with_stat(self, file_id, path=None): - try: - content = self._content_provider(file_id) - except KeyError: - # The content wasn't shown as 'new'. Just validate this fact - assert file_id not in self._new_info_by_id - old_ie = self._basis_inv[file_id] - old_text_key = (file_id, old_ie.revision) - stream = self._repo.texts.get_record_stream([old_text_key], - 'unordered', True) - content = stream.next().get_bytes_as('fulltext') - sio = cStringIO.StringIO(content) - return sio, None - - def get_symlink_target(self, file_id): - if file_id in self._new_info_by_id: - ie = self._new_info_by_id[file_id][1] - return ie.symlink_target - return self._basis_inv[file_id].symlink_target - - def get_reference_revision(self, file_id, path=None): - raise NotImplementedError(_TreeShim.get_reference_revision) - - def _delta_to_iter_changes(self): - """Convert the inv_delta into an iter_changes repr.""" - # iter_changes is: - # (file_id, - # (old_path, new_path), - # content_changed, - # (old_versioned, new_versioned), - # (old_parent_id, new_parent_id), - # (old_name, new_name), - # (old_kind, new_kind), - # (old_exec, new_exec), - # ) - basis_inv = self._basis_inv - for old_path, new_path, file_id, ie in self._inv_delta: - # Perf: Would this be faster if we did 'if file_id in basis_inv'? - # Since the *very* common case is that the file already exists, it - # probably is better to optimize for that - try: - old_ie = basis_inv[file_id] - except errors.NoSuchId: - old_ie = None - if ie is None: - raise AssertionError('How is both old and new None?') - change = (file_id, - (old_path, new_path), - False, - (False, False), - (None, None), - (None, None), - (None, None), - (None, None), - ) - change = (file_id, - (old_path, new_path), - True, - (False, True), - (None, ie.parent_id), - (None, ie.name), - (None, ie.kind), - (None, ie.executable), - ) - else: - if ie is None: - change = (file_id, - (old_path, new_path), - True, - (True, False), - (old_ie.parent_id, None), - (old_ie.name, None), - (old_ie.kind, None), - (old_ie.executable, None), - ) - else: - content_modified = (ie.text_sha1 != old_ie.text_sha1 - or ie.text_size != old_ie.text_size) - # TODO: ie.kind != old_ie.kind - # TODO: symlinks changing targets, content_modified? - change = (file_id, - (old_path, new_path), - content_modified, - (True, True), - (old_ie.parent_id, ie.parent_id), - (old_ie.name, ie.name), - (old_ie.kind, ie.kind), - (old_ie.executable, ie.executable), - ) - yield change - - -class AbstractRevisionStore(object): - - def __init__(self, repo): - """An object responsible for loading revisions into a repository. - - NOTE: Repository locking is not managed by this class. Clients - should take a write lock, call load() multiple times, then release - the lock. - - :param repository: the target repository - """ - self.repo = repo - self._graph = None - self._use_known_graph = True - self._supports_chks = getattr(repo._format, 'supports_chks', False) - - def expects_rich_root(self): - """Does this store expect inventories with rich roots?""" - return self.repo.supports_rich_root() - - def init_inventory(self, revision_id): - """Generate an inventory for a parentless revision.""" - if self._supports_chks: - inv = self._init_chk_inventory(revision_id, inventory.ROOT_ID) - else: - inv = inventory.Inventory(revision_id=revision_id) - if self.expects_rich_root(): - # The very first root needs to have the right revision - inv.root.revision = revision_id - return inv - - def _init_chk_inventory(self, revision_id, root_id): - """Generate a CHKInventory for a parentless revision.""" - from bzrlib import chk_map - # Get the creation parameters - chk_store = self.repo.chk_bytes - serializer = self.repo._format._serializer - search_key_name = serializer.search_key_name - maximum_size = serializer.maximum_size - - # Maybe the rest of this ought to be part of the CHKInventory API? - inv = inventory.CHKInventory(search_key_name) - inv.revision_id = revision_id - inv.root_id = root_id - search_key_func = chk_map.search_key_registry.get(search_key_name) - inv.id_to_entry = chk_map.CHKMap(chk_store, None, search_key_func) - inv.id_to_entry._root_node.set_maximum_size(maximum_size) - inv.parent_id_basename_to_file_id = chk_map.CHKMap(chk_store, - None, search_key_func) - inv.parent_id_basename_to_file_id._root_node.set_maximum_size( - maximum_size) - inv.parent_id_basename_to_file_id._root_node._key_width = 2 - return inv - - def get_inventory(self, revision_id): - """Get a stored inventory.""" - return self.repo.get_inventory(revision_id) - - def get_file_text(self, revision_id, file_id): - """Get the text stored for a file in a given revision.""" - revtree = self.repo.revision_tree(revision_id) - return revtree.get_file_text(file_id) - - def get_file_lines(self, revision_id, file_id): - """Get the lines stored for a file in a given revision.""" - revtree = self.repo.revision_tree(revision_id) - return osutils.split_lines(revtree.get_file_text(file_id)) - - def start_new_revision(self, revision, parents, parent_invs): - """Init the metadata needed for get_parents_and_revision_for_entry(). - - :param revision: a Revision object - """ - self._current_rev_id = revision.revision_id - self._rev_parents = parents - self._rev_parent_invs = parent_invs - # We don't know what the branch will be so there's no real BranchConfig. - # That means we won't be triggering any hooks and that's a good thing. - # Without a config though, we must pass in the committer below so that - # the commit builder doesn't try to look up the config. - config = None - # We can't use self.repo.get_commit_builder() here because it starts a - # new write group. We want one write group around a batch of imports - # where the default batch size is currently 10000. IGC 20090312 - self._commit_builder = self.repo._commit_builder_class(self.repo, - parents, config, timestamp=revision.timestamp, - timezone=revision.timezone, committer=revision.committer, - revprops=revision.properties, revision_id=revision.revision_id) - - def get_parents_and_revision_for_entry(self, ie): - """Get the parents and revision for an inventory entry. - - :param ie: the inventory entry - :return parents, revision_id where - parents is the tuple of parent revision_ids for the per-file graph - revision_id is the revision_id to use for this entry - """ - # Check for correct API usage - if self._current_rev_id is None: - raise AssertionError("start_new_revision() must be called" - " before get_parents_and_revision_for_entry()") - if ie.revision != self._current_rev_id: - raise AssertionError("start_new_revision() registered a different" - " revision (%s) to that in the inventory entry (%s)" % - (self._current_rev_id, ie.revision)) - - # Find the heads. This code is lifted from - # repository.CommitBuilder.record_entry_contents(). - parent_candidate_entries = ie.parent_candidates(self._rev_parent_invs) - head_set = self._commit_builder._heads(ie.file_id, - parent_candidate_entries.keys()) - heads = [] - for inv in self._rev_parent_invs: - if ie.file_id in inv: - old_rev = inv[ie.file_id].revision - if old_rev in head_set: - rev_id = inv[ie.file_id].revision - heads.append(rev_id) - head_set.remove(rev_id) - - # Find the revision to use. If the content has not changed - # since the parent, record the parent's revision. - if len(heads) == 0: - return (), ie.revision - parent_entry = parent_candidate_entries[heads[0]] - changed = False - if len(heads) > 1: - changed = True - elif (parent_entry.name != ie.name or parent_entry.kind != ie.kind or - parent_entry.parent_id != ie.parent_id): - changed = True - elif ie.kind == 'file': - if (parent_entry.text_sha1 != ie.text_sha1 or - parent_entry.executable != ie.executable): - changed = True - elif ie.kind == 'symlink': - if parent_entry.symlink_target != ie.symlink_target: - changed = True - if changed: - rev_id = ie.revision - else: - rev_id = parent_entry.revision - return tuple(heads), rev_id - - def load(self, rev, inv, signature, text_provider, parents_provider, - inventories_provider=None): - """Load a revision. - - :param rev: the Revision - :param inv: the inventory - :param signature: signing information - :param text_provider: a callable expecting a file_id parameter - that returns the text for that file-id - :param parents_provider: a callable expecting a file_id parameter - that return the list of parent-ids for that file-id - :param inventories_provider: a callable expecting a repository and - a list of revision-ids, that returns: - * the list of revision-ids present in the repository - * the list of inventories for the revision-id's, - including an empty inventory for the missing revisions - If None, a default implementation is provided. - """ - # NOTE: This is bzrlib.repository._install_revision refactored to - # to provide more flexibility in how previous revisions are cached, - # data is feed in, etc. - - # Get the non-ghost parents and their inventories - if inventories_provider is None: - inventories_provider = self._default_inventories_provider - present_parents, parent_invs = inventories_provider(rev.parent_ids) - - # Load the inventory - try: - rev.inventory_sha1 = self._add_inventory(rev.revision_id, - inv, present_parents, parent_invs) - except errors.RevisionAlreadyPresent: - pass - - # Load the texts, signature and revision - entries = self._non_root_entries_iter(inv, rev.revision_id) - self._load_texts(rev.revision_id, entries, text_provider, - parents_provider) - if signature is not None: - self.repo.add_signature_text(rev.revision_id, signature) - self._add_revision(rev, inv) - - def load_using_delta(self, rev, basis_inv, inv_delta, signature, - text_provider, parents_provider, inventories_provider=None): - """Load a revision by applying a delta to a (CHK)Inventory. - - :param rev: the Revision - :param basis_inv: the basis Inventory or CHKInventory - :param inv_delta: the inventory delta - :param signature: signing information - :param text_provider: a callable expecting a file_id parameter - that returns the text for that file-id - :param parents_provider: a callable expecting a file_id parameter - that return the list of parent-ids for that file-id - :param inventories_provider: a callable expecting a repository and - a list of revision-ids, that returns: - * the list of revision-ids present in the repository - * the list of inventories for the revision-id's, - including an empty inventory for the missing revisions - If None, a default implementation is provided. - """ - # TODO: set revision_id = rev.revision_id - builder = self.repo._commit_builder_class(self.repo, - parents=rev.parent_ids, config=None, timestamp=rev.timestamp, - timezone=rev.timezone, committer=rev.committer, - revprops=rev.properties, revision_id=rev.revision_id) - if self._graph is None and self._use_known_graph: - if (getattr(_mod_graph, 'GraphThunkIdsToKeys', None) is None - or getattr(_mod_graph.KnownGraph, 'add_node', None) is None): - self._use_known_graph = False - else: - self._graph = self.repo.revisions.get_known_graph_ancestry( - [(r,) for r in rev.parent_ids]) - if self._graph is not None: - orig_heads = builder._heads - def thunked_heads(file_id, revision_ids): - # self._graph thinks in terms of keys, not ids, so translate - # them - # old_res = orig_heads(file_id, revision_ids) - if len(revision_ids) < 2: - res = set(revision_ids) - else: - res = set([h[0] for h in - self._graph.heads([(r,) for r in revision_ids])]) - # if old_res != res: - # import pdb; pdb.set_trace() - return res - builder._heads = thunked_heads - - if rev.parent_ids: - basis_rev_id = rev.parent_ids[0] - else: - basis_rev_id = _mod_revision.NULL_REVISION - tree = _TreeShim(self.repo, basis_inv, inv_delta, text_provider) - changes = tree._delta_to_iter_changes() - for (file_id, path, fs_hash) in builder.record_iter_changes( - tree, basis_rev_id, changes): - # So far, we don't *do* anything with the result - pass - builder.finish_inventory() - # TODO: This is working around a bug in the bzrlib code base. - # 'builder.finish_inventory()' ends up doing: - # self.inv_sha1 = self.repository.add_inventory_by_delta(...) - # However, add_inventory_by_delta returns (sha1, inv) - # And we *want* to keep a handle on both of those objects - if isinstance(builder.inv_sha1, tuple): - builder.inv_sha1, builder.new_inventory = builder.inv_sha1 - # This is a duplicate of Builder.commit() since we already have the - # Revision object, and we *don't* want to call commit_write_group() - rev.inv_sha1 = builder.inv_sha1 - builder.repository.add_revision(builder._new_revision_id, rev, - builder.new_inventory, builder._config) - if self._graph is not None: - # TODO: Use StaticTuple and .intern() for these things - self._graph.add_node((builder._new_revision_id,), - [(p,) for p in rev.parent_ids]) - - if signature is not None: - raise AssertionError('signatures not guaranteed yet') - self.repo.add_signature_text(rev_id, signature) - # self._add_revision(rev, inv) - return builder.revision_tree().inventory - - def _non_root_entries_iter(self, inv, revision_id): - if hasattr(inv, 'iter_non_root_entries'): - entries = inv.iter_non_root_entries() - else: - path_entries = inv.iter_entries() - # Backwards compatibility hack: skip the root id. - if not self.repo.supports_rich_root(): - path, root = path_entries.next() - if root.revision != revision_id: - raise errors.IncompatibleRevision(repr(self.repo)) - entries = iter([ie for path, ie in path_entries]) - return entries - - def _load_texts(self, revision_id, entries, text_provider, - parents_provider): - """Load texts to a repository for inventory entries. - - This method is provided for subclasses to use or override. - - :param revision_id: the revision identifier - :param entries: iterator over the inventory entries - :param text_provider: a callable expecting a file_id parameter - that returns the text for that file-id - :param parents_provider: a callable expecting a file_id parameter - that return the list of parent-ids for that file-id - """ - raise NotImplementedError(self._load_texts) - - def _add_inventory(self, revision_id, inv, parents, parent_invs): - """Add the inventory inv to the repository as revision_id. - - :param parents: The revision ids of the parents that revision_id - is known to have and are in the repository already. - :param parent_invs: the parent inventories - - :returns: The validator(which is a sha1 digest, though what is sha'd is - repository format specific) of the serialized inventory. - """ - return self.repo.add_inventory(revision_id, inv, parents) - - def _add_inventory_by_delta(self, revision_id, basis_inv, inv_delta, - parents, parent_invs): - """Add the inventory to the repository as revision_id. - - :param basis_inv: the basis Inventory or CHKInventory - :param inv_delta: the inventory delta - :param parents: The revision ids of the parents that revision_id - is known to have and are in the repository already. - :param parent_invs: the parent inventories - - :returns: (validator, inv) where validator is the validator - (which is a sha1 digest, though what is sha'd is repository format - specific) of the serialized inventory; - inv is the generated inventory - """ - if len(parents): - if self._supports_chks: - try: - validator, new_inv = self.repo.add_inventory_by_delta(parents[0], - inv_delta, revision_id, parents, basis_inv=basis_inv, - propagate_caches=False) - except errors.InconsistentDelta: - #print "BASIS INV IS\n%s\n" % "\n".join([str(i) for i in basis_inv.iter_entries_by_dir()]) - trace.mutter("INCONSISTENT DELTA IS:\n%s\n" % "\n".join([str(i) for i in inv_delta])) - raise - else: - validator, new_inv = self.repo.add_inventory_by_delta(parents[0], - inv_delta, revision_id, parents) - else: - if isinstance(basis_inv, inventory.CHKInventory): - new_inv = basis_inv.create_by_apply_delta(inv_delta, revision_id) - else: - new_inv = inventory.Inventory(revision_id=revision_id) - # This is set in the delta so remove it to prevent a duplicate - del new_inv[inventory.ROOT_ID] - new_inv.apply_delta(inv_delta) - validator = self.repo.add_inventory(revision_id, new_inv, parents) - return validator, new_inv - - def _add_revision(self, rev, inv): - """Add a revision and its inventory to a repository. - - :param rev: the Revision - :param inv: the inventory - """ - self.repo.add_revision(rev.revision_id, rev, inv) - - def _default_inventories_provider(self, revision_ids): - """An inventories provider that queries the repository.""" - present = [] - inventories = [] - for revision_id in revision_ids: - if self.repo.has_revision(revision_id): - present.append(revision_id) - rev_tree = self.repo.revision_tree(revision_id) - else: - rev_tree = self.repo.revision_tree(None) - inventories.append(rev_tree.inventory) - return present, inventories - - -class RevisionStore1(AbstractRevisionStore): - """A RevisionStore that uses the old bzrlib Repository API. - - The old API was present until bzr.dev rev 3510. - """ - - def _load_texts(self, revision_id, entries, text_provider, parents_provider): - """See RevisionStore._load_texts().""" - # Add the texts that are not already present - tx = self.repo.get_transaction() - for ie in entries: - # This test is *really* slow: over 50% of import time - #w = self.repo.weave_store.get_weave_or_empty(ie.file_id, tx) - #if ie.revision in w: - # continue - # Try another way, realising that this assumes that the - # version is not already there. In the general case, - # a shared repository might already have the revision but - # we arguably don't need that check when importing from - # a foreign system. - if ie.revision != revision_id: - continue - file_id = ie.file_id - text_parents = [(file_id, p) for p in parents_provider(file_id)] - lines = text_provider(file_id) - vfile = self.repo.weave_store.get_weave_or_empty(file_id, tx) - vfile.add_lines(revision_id, text_parents, lines) - - def get_file_lines(self, revision_id, file_id): - tx = self.repo.get_transaction() - w = self.repo.weave_store.get_weave(file_id, tx) - return w.get_lines(revision_id) - - def _add_revision(self, rev, inv): - # There's no need to do everything repo.add_revision does and - # doing so (since bzr.dev 3392) can be pretty slow for long - # delta chains on inventories. Just do the essentials here ... - _mod_revision.check_not_reserved_id(rev.revision_id) - self.repo._revision_store.add_revision(rev, self.repo.get_transaction()) - - -class RevisionStore2(AbstractRevisionStore): - """A RevisionStore that uses the new bzrlib Repository API.""" - - def _load_texts(self, revision_id, entries, text_provider, parents_provider): - """See RevisionStore._load_texts().""" - text_keys = {} - for ie in entries: - text_keys[(ie.file_id, ie.revision)] = ie - text_parent_map = self.repo.texts.get_parent_map(text_keys) - missing_texts = set(text_keys) - set(text_parent_map) - self._load_texts_for_file_rev_ids(missing_texts, text_provider, - parents_provider) - - def _load_texts_for_file_rev_ids(self, file_rev_ids, text_provider, - parents_provider): - """Load texts to a repository for file-ids, revision-id tuples. - - :param file_rev_ids: iterator over the (file_id, revision_id) tuples - :param text_provider: a callable expecting a file_id parameter - that returns the text for that file-id - :param parents_provider: a callable expecting a file_id parameter - that return the list of parent-ids for that file-id - """ - for file_id, revision_id in file_rev_ids: - text_key = (file_id, revision_id) - text_parents = [(file_id, p) for p in parents_provider(file_id)] - lines = text_provider(file_id) - #print "adding text for %s\n\tparents:%s" % (text_key,text_parents) - self.repo.texts.add_lines(text_key, text_parents, lines) - - def get_file_lines(self, revision_id, file_id): - record = self.repo.texts.get_record_stream([(file_id, revision_id)], - 'unordered', True).next() - if record.storage_kind == 'absent': - raise errors.RevisionNotPresent(record.key, self.repo) - return osutils.split_lines(record.get_bytes_as('fulltext')) - - # This is breaking imports into brisbane-core currently - #def _add_revision(self, rev, inv): - # # There's no need to do everything repo.add_revision does and - # # doing so (since bzr.dev 3392) can be pretty slow for long - # # delta chains on inventories. Just do the essentials here ... - # _mod_revision.check_not_reserved_id(rev.revision_id) - # self.repo._add_revision(rev) - - -class ImportRevisionStore1(RevisionStore1): - """A RevisionStore (old Repository API) optimised for importing. - - This implementation caches serialised inventory texts and provides - fine-grained control over when inventories are stored as fulltexts. - """ - - def __init__(self, repo, parent_texts_to_cache=1, fulltext_when=None, - random_ids=True): - """See AbstractRevisionStore.__init__. - - :param repository: the target repository - :param parent_text_to_cache: the number of parent texts to cache - :para fulltext_when: if non None, a function to call to decide - whether to fulltext the inventory or not. The revision count - is passed as a parameter and the result is treated as a boolean. - """ - RevisionStore1.__init__(self, repo) - self.inv_parent_texts = lru_cache.LRUCache(parent_texts_to_cache) - self.fulltext_when = fulltext_when - self.random_ids = random_ids - self.revision_count = 0 - - def _add_inventory(self, revision_id, inv, parents, parent_invs): - """See RevisionStore._add_inventory.""" - # Code taken from bzrlib.repository.add_inventory - assert self.repo.is_in_write_group() - _mod_revision.check_not_reserved_id(revision_id) - assert inv.revision_id is None or inv.revision_id == revision_id, \ - "Mismatch between inventory revision" \ - " id and insertion revid (%r, %r)" % (inv.revision_id, revision_id) - assert inv.root is not None - inv_lines = self.repo._serialise_inventory_to_lines(inv) - inv_vf = self.repo.get_inventory_weave() - sha1, num_bytes, parent_text = self._inventory_add_lines(inv_vf, - revision_id, parents, inv_lines, self.inv_parent_texts) - self.inv_parent_texts[revision_id] = parent_text - return sha1 - - def _inventory_add_lines(self, inv_vf, version_id, parents, lines, - parent_texts): - """See Repository._inventory_add_lines().""" - # setup parameters used in original code but not this API - self.revision_count += 1 - if self.fulltext_when is not None: - delta = not self.fulltext_when(self.revision_count) - else: - delta = inv_vf.delta - left_matching_blocks = None - random_id = self.random_ids - check_content = False - - # bzrlib.knit.add_lines() but error checking optimised - inv_vf._check_add(version_id, lines, random_id, check_content) - - #################################################################### - # bzrlib.knit._add() but skip checking if fulltext better than delta - #################################################################### - - line_bytes = ''.join(lines) - digest = osutils.sha_string(line_bytes) - present_parents = [] - for parent in parents: - if inv_vf.has_version(parent): - present_parents.append(parent) - if parent_texts is None: - parent_texts = {} - - # can only compress against the left most present parent. - if (delta and - (len(present_parents) == 0 or - present_parents[0] != parents[0])): - delta = False - - text_length = len(line_bytes) - options = [] - if lines: - if lines[-1][-1] != '\n': - # copy the contents of lines. - lines = lines[:] - options.append('no-eol') - lines[-1] = lines[-1] + '\n' - line_bytes += '\n' - - #if delta: - # # To speed the extract of texts the delta chain is limited - # # to a fixed number of deltas. This should minimize both - # # I/O and the time spend applying deltas. - # delta = inv_vf._check_should_delta(present_parents) - - assert isinstance(version_id, str) - content = inv_vf.factory.make(lines, version_id) - if delta or (inv_vf.factory.annotated and len(present_parents) > 0): - # Merge annotations from parent texts if needed. - delta_hunks = inv_vf._merge_annotations(content, present_parents, - parent_texts, delta, inv_vf.factory.annotated, - left_matching_blocks) - - if delta: - options.append('line-delta') - store_lines = inv_vf.factory.lower_line_delta(delta_hunks) - size, bytes = inv_vf._data._record_to_data(version_id, digest, - store_lines) - else: - options.append('fulltext') - # isinstance is slower and we have no hierarchy. - if inv_vf.factory.__class__ == knit.KnitPlainFactory: - # Use the already joined bytes saving iteration time in - # _record_to_data. - size, bytes = inv_vf._data._record_to_data(version_id, digest, - lines, [line_bytes]) - else: - # get mixed annotation + content and feed it into the - # serialiser. - store_lines = inv_vf.factory.lower_fulltext(content) - size, bytes = inv_vf._data._record_to_data(version_id, digest, - store_lines) - - access_memo = inv_vf._data.add_raw_records([size], bytes)[0] - inv_vf._index.add_versions( - ((version_id, options, access_memo, parents),), - random_id=random_id) - return digest, text_length, content @@ -1,24 +1,11 @@ #!/usr/bin/env python from distutils.core import setup -bzr_plugin_name = 'fastimport' - -bzr_plugin_version = (0, 9, 0, 'dev', 0) -bzr_minimum_version = (1, 1, 0) -bzr_maximum_version = None - -if __name__ == '__main__': - setup(name="fastimport", - version="0.9.0dev0", - description="stream-based import into and export from Bazaar.", - author="Canonical Ltd", - author_email="bazaar@lists.canonical.com", - license = "GNU GPL v2", - url="https://launchpad.net/bzr-fastimport", - scripts=[], - packages=['bzrlib.plugins.fastimport', - 'bzrlib.plugins.fastimport.exporters', - 'bzrlib.plugins.fastimport.processors', - 'bzrlib.plugins.fastimport.tests', - ], - package_dir={'bzrlib.plugins.fastimport': '.'}) +setup(name="fastimport", + version="0.9.0dev0", + description="VCS fastimport/fsatexport parser", + author="Canonical Ltd", + author_email="bazaar@lists.canonical.com", + license = "GNU GPL v2", + url="https://launchpad.net/python-fastimport", + packages=['fastimport', 'fastimport.tests']) diff --git a/tests/__init__.py b/tests/__init__.py deleted file mode 100644 index 549cf3e..0000000 --- a/tests/__init__.py +++ /dev/null @@ -1,36 +0,0 @@ -# Copyright (C) 2008 Canonical Limited. -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; version 2 of the License. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -# - -"""Tests for bzr-fastimport.""" - - -from bzrlib.tests.TestUtil import TestLoader, TestSuite - - -def test_suite(): - module_names = ['bzrlib.plugins.fastimport.' + x for x in [ - 'fastimport.tests.test_commands', - 'fastimport.tests.test_errors', - 'tests.test_branch_mapper', - 'tests.test_filter_processor', - 'tests.test_generic_processor', - 'tests.test_head_tracking', - 'tests.test_helpers', - 'tests.test_parser', - 'tests.test_revision_store', - ]] - loader = TestLoader() - return loader.loadTestsFromModuleNames(module_names) diff --git a/tests/test_branch_mapper.py b/tests/test_branch_mapper.py deleted file mode 100644 index 00450c9..0000000 --- a/tests/test_branch_mapper.py +++ /dev/null @@ -1,64 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Test the BranchMapper methods.""" - -from bzrlib import tests - -from bzrlib.plugins.fastimport import ( - branch_mapper, - ) - - -class TestBranchMapper(tests.TestCase): - - def test_git_to_bzr(self): - m = branch_mapper.BranchMapper() - for git, bzr in { - 'refs/heads/master': 'trunk', - 'refs/heads/foo': 'foo', - 'refs/tags/master': 'trunk.tag', - 'refs/tags/foo': 'foo.tag', - 'refs/remotes/origin/master': 'trunk.remote', - 'refs/remotes/origin/foo': 'foo.remote', - }.items(): - self.assertEqual(m.git_to_bzr(git), bzr) - - def test_git_to_bzr_with_slashes(self): - m = branch_mapper.BranchMapper() - for git, bzr in { - 'refs/heads/master/slave': 'master/slave', - 'refs/heads/foo/bar': 'foo/bar', - 'refs/tags/master/slave': 'master/slave.tag', - 'refs/tags/foo/bar': 'foo/bar.tag', - 'refs/remotes/origin/master/slave': 'master/slave.remote', - 'refs/remotes/origin/foo/bar': 'foo/bar.remote', - }.items(): - self.assertEqual(m.git_to_bzr(git), bzr) - - def test_git_to_bzr_for_trunk(self): - # As 'master' in git is mapped to trunk in bzr, we need to handle - # 'trunk' in git in a sensible way. - m = branch_mapper.BranchMapper() - for git, bzr in { - 'refs/heads/trunk': 'git-trunk', - 'refs/tags/trunk': 'git-trunk.tag', - 'refs/remotes/origin/trunk': 'git-trunk.remote', - 'refs/heads/git-trunk': 'git-git-trunk', - 'refs/tags/git-trunk': 'git-git-trunk.tag', - 'refs/remotes/origin/git-trunk':'git-git-trunk.remote', - }.items(): - self.assertEqual(m.git_to_bzr(git), bzr) diff --git a/tests/test_filter_processor.py b/tests/test_filter_processor.py deleted file mode 100644 index da5fdf2..0000000 --- a/tests/test_filter_processor.py +++ /dev/null @@ -1,879 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Test FilterProcessor""" - -from cStringIO import StringIO - -from testtools import TestCase - -from bzrlib.plugins.fastimport.fastimport import ( - parser, - ) - -from bzrlib.plugins.fastimport.processors import ( - filter_processor, - ) - - -# A sample input stream containing all (top level) import commands -_SAMPLE_ALL = \ -"""blob -mark :1 -data 4 -foo -commit refs/heads/master -mark :2 -committer Joe <joe@example.com> 1234567890 +1000 -data 14 -Initial import -M 644 :1 COPYING -checkpoint -progress first import done -reset refs/remote/origin/master -from :2 -tag v0.1 -from :2 -tagger Joe <joe@example.com> 1234567890 +1000 -data 12 -release v0.1 -""" - - -# A sample input stream creating the following tree: -# -# NEWS -# doc/README.txt -# doc/index.txt -_SAMPLE_WITH_DIR = \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 doc/README.txt -blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/master -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :2 NEWS -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :101 -M 644 :3 doc/README.txt -M 644 :4 doc/index.txt -""" - - -class TestCaseWithFiltering(TestCase): - - def assertFiltering(self, input, params, expected): - outf = StringIO() - proc = filter_processor.FilterProcessor( - None, params=params) - proc.outf = outf - s = StringIO(input) - p = parser.ImportParser(s) - proc.process(p.iter_commands) - out = outf.getvalue() - self.assertEquals(expected, out) - - -class TestNoFiltering(TestCaseWithFiltering): - - def test_params_not_given(self): - self.assertFiltering(_SAMPLE_ALL, None, _SAMPLE_ALL) - - def test_params_are_none(self): - params = {'include_paths': None, 'exclude_paths': None} - self.assertFiltering(_SAMPLE_ALL, params, _SAMPLE_ALL) - - -class TestIncludePaths(TestCaseWithFiltering): - - def test_file_in_root(self): - # Things to note: - # * only referenced blobs are retained - # * from clause is dropped from the first command - params = {'include_paths': ['NEWS']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/master -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -M 644 :2 NEWS -""") - - def test_file_in_subdir(self): - # Additional things to note: - # * new root: path is now index.txt, not doc/index.txt - # * other files changed in matching commits are excluded - params = {'include_paths': ['doc/index.txt']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -M 644 :4 index.txt -""") - - def test_file_with_changes(self): - # Additional things to note: - # * from updated to reference parents in the output - params = {'include_paths': ['doc/README.txt']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -""") - - def test_subdir(self): - params = {'include_paths': ['doc/']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -M 644 :4 index.txt -""") - - def test_multiple_files_in_subdir(self): - # The new root should be the subdrectory - params = {'include_paths': ['doc/README.txt', 'doc/index.txt']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -M 644 :4 index.txt -""") - - -class TestExcludePaths(TestCaseWithFiltering): - - def test_file_in_root(self): - params = {'exclude_paths': ['NEWS']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 doc/README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 doc/README.txt -M 644 :4 doc/index.txt -""") - - def test_file_in_subdir(self): - params = {'exclude_paths': ['doc/README.txt']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/master -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -M 644 :2 NEWS -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :101 -M 644 :4 doc/index.txt -""") - - def test_subdir(self): - params = {'exclude_paths': ['doc/']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/master -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -M 644 :2 NEWS -""") - - def test_multple_files(self): - params = {'exclude_paths': ['doc/index.txt', 'NEWS']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 doc/README.txt -blob -mark :3 -data 19 -Welcome! -my friend -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 doc/README.txt -""") - - -class TestIncludeAndExcludePaths(TestCaseWithFiltering): - - def test_included_dir_and_excluded_file(self): - params = {'include_paths': ['doc/'], 'exclude_paths': ['doc/index.txt']} - self.assertFiltering(_SAMPLE_WITH_DIR, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -""") - - -# A sample input stream creating the following tree: -# -# NEWS -# doc/README.txt -# doc/index.txt -# -# It then renames doc/README.txt => doc/README -_SAMPLE_WITH_RENAME_INSIDE = _SAMPLE_WITH_DIR + \ -"""commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -R doc/README.txt doc/README -""" - -# A sample input stream creating the following tree: -# -# NEWS -# doc/README.txt -# doc/index.txt -# -# It then renames doc/README.txt => README -_SAMPLE_WITH_RENAME_TO_OUTSIDE = _SAMPLE_WITH_DIR + \ -"""commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -R doc/README.txt README -""" - -# A sample input stream creating the following tree: -# -# NEWS -# doc/README.txt -# doc/index.txt -# -# It then renames NEWS => doc/NEWS -_SAMPLE_WITH_RENAME_TO_INSIDE = _SAMPLE_WITH_DIR + \ -"""commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -R NEWS doc/NEWS -""" - -class TestIncludePathsWithRenames(TestCaseWithFiltering): - - def test_rename_all_inside(self): - # These rename commands ought to be kept but adjusted for the new root - params = {'include_paths': ['doc/']} - self.assertFiltering(_SAMPLE_WITH_RENAME_INSIDE, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -M 644 :4 index.txt -commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -R README.txt README -""") - - def test_rename_to_outside(self): - # These rename commands become deletes - params = {'include_paths': ['doc/']} - self.assertFiltering(_SAMPLE_WITH_RENAME_TO_OUTSIDE, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -M 644 :4 index.txt -commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -D README.txt -""") - - def test_rename_to_inside(self): - # This ought to create a new file but doesn't yet - params = {'include_paths': ['doc/']} - self.assertFiltering(_SAMPLE_WITH_RENAME_TO_INSIDE, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -M 644 :4 index.txt -""") - - -# A sample input stream creating the following tree: -# -# NEWS -# doc/README.txt -# doc/index.txt -# -# It then copies doc/README.txt => doc/README -_SAMPLE_WITH_COPY_INSIDE = _SAMPLE_WITH_DIR + \ -"""commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -C doc/README.txt doc/README -""" - -# A sample input stream creating the following tree: -# -# NEWS -# doc/README.txt -# doc/index.txt -# -# It then copies doc/README.txt => README -_SAMPLE_WITH_COPY_TO_OUTSIDE = _SAMPLE_WITH_DIR + \ -"""commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -C doc/README.txt README -""" - -# A sample input stream creating the following tree: -# -# NEWS -# doc/README.txt -# doc/index.txt -# -# It then copies NEWS => doc/NEWS -_SAMPLE_WITH_COPY_TO_INSIDE = _SAMPLE_WITH_DIR + \ -"""commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -C NEWS doc/NEWS -""" - - -class TestIncludePathsWithCopies(TestCaseWithFiltering): - - def test_copy_all_inside(self): - # These copy commands ought to be kept but adjusted for the new root - params = {'include_paths': ['doc/']} - self.assertFiltering(_SAMPLE_WITH_COPY_INSIDE, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -M 644 :4 index.txt -commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 10 -move intro -from :102 -C README.txt README -""") - - def test_copy_to_outside(self): - # This can be ignored - params = {'include_paths': ['doc/']} - self.assertFiltering(_SAMPLE_WITH_COPY_TO_OUTSIDE, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -M 644 :4 index.txt -""") - - def test_copy_to_inside(self): - # This ought to create a new file but doesn't yet - params = {'include_paths': ['doc/']} - self.assertFiltering(_SAMPLE_WITH_COPY_TO_INSIDE, params, \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 README.txt -M 644 :4 index.txt -""") - - -# A sample input stream with deleteall's creating the following tree: -# -# NEWS -# doc/README.txt -# doc/index.txt -_SAMPLE_WITH_DELETEALL = \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -deleteall -M 644 :1 doc/README.txt -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -deleteall -M 644 :3 doc/README.txt -M 644 :4 doc/index.txt -""" - - -class TestIncludePathsWithDeleteAll(TestCaseWithFiltering): - - def test_deleteall(self): - params = {'include_paths': ['doc/index.txt']} - self.assertFiltering(_SAMPLE_WITH_DELETEALL, params, \ -"""blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -deleteall -M 644 :4 index.txt -""") - - -_SAMPLE_WITH_TAGS = _SAMPLE_WITH_DIR + \ -"""tag v0.1 -from :100 -tagger d <b@c> 1234798653 +0000 -data 12 -release v0.1 -tag v0.2 -from :102 -tagger d <b@c> 1234798653 +0000 -data 12 -release v0.2 -""" - -class TestIncludePathsWithTags(TestCaseWithFiltering): - - def test_tag_retention(self): - # If a tag references a commit with a parent we kept, - # keep the tag but adjust 'from' accordingly. - # Otherwise, delete the tag command. - params = {'include_paths': ['NEWS']} - self.assertFiltering(_SAMPLE_WITH_TAGS, params, \ -"""blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/master -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -M 644 :2 NEWS -tag v0.2 -from :101 -tagger d <b@c> 1234798653 +0000 -data 12 -release v0.2 -""") - - -_SAMPLE_WITH_RESETS = _SAMPLE_WITH_DIR + \ -"""reset refs/heads/foo -reset refs/heads/bar -from :102 -""" - -class TestIncludePathsWithResets(TestCaseWithFiltering): - - def test_reset_retention(self): - # Resets init'ing a branch (without a from) are passed through. - # If a reset references a commit with a parent we kept, - # keep the reset but adjust 'from' accordingly. - params = {'include_paths': ['NEWS']} - self.assertFiltering(_SAMPLE_WITH_RESETS, params, \ -"""blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/master -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -M 644 :2 NEWS -reset refs/heads/foo -reset refs/heads/bar -from :101 -""") diff --git a/tests/test_generic_processor.py b/tests/test_generic_processor.py deleted file mode 100644 index d479d09..0000000 --- a/tests/test_generic_processor.py +++ /dev/null @@ -1,1884 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -import time - -from bzrlib import ( - branch, - tests, - ) - -from bzrlib.plugins.fastimport.fastimport import ( - commands, - ) - -from bzrlib.plugins.fastimport.processors import ( - generic_processor, - ) - - -def load_tests(standard_tests, module, loader): - """Parameterize tests for all versions of groupcompress.""" - scenarios = [ - ('pack-0.92', {'branch_format': 'pack-0.92'}), - ('1.9-rich-root', {'branch_format': '1.9-rich-root'}), - ] - try: - from bzrlib.repofmt.groupcompress_repo import RepositoryFormat2a - scenarios.append(('2a', {'branch_format': '2a'})) - except ImportError: - pass - suite = loader.suiteClass() - result = tests.multiply_tests(standard_tests, scenarios, suite) - return result - - -class TestCaseForGenericProcessor(tests.TestCaseWithTransport): - - branch_format = "pack-0.92" - - def get_handler(self): - branch = self.make_branch('.', format=self.branch_format) - handler = generic_processor.GenericProcessor(branch.bzrdir) - return handler, branch - - # FIXME: [] as a default is bad, as it is mutable, but I want - # to use None to mean "don't check this". - def assertChanges(self, branch, revno, expected_added=[], - expected_removed=[], expected_modified=[], - expected_renamed=[], expected_kind_changed=[]): - """Check the changes introduced in a revision of a branch. - - This method checks that a revision introduces expected changes. - The required changes are passed in as a list, where - each entry contains the needed information about the change. - - If you do not wish to assert anything about a particular - category then pass None instead. - - branch: The branch. - revno: revision number of revision to check. - expected_added: a list of (filename,) tuples that must have - been added in the delta. - expected_removed: a list of (filename,) tuples that must have - been removed in the delta. - expected_modified: a list of (filename,) tuples that must have - been modified in the delta. - expected_renamed: a list of (old_path, new_path) tuples that - must have been renamed in the delta. - expected_kind_changed: a list of (path, old_kind, new_kind) tuples - that must have been changed in the delta. - :return: revtree1, revtree2 - """ - repo = branch.repository - revtree1 = repo.revision_tree(branch.get_rev_id(revno - 1)) - revtree2 = repo.revision_tree(branch.get_rev_id(revno)) - changes = revtree2.changes_from(revtree1) - self._check_changes(changes, expected_added, expected_removed, - expected_modified, expected_renamed, expected_kind_changed) - return revtree1, revtree2 - - def _check_changes(self, changes, expected_added=[], - expected_removed=[], expected_modified=[], - expected_renamed=[], expected_kind_changed=[]): - """Check the changes in a TreeDelta - - This method checks that the TreeDelta contains the expected - modifications between the two trees that were used to generate - it. The required changes are passed in as a list, where - each entry contains the needed information about the change. - - If you do not wish to assert anything about a particular - category then pass None instead. - - changes: The TreeDelta to check. - expected_added: a list of (filename,) tuples that must have - been added in the delta. - expected_removed: a list of (filename,) tuples that must have - been removed in the delta. - expected_modified: a list of (filename,) tuples that must have - been modified in the delta. - expected_renamed: a list of (old_path, new_path) tuples that - must have been renamed in the delta. - expected_kind_changed: a list of (path, old_kind, new_kind) tuples - that must have been changed in the delta. - """ - renamed = changes.renamed - added = changes.added - removed = changes.removed - modified = changes.modified - kind_changed = changes.kind_changed - if expected_renamed is not None: - self.assertEquals(len(renamed), len(expected_renamed), - "%s is renamed, expected %s" % (renamed, expected_renamed)) - renamed_files = [(item[0], item[1]) for item in renamed] - for expected_renamed_entry in expected_renamed: - self.assertTrue(expected_renamed_entry in renamed_files, - "%s is not renamed, %s are" % (str(expected_renamed_entry), - renamed_files)) - if expected_added is not None: - self.assertEquals(len(added), len(expected_added), - "%s is added" % str(added)) - added_files = [(item[0],) for item in added] - for expected_added_entry in expected_added: - self.assertTrue(expected_added_entry in added_files, - "%s is not added, %s are" % (str(expected_added_entry), - added_files)) - if expected_removed is not None: - self.assertEquals(len(removed), len(expected_removed), - "%s is removed" % str(removed)) - removed_files = [(item[0],) for item in removed] - for expected_removed_entry in expected_removed: - self.assertTrue(expected_removed_entry in removed_files, - "%s is not removed, %s are" % (str(expected_removed_entry), - removed_files)) - if expected_modified is not None: - self.assertEquals(len(modified), len(expected_modified), - "%s is modified" % str(modified)) - modified_files = [(item[0],) for item in modified] - for expected_modified_entry in expected_modified: - self.assertTrue(expected_modified_entry in modified_files, - "%s is not modified, %s are" % ( - str(expected_modified_entry), modified_files)) - if expected_kind_changed is not None: - self.assertEquals(len(kind_changed), len(expected_kind_changed), - "%s is kind-changed, expected %s" % (kind_changed, - expected_kind_changed)) - kind_changed_files = [(item[0], item[2], item[3]) - for item in kind_changed] - for expected_kind_changed_entry in expected_kind_changed: - self.assertTrue(expected_kind_changed_entry in - kind_changed_files, "%s is not kind-changed, %s are" % ( - str(expected_kind_changed_entry), kind_changed_files)) - - def assertContent(self, branch, tree, path, content): - file_id = tree.inventory.path2id(path) - branch.lock_read() - self.addCleanup(branch.unlock) - self.assertEqual(tree.get_file_text(file_id), content) - - def assertSymlinkTarget(self, branch, tree, path, target): - file_id = tree.inventory.path2id(path) - branch.lock_read() - self.addCleanup(branch.unlock) - self.assertEqual(tree.get_symlink_target(file_id), target) - - def assertExecutable(self, branch, tree, path, executable): - file_id = tree.inventory.path2id(path) - branch.lock_read() - self.addCleanup(branch.unlock) - self.assertEqual(tree.is_executable(file_id), executable) - - def assertRevisionRoot(self, revtree, path): - self.assertEqual(revtree.get_revision_id(), - revtree.inventory.root.children[path].revision) - - -class TestImportToPackModify(TestCaseForGenericProcessor): - - def file_command_iter(self, path, kind='file', content='aaa', - executable=False, to_kind=None, to_content='bbb', to_executable=None): - # Revno 1: create a file or symlink - # Revno 2: modify it - if to_kind is None: - to_kind = kind - if to_executable is None: - to_executable = executable - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(path, kind, executable, - None, content) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileModifyCommand(path, to_kind, to_executable, - None, to_content) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_modify_file_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(path,)]) - self.assertContent(branch, revtree1, path, "aaa") - self.assertContent(branch, revtree2, path, "bbb") - self.assertRevisionRoot(revtree1, path) - self.assertRevisionRoot(revtree2, path) - - def test_modify_file_in_subdir(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(path,)]) - self.assertContent(branch, revtree1, path, "aaa") - self.assertContent(branch, revtree2, path, "bbb") - - def test_modify_symlink_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path, kind='symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(path,)]) - self.assertSymlinkTarget(branch, revtree1, path, "aaa") - self.assertSymlinkTarget(branch, revtree2, path, "bbb") - self.assertRevisionRoot(revtree1, path) - self.assertRevisionRoot(revtree2, path) - - def test_modify_symlink_in_subdir(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path, kind='symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(path,)]) - self.assertSymlinkTarget(branch, revtree1, path, "aaa") - self.assertSymlinkTarget(branch, revtree2, path, "bbb") - - def test_modify_file_becomes_symlink(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path, - kind='file', to_kind='symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_kind_changed=[(path, 'file', 'symlink')]) - self.assertContent(branch, revtree1, path, "aaa") - self.assertSymlinkTarget(branch, revtree2, path, "bbb") - - def test_modify_symlink_becomes_file(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path, - kind='symlink', to_kind='file')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_kind_changed=[(path, 'symlink', 'file')]) - self.assertSymlinkTarget(branch, revtree1, path, "aaa") - self.assertContent(branch, revtree2, path, "bbb") - - def test_modify_file_now_executable(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path, - executable=False, to_executable=True, to_content='aaa')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(path,)]) - self.assertExecutable(branch, revtree1, path, False) - self.assertExecutable(branch, revtree2, path, True) - - def test_modify_file_no_longer_executable(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path, - executable=True, to_executable=False, to_content='aaa')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(path,)]) - self.assertExecutable(branch, revtree1, path, True) - self.assertExecutable(branch, revtree2, path, False) - - -class TestImportToPackModifyTwice(TestCaseForGenericProcessor): - """This tests when the same file is modified twice in the one commit. - - Note: hg-fast-export produces data like this on occasions. - """ - - def file_command_iter(self, path, kind='file', content='aaa', - executable=False, to_kind=None, to_content='bbb', to_executable=None): - # Revno 1: create a file twice - if to_kind is None: - to_kind = kind - if to_executable is None: - to_executable = executable - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(path, kind, executable, - None, content) - yield commands.FileModifyCommand(path, to_kind, to_executable, - None, to_content) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - return command_list - - def test_modify_file_twice_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(path,)]) - self.assertContent(branch, revtree1, path, "aaa") - self.assertRevisionRoot(revtree1, path) - - -class TestImportToPackModifyTricky(TestCaseForGenericProcessor): - - def file_command_iter(self, path1, path2, kind='file'): - # Revno 1: create a file or symlink in a directory - # Revno 2: create a second file that implicitly deletes the - # first one because either: - # * the new file is a in directory with the old file name - # * the new file has the same name as the directory of the first - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(path1, kind, False, - None, "aaa") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileModifyCommand(path2, kind, False, - None, "bbb") - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - - def test_modify_file_becomes_directory(self): - handler, branch = self.get_handler() - path1 = 'a/b' - path2 = 'a/b/c' - handler.process(self.file_command_iter(path1, path2)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path1,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_added=[(path2,)], - expected_kind_changed=[(path1, 'file', 'directory')]) - self.assertContent(branch, revtree1, path1, "aaa") - self.assertContent(branch, revtree2, path2, "bbb") - - def test_modify_directory_becomes_file(self): - handler, branch = self.get_handler() - path1 = 'a/b/c' - path2 = 'a/b' - handler.process(self.file_command_iter(path1, path2)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), ('a/b',), (path1,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(path1,),], - expected_kind_changed=[(path2, 'directory', 'file')]) - self.assertContent(branch, revtree1, path1, "aaa") - self.assertContent(branch, revtree2, path2, "bbb") - - def test_modify_symlink_becomes_directory(self): - handler, branch = self.get_handler() - path1 = 'a/b' - path2 = 'a/b/c' - handler.process(self.file_command_iter(path1, path2, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path1,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_added=[(path2,)], - expected_kind_changed=[(path1, 'symlink', 'directory')]) - self.assertSymlinkTarget(branch, revtree1, path1, "aaa") - self.assertSymlinkTarget(branch, revtree2, path2, "bbb") - - def test_modify_directory_becomes_symlink(self): - handler, branch = self.get_handler() - path1 = 'a/b/c' - path2 = 'a/b' - handler.process(self.file_command_iter(path1, path2, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), ('a/b',), (path1,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(path1,),], - expected_kind_changed=[(path2, 'directory', 'symlink')]) - self.assertSymlinkTarget(branch, revtree1, path1, "aaa") - self.assertSymlinkTarget(branch, revtree2, path2, "bbb") - - -class TestImportToPackDelete(TestCaseForGenericProcessor): - - def file_command_iter(self, path, kind='file'): - # Revno 1: create a file or symlink - # Revno 2: delete it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(path, kind, False, - None, "aaa") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileDeleteCommand(path) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_delete_file_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(path,)]) - self.assertContent(branch, revtree1, path, "aaa") - self.assertRevisionRoot(revtree1, path) - - def test_delete_file_in_subdir(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('a',), (path,)]) - self.assertContent(branch, revtree1, path, "aaa") - - def test_delete_symlink_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path, kind='symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(path,)]) - self.assertSymlinkTarget(branch, revtree1, path, "aaa") - self.assertRevisionRoot(revtree1, path) - - def test_delete_symlink_in_subdir(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path, kind='symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('a',), (path,)]) - self.assertSymlinkTarget(branch, revtree1, path, "aaa") - - def test_delete_file_in_deep_subdir(self): - handler, branch = self.get_handler() - path = 'a/b/c/d' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), ('a/b',), ('a/b/c',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('a',), ('a/b',), ('a/b/c',), (path,)]) - self.assertContent(branch, revtree1, path, "aaa") - - -class TestImportToPackDeleteNew(TestCaseForGenericProcessor): - """Test deletion of a newly added file.""" - - def file_command_iter(self, path, kind='file'): - # Revno 1: create a file or symlink then delete it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(path, kind, False, - None, "aaa") - yield commands.FileDeleteCommand(path) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - return command_list - - def test_delete_new_file_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1,) - - def test_delete_new_file_in_subdir(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1,) - - def test_delete_new_symlink_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path, kind='symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1,) - - def test_delete_new_symlink_in_subdir(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path, kind='symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1,) - - def test_delete_new_file_in_deep_subdir(self): - handler, branch = self.get_handler() - path = 'a/b/c/d' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1,) - - -class TestImportToPackDeleteMultiLevel(TestCaseForGenericProcessor): - - def file_command_iter(self, paths, paths_to_delete): - # Revno 1: create multiple files - # Revno 2: delete multiple files - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - for i, path in enumerate(paths): - yield commands.FileModifyCommand(path, 'file', False, - None, "aaa%d" % i) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - for path in paths_to_delete: - yield commands.FileDeleteCommand(path) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_delete_files_in_multiple_levels(self): - handler, branch = self.get_handler() - paths = ['a/b/c', 'a/b/d/e'] - paths_to_delete = ['a/b/c', 'a/b/d/e'] - handler.process(self.file_command_iter(paths, paths_to_delete)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[ - ('a',), ('a/b',), ('a/b/c',), - ('a/b/d',), ('a/b/d/e',), - ]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[ - ('a',), ('a/b',), ('a/b/c',), - ('a/b/d',), ('a/b/d/e',), - ]) - - def test_delete_file_single_level(self): - handler, branch = self.get_handler() - paths = ['a/b/c', 'a/b/d/e'] - paths_to_delete = ['a/b/d/e'] - handler.process(self.file_command_iter(paths, paths_to_delete)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[ - ('a',), ('a/b',), ('a/b/c',), - ('a/b/d',), ('a/b/d/e',), - ]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[ - ('a/b/d',), ('a/b/d/e',), - ]) - - def test_delete_file_complex_level(self): - handler, branch = self.get_handler() - paths = ['a/b/c', 'a/b/d/e', 'a/f/g', 'a/h', 'a/b/d/i/j'] - paths_to_delete = ['a/b/c', 'a/b/d/e', 'a/f/g', 'a/b/d/i/j'] - handler.process(self.file_command_iter(paths, paths_to_delete)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[ - ('a',), ('a/b',), ('a/b/c',), - ('a/b/d',), ('a/b/d/e',), - ('a/f',), ('a/f/g',), - ('a/h',), - ('a/b/d/i',), ('a/b/d/i/j',), - ]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[ - ('a/b',), ('a/b/c',), - ('a/b/d',), ('a/b/d/e',), - ('a/f',), ('a/f/g',), - ('a/b/d/i',), ('a/b/d/i/j',), - ]) - -class TestImportToPackDeleteThenAdd(TestCaseForGenericProcessor): - """Test delete followed by an add. Merges can cause this.""" - - def file_command_iter(self, path, kind='file', content='aaa', - executable=False, to_kind=None, to_content='bbb', to_executable=None): - # Revno 1: create a file or symlink - # Revno 2: delete it and add it - if to_kind is None: - to_kind = kind - if to_executable is None: - to_executable = executable - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(path, kind, executable, - None, content) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileDeleteCommand(path) - yield commands.FileModifyCommand(path, to_kind, to_executable, - None, to_content) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_delete_then_add_file_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(path,)], - expected_added=[(path,)]) - self.assertContent(branch, revtree1, path, "aaa") - self.assertContent(branch, revtree2, path, "bbb") - self.assertRevisionRoot(revtree1, path) - self.assertRevisionRoot(revtree2, path) - - def test_delete_then_add_file_in_subdir(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(path,)], - expected_added=[(path,)]) - self.assertContent(branch, revtree1, path, "aaa") - self.assertContent(branch, revtree2, path, "bbb") - - def test_delete_then_add_symlink_in_root(self): - handler, branch = self.get_handler() - path = 'a' - handler.process(self.file_command_iter(path, kind='symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(path,)], - expected_added=[(path,)]) - self.assertSymlinkTarget(branch, revtree1, path, "aaa") - self.assertSymlinkTarget(branch, revtree2, path, "bbb") - self.assertRevisionRoot(revtree1, path) - self.assertRevisionRoot(revtree2, path) - - def test_delete_then_add_symlink_in_subdir(self): - handler, branch = self.get_handler() - path = 'a/a' - handler.process(self.file_command_iter(path, kind='symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(path,)], - expected_added=[(path,)]) - self.assertSymlinkTarget(branch, revtree1, path, "aaa") - self.assertSymlinkTarget(branch, revtree2, path, "bbb") - - -class TestImportToPackDeleteDirectory(TestCaseForGenericProcessor): - - def file_command_iter(self, paths, dir): - # Revno 1: create multiple files - # Revno 2: delete a directory holding those files - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - for i, path in enumerate(paths): - yield commands.FileModifyCommand(path, 'file', False, - None, "aaa%d" % i) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileDeleteCommand(dir) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_delete_dir(self): - handler, branch = self.get_handler() - paths = ['a/b/c', 'a/b/d', 'a/b/e/f', 'a/g'] - dir = 'a/b' - handler.process(self.file_command_iter(paths, dir)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[ - ('a',), ('a/b',), ('a/b/c',), - ('a/b/d',), - ('a/b/e',), ('a/b/e/f',), - ('a/g',), - ]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[ - ('a/b',), ('a/b/c',), - ('a/b/d',), - ('a/b/e',), ('a/b/e/f',), - ]) - - -class TestImportToPackDeleteDirectoryThenAddFile(TestCaseForGenericProcessor): - """Test deleting a directory then adding a file in the same commit.""" - - def file_command_iter(self, paths, dir, new_path, kind='file'): - # Revno 1: create files in a directory - # Revno 2: delete the directory then add a file into it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - for i, path in enumerate(paths): - yield commands.FileModifyCommand(path, kind, False, - None, "aaa%d" % i) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileDeleteCommand(dir) - yield commands.FileModifyCommand(new_path, kind, False, - None, "bbb") - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_delete_dir_then_add_file(self): - handler, branch = self.get_handler() - paths = ['a/b/c', 'a/b/d'] - dir = 'a/b' - new_path = 'a/b/z' - handler.process(self.file_command_iter(paths, dir, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), ('a/b',), ('a/b/c',), ('a/b/d',),]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('a/b',), ('a/b/c',), ('a/b/d',)], - expected_added=[('a/b',), ('a/b/z',)]) - self.assertContent(branch, revtree2, new_path, "bbb") - - def test_delete_dir_then_add_symlink(self): - handler, branch = self.get_handler() - paths = ['a/b/c', 'a/b/d'] - dir = 'a/b' - new_path = 'a/b/z' - handler.process(self.file_command_iter(paths, dir, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), ('a/b',), ('a/b/c',), ('a/b/d',),]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('a/b',), ('a/b/c',), ('a/b/d',)], - expected_added=[('a/b',), ('a/b/z',)]) - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - - -class TestImportToPackRename(TestCaseForGenericProcessor): - - def get_command_iter(self, old_path, new_path, kind='file'): - # Revno 1: create a file or symlink - # Revno 2: rename it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(old_path, kind, False, - None, "aaa") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileRenameCommand(old_path, new_path) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_rename_file_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree2, new_path) - - def test_rename_symlink_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree2, new_path) - - def test_rename_file_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'a/a' - new_path = 'a/b' - handler.process(self.get_command_iter(old_path, new_path)) - self.assertChanges(branch, 2, expected_renamed=[(old_path, new_path)]) - - def test_rename_symlink_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'a/a' - new_path = 'a/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - self.assertChanges(branch, 2, expected_renamed=[(old_path, new_path)]) - - def test_rename_file_to_new_dir(self): - handler, branch = self.get_handler() - old_path = 'a/a' - new_path = 'b/a' - handler.process(self.get_command_iter(old_path, new_path)) - self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)], - expected_added=[('b',)], - expected_removed=[('a',)]) - - def test_rename_symlink_to_new_dir(self): - handler, branch = self.get_handler() - old_path = 'a/a' - new_path = 'b/a' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)], - expected_added=[('b',)], - expected_removed=[('a',)]) - - -class TestImportToPackRenameNew(TestCaseForGenericProcessor): - """Test rename of a newly added file.""" - - def get_command_iter(self, old_path, new_path, kind='file'): - # Revno 1: create a file and rename it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(old_path, kind, False, - None, "aaa") - yield commands.FileRenameCommand(old_path, new_path) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - return command_list - - def test_rename_new_file_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(new_path,)]) - self.assertRevisionRoot(revtree1, new_path) - - def test_rename_new_symlink_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(new_path,)]) - self.assertRevisionRoot(revtree1, new_path) - - def test_rename_new_file_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'a/a' - new_path = 'a/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (new_path,)]) - - def test_rename_new_symlink_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'a/a' - new_path = 'a/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (new_path,)]) - - -class TestImportToPackRenameToDeleted(TestCaseForGenericProcessor): - """Test rename to a destination path deleted in this commit.""" - - def get_command_iter(self, old_path, new_path, kind='file'): - # Revno 1: create two files - # Revno 2: delete one, rename the other one to that path - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(old_path, kind, False, - None, "aaa") - yield commands.FileModifyCommand(new_path, kind, False, - None, "bbb") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileDeleteCommand(new_path) - yield commands.FileRenameCommand(old_path, new_path) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_rename_to_deleted_file_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(old_path,), (new_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree1, new_path, "bbb") - self.assertContent(branch, revtree2, new_path, "aaa") - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree1, new_path) - - def test_rename_to_deleted_symlink_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(old_path,), (new_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, new_path, "bbb") - self.assertSymlinkTarget(branch, revtree2, new_path, "aaa") - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree1, new_path) - - def test_rename_to_deleted_file_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'd/a' - new_path = 'd/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (old_path,), (new_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree1, new_path, "bbb") - self.assertContent(branch, revtree2, new_path, "aaa") - - def test_rename_to_deleted_symlink_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'd/a' - new_path = 'd/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (old_path,), (new_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, new_path, "bbb") - self.assertSymlinkTarget(branch, revtree2, new_path, "aaa") - - def test_rename_to_deleted_file_in_new_dir(self): - handler, branch = self.get_handler() - old_path = 'd1/a' - new_path = 'd2/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d1',), (old_path,), ('d2',), (new_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('d1',), (new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree1, new_path, "bbb") - self.assertContent(branch, revtree2, new_path, "aaa") - - def test_rename_to_deleted_symlink_in_new_dir(self): - handler, branch = self.get_handler() - old_path = 'd1/a' - new_path = 'd2/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d1',), (old_path,), ('d2',), (new_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('d1',), (new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, new_path, "bbb") - self.assertSymlinkTarget(branch, revtree2, new_path, "aaa") - - -class TestImportToPackRenameModified(TestCaseForGenericProcessor): - """Test rename of a path previously modified in this commit.""" - - def get_command_iter(self, old_path, new_path, kind='file'): - # Revno 1: create a file or symlink - # Revno 2: modify then rename it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(old_path, kind, False, - None, "aaa") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileModifyCommand(old_path, kind, False, - None, "bbb") - yield commands.FileRenameCommand(old_path, new_path) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_rename_of_modified_file_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree2, new_path, "bbb") - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree2, new_path) - - def test_rename_of_modified_symlink_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree2, new_path) - - def test_rename_of_modified_file_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'd/a' - new_path = 'd/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree2, new_path, "bbb") - - def test_rename_of_modified_symlink_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'd/a' - new_path = 'd/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - - def test_rename_of_modified_file_to_new_dir(self): - handler, branch = self.get_handler() - old_path = 'd1/a' - new_path = 'd2/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d1',), (old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)], - expected_added=[('d2',)], - expected_removed=[('d1',)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree2, new_path, "bbb") - - def test_rename_of_modified_symlink_to_new_dir(self): - handler, branch = self.get_handler() - old_path = 'd1/a' - new_path = 'd2/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d1',), (old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)], - expected_added=[('d2',)], - expected_removed=[('d1',)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - - -class TestImportToPackRenameThenModify(TestCaseForGenericProcessor): - """Test rename of a path then modfy the new-path in the same commit.""" - - def get_command_iter(self, old_path, new_path, kind='file'): - # Revno 1: create a file or symlink - # Revno 2: rename it then modify the newly created path - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(old_path, kind, False, - None, "aaa") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileRenameCommand(old_path, new_path) - yield commands.FileModifyCommand(new_path, kind, False, - None, "bbb") - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_rename_then_modify_file_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree2, new_path, "bbb") - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree2, new_path) - - def test_rename_then_modify_file_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'd/a' - new_path = 'd/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree2, new_path, "bbb") - - def test_rename_then_modify_file_in_new_dir(self): - handler, branch = self.get_handler() - old_path = 'd1/a' - new_path = 'd2/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d1',), (old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)], - expected_added=[('d2',)], - expected_removed=[('d1',)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree2, new_path, "bbb") - - def test_rename_then_modify_symlink_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree2, new_path) - - def test_rename_then_modify_symlink_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'd/a' - new_path = 'd/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - - def test_rename_then_modify_symlink_in_new_dir(self): - handler, branch = self.get_handler() - old_path = 'd1/a' - new_path = 'd2/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d1',), (old_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path, new_path)], - expected_added=[('d2',)], - expected_removed=[('d1',)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - - -class TestImportToPackDeleteRenameThenModify(TestCaseForGenericProcessor): - """Test rename of to a deleted path then modfy the new-path in the same commit.""" - - def get_command_iter(self, old_path, new_path, kind='file'): - # Revno 1: create two files or symlinks - # Revno 2: delete one, rename the other to it then modify the newly created path - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(old_path, kind, False, - None, "aaa") - yield commands.FileModifyCommand(new_path, kind, False, - None, "zzz") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileDeleteCommand(new_path) - yield commands.FileRenameCommand(old_path, new_path) - yield commands.FileModifyCommand(new_path, kind, False, - None, "bbb") - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_delete_rename_then_modify_file_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(old_path,), (new_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree1, new_path, "zzz") - self.assertContent(branch, revtree2, new_path, "bbb") - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree1, new_path) - self.assertRevisionRoot(revtree2, new_path) - - def test_delete_rename_then_modify_file_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'd/a' - new_path = 'd/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (old_path,), (new_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree1, new_path, "zzz") - self.assertContent(branch, revtree2, new_path, "bbb") - - def test_delete_rename_then_modify_file_in_new_dir(self): - handler, branch = self.get_handler() - old_path = 'd1/a' - new_path = 'd2/b' - handler.process(self.get_command_iter(old_path, new_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d1',), ('d2',), (old_path,), (new_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('d1',), (new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertContent(branch, revtree1, old_path, "aaa") - self.assertContent(branch, revtree1, new_path, "zzz") - self.assertContent(branch, revtree2, new_path, "bbb") - - def test_delete_rename_then_modify_symlink_in_root(self): - handler, branch = self.get_handler() - old_path = 'a' - new_path = 'b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(old_path,), (new_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, new_path, "zzz") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - self.assertRevisionRoot(revtree1, old_path) - self.assertRevisionRoot(revtree1, new_path) - self.assertRevisionRoot(revtree2, new_path) - - def test_delete_rename_then_modify_symlink_in_subdir(self): - handler, branch = self.get_handler() - old_path = 'd/a' - new_path = 'd/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (old_path,), (new_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, new_path, "zzz") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - - def test_delete_rename_then_modify_symlink_in_new_dir(self): - handler, branch = self.get_handler() - old_path = 'd1/a' - new_path = 'd2/b' - handler.process(self.get_command_iter(old_path, new_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d1',), ('d2',), (old_path,), (new_path,)]) - # Note: the delta doesn't show the modification? - # The actual new content is validated in the assertions following. - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[('d1',), (new_path,)], - expected_renamed=[(old_path, new_path)]) - self.assertSymlinkTarget(branch, revtree1, old_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, new_path, "zzz") - self.assertSymlinkTarget(branch, revtree2, new_path, "bbb") - - -class TestImportToPackRenameTricky(TestCaseForGenericProcessor): - - def file_command_iter(self, path1, old_path2, new_path2, kind='file'): - # Revno 1: create two files or symlinks in a directory - # Revno 2: rename the second file so that it implicitly deletes the - # first one because either: - # * the new file is a in directory with the old file name - # * the new file has the same name as the directory of the first - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(path1, kind, False, - None, "aaa") - yield commands.FileModifyCommand(old_path2, kind, False, - None, "bbb") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileRenameCommand(old_path2, new_path2) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_rename_file_becomes_directory(self): - handler, branch = self.get_handler() - old_path2 = 'foo' - path1 = 'a/b' - new_path2 = 'a/b/c' - handler.process(self.file_command_iter(path1, old_path2, new_path2)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path1,), (old_path2,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path2, new_path2)], - expected_kind_changed=[(path1, 'file', 'directory')]) - self.assertContent(branch, revtree1, path1, "aaa") - self.assertContent(branch, revtree2, new_path2, "bbb") - - def test_rename_directory_becomes_file(self): - handler, branch = self.get_handler() - old_path2 = 'foo' - path1 = 'a/b/c' - new_path2 = 'a/b' - handler.process(self.file_command_iter(path1, old_path2, new_path2)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), ('a/b',), (path1,), (old_path2,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path2, new_path2)], - expected_removed=[(path1,), (new_path2,)]) - self.assertContent(branch, revtree1, path1, "aaa") - self.assertContent(branch, revtree2, new_path2, "bbb") - - def test_rename_symlink_becomes_directory(self): - handler, branch = self.get_handler() - old_path2 = 'foo' - path1 = 'a/b' - new_path2 = 'a/b/c' - handler.process(self.file_command_iter(path1, old_path2, new_path2, - 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (path1,), (old_path2,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path2, new_path2)], - expected_kind_changed=[(path1, 'symlink', 'directory')]) - self.assertSymlinkTarget(branch, revtree1, path1, "aaa") - self.assertSymlinkTarget(branch, revtree2, new_path2, "bbb") - - def test_rename_directory_becomes_symlink(self): - handler, branch = self.get_handler() - old_path2 = 'foo' - path1 = 'a/b/c' - new_path2 = 'a/b' - handler.process(self.file_command_iter(path1, old_path2, new_path2, - 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), ('a/b',), (path1,), (old_path2,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_renamed=[(old_path2, new_path2)], - expected_removed=[(path1,), (new_path2,)]) - self.assertSymlinkTarget(branch, revtree1, path1, "aaa") - self.assertSymlinkTarget(branch, revtree2, new_path2, "bbb") - - -class TestImportToPackCopy(TestCaseForGenericProcessor): - - def file_command_iter(self, src_path, dest_path, kind='file'): - # Revno 1: create a file or symlink - # Revno 2: copy it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(src_path, kind, False, - None, "aaa") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileCopyCommand(src_path, dest_path) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_copy_file_in_root(self): - handler, branch = self.get_handler() - src_path = 'a' - dest_path = 'b' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_added=[(dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree2, src_path, "aaa") - self.assertContent(branch, revtree2, dest_path, "aaa") - self.assertRevisionRoot(revtree1, src_path) - self.assertRevisionRoot(revtree2, dest_path) - - def test_copy_file_in_subdir(self): - handler, branch = self.get_handler() - src_path = 'a/a' - dest_path = 'a/b' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_added=[(dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree2, src_path, "aaa") - self.assertContent(branch, revtree2, dest_path, "aaa") - - def test_copy_file_to_new_dir(self): - handler, branch = self.get_handler() - src_path = 'a/a' - dest_path = 'b/a' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_added=[('b',), (dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree2, src_path, "aaa") - self.assertContent(branch, revtree2, dest_path, "aaa") - - def test_copy_symlink_in_root(self): - handler, branch = self.get_handler() - src_path = 'a' - dest_path = 'b' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_added=[(dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, dest_path, "aaa") - self.assertRevisionRoot(revtree1, src_path) - self.assertRevisionRoot(revtree2, dest_path) - - def test_copy_symlink_in_subdir(self): - handler, branch = self.get_handler() - src_path = 'a/a' - dest_path = 'a/b' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_added=[(dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, dest_path, "aaa") - - def test_copy_symlink_to_new_dir(self): - handler, branch = self.get_handler() - src_path = 'a/a' - dest_path = 'b/a' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_added=[('b',), (dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, dest_path, "aaa") - - -class TestImportToPackCopyNew(TestCaseForGenericProcessor): - """Test copy of a newly added file.""" - - def file_command_iter(self, src_path, dest_path, kind='file'): - # Revno 1: create a file or symlink and copy it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(src_path, kind, False, - None, "aaa") - yield commands.FileCopyCommand(src_path, dest_path) - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - return command_list - - def test_copy_new_file_in_root(self): - handler, branch = self.get_handler() - src_path = 'a' - dest_path = 'b' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(src_path,), (dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree1, dest_path, "aaa") - self.assertRevisionRoot(revtree1, src_path) - self.assertRevisionRoot(revtree1, dest_path) - - def test_copy_new_file_in_subdir(self): - handler, branch = self.get_handler() - src_path = 'a/a' - dest_path = 'a/b' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (src_path,), (dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree1, dest_path, "aaa") - - def test_copy_new_file_to_new_dir(self): - handler, branch = self.get_handler() - src_path = 'a/a' - dest_path = 'b/a' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (src_path,), ('b',), (dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree1, dest_path, "aaa") - - def test_copy_new_symlink_in_root(self): - handler, branch = self.get_handler() - src_path = 'a' - dest_path = 'b' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(src_path,), (dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, dest_path, "aaa") - self.assertRevisionRoot(revtree1, src_path) - self.assertRevisionRoot(revtree1, dest_path) - - def test_copy_new_symlink_in_subdir(self): - handler, branch = self.get_handler() - src_path = 'a/a' - dest_path = 'a/b' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (src_path,), (dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, dest_path, "aaa") - - def test_copy_new_symlink_to_new_dir(self): - handler, branch = self.get_handler() - src_path = 'a/a' - dest_path = 'b/a' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('a',), (src_path,), ('b',), (dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, dest_path, "aaa") - - -class TestImportToPackCopyToDeleted(TestCaseForGenericProcessor): - - def file_command_iter(self, src_path, dest_path, kind='file'): - # Revno 1: create two files or symlinks - # Revno 2: delete one and copy the other one to its path - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(src_path, kind, False, - None, "aaa") - yield commands.FileModifyCommand(dest_path, kind, False, - None, "bbb") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileDeleteCommand(dest_path) - yield commands.FileCopyCommand(src_path, dest_path) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_copy_to_deleted_file_in_root(self): - handler, branch = self.get_handler() - src_path = 'a' - dest_path = 'b' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(src_path,), (dest_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(dest_path,)], - expected_added=[(dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree1, dest_path, "bbb") - self.assertContent(branch, revtree2, src_path, "aaa") - self.assertContent(branch, revtree2, dest_path, "aaa") - self.assertRevisionRoot(revtree1, src_path) - self.assertRevisionRoot(revtree1, dest_path) - - def test_copy_to_deleted_symlink_in_root(self): - handler, branch = self.get_handler() - src_path = 'a' - dest_path = 'b' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[(src_path,), (dest_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(dest_path,)], - expected_added=[(dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, dest_path, "bbb") - self.assertSymlinkTarget(branch, revtree2, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, dest_path, "aaa") - self.assertRevisionRoot(revtree1, src_path) - self.assertRevisionRoot(revtree1, dest_path) - - def test_copy_to_deleted_file_in_subdir(self): - handler, branch = self.get_handler() - src_path = 'd/a' - dest_path = 'd/b' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (src_path,), (dest_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(dest_path,)], - expected_added=[(dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree1, dest_path, "bbb") - self.assertContent(branch, revtree2, src_path, "aaa") - self.assertContent(branch, revtree2, dest_path, "aaa") - - def test_copy_to_deleted_symlink_in_subdir(self): - handler, branch = self.get_handler() - src_path = 'd/a' - dest_path = 'd/b' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree0, revtree1 = self.assertChanges(branch, 1, - expected_added=[('d',), (src_path,), (dest_path,)]) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_removed=[(dest_path,)], - expected_added=[(dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree1, dest_path, "bbb") - self.assertSymlinkTarget(branch, revtree2, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, dest_path, "aaa") - - -class TestImportToPackCopyModified(TestCaseForGenericProcessor): - """Test copy of file/symlink already modified in this commit.""" - - def file_command_iter(self, src_path, dest_path, kind='file'): - # Revno 1: create a file or symlink - # Revno 2: modify and copy it - def command_list(): - author = ['', 'bugs@a.com', time.time(), time.timezone] - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(src_path, kind, False, - None, "aaa") - yield commands.CommitCommand('head', '1', author, - committer, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileModifyCommand(src_path, kind, False, - None, "bbb") - yield commands.FileCopyCommand(src_path, dest_path) - yield commands.CommitCommand('head', '2', author, - committer, "commit 2", ":1", [], files_two) - return command_list - - def test_copy_of_modified_file_in_root(self): - handler, branch = self.get_handler() - src_path = 'a' - dest_path = 'b' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(src_path,)], - expected_added=[(dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree2, src_path, "bbb") - self.assertContent(branch, revtree2, dest_path, "bbb") - self.assertRevisionRoot(revtree1, src_path) - self.assertRevisionRoot(revtree2, dest_path) - - def test_copy_of_modified_file_in_subdir(self): - handler, branch = self.get_handler() - src_path = 'd/a' - dest_path = 'd/b' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(src_path,)], - expected_added=[(dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree2, src_path, "bbb") - self.assertContent(branch, revtree2, dest_path, "bbb") - - def test_copy_of_modified_file_to_new_dir(self): - handler, branch = self.get_handler() - src_path = 'd1/a' - dest_path = 'd2/a' - handler.process(self.file_command_iter(src_path, dest_path)) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(src_path,)], - expected_added=[('d2',), (dest_path,)]) - self.assertContent(branch, revtree1, src_path, "aaa") - self.assertContent(branch, revtree2, src_path, "bbb") - self.assertContent(branch, revtree2, dest_path, "bbb") - - def test_copy_of_modified_symlink_in_root(self): - handler, branch = self.get_handler() - src_path = 'a' - dest_path = 'b' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(src_path,)], - expected_added=[(dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, src_path, "bbb") - self.assertSymlinkTarget(branch, revtree2, dest_path, "bbb") - self.assertRevisionRoot(revtree1, src_path) - self.assertRevisionRoot(revtree2, dest_path) - - def test_copy_of_modified_symlink_in_subdir(self): - handler, branch = self.get_handler() - src_path = 'd/a' - dest_path = 'd/b' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(src_path,)], - expected_added=[(dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, src_path, "bbb") - self.assertSymlinkTarget(branch, revtree2, dest_path, "bbb") - - def test_copy_of_modified_symlink_to_new_dir(self): - handler, branch = self.get_handler() - src_path = 'd1/a' - dest_path = 'd2/a' - handler.process(self.file_command_iter(src_path, dest_path, 'symlink')) - revtree1, revtree2 = self.assertChanges(branch, 2, - expected_modified=[(src_path,)], - expected_added=[('d2',), (dest_path,)]) - self.assertSymlinkTarget(branch, revtree1, src_path, "aaa") - self.assertSymlinkTarget(branch, revtree2, src_path, "bbb") - self.assertSymlinkTarget(branch, revtree2, dest_path, "bbb") - - -class TestImportToPackFileKinds(TestCaseForGenericProcessor): - - def get_command_iter(self, path, kind, content): - def command_list(): - committer = ['', 'elmer@a.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand(path, kind, False, - None, content) - yield commands.CommitCommand('head', '1', None, - committer, "commit 1", None, [], files_one) - return command_list - - def test_import_plainfile(self): - handler, branch = self.get_handler() - handler.process(self.get_command_iter('foo', 'file', 'aaa')) - - def test_import_symlink(self): - handler, branch = self.get_handler() - handler.process(self.get_command_iter('foo', 'symlink', 'bar')) - - -class TestModifyRevertInBranch(TestCaseForGenericProcessor): - - def file_command_iter(self): - # A add 'foo' - # |\ - # | B modify 'foo' - # | | - # | C revert 'foo' back to A - # |/ - # D merge 'foo' - def command_list(): - committer_a = ['', 'a@elmer.com', time.time(), time.timezone] - committer_b = ['', 'b@elmer.com', time.time(), time.timezone] - committer_c = ['', 'c@elmer.com', time.time(), time.timezone] - committer_d = ['', 'd@elmer.com', time.time(), time.timezone] - def files_one(): - yield commands.FileModifyCommand('foo', 'file', False, - None, "content A\n") - yield commands.CommitCommand('head', '1', None, - committer_a, "commit 1", None, [], files_one) - def files_two(): - yield commands.FileModifyCommand('foo', 'file', False, - None, "content B\n") - yield commands.CommitCommand('head', '2', None, - committer_b, "commit 2", ":1", [], files_two) - def files_three(): - yield commands.FileModifyCommand('foo', 'file', False, - None, "content A\n") - yield commands.CommitCommand('head', '3', None, - committer_c, "commit 3", ":2", [], files_three) - yield commands.CommitCommand('head', '4', None, - committer_d, "commit 4", ":1", [':3'], lambda: []) - return command_list - - def test_modify_revert(self): - handler, branch = self.get_handler() - handler.process(self.file_command_iter()) - branch.lock_read() - self.addCleanup(branch.unlock) - rev_d = branch.last_revision() - rev_a, rev_c = branch.repository.get_parent_map([rev_d])[rev_d] - rev_b = branch.repository.get_parent_map([rev_c])[rev_c][0] - rtree_a, rtree_b, rtree_c, rtree_d = branch.repository.revision_trees([ - rev_a, rev_b, rev_c, rev_d]) - foo_id = rtree_a.path2id('foo') - self.assertEqual(rev_a, rtree_a.inventory[foo_id].revision) - self.assertEqual(rev_b, rtree_b.inventory[foo_id].revision) - self.assertEqual(rev_c, rtree_c.inventory[foo_id].revision) - self.assertEqual(rev_c, rtree_d.inventory[foo_id].revision) diff --git a/tests/test_head_tracking.py b/tests/test_head_tracking.py deleted file mode 100644 index e88e366..0000000 --- a/tests/test_head_tracking.py +++ /dev/null @@ -1,257 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Test tracking of heads""" - -from cStringIO import StringIO - -from bzrlib import tests - -from bzrlib.plugins.fastimport.fastimport import ( - commands, - parser, - ) -from bzrlib.plugins.fastimport.cache_manager import CacheManager - - -# A sample input stream that only adds files to a branch -_SAMPLE_MAINLINE = \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 doc/README.txt -blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/master -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :2 NEWS -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :101 -M 644 :3 doc/README.txt -M 644 :4 doc/index.txt -""" - -# A sample input stream that adds files to two branches -_SAMPLE_TWO_HEADS = \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 doc/README.txt -blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/mybranch -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :2 NEWS -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 doc/README.txt -M 644 :4 doc/index.txt -""" - -# A sample input stream that adds files to two branches -_SAMPLE_TWO_BRANCHES_MERGED = \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 doc/README.txt -blob -mark :2 -data 17 -Life -is -good ... -commit refs/heads/mybranch -mark :101 -committer a <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :2 NEWS -blob -mark :3 -data 19 -Welcome! -my friend -blob -mark :4 -data 11 -== Docs == -commit refs/heads/master -mark :102 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -M 644 :3 doc/README.txt -M 644 :4 doc/index.txt -commit refs/heads/master -mark :103 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :102 -merge :101 -D doc/index.txt -""" - -# A sample input stream that contains a reset -_SAMPLE_RESET = \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 doc/README.txt -reset refs/remotes/origin/master -from :100 -""" - -# A sample input stream that contains a reset and more commits -_SAMPLE_RESET_WITH_MORE_COMMITS = \ -"""blob -mark :1 -data 9 -Welcome! -commit refs/heads/master -mark :100 -committer a <b@c> 1234798653 +0000 -data 4 -test -M 644 :1 doc/README.txt -reset refs/remotes/origin/master -from :100 -commit refs/remotes/origin/master -mark :101 -committer d <b@c> 1234798653 +0000 -data 8 -test -ing -from :100 -D doc/README.txt -""" - -class TestHeadTracking(tests.TestCase): - - def assertHeads(self, input, expected): - s = StringIO(input) - p = parser.ImportParser(s) - cm = CacheManager() - for cmd in p.iter_commands(): - if isinstance(cmd, commands.CommitCommand): - cm.track_heads(cmd) - # eat the file commands - list(cmd.file_iter()) - elif isinstance(cmd, commands.ResetCommand): - if cmd.from_ is not None: - cm.track_heads_for_ref(cmd.ref, cmd.from_) - self.assertEqual(cm.heads, expected) - - def test_mainline(self): - self.assertHeads(_SAMPLE_MAINLINE, { - ':102': set(['refs/heads/master']), - }) - - def test_two_heads(self): - self.assertHeads(_SAMPLE_TWO_HEADS, { - ':101': set(['refs/heads/mybranch']), - ':102': set(['refs/heads/master']), - }) - - def test_two_branches_merged(self): - self.assertHeads(_SAMPLE_TWO_BRANCHES_MERGED, { - ':103': set(['refs/heads/master']), - }) - - def test_reset(self): - self.assertHeads(_SAMPLE_RESET, { - ':100': set(['refs/heads/master', 'refs/remotes/origin/master']), - }) - - def test_reset_with_more_commits(self): - self.assertHeads(_SAMPLE_RESET_WITH_MORE_COMMITS, { - ':101': set(['refs/remotes/origin/master']), - }) diff --git a/tests/test_helpers.py b/tests/test_helpers.py deleted file mode 100644 index 89009d1..0000000 --- a/tests/test_helpers.py +++ /dev/null @@ -1,56 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Test the helper functions.""" - -from bzrlib import tests - -from bzrlib.plugins.fastimport import ( - helpers, - ) - - -class TestCommonDirectory(tests.TestCase): - - def test_no_paths(self): - c = helpers.common_directory(None) - self.assertEqual(c, None) - c = helpers.common_directory([]) - self.assertEqual(c, None) - - def test_one_path(self): - c = helpers.common_directory(['foo']) - self.assertEqual(c, '') - c = helpers.common_directory(['foo/']) - self.assertEqual(c, 'foo/') - c = helpers.common_directory(['foo/bar']) - self.assertEqual(c, 'foo/') - - def test_two_paths(self): - c = helpers.common_directory(['foo', 'bar']) - self.assertEqual(c, '') - c = helpers.common_directory(['foo/', 'bar']) - self.assertEqual(c, '') - c = helpers.common_directory(['foo/', 'foo/bar']) - self.assertEqual(c, 'foo/') - c = helpers.common_directory(['foo/bar/x', 'foo/bar/y']) - self.assertEqual(c, 'foo/bar/') - c = helpers.common_directory(['foo/bar/aa_x', 'foo/bar/aa_y']) - self.assertEqual(c, 'foo/bar/') - - def test_lots_of_paths(self): - c = helpers.common_directory(['foo/bar/x', 'foo/bar/y', 'foo/bar/z']) - self.assertEqual(c, 'foo/bar/') diff --git a/tests/test_parser.py b/tests/test_parser.py deleted file mode 100644 index 18475e6..0000000 --- a/tests/test_parser.py +++ /dev/null @@ -1,284 +0,0 @@ -# Copyright (C) 2008 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -"""Test the Import parsing""" - -import StringIO - -from bzrlib import tests - -from bzrlib.plugins.fastimport.fastimport import ( - errors, - parser, - ) - - -class TestLineBasedParser(tests.TestCase): - - def test_push_line(self): - s = StringIO.StringIO("foo\nbar\nbaz\n") - p = parser.LineBasedParser(s) - self.assertEqual('foo', p.next_line()) - self.assertEqual('bar', p.next_line()) - p.push_line('bar') - self.assertEqual('bar', p.next_line()) - self.assertEqual('baz', p.next_line()) - self.assertEqual(None, p.next_line()) - - def test_read_bytes(self): - s = StringIO.StringIO("foo\nbar\nbaz\n") - p = parser.LineBasedParser(s) - self.assertEqual('fo', p.read_bytes(2)) - self.assertEqual('o\nb', p.read_bytes(3)) - self.assertEqual('ar', p.next_line()) - # Test that the line buffer is ignored - p.push_line('bar') - self.assertEqual('baz', p.read_bytes(3)) - # Test missing bytes - self.assertRaises(errors.MissingBytes, p.read_bytes, 10) - - def test_read_until(self): - # TODO - return - s = StringIO.StringIO("foo\nbar\nbaz\nabc\ndef\nghi\n") - p = parser.LineBasedParser(s) - self.assertEqual('foo\nbar', p.read_until('baz')) - self.assertEqual('abc', p.next_line()) - # Test that the line buffer is ignored - p.push_line('abc') - self.assertEqual('def', p.read_until('ghi')) - # Test missing terminator - self.assertRaises(errors.MissingTerminator, p.read_until('>>>')) - - -# Sample text -_sample_import_text = """ -progress completed -# Test blob formats -blob -mark :1 -data 4 -aaaablob -data 5 -bbbbb -# Commit formats -commit refs/heads/master -mark :2 -committer bugs bunny <bugs@bunny.org> now -data 14 -initial import -M 644 inline README -data 18 -Welcome from bugs -commit refs/heads/master -committer <bugs@bunny.org> now -data 13 -second commit -from :2 -M 644 inline README -data 23 -Welcome from bugs, etc. -# Miscellaneous -checkpoint -progress completed -# Test a commit without sub-commands (bug #351717) -commit refs/heads/master -mark :3 -author <bugs@bunny.org> now -committer <bugs@bunny.org> now -data 20 -first commit, empty -# Test a commit with a heredoc-style (delimited_data) messsage (bug #400960) -commit refs/heads/master -mark :4 -author <bugs@bunny.org> now -committer <bugs@bunny.org> now -data <<EOF -Commit with heredoc-style message -EOF -# Test a "submodule"/tree-reference -commit refs/heads/master -mark :5 -author <bugs@bunny.org> now -committer <bugs@bunny.org> now -data 15 -submodule test -M 160000 rev-id tree-id -# Test features -feature whatever -feature foo=bar -# Test commit with properties -commit refs/heads/master -mark :6 -committer <bugs@bunny.org> now -data 18 -test of properties -property p1 -property p2 5 hohum -property p3 16 alpha -beta -gamma -property p4 8 whatever -# Test a commit with multiple authors -commit refs/heads/master -mark :7 -author Fluffy <fluffy@bunny.org> now -author Daffy <daffy@duck.org> now -author Donald <donald@duck.org> now -committer <bugs@bunny.org> now -data 17 -multi-author test -""" - - -class TestImportParser(tests.TestCase): - - def test_iter_commands(self): - s = StringIO.StringIO(_sample_import_text) - p = parser.ImportParser(s) - result = [] - for cmd in p.iter_commands(): - result.append(cmd) - if cmd.name == 'commit': - for fc in cmd.file_iter(): - result.append(fc) - self.assertEqual(len(result), 17) - cmd1 = result.pop(0) - self.assertEqual('progress', cmd1.name) - self.assertEqual('completed', cmd1.message) - cmd2 = result.pop(0) - self.assertEqual('blob', cmd2.name) - self.assertEqual('1', cmd2.mark) - self.assertEqual(':1', cmd2.id) - self.assertEqual('aaaa', cmd2.data) - self.assertEqual(4, cmd2.lineno) - cmd3 = result.pop(0) - self.assertEqual('blob', cmd3.name) - self.assertEqual('@7', cmd3.id) - self.assertEqual(None, cmd3.mark) - self.assertEqual('bbbbb', cmd3.data) - self.assertEqual(7, cmd3.lineno) - cmd4 = result.pop(0) - self.assertEqual('commit', cmd4.name) - self.assertEqual('2', cmd4.mark) - self.assertEqual(':2', cmd4.id) - self.assertEqual('initial import', cmd4.message) - self.assertEqual('bugs bunny', cmd4.committer[0]) - self.assertEqual('bugs@bunny.org', cmd4.committer[1]) - # FIXME: check timestamp and timezone as well - self.assertEqual(None, cmd4.author) - self.assertEqual(11, cmd4.lineno) - self.assertEqual('refs/heads/master', cmd4.ref) - self.assertEqual(None, cmd4.from_) - self.assertEqual([], cmd4.merges) - file_cmd1 = result.pop(0) - self.assertEqual('filemodify', file_cmd1.name) - self.assertEqual('README', file_cmd1.path) - self.assertEqual('file', file_cmd1.kind) - self.assertEqual(False, file_cmd1.is_executable) - self.assertEqual('Welcome from bugs\n', file_cmd1.data) - cmd5 = result.pop(0) - self.assertEqual('commit', cmd5.name) - self.assertEqual(None, cmd5.mark) - self.assertEqual('@19', cmd5.id) - self.assertEqual('second commit', cmd5.message) - self.assertEqual('', cmd5.committer[0]) - self.assertEqual('bugs@bunny.org', cmd5.committer[1]) - # FIXME: check timestamp and timezone as well - self.assertEqual(None, cmd5.author) - self.assertEqual(19, cmd5.lineno) - self.assertEqual('refs/heads/master', cmd5.ref) - self.assertEqual(':2', cmd5.from_) - self.assertEqual([], cmd5.merges) - file_cmd2 = result.pop(0) - self.assertEqual('filemodify', file_cmd2.name) - self.assertEqual('README', file_cmd2.path) - self.assertEqual('file', file_cmd2.kind) - self.assertEqual(False, file_cmd2.is_executable) - self.assertEqual('Welcome from bugs, etc.', file_cmd2.data) - cmd6 = result.pop(0) - self.assertEqual(cmd6.name, 'checkpoint') - cmd7 = result.pop(0) - self.assertEqual('progress', cmd7.name) - self.assertEqual('completed', cmd7.message) - cmd = result.pop(0) - self.assertEqual('commit', cmd.name) - self.assertEqual('3', cmd.mark) - self.assertEqual(None, cmd.from_) - cmd = result.pop(0) - self.assertEqual('commit', cmd.name) - self.assertEqual('4', cmd.mark) - self.assertEqual('Commit with heredoc-style message\n', cmd.message) - cmd = result.pop(0) - self.assertEqual('commit', cmd.name) - self.assertEqual('5', cmd.mark) - self.assertEqual('submodule test\n', cmd.message) - file_cmd1 = result.pop(0) - self.assertEqual('filemodify', file_cmd1.name) - self.assertEqual('tree-id', file_cmd1.path) - self.assertEqual('tree-reference', file_cmd1.kind) - self.assertEqual(False, file_cmd1.is_executable) - self.assertEqual("rev-id", file_cmd1.dataref) - cmd = result.pop(0) - self.assertEqual('feature', cmd.name) - self.assertEqual('whatever', cmd.feature_name) - self.assertEqual(None, cmd.value) - cmd = result.pop(0) - self.assertEqual('feature', cmd.name) - self.assertEqual('foo', cmd.feature_name) - self.assertEqual('bar', cmd.value) - cmd = result.pop(0) - self.assertEqual('commit', cmd.name) - self.assertEqual('6', cmd.mark) - self.assertEqual('test of properties', cmd.message) - self.assertEqual({ - 'p1': None, - 'p2': u'hohum', - 'p3': u'alpha\nbeta\ngamma', - 'p4': u'whatever', - }, cmd.properties) - cmd = result.pop(0) - self.assertEqual('commit', cmd.name) - self.assertEqual('7', cmd.mark) - self.assertEqual('multi-author test', cmd.message) - self.assertEqual('', cmd.committer[0]) - self.assertEqual('bugs@bunny.org', cmd.committer[1]) - self.assertEqual('Fluffy', cmd.author[0]) - self.assertEqual('fluffy@bunny.org', cmd.author[1]) - self.assertEqual('Daffy', cmd.more_authors[0][0]) - self.assertEqual('daffy@duck.org', cmd.more_authors[0][1]) - self.assertEqual('Donald', cmd.more_authors[1][0]) - self.assertEqual('donald@duck.org', cmd.more_authors[1][1]) - - -class TestStringParsing(tests.TestCase): - - def test_unquote(self): - s = r'hello \"sweet\" wo\\r\tld' - self.assertEquals(r'hello "sweet" wo\r' + "\tld", - parser._unquote_c_string(s)) - - -class TestPathPairParsing(tests.TestCase): - - def test_path_pair_simple(self): - p = parser.ImportParser("") - self.assertEqual(['foo', 'bar'], p._path_pair("foo bar")) - - def test_path_pair_spaces_in_first(self): - p = parser.ImportParser("") - self.assertEqual(['foo bar', 'baz'], - p._path_pair('"foo bar" baz')) diff --git a/tests/test_revision_store.py b/tests/test_revision_store.py deleted file mode 100644 index d850c95..0000000 --- a/tests/test_revision_store.py +++ /dev/null @@ -1,147 +0,0 @@ -# Copyright (C) 2008, 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - -"""Direct tests of the revision_store classes.""" - -from bzrlib import ( - branch, - errors, - inventory, - osutils, - tests, - ) - -from bzrlib.plugins.fastimport import ( - revision_store, - ) - - -class Test_TreeShim(tests.TestCase): - - def invAddEntry(self, inv, path, file_id=None): - if path.endswith('/'): - path = path[:-1] - kind = 'directory' - else: - kind = 'file' - parent_path, basename = osutils.split(path) - parent_id = inv.path2id(parent_path) - inv.add(inventory.make_entry(kind, basename, parent_id, file_id)) - - def make_trivial_basis_inv(self): - basis_inv = inventory.Inventory('TREE_ROOT') - self.invAddEntry(basis_inv, 'foo', 'foo-id') - self.invAddEntry(basis_inv, 'bar/', 'bar-id') - self.invAddEntry(basis_inv, 'bar/baz', 'baz-id') - return basis_inv - - def test_id2path_no_delta(self): - basis_inv = self.make_trivial_basis_inv() - shim = revision_store._TreeShim(repo=None, basis_inv=basis_inv, - inv_delta=[], content_provider=None) - self.assertEqual('', shim.id2path('TREE_ROOT')) - self.assertEqual('foo', shim.id2path('foo-id')) - self.assertEqual('bar', shim.id2path('bar-id')) - self.assertEqual('bar/baz', shim.id2path('baz-id')) - self.assertRaises(errors.NoSuchId, shim.id2path, 'qux-id') - - def test_id2path_with_delta(self): - basis_inv = self.make_trivial_basis_inv() - foo_entry = inventory.make_entry('file', 'foo2', 'TREE_ROOT', 'foo-id') - inv_delta = [('foo', 'foo2', 'foo-id', foo_entry), - ('bar/baz', None, 'baz-id', None), - ] - - shim = revision_store._TreeShim(repo=None, basis_inv=basis_inv, - inv_delta=inv_delta, - content_provider=None) - self.assertEqual('', shim.id2path('TREE_ROOT')) - self.assertEqual('foo2', shim.id2path('foo-id')) - self.assertEqual('bar', shim.id2path('bar-id')) - self.assertRaises(errors.NoSuchId, shim.id2path, 'baz-id') - - def test_path2id(self): - basis_inv = self.make_trivial_basis_inv() - shim = revision_store._TreeShim(repo=None, basis_inv=basis_inv, - inv_delta=[], content_provider=None) - self.assertEqual('TREE_ROOT', shim.path2id('')) - # We don't want to ever give a wrong value, so for now we just raise - # NotImplementedError - self.assertRaises(NotImplementedError, shim.path2id, 'bar') - - def test_get_file_with_stat_content_in_stream(self): - basis_inv = self.make_trivial_basis_inv() - - def content_provider(file_id): - return 'content of\n' + file_id + '\n' - - shim = revision_store._TreeShim(repo=None, basis_inv=basis_inv, - inv_delta=[], - content_provider=content_provider) - f_obj, stat_val = shim.get_file_with_stat('baz-id') - self.assertIs(None, stat_val) - self.assertEqualDiff('content of\nbaz-id\n', f_obj.read()) - - # TODO: Test when the content isn't in the stream, and we fall back to the - # repository that was passed in - - def test_get_symlink_target(self): - basis_inv = self.make_trivial_basis_inv() - ie = inventory.make_entry('symlink', 'link', 'TREE_ROOT', 'link-id') - ie.symlink_target = u'link-target' - basis_inv.add(ie) - shim = revision_store._TreeShim(repo=None, basis_inv=basis_inv, - inv_delta=[], content_provider=None) - self.assertEqual(u'link-target', shim.get_symlink_target('link-id')) - - def test_get_symlink_target_from_delta(self): - basis_inv = self.make_trivial_basis_inv() - ie = inventory.make_entry('symlink', 'link', 'TREE_ROOT', 'link-id') - ie.symlink_target = u'link-target' - inv_delta = [(None, 'link', 'link-id', ie)] - shim = revision_store._TreeShim(repo=None, basis_inv=basis_inv, - inv_delta=inv_delta, - content_provider=None) - self.assertEqual(u'link-target', shim.get_symlink_target('link-id')) - - def test__delta_to_iter_changes(self): - basis_inv = self.make_trivial_basis_inv() - foo_entry = inventory.make_entry('file', 'foo2', 'bar-id', 'foo-id') - link_entry = inventory.make_entry('symlink', 'link', 'TREE_ROOT', - 'link-id') - link_entry.symlink_target = u'link-target' - inv_delta = [('foo', 'bar/foo2', 'foo-id', foo_entry), - ('bar/baz', None, 'baz-id', None), - (None, 'link', 'link-id', link_entry), - ] - shim = revision_store._TreeShim(repo=None, basis_inv=basis_inv, - inv_delta=inv_delta, - content_provider=None) - changes = list(shim._delta_to_iter_changes()) - expected = [('foo-id', ('foo', 'bar/foo2'), False, (True, True), - ('TREE_ROOT', 'bar-id'), ('foo', 'foo2'), - ('file', 'file'), (False, False)), - ('baz-id', ('bar/baz', None), True, (True, False), - ('bar-id', None), ('baz', None), - ('file', None), (False, None)), - ('link-id', (None, 'link'), True, (False, True), - (None, 'TREE_ROOT'), (None, 'link'), - (None, 'symlink'), (None, False)), - ] - # from pprint import pformat - # self.assertEqualDiff(pformat(expected), pformat(changes)) - self.assertEqual(expected, changes) - diff --git a/user_mapper.py b/user_mapper.py deleted file mode 100644 index 4fcf4a4..0000000 --- a/user_mapper.py +++ /dev/null @@ -1,81 +0,0 @@ -# Copyright (C) 2009 Canonical Ltd -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -from email import Utils - - -class UserMapper(object): - - def __init__(self, lines): - """Create a user-mapper from a list of lines. - - Blank lines and comment lines (starting with #) are ignored. - Otherwise lines are of the form: - - old-id = new-id - - Each id may be in the following forms: - - name <email> - name - - If old-id has the value '@', then new-id is the domain to use - when generating an email from a user-id. - """ - self._parse(lines) - - def _parse(self, lines): - self._user_map = {} - self._default_domain = None - for line in lines: - line = line.strip() - if len(line) == 0 or line.startswith('#'): - continue - old, new = line.split('=', 1) - old = old.strip() - new = new.strip() - if old == '@': - self._default_domain = new - continue - # Parse each id into a name and email address - old_name, old_email = self._parse_id(old) - new_name, new_email = self._parse_id(new) - #print "found user map: %s => %s" % ((old_name, old_email), (new_name, new_email)) - self._user_map[(old_name, old_email)] = (new_name, new_email) - - def _parse_id(self, id): - if id.find('<') == -1: - return id, None - else: - return Utils.parseaddr(id) - - def map_name_and_email(self, name, email): - """Map a name and an email to the preferred name and email. - - :param name: the current name - :param email: the current email - :result: the preferred name and email - """ - try: - new_name, new_email = self._user_map[(name, email)] - except KeyError: - new_name = name - if self._default_domain and not email: - new_email = "%s@%s" % (name, self._default_domain) - else: - new_email = email - #print "converted '%s <%s>' to '%s <%s>'" % (name, email, new_name, new_email) - return new_name, new_email |
