summaryrefslogtreecommitdiff
path: root/baserockimport
Commit message (Collapse)AuthorAgeFilesLines
* Add perl importer baseRichard Ipsum2015-08-251-0/+70
| | | | Change-Id: Ieb1a0d2047746e68db0e494d9dc73fe5aae782a3
* Add pathname2distinfo utilRichard Ipsum2015-08-251-0/+46
| | | | Change-Id: I89ea4bbb26cff27a6a2dcb71e3101e52b8f91002
* Clone source repo into tempdirRichard Ipsum2015-08-251-30/+33
| | | | | | | | | | | Extensions will now be given a path to a clone of the checkout rather than a path to the checkout itself. This prevents untracked modifications and protects any data the user might have in the checkout dirs from the extensions themselves. It also helps prevent a possible bug where the extension corrupts a checkout so that on the second import the result differs from the first import. Change-Id: I95d551a02aa77d665ba07fb7ed5b01be330e24d4
* Add an ignore_version_field option to PackageRichard Ipsum2015-08-252-6/+14
| | | | | | | If we're using an external tool to resolve dependencies then we may be want to ignore the version field. Change-Id: Ic41f5b5ac080a5e9a0b4e1b0607464faabb1ec94
* Let parent packages pass metadata to their childrenRichard Ipsum2015-08-251-14/+47
| | | | | | | | | With this packages can pass metadata to their dependencies (children) by putting the path to the parent's foreign-dependencies file into the child's environment. This provides a flexible means for extensions to pass around extra data where necessary. Change-Id: I6446d02abd1a8d703c8ff88fe12b92345a0ee461
* Rework tag searchRichard Ipsum2015-08-251-12/+13
| | | | | | This should make it easier to search through a wider range of tag formats. Change-Id: Ie124a054d55b4a8800f11479e0389c16c97c53a8
* Rename lambda parametersRichard Ipsum2015-08-251-6/+8
| | | | Change-Id: I6ee07010fad5164c9a3a3d40f0b951a6c9d676f1
* Show errors encountered during import at the endRichard Ipsum2015-08-251-0/+5
| | | | Change-Id: I86d38e0116fc8b95dcd20c1d52b1b801cf16466c
* Simplify accessors in packageRichard Ipsum2015-08-242-18/+18
| | | | Change-Id: I38dc72c2348883abea56bd58a3195df4dc197c8a
* Make to_chunk stage optionalRichard Ipsum2015-08-242-48/+50
| | | | | | | | | Most CPAN distributions can be installed with a standard set of commands so we don't want to generate chunk morphs for each chunk. This also fixes stratum build dependency generation Change-Id: Ica51721b9b3809f91b8016c959202694d4bc6ea7
* python.find_deps: On failure, show stdout and stderr output from PipSam Thursfield2015-07-131-2/+5
| | | | | | | | | | | | | | | | | | | | | | It's nice to have some idea right away of what went wrong. A common error is running the import tool on a system without the patched version of Pip. This now produces the following sort of message: baserockimport/exts/python.find_deps: Failed to get runtime dependencies for numpy at checkouts/python_numpy-tarball/. Output from Pip: Usage: pip install [options] <requirement specifier> [package-index-options] ... pip install [options] -r <requirements file> [package-index-options] ... pip install [options] [-e] <vcs project url> ... pip install [options] [-e] <local project path> ... pip install [options] <archive url/path> ... no such option: --list-dependencies Change-Id: I902920b46b29b1e2b18736b2ff2b5f4f4cdb42df
* Clean up data so url is validZara Zaimeche2015-06-101-1/+6
| | | | | | | | | Some people now put urls in the registry with '+git' at the start. The tool doesn't recognise this, but these urls still end in '.git', so this change just checks for such urls and cuts out the '+git'. The tool finds the repos once more. Panic over. Change-Id: I4e807797ed914fa1dbe96cc7e05263228785c86d
* Fix: honour version flag in npm.to_lorry extensionZara Zaimeche2015-06-091-1/+8
| | | | | | | | | | | | | | | This fixes a version bug in npm.to_lorry that meant the most recent db entry was used for repo location, regardless of the version specified. The npm registry is a database of all versions of all packages, listing names, source locations, etc. There was a fun bug in npm.to_lorry that meant that the import tool would try access the repo listed in the database entry for the latest version of the package, rather than for the version specified. This meant that if version 1.0.1 of a package had the right url listed, but something was wrong with the url listed for version 1.4.6, the tool would error out, even if version 1.0.1 was requested. Change-Id: If40c8b4c85f5e27fee07ee88daa1e5d2d347944f
* Don't put version in error message if it's NoneRichard Ipsum2015-06-031-2/+2
| | | | Change-Id: Id7b0713263f3d68773f69bc4c2dadec414e1d902
* Allow import tool to run as non-rootRichard Ipsum2015-06-031-1/+2
| | | | | | | | | Morph runs extensions in a separate mount namespace by default, but constructing this namespace requires root privileges. The import tool has no need for a separate mount namespace, so this commit disables it. Change-Id: I952885860100453e4a531589b6acca7e5dc31435
* Import warn() and error() from utilsRichard Ipsum2015-06-031-0/+1
| | | | | | python.find_deps uses warn and error, which have moved into the utils module Change-Id: I7ce58c034cb83b7fc486487d6234282b650edfc1
* Don't put "build-depends: []" in strataRichard Ipsum2015-05-291-1/+4
| | | | Change-Id: I53d655e09726b57abb69e04264ef685f26519356
* Make the python ext use cachingRichard Ipsum2015-05-291-12/+14
| | | | | | | | This doesn't provide much performance benefit, since most of our queries go through xmlrpclib, but caching here does no harm. Change-Id: Id740a7ffab56defeddb3a6f3f481d81498a4411a
* Move WebServiceClient into ImporterBaseRichard Ipsum2015-05-292-34/+37
| | | | | | | | | | | | | | | | | | | To allow all extensions to use it, this also modifies the client so that the cache gets expired after 5 minutes by default. The error message returned by the rubygems extension on failure to get gem data will also be slightly more detailed, old message, ERROR: Request to http://rubygems.org/api/v1/gems/kittens.json failed: Not Found new message, ERROR: Request to http://rubygems.org/api/v1/gems/kittens.json failed: 404 Client Error: Not Found Change-Id: I63354fc7682bb01b1122007c1435bf35975db1aa
* Move common functions into a utils moduleRichard Ipsum2015-05-294-60/+88
| | | | | | These functions will also be useful for our forthcoming cpan extension Change-Id: I9df87dee09bbcf43dd0868f062fb873632f1f5ae
* Add check scriptRichard Ipsum2015-05-192-6/+10
| | | | | | | We switch to unittest.main() here so that the exit code gets set by unittest, which in turn sets check's exit code Change-Id: If9ace26ba5373cb78192b30922ee1bd0ea91f36f
* Fix str_repo_lorry for svn reposbaserock/richardipsum/import-lorry-fixRichard Ipsum2015-01-161-4/+7
| | | | | | Most svn repos use a standard layout, we assume svn repos we want to lorry are using a standard layout, so make str_repo_lorry add 'layout': 'standard' for svn repos
* Add more possible namesbaserock/richardipsum/add-more-possible-namesRichard Ipsum2015-01-141-5/+8
| | | | | | This fixes the tool so that it can find refs such as foobar-1.4.0 when the version has been passed as 1.4 and refs such as foobar-1.4 when the version has been passed 1.4.0
* Update tests for python.to_lorrybaserock/richardipsum/fix-lorry-bug-3Richard Ipsum2015-01-131-4/+7
|
* Make npm lorry ext work with new interfaceRichard Ipsum2015-01-131-5/+7
|
* Make rubygems lorry ext work with new interfaceRichard Ipsum2015-01-131-2/+3
|
* Move get_releases into common and make all extensions use itRichard Ipsum2015-01-132-2/+11
| | | | | | | We also call package_releases with True, so that we also get versions of releases that have been hidden. pip is willing to install from hidden releases so we should too, the concept of hidden releases will eventually disappear from pypi as well.
* Fix python.to_lorry to use version passed on cmdlineRichard Ipsum2015-01-131-33/+31
|
* Make import tool pass version to to_lorry extRichard Ipsum2015-01-121-5/+9
|
* Add debug msg stating requirementRichard Ipsum2015-01-121-0/+2
|
* Add option for debugging vcssRichard Ipsum2015-01-121-1/+4
| | | | | It's sometime useful to see the output of the vcs, but having this enabled all the time clutters the log
* Remove generic PythonExtension classRichard Ipsum2015-01-122-28/+25
| | | | The two extensions have diverged so this generic class is no longer useful.
* Fix bug in name_or_closestRichard Ipsum2015-01-121-0/+1
|
* Don't convert package names to lower caseRichard Ipsum2015-01-121-4/+4
| | | | This is redundant.
* Use comparison callback in python importerRichard Ipsum2015-01-121-1/+6
| | | | | | | As well as being case insensitive project names on python treat '-' and '_' as equivalent characters [1] [1]: http://legacy.python.org/dev/peps/pep-0426/#name
* Add package name comparison callbackRichard Ipsum2015-01-122-3/+11
| | | | | | This lets the importer specify how package names will be compared, to do this the importer optionally supplies a callback, if no callback is supplied then the tool will default to == for string comparison.
* Make python.to_lorry output x-products-python fieldRichard Ipsum2015-01-121-2/+9
|
* Read lorry prefix from yaml fileRichard Ipsum2015-01-121-30/+39
| | | | | | | | | | | | This adds PythonLorryExtension class to python.to_lorry to run the extension in a more conventional way. Previously the PythonExtension class was used to execute any of the extensions (it would call the extensions main() function). We move away from this so the extension can access useful methods, such as local_data_path(), that are provided by the ImportExtension class. This also removes use of pkg_resources.parse_requirement which is redundant. This also removes the unused import of select.
* Add python.yamlRichard Ipsum2015-01-121-0/+2
| | | | This adds a config file for the python extensions
* Add npm to_lorryZara Zaimeche2015-01-091-0/+51
|
* Add npm to_chunkZara Zaimeche2015-01-091-0/+72
|
* Add npm find_depsZara Zaimeche2015-01-091-0/+91
|
* Add helper functions for npm extensionsZara Zaimeche2015-01-091-0/+24
|
* Add npm subcommand to import tool via app.pyZara Zaimeche2015-01-091-0/+18
|
* fix app.py so that ruby doesn't always import from masterZara Zaimeche2015-01-071-1/+1
|
* Update copyrightsbaserock/richardipsum/fix-hyphen-bugRichard Ipsum2015-01-053-3/+6
|
* Treat hyphens and underscores equivalentlyRichard Ipsum2015-01-052-7/+22
| | | | | | | | | | | | | | There isn't yet an official spec for distribution names in python, however there is a draft at http://legacy.python.org/dev/peps/pep-0426/#name In particular, "All comparisons of distribution names MUST be case insensitive, and MUST consider hyphens and underscores to be equivalent." pkg_resource.parse_requirements will replace any underscores in the package name as hyphens, so when we search pypi we need to look for the package name with underscores as well as with hyphens.
* Add python.to_chunkRichard Ipsum2014-12-051-0/+33
| | | | | | Morph doesn't need a chunk morph to build/install setuptools packages, but the import tool needs to grow the ability to have 'to_chunk' as an optional stage before we can remove this part of the python extension.
* Add tests for requirement conflict detectionRichard Ipsum2014-12-051-0/+362
| | | | | python.find_deps does some pretty basic validation of the requirement specs, this commit adds the tests for this validation.
* Add python.find_depsRichard Ipsum2014-12-051-0/+352
| | | | | This takes source_dir, name, version and returns the dependencies for the package as json on stdout.
View Lorry Controller Status