summaryrefslogtreecommitdiff
path: root/tools
Commit message (Collapse)AuthorAgeFilesLines
* Increase integration test timeoutClark Boylan2020-08-171-1/+1
| | | | | | | | We're timing out installing some packages and it seems we're right up against the timeout based on logs. Increase the timeout by 50% to give us a better chance of running these jobs successfully. Change-Id: I33ce50f57af2a1d366148e24b49cae5eda2d3dfb
* Begin work to modernize pbr's integration testingClark Boylan2020-08-061-4/+4
| | | | | | | | | | | | | | | | | | PBR does integration testing by installing all of openstack's python projects to ensure we can install all of them in a variety of different ways with different pip versions. Unfortunately, these jobs are still using devstack-gate. We can simplify them significantly because zuul is now doing what devstack-gate does and provides repos configured the way we want them to be. This change updates the jobs to drop devstack-gate and simply consume zuul's repo prep. A follow up will remove the old legacy jobs as they need to be removed from project-config first. Needed-By: https://review.opendev.org/745189 Change-Id: I77a38c4611dc28db79d6f25d96ad0de36b224c98
* Add support for virtualenv 20.xStephen Finucane2020-08-051-11/+10
| | | | | | | | | | | | | | | | | | | | virtualenv is undergoing a rewrite and has changed how the programmatic API works [1]. Switch to the "new way". While we're here, we also need to get Python 2.7 tests passing again. That requires dropping support for upper-constraints and using our own, limited local constraints based on supported Python versions. We also need to migrate integration tests since those run with Python 3 now. Update the scenarios for pip/setuptools in integration testing similarly. Finally clean up the installation of all devstack repos as requirements are managed different now. Instead of worrying about syncing them we use constraints. [1] https://github.com/pypa/virtualenv/issues/1585#issuecomment-585228492 Change-Id: I493e88985d2c4d09612fea4d20d8ffa20043a0cb Signed-off-by: Stephen Finucane <sfinucan@redhat.com> Depends-On: https://review.opendev.org/739014
* Install more dependencies for integration testingBen Nemec2019-05-211-1/+1
| | | | | | | | When running this locally I needed to install a few more distro packages to provide build dependencies for pip packages. This change adds them to the list of things installed in tools/integration.sh. Change-Id: I2ae04b893c7dfddcb6d94b724f53ee08436bf577
* Remove libzmq-dev from integration.sh package installBen Nemec2019-03-191-1/+1
| | | | | | | | | It appears this no longer exists on bionic, and we shouldn't need it for pbr tests. I suspect this may have been some copy-pasta from another project's integration tests. Change-Id: Ife631f77a92ee0b34c19e77cad782d94d18f2e74 Closes-Bug: 1820855
* trivial: Remove 'tools/releasenotes_tox.sh'Stephen Finucane2018-07-181-28/+0
| | | | | | | | | This is not needed since reno 2.1.0 (commit f8fc8f97) [1]. [1] https://github.com/openstack/reno/commit/f8fc8f97 Change-Id: I335bba8c8bf4f2d73aba32ae762607e67ee3814c Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* Don't poke in pip for requestsIan Wienand2018-04-301-6/+2
| | | | | | | Pip10 has removed the ability to poke inside it for things like requests. Explicitly add it to the venv. Change-Id: Ie12168091dc4c737f3913720c1743bea666b5d9c
* Avoid tox_install.sh for constraints supportAndreas Jaeger2017-12-021-30/+0
| | | | | | | | | | | We do not need tox_install.sh, pip can handle constraints itself and install the project correctly. Thus update tox.ini and remove the now obsolete tools/tox_install.sh file. This follows https://review.openstack.org/#/c/508061 to remove tools/tox_install.sh. Change-Id: I0a80865a8a18c49b4d883850488cc0c73fe1bfcd
* Add reno for release notes managementStephen Finucane2017-10-101-0/+28
| | | | Change-Id: I36b8193b22123a409b729c7b9f4309507379f1ad
* Add Constraints supportDirk Mueller2017-01-031-0/+30
| | | | | | | | | | | | Adding constraints support to clients is slightly more complex than services as the clients themselves are listed in upper-constraints.txt which leads to errors that you can't install a specific version and a constrained version. This change adds constraints support by also adding a helper script to edit the constraints to remove pbr. Change-Id: I80a10f80e2c5efafb099a6c1dcebe8f0935ace09
* Use apt-cache generated packages to provide build depsSachi King2016-03-081-2/+8
| | | | | | | Apt config does not have sources anymore, so we copy apt-cache and run apt-get install ourselves to provide our dependencies Change-Id: Ie70cb124120d1012d38e5189044c21b83be20819
* Add libjpeg and liberasurecode for testsMonty Taylor2016-01-131-1/+1
| | | | | | | | Pilow needs libjpeg and liberasurecode is needed by PyECLib. In the future we should be able to get this done with bindep, but we're not quite there yet. Change-Id: Ib56d9fdd99731e3eefc9f69f07d0f002d5415868
* passenv integration environment variables re-enabling integration testsSachi King2015-12-111-0/+2
| | | | Change-Id: I87e0c7765df1fa9ae336d8e9052c3947c9fc8392
* Add kerberos deps to build the kerberos wheel.Robert Collins2015-05-201-1/+1
| | | | Change-Id: If9b4db43cd260da731ec5a14599eddedfc33808a
* Bump integration test timeouts.Robert Collins2015-05-141-1/+1
| | | | | | Node load is too variable. Change-Id: If03cd577ea18bfad4b53fcddb7a87e48eefe84c1
* Parallelise integration tests.Robert Collins2015-05-021-65/+16
| | | | | | | This involves moving the inner loop to python because existing-tooling. Change-Id: Iaad811a0248a3f700e655bd8be656d183deead93
* Issue #1450210: Preversioning rc tag handling0.11.0Robert Collins2015-04-301-0/+1
| | | | | | | | | | We were generating versions lower than the last tag when preversioning was in use and pre-release versions had been tagged. Also attempt to run the script without devstack. Change-Id: If984939003ae7c513ee76b52c6808db0c1766050 Closes-Bug: #1450210
* Stop testing setup.py easy_install behaviourRobert Collins2015-04-291-14/+4
| | | | | | | | | Instead test pip install more thoroughly. Also document that we don't support setup.py install's easy-install behaviour. Change-Id: I87872a69425f9d4f17047e584c12516508d57b16
* Test pip install -e of projects.Robert Collins2015-04-281-0/+5
| | | | Change-Id: Ie57d12f32085036079c60edfbe7aecfc7a077f5b
* Build all the wheels in one pass rather than many.Robert Collins2015-04-281-4/+2
| | | | | | | | pip tries to build all wheels, and can avoid re-processing all the requirements over the network N times - basically quadratic down to linear scaling. Change-Id: I69441b2378bf4f165d0b475cd16a9e56797df575
* Improve integration.sh.Robert Collins2015-04-281-18/+23
| | | | | | | | | | | | | | Add global flags to pip install to see more about what its doing (when set). pip is too verbose to enable -v mode by default as yet. Document the environment variable parameters the script takes. Change the way we enable the wheelhouse: rather than magic environment variables which require action-at-a-distance knowledge, use actual parameters. This means that the easy_install invocation which recurses back into pip won't use the wheel house... but we're removing that reentry anyway. Change-Id: Ie7ed8d60a6dd6b5766a2c7116b860b8e98a1f1f1
* Honour new test variable PIPVERSIONRobert Collins2015-04-251-1/+16
| | | | | | | This allows testing with unreleased pips, which is important to know if something is going to cause us or our users trouble. Change-Id: Icb3a9c1486c76b673d25a3d18792baad3d26c763
* Support script text override for newer developMonty Taylor2015-01-281-0/+7
| | | | | | | | setuptools > 12 has a new flow for writing out generated script text. It's nicer, actually, because it means we can just subclass and extend one method instead of monekypatching. Change-Id: I56e7bea60df8a59d859575d426ce93c45ffee314
* Be more aggressive about building wheelsDoug Hellmann2015-01-121-1/+4
| | | | | | | | | | When "pip wheel" hits a package where building the egg-info metadata fails, it completely falls over and stops trying to build more wheels. Loop over our requirements one by one, so we can ignore the packages that fail to build wheels, and build any that come in the list after those. Change-Id: Ia23672b851d50af38c61823936f19b8572a78c4f
* Use a wheelhouse in the integration testClark Boylan2015-01-051-1/+12
| | | | | | | | | | The integration test can take >2 hours running into a timeout. Much of the time spent in this test is consumed by recompiling dependencies like lxml. Reduce this overhead by compiling these items once in a wheelhouse then installing all dependencies from that wheelhouse preventing recompilation. Change-Id: I40cd7587ff32f1a4cf881939069dde015dc9589d
* Build a wheel when integration testing PBR changesJeremy Stanley2014-12-241-3/+5
| | | | Change-Id: Id6ac57a5fd2ab9089bdf6865c1e203fcf591f08d
* Port in git sha changes from 0.10 lineMonty Taylor2014-12-211-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Stop including git sha in version strings We include it in pbr.json now. Including it is contentious in the world of python, and it's up for debate as to whether or not it provides value. Write and read more complex git sha info Instead of encoding the git sha into the version string, add it to a metadata file. This will allow us to get out of the business of arguing with pip and setuptools about version info. In order to make this really nice, provide a command line utility called "pbr" that has subcommands to print out the metadata that we're now including in the egg-info dir. Only import sphinx during hook processing When pbr is imported to handle writing the egg_info file because of the entry point, it's causing sphinx to get imported. This has a cascading effect once docutils is trying to be installed on a system with pbr installed. If some of the imports fail along the way, allow pbr to continue usefully but without the Sphinx extensions available. Eventually, when everything is installed, those extensions will work again when the commands for build_sphinx, etc. are run separately. Also slip in a change to reorder the default list of environments run by tox so the testr database is created using a dbm format available to all python versions. Integration test PBR commits Make sure that if a PBR commit is being tested then we install and use that source rather than the latest PBR release. Change-Id: Ie121e795be2eef30822daaa5fe8ab1c2315577ae (cherry picked from commit 65f4fafd907a16ea1952ab7072676db2e9e0c51d) (cherry picked from commit cd7da23937b66fea3ec42fa2f5a128f363a97e7e) Closes-Bug: #1403510 Co-Authored-By: Clark Boylan <clark.boylan@gmail.com> Co-Authored-By: Doug Hellmann <doug@doughellmann.com> Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
* Retry the integration setup on connection errorMike Heald2014-09-121-3/+19
| | | | | | | | | | | Sometimes there are network errors communicating with pypi during a long running (1h+) integration gate job. This causes the job to fail, and wastes a lot of time re-running the job to get it to pass. This change makes it less likely to fail, as if there is a timeout or socket error, the integration setup will retry. Change-Id: Ic4c888626699eb710e85e3bdc48dd6ee74bdf024
* Remove mirror testing from the integration scriptDoug Hellmann2014-07-101-140/+0
| | | | | | | | We don't need to test the mirroring tool in this integration script any more, because we're using a different tool to build the mirror now anyway. Change-Id: I1a3f739a9530f4f5cce18f64b0705ae9ae77704d
* Update integration script for Apache 2.4Sean Dague2014-06-261-4/+24
| | | | | | | | | | | | | | | | | | | | | | | | Apache 2.4 requires site configuration files to have a ".conf" extension, and Apache 2.2 does not want the extension. Add logic to figure out the right name for the file so we can run the tests against both versions of Apache. Fixes bug: #1334326 install required libraries for mixed c things use build-dep to install the required libraries needed for the python build deps. Without this we can get cryptic library compilation issues. And use ccache explicitly Change-Id: Ia750e4221b119097521cf373752aae364759913b squashed into this one because we can't move forward without both. Co-Authored-By: Sean Dague <sean@dague.net> Change-Id: I5ab8898bd3cc2de18681fe3262cb784f7d9519ab
* Make tools/integration.sh take a branchJoe Gordon2014-03-181-2/+4
| | | | | | | | Instead of always using master branch support the option to use stable branches. Read the branch via the $OVERRIDE_ZUUL_BRANCH environmental variable that devstack-gate sets Change-Id: Iab4ae1efddc4dca38dcd3779dae1a79ddc2b0852
* Build mirror using proper requirements.Clark Boylan2013-12-051-3/+7
| | | | | | | | | | | | | * tools/integration.sh: Use the --no-update flag to run-mirror to make sure that the repo checked out by devstack-gate is the repo used to build the mirror. Otherwise `git reset --hard remote/origin/master` is used which we do not want. Also, we can't update the requirements repo until integration.sh is working and for that we need to install d2to1 into the mirror. Usually this would require a requirements repo change, but chicken and egg. Force it in along with pip and setuptools instead. This can be removed when d2to1 is added to the requirements repo. Change-Id: Id5c2c902832e475f28310fdd547672464bf42c0c
* Use wheels for installationMonty Taylor2013-12-041-1/+0
| | | | | | | | Add an option to pbr to use wheels when installing things from pip. The speed boost this gives is quite sexy, as it can avoid having to re-build C things all the time. Change-Id: Id9b9b21ef949ff44c7c962e81576266661264a13
* show diff between pip installed packages and requirementsKhai Do2013-12-031-2/+35
| | | | | | | | | | | This change will display the difference between which python packages pip installed and the packages from the requirements lists. It will print the list of dependencies to the console (not including the directly declared dependencies from the requirements list). Closes-Bug: #1172417 Change-Id: I9363b50d6d7f5ef19c42bb2e30de45e352060ccd
* Enable wheel processing in the testsMonty Taylor2013-11-291-1/+3
| | | | | | | In advance of enabling wheel processing in pbr overall, enable it in the integration testing. Change-Id: Ide460e000c872902590726639d42a5ecac9e4673
* Clean up integration scriptMonty Taylor2013-11-291-47/+27
| | | | | | | | There are several things in the script that are no longer necessary. Remove them to clean things up so that we can reason about this script with less stress. Change-Id: I02863c8ab98d7e403fada2973a0b5807e1b75595
* Serve local mirror using apacheMonty Taylor2013-11-221-3/+33
| | | | | | | | | | There are enough behavioral differences between http and file urls for pypi mirrors that we should do the local mirror via apache. That way we're testing what we care about - consumption of changes in the mirror context - and not what we don't - how well pip deals with file urls. Change-Id: Icb16e8bbe77c03f224d06010eeb6bd0877b0d6ca
* Add wheel mirror structure awarenessMonty Taylor2013-10-311-0/+5
| | | | | | | | | We are going to start having wheels in the gate. Having them there should not break folks who are not using wheels. Don't start actually consuming wheels yet, but add awareness about the wheel structure to the pypi config. Change-Id: Id29bb1bb31fc1647f8e1ea00bab487e447ba2838
* Remove pip as an argument from mkvenvMonty Taylor2013-10-311-9/+8
| | | | | | | We no longer care about or support old pip, because it turns out that evil happens. Remove it from the mkvenv function. Change-Id: I27dbef7816eaac856e97ec5a036f6d5ca9b183af
* Use pypi-mirror instead of jeepybMonty Taylor2013-10-311-10/+10
| | | | Change-Id: I56dffb73526bd56ac544d320d9d6f2baa6175a2a
* Get rid of PyPI URL override in integration testJeremy Stanley2013-10-071-15/+0
| | | | | | | | * tools/integration.sh: Overriding the PyPI URL is logic best left to devstack-gate since it does those things already. Remove the similar dance found here. Change-Id: Ie2fcabb11b30dc3ab6564bdedd55b2b173c1bf2a
* Fix python-ldap mirroring.Robert Collins2013-08-141-1/+1
| | | | | | python-ldap wants ldap headers to compile. Change-Id: I57dc54a3b745702ee8c1f8f8f91fc4ef1aa169fd
* Update requirements in integration testMonty Taylor2013-08-101-0/+7
| | | | | | | So that we don't test combinations of things that we won't see in the gate, run the same requirements sync job that devstack does. Change-Id: I0ee375b4e41aa214acf4cd3bfeb1b3a8ccb08dff
* Clean up some style warningsMonty Taylor2013-08-031-3/+3
| | | | | | | | ksh -n, weirdly enough, can be used as a rudimentary syntax and style checker for bash. tools/integration.sh was mostly clean, but it suggested that `` should be replaced with $(). Sure, why not. Change-Id: I76beff1194133444284c1e8fbbd5d87db369e47f
* Consume the OpenStack mirror by defaultMonty Taylor2013-08-031-4/+23
| | | | | | | | | | | | | | PyPI upstream obviously breaks all the time. Like, literally, I've spent my whole life dealing with it this past week. In this test, we were avoiding the OpenStack mirror so that we could be sure to be testing that we could build the mirror when new requirements are added. However, the only time that's a concern is on a requirements repo change. Add an option --no-mirror that will trigger the "don't use OpenStack's mirrors as an upstream" behavior, and we'll add that option to the invocation on requirements changes. Change-Id: I8d01c1fbb8a5df5aa83ffba3b6489e374abef1af
* Merge "Revert include_package_data change"Jenkins2013-07-271-0/+5
|\
| * Revert include_package_data changeMonty Taylor2013-07-231-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | Setting the value directly in kwargs is not doing what we want it to do. This is largely because of how the hook system works. When the hooks process backwards_compat, they do a config.get('backwards_compat', dict()), which is then written back to the config dict - which means that blank override values are being fed in. Later we can go through and just re-engineer how that works. For now, unbreak nova. Change-Id: I0c6055253cbc89b6884553e6f2fbfe8a7bbd1953
* | Install jeepyb and requirements from REPODIRMonty Taylor2013-07-251-2/+5
| | | | | | | | | | | | | | | | integration.sh already expects there to be a pbr source dir in $REPODIR/pbr. If we pull jeepyb and requirements from the same place, then we can gate them as wel as pbr. Change-Id: Ie0eb513c2deade957d480a25dfba2c5486d14876
* | Stop trying to install old pip and setuptoolsMonty Taylor2013-07-251-3/+2
|/ | | | | | | | | | | | | Now that pip 1.4 is out, setuptools<0.7 is invalid because 1.4 will not install 'pre-release' software without being explicitly asked, and all of the pre 0.7 setuptools releases have a letter in them. Just stop trying to do it, because we actually don't use it anyway. Additionally, we were trying to intsall pip 1.3 for a test but were not installing that into the local mirror. Match them so that they're equal and can install. Change-Id: I61b9b810e64b40ceccb5eb0f4b2ddc4faefd8e04
* Merge "Do not assume the tests run as jenkins"Jenkins2013-07-221-2/+2
|\
View Lorry Controller Status