diff options
34 files changed, 492 insertions, 491 deletions
diff --git a/README.rst b/README.rst index 425a6652e..84b9b7afa 100644 --- a/README.rst +++ b/README.rst @@ -118,7 +118,7 @@ the following: Construct a test to fully simulate the series of events you want to see, then run it in the foreground. For example:: - .tox/py27/bin/python -m testtools.run tests.test_scheduler.TestScheduler.test_jobs_launched + .tox/py27/bin/python -m testtools.run tests.test_scheduler.TestScheduler.test_jobs_executed See TESTING.rst for more information. diff --git a/TESTING.rst b/TESTING.rst index d2cd4c14e..0786ebf40 100644 --- a/TESTING.rst +++ b/TESTING.rst @@ -64,12 +64,12 @@ To run individual tests with tox:: For example, to *run the basic Zuul test*:: - tox -e py27 -- tests.unit.test_scheduler.TestScheduler.test_jobs_launched + tox -e py27 -- tests.unit.test_scheduler.TestScheduler.test_jobs_executed To *run one test in the foreground* (after previously having run tox to set up the virtualenv):: - .tox/py27/bin/python -m testtools.run tests.unit.test_scheduler.TestScheduler.test_jobs_launched + .tox/py27/bin/python -m testtools.run tests.unit.test_scheduler.TestScheduler.test_jobs_executed List Failing Tests ------------------ diff --git a/doc/source/launchers.rst b/doc/source/executors.rst index c9dbd992f..5f85f5828 100644 --- a/doc/source/launchers.rst +++ b/doc/source/executors.rst @@ -1,4 +1,4 @@ -:title: Launchers +:title: Executors .. _Gearman: http://gearman.org/ @@ -11,27 +11,27 @@ .. _`Turbo-Hipster Documentation`: http://turbo-hipster.rtfd.org/ -.. _launchers: +.. _executors: -Launchers +Executors ========= -Zuul has a modular architecture for launching jobs. Currently, the +Zuul has a modular architecture for executing jobs. Currently, the only supported module interfaces with Gearman_. This design allows any system to run jobs for Zuul simply by interfacing with a Gearman server. The recommended way of integrating a new job-runner with Zuul is via this method. -If Gearman is unsuitable, Zuul may be extended with a new launcher +If Gearman is unsuitable, Zuul may be extended with a new executor module. Zuul makes very few assumptions about the interface to a -launcher -- if it can trigger jobs, cancel them, and receive success +executor -- if it can trigger jobs, cancel them, and receive success or failure reports, it should be able to be used with Zuul. Patches to this effect are welcome. Zuul Parameters --------------- -Zuul will pass some parameters with every job it launches. These are +Zuul will pass some parameters with every job it executes. These are for workers to be able to get the repositories into the state they are intended to be tested in. Builds can be triggered either by an action on a change or by a reference update. Both events share a common set diff --git a/doc/source/index.rst b/doc/source/index.rst index 784fc4d96..3f903db46 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -21,7 +21,7 @@ Contents: zuul merger cloner - launchers + executors statsd client developer diff --git a/doc/source/merger.rst b/doc/source/merger.rst index 82e204b2c..d8de7028d 100644 --- a/doc/source/merger.rst +++ b/doc/source/merger.rst @@ -30,7 +30,7 @@ cherry-picking changes as required and identifies the result with a Git reference of the form ``refs/zuul/<branch>/Z<random sha1>``. Preparing the workspace is then a simple matter of fetching that ref and checking it out. The parameters that provide this information are -described in :ref:`launchers`. +described in :ref:`executors`. These references need to be made available via a Git repository that is available to workers (such as Jenkins). This is accomplished by diff --git a/doc/source/quick-start.rst b/doc/source/quick-start.rst index 82779c670..6d16c92a6 100644 --- a/doc/source/quick-start.rst +++ b/doc/source/quick-start.rst @@ -19,7 +19,7 @@ Zuul Components Zuul provides the following components: - **zuul-server**: scheduler daemon which communicates with Gerrit and - Gearman. Handles receiving events, launching jobs, collecting results + Gearman. Handles receiving events, executing jobs, collecting results and postingreports. - **zuul-merger**: speculative-merger which communicates with Gearman. Prepares Git repositories for jobs to test against. This additionally diff --git a/doc/source/statsd.rst b/doc/source/statsd.rst index b3bf99f32..fb6989ef4 100644 --- a/doc/source/statsd.rst +++ b/doc/source/statsd.rst @@ -17,7 +17,7 @@ the statsd python module, so an existing Zuul installation may be missing it. The configuration is done via environment variables STATSD_HOST and STATSD_PORT. They are interpreted by the statsd module directly and there is no such parameter in zuul.conf yet. Your init script will have to initialize both -of them before launching Zuul. +of them before executing Zuul. Your init script most probably loads a configuration file named ``/etc/default/zuul`` which would contain the environment variables:: @@ -61,7 +61,7 @@ The metrics are emitted by the Zuul scheduler (`zuul/scheduler.py`): #. **job.<jobname>** subtree detailing per job statistics: #. **wait_time** counter and timer of the wait time, with the - difference of the job start time and the launch time, in + difference of the job start time and the execute time, in milliseconds. **zuul.pipeline.** @@ -88,7 +88,7 @@ The metrics are emitted by the Zuul scheduler (`zuul/scheduler.py`): #. **total_changes** counter of the number of change proceeding since Zuul started. #. **wait_time** counter and timer of the wait time, with the difference - of the job start time and the launch time, in milliseconds. + of the job start time and the execute time, in milliseconds. Additionally, the `zuul.pipeline.<pipeline name>` hierarchy contains `current_changes` (gauge), `resident_time` (timing) and `total_changes` diff --git a/doc/source/zuul.rst b/doc/source/zuul.rst index 4f43596b0..8b325baa6 100644 --- a/doc/source/zuul.rst +++ b/doc/source/zuul.rst @@ -635,9 +635,9 @@ each job as it builds a list from the project specification. **hold-following-changes (optional)** This is a boolean that indicates that changes that follow this change in a dependent change pipeline should wait until this job - succeeds before launching. If this is applied to a very short job + succeeds before executing. If this is applied to a very short job that can predict whether longer jobs will fail early, this can be - used to reduce the number of jobs that Zuul will launch and + used to reduce the number of jobs that Zuul will execute and ultimately have to cancel. In that case, a small amount of parallelization of jobs is traded for more efficient use of testing resources. On the other hand, to apply this to a long running job @@ -709,7 +709,7 @@ each job as it builds a list from the project specification. a job is voting or not. Default: ``true``. **attempts (optional)** - Number of attempts zuul will launch a job. Once reached, zuul will report + Number of attempts zuul will execute a job. Once reached, zuul will report RETRY_LIMIT as the job result. Defaults to 3. diff --git a/playbooks/base-post.yaml b/playbooks/base-post.yaml index 2b3450d26..ed3f7b8c9 100644 --- a/playbooks/base-post.yaml +++ b/playbooks/base-post.yaml @@ -2,12 +2,12 @@ tasks: - name: Collect console log. synchronize: - dest: "{{ zuul.launcher.log_root }}" + dest: "{{ zuul.executor.log_root }}" mode: pull src: "/tmp/console.log" - name: Publish logs. copy: dest: "/opt/zuul-logs/{{ zuul.uuid}}" - src: "{{ zuul.launcher.log_root }}/" + src: "{{ zuul.executor.log_root }}/" delegate_to: 127.0.0.1 diff --git a/playbooks/roles/prepare-workspace/tasks/main.yaml b/playbooks/roles/prepare-workspace/tasks/main.yaml index 5fcc67979..26d067009 100644 --- a/playbooks/roles/prepare-workspace/tasks/main.yaml +++ b/playbooks/roles/prepare-workspace/tasks/main.yaml @@ -18,7 +18,7 @@ - name: Synchronize src repos to workspace directory. synchronize: dest: "{{ prepare_workspace_root }}" - src: "{{ zuul.launcher.src_root }}" + src: "{{ zuul.executor.src_root }}" - name: Run configure_mirror.sh shell: /opt/nodepool-scripts/configure_mirror.sh diff --git a/playbooks/tox-post.yaml b/playbooks/tox-post.yaml index 786a984d2..697b727fe 100644 --- a/playbooks/tox-post.yaml +++ b/playbooks/tox-post.yaml @@ -11,7 +11,7 @@ - name: Collect tox logs. synchronize: - dest: "{{ zuul.launcher.log_root }}/tox" + dest: "{{ zuul.executor.log_root }}/tox" mode: pull src: "{{ item.path }}/log/" with_items: "{{ result.files }}" @@ -25,7 +25,7 @@ console_scripts = zuul-merger = zuul.cmd.merger:main zuul = zuul.cmd.client:main zuul-cloner = zuul.cmd.cloner:main - zuul-launcher = zuul.cmd.launcher:main + zuul-executor = zuul.cmd.executor:main [build_sphinx] source-dir = doc/source diff --git a/tests/base.py b/tests/base.py index 3fe29b210..a2a70c2bf 100755 --- a/tests/base.py +++ b/tests/base.py @@ -57,8 +57,8 @@ import zuul.connection.sql import zuul.scheduler import zuul.webapp import zuul.rpclistener -import zuul.launcher.server -import zuul.launcher.client +import zuul.executor.server +import zuul.executor.client import zuul.lib.connections import zuul.merger.client import zuul.merger.merger @@ -570,9 +570,9 @@ class FakeStatsd(threading.Thread): class FakeBuild(object): log = logging.getLogger("zuul.test") - def __init__(self, launch_server, job): + def __init__(self, executor_server, job): self.daemon = True - self.launch_server = launch_server + self.executor_server = executor_server self.job = job self.jobdir = None self.uuid = job.unique @@ -638,7 +638,7 @@ class FakeBuild(object): def run(self): self.log.debug('Running build %s' % self.unique) - if self.launch_server.hold_jobs_in_build: + if self.executor_server.hold_jobs_in_build: self.log.debug('Holding build %s' % self.unique) self._wait() self.log.debug("Build %s continuing" % self.unique) @@ -654,7 +654,7 @@ class FakeBuild(object): return result def shouldFail(self): - changes = self.launch_server.fail_tests.get(self.name, []) + changes = self.executor_server.fail_tests.get(self.name, []) for change in changes: if self.hasChanges(change): return True @@ -691,21 +691,21 @@ class FakeBuild(object): return True -class RecordingLaunchServer(zuul.launcher.server.LaunchServer): - """An Ansible launcher to be used in tests. +class RecordingExecutorServer(zuul.executor.server.ExecutorServer): + """An Ansible executor to be used in tests. - :ivar bool hold_jobs_in_build: If true, when jobs are launched + :ivar bool hold_jobs_in_build: If true, when jobs are executed they will report that they have started but then pause until released before reporting completion. This attribute may be changed at any time and will take effect for subsequently - launched builds, but previously held builds will still need to + executed builds, but previously held builds will still need to be explicitly released. """ def __init__(self, *args, **kw): self._run_ansible = kw.pop('_run_ansible', False) self._test_root = kw.pop('_test_root', False) - super(RecordingLaunchServer, self).__init__(*args, **kw) + super(RecordingExecutorServer, self).__init__(*args, **kw) self.hold_jobs_in_build = False self.lock = threading.Lock() self.running_builds = [] @@ -714,7 +714,7 @@ class RecordingLaunchServer(zuul.launcher.server.LaunchServer): self.job_builds = {} def failJob(self, name, change): - """Instruct the launcher to report matching builds as failures. + """Instruct the executor to report matching builds as failures. :arg str name: The name of the job to fail. :arg Change change: The :py:class:`~tests.base.FakeChange` @@ -748,7 +748,7 @@ class RecordingLaunchServer(zuul.launcher.server.LaunchServer): self.log.debug("Done releasing builds %s (%s)" % (regex, len(self.running_builds))) - def launchJob(self, job): + def executeJob(self, job): build = FakeBuild(self, job) job.build = build self.running_builds.append(build) @@ -767,32 +767,32 @@ class RecordingLaunchServer(zuul.launcher.server.LaunchServer): if build.unique == uuid: build.aborted = True build.release() - super(RecordingLaunchServer, self).stopJob(job) + super(RecordingExecutorServer, self).stopJob(job) -class RecordingAnsibleJob(zuul.launcher.server.AnsibleJob): +class RecordingAnsibleJob(zuul.executor.server.AnsibleJob): def runPlaybooks(self, args): - build = self.launcher_server.job_builds[self.job.unique] + build = self.executor_server.job_builds[self.job.unique] build.jobdir = self.jobdir result = super(RecordingAnsibleJob, self).runPlaybooks(args) - self.launcher_server.lock.acquire() - self.launcher_server.build_history.append( + self.executor_server.lock.acquire() + self.executor_server.build_history.append( BuildHistory(name=build.name, result=result, changes=build.changes, node=build.node, uuid=build.unique, parameters=build.parameters, jobdir=build.jobdir, pipeline=build.parameters['ZUUL_PIPELINE']) ) - self.launcher_server.running_builds.remove(build) - del self.launcher_server.job_builds[self.job.unique] - self.launcher_server.lock.release() + self.executor_server.running_builds.remove(build) + del self.executor_server.job_builds[self.job.unique] + self.executor_server.lock.release() return result def runAnsible(self, cmd, timeout, trusted=False): - build = self.launcher_server.job_builds[self.job.unique] + build = self.executor_server.job_builds[self.job.unique] - if self.launcher_server._run_ansible: + if self.executor_server._run_ansible: result = super(RecordingAnsibleJob, self).runAnsible( cmd, timeout, trusted=trusted) else: @@ -828,7 +828,7 @@ class FakeGearmanServer(gear.Server): for queue in [self.high_queue, self.normal_queue, self.low_queue]: for job in queue: if not hasattr(job, 'waiting'): - if job.name.startswith('launcher:launch'): + if job.name.startswith('executor:execute'): job.waiting = self.hold_jobs_in_queue else: job.waiting = False @@ -855,7 +855,7 @@ class FakeGearmanServer(gear.Server): len(self.low_queue)) self.log.debug("releasing queued job %s (%s)" % (regex, qlen)) for job in self.getQueue(): - if job.name != 'launcher:launch': + if job.name != 'executor:execute': continue parameters = json.loads(job.arguments) if not regex or re.match(regex, parameters.get('job')): @@ -991,7 +991,7 @@ class FakeNodepool(object): created_time=now, updated_time=now, image_id=None, - launcher='fake-nodepool') + executor='fake-nodepool') data = json.dumps(data) path = self.client.create(path, data, makepath=True, @@ -1223,13 +1223,13 @@ class ZuulTestCase(BaseTestCase): server that all of the Zuul components in this test use to communicate with each other. - :ivar RecordingLaunchServer launch_server: An instance of - :py:class:`~tests.base.RecordingLaunchServer` which is the - Ansible launch server used to run jobs for this test. + :ivar RecordingExecutorServer executor_server: An instance of + :py:class:`~tests.base.RecordingExecutorServer` which is the + Ansible execute server used to run jobs for this test. :ivar list builds: A list of :py:class:`~tests.base.FakeBuild` objects representing currently running builds. They are appended to - the list in the order they are launched, and removed from this + the list in the order they are executed, and removed from this list upon completion. :ivar list history: A list of :py:class:`~tests.base.BuildHistory` @@ -1261,7 +1261,7 @@ class ZuulTestCase(BaseTestCase): self.test_root = os.path.join(tmp_root, "zuul-test") self.upstream_root = os.path.join(self.test_root, "upstream") self.merger_src_root = os.path.join(self.test_root, "merger-git") - self.launcher_src_root = os.path.join(self.test_root, "launcher-git") + self.executor_src_root = os.path.join(self.test_root, "executor-git") self.state_root = os.path.join(self.test_root, "lib") if os.path.exists(self.test_root): @@ -1276,7 +1276,7 @@ class ZuulTestCase(BaseTestCase): os.path.join(FIXTURE_DIR, self.config.get('zuul', 'tenant_config'))) self.config.set('merger', 'git_dir', self.merger_src_root) - self.config.set('launcher', 'git_dir', self.launcher_src_root) + self.config.set('executor', 'git_dir', self.executor_src_root) self.config.set('zuul', 'state_dir', self.state_root) # For each project in config: @@ -1337,17 +1337,17 @@ class ZuulTestCase(BaseTestCase): self._startMerger() - self.launch_server = RecordingLaunchServer( + self.executor_server = RecordingExecutorServer( self.config, self.connections, jobdir_root=self.test_root, _run_ansible=self.run_ansible, _test_root=self.test_root, keep_jobdir=KEEP_TEMPDIRS) - self.launch_server.start() - self.history = self.launch_server.build_history - self.builds = self.launch_server.running_builds + self.executor_server.start() + self.history = self.executor_server.build_history + self.builds = self.executor_server.running_builds - self.launch_client = zuul.launcher.client.LaunchClient( + self.executor_client = zuul.executor.client.ExecutorClient( self.config, self.sched) self.merge_client = zuul.merger.client.MergeClient( self.config, self.sched) @@ -1360,7 +1360,7 @@ class ZuulTestCase(BaseTestCase): self.zk_chroot_fixture.zookeeper_port, self.zk_chroot_fixture.zookeeper_chroot) - self.sched.setLauncher(self.launch_client) + self.sched.setExecutor(self.executor_client) self.sched.setMerger(self.merge_client) self.sched.setNodepool(self.nodepool) self.sched.setZooKeeper(self.zk) @@ -1372,7 +1372,7 @@ class ZuulTestCase(BaseTestCase): self.sched.start() self.webapp.start() self.rpc.start() - self.launch_client.gearman.waitForServer() + self.executor_client.gearman.waitForServer() self.addCleanup(self.shutdown) self.sched.reconfigure(self.config) @@ -1488,11 +1488,11 @@ class ZuulTestCase(BaseTestCase): def shutdown(self): self.log.debug("Shutting down after tests") - self.launch_client.stop() + self.executor_client.stop() self.merge_server.stop() self.merge_server.join() self.merge_client.stop() - self.launch_server.stop() + self.executor_server.stop() self.sched.stop() self.sched.join() self.statsd.stop() @@ -1579,29 +1579,29 @@ class ZuulTestCase(BaseTestCase): def haveAllBuildsReported(self): # See if Zuul is waiting on a meta job to complete - if self.launch_client.meta_jobs: + if self.executor_client.meta_jobs: return False # Find out if every build that the worker has completed has been # reported back to Zuul. If it hasn't then that means a Gearman # event is still in transit and the system is not stable. for build in self.history: - zbuild = self.launch_client.builds.get(build.uuid) + zbuild = self.executor_client.builds.get(build.uuid) if not zbuild: # It has already been reported continue # It hasn't been reported yet. return False # Make sure that none of the worker connections are in GRAB_WAIT - for connection in self.launch_server.worker.active_connections: + for connection in self.executor_server.worker.active_connections: if connection.state == 'GRAB_WAIT': return False return True def areAllBuildsWaiting(self): - builds = self.launch_client.builds.values() + builds = self.executor_client.builds.values() for build in builds: client_job = None - for conn in self.launch_client.gearman.active_connections: + for conn in self.executor_client.gearman.active_connections: for j in conn.related_jobs.values(): if j.unique == build.uuid: client_job = j @@ -1626,7 +1626,8 @@ class ZuulTestCase(BaseTestCase): if build.url is None: self.log.debug("%s has not reported start" % build) return False - worker_build = self.launch_server.job_builds.get(server_job.unique) + worker_build = self.executor_server.job_builds.get( + server_job.unique) if worker_build: if worker_build.isWaiting(): continue @@ -1673,7 +1674,7 @@ class ZuulTestCase(BaseTestCase): raise Exception("Timeout waiting for Zuul to settle") # Make sure no new events show up while we're checking - self.launch_server.lock.acquire() + self.executor_server.lock.acquire() # have all build states propogated to zuul? if self.haveAllBuildsReported(): # Join ensures that the queue is empty _and_ events have been @@ -1691,11 +1692,11 @@ class ZuulTestCase(BaseTestCase): # components were stable, we don't erroneously # report that we are settled. self.sched.run_handler_lock.release() - self.launch_server.lock.release() + self.executor_server.lock.release() self.log.debug("...settled.") return self.sched.run_handler_lock.release() - self.launch_server.lock.release() + self.executor_server.lock.release() self.sched.wake_event.wait(0.1) def countJobResults(self, jobs, result): @@ -1912,7 +1913,7 @@ class ZuulTestCase(BaseTestCase): class AnsibleZuulTestCase(ZuulTestCase): - """ZuulTestCase but with an actual ansible launcher running""" + """ZuulTestCase but with an actual ansible executor running""" run_ansible = True diff --git a/tests/fixtures/zuul-connections-multiple-gerrits.conf b/tests/fixtures/zuul-connections-multiple-gerrits.conf index c1a335df5..b3182d7d9 100644 --- a/tests/fixtures/zuul-connections-multiple-gerrits.conf +++ b/tests/fixtures/zuul-connections-multiple-gerrits.conf @@ -12,8 +12,8 @@ git_user_email=zuul@example.com git_user_name=zuul zuul_url=http://zuul.example.com/p -[launcher] -git_dir=/tmp/zuul-test/launcher-git +[executor] +git_dir=/tmp/zuul-test/executor-git [connection review_gerrit] driver=gerrit diff --git a/tests/fixtures/zuul-connections-same-gerrit.conf b/tests/fixtures/zuul-connections-same-gerrit.conf index 5c1044451..69f523982 100644 --- a/tests/fixtures/zuul-connections-same-gerrit.conf +++ b/tests/fixtures/zuul-connections-same-gerrit.conf @@ -12,8 +12,8 @@ git_user_email=zuul@example.com git_user_name=zuul zuul_url=http://zuul.example.com/p -[launcher] -git_dir=/tmp/zuul-test/launcher-git +[executor] +git_dir=/tmp/zuul-test/executor-git [connection review_gerrit] driver=gerrit diff --git a/tests/fixtures/zuul-git-driver.conf b/tests/fixtures/zuul-git-driver.conf index 868e2725f..936c5302a 100644 --- a/tests/fixtures/zuul-git-driver.conf +++ b/tests/fixtures/zuul-git-driver.conf @@ -12,8 +12,8 @@ git_user_email=zuul@example.com git_user_name=zuul zuul_url=http://zuul.example.com/p -[launcher] -git_dir=/tmp/zuul-test/launcher-git +[executor] +git_dir=/tmp/zuul-test/executor-git [swift] authurl=https://identity.api.example.org/v2.0/ diff --git a/tests/fixtures/zuul.conf b/tests/fixtures/zuul.conf index f0b606825..516ce089f 100644 --- a/tests/fixtures/zuul.conf +++ b/tests/fixtures/zuul.conf @@ -12,8 +12,8 @@ git_user_email=zuul@example.com git_user_name=zuul zuul_url=http://zuul.example.com/p -[launcher] -git_dir=/tmp/zuul-test/launcher-git +[executor] +git_dir=/tmp/zuul-test/executor-git [swift] authurl=https://identity.api.example.org/v2.0/ diff --git a/tests/unit/test_cloner.py b/tests/unit/test_cloner.py index da0f774c3..e65904bf0 100644 --- a/tests/unit/test_cloner.py +++ b/tests/unit/test_cloner.py @@ -610,7 +610,7 @@ class TestCloner(ZuulTestCase): # Start a periodic job self.worker.hold_jobs_in_build = True - self.launcher.negative_function_cache_ttl = 0 + self.executor.negative_function_cache_ttl = 0 self.config.set('zuul', 'layout_config', 'tests/fixtures/layout-timer.yaml') self.sched.reconfigure(self.config) diff --git a/tests/unit/test_connection.py b/tests/unit/test_connection.py index 895483297..22cf33103 100644 --- a/tests/unit/test_connection.py +++ b/tests/unit/test_connection.py @@ -46,7 +46,7 @@ class TestConnections(ZuulTestCase): 'jenkins') B = self.fake_review_gerrit.addFakeChange('org/project', 'master', 'B') - self.launch_server.failJob('project-test2', B) + self.executor_server.failJob('project-test2', B) self.addEvent('review_gerrit', B.getPatchsetCreatedEvent(1)) self.waitUntilSettled() @@ -239,7 +239,7 @@ class TestMultipleGerrits(ZuulTestCase): tenant_config_file = 'config/zuul-connections-multiple-gerrits/main.yaml' def test_multiple_project_separate_gerrits(self): - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_another_gerrit.addFakeChange( 'org/project1', 'master', 'A') @@ -276,6 +276,6 @@ class TestMultipleGerrits(ZuulTestCase): pipeline='review_check'), ]) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() diff --git a/tests/unit/test_openstack.py b/tests/unit/test_openstack.py index 670e578c7..c7a47ec3b 100644 --- a/tests/unit/test_openstack.py +++ b/tests/unit/test_openstack.py @@ -58,7 +58,7 @@ class TestOpenStack(AnsibleZuulTestCase): 'ubuntu-trusty') def test_dsvm_keystone_repo(self): - self.launch_server.keep_jobdir = True + self.executor_server.keep_jobdir = True A = self.fake_gerrit.addFakeChange('openstack/nova', 'master', 'A') self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) self.waitUntilSettled() @@ -68,9 +68,9 @@ class TestOpenStack(AnsibleZuulTestCase): build = self.getJobFromHistory('dsvm') # Check that a change to nova triggered a keystone clone - launcher_git_dir = os.path.join(self.launcher_src_root, + executor_git_dir = os.path.join(self.executor_src_root, 'openstack', 'keystone', '.git') - self.assertTrue(os.path.exists(launcher_git_dir), + self.assertTrue(os.path.exists(executor_git_dir), msg='openstack/keystone should be cloned.') jobdir_git_dir = os.path.join(build.jobdir.src_root, @@ -79,7 +79,7 @@ class TestOpenStack(AnsibleZuulTestCase): msg='openstack/keystone should be cloned.') def test_dsvm_nova_repo(self): - self.launch_server.keep_jobdir = True + self.executor_server.keep_jobdir = True A = self.fake_gerrit.addFakeChange('openstack/keystone', 'master', 'A') self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) self.waitUntilSettled() @@ -89,9 +89,9 @@ class TestOpenStack(AnsibleZuulTestCase): build = self.getJobFromHistory('dsvm') # Check that a change to keystone triggered a nova clone - launcher_git_dir = os.path.join(self.launcher_src_root, + executor_git_dir = os.path.join(self.executor_src_root, 'openstack', 'nova', '.git') - self.assertTrue(os.path.exists(launcher_git_dir), + self.assertTrue(os.path.exists(executor_git_dir), msg='openstack/nova should be cloned.') jobdir_git_dir = os.path.join(build.jobdir.src_root, diff --git a/tests/unit/test_scheduler.py b/tests/unit/test_scheduler.py index 1e56faecb..e32e41bc0 100755 --- a/tests/unit/test_scheduler.py +++ b/tests/unit/test_scheduler.py @@ -39,8 +39,8 @@ from tests.base import ( class TestScheduler(ZuulTestCase): tenant_config_file = 'config/single-tenant/main.yaml' - def test_jobs_launched(self): - "Test that jobs are launched and a change is merged" + def test_jobs_executed(self): + "Test that jobs are executed and a change is merged" A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) @@ -106,7 +106,7 @@ class TestScheduler(ZuulTestCase): def test_parallel_changes(self): "Test that changes are tested in parallel and merged in series" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -123,7 +123,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[0].name, 'project-merge') self.assertTrue(self.builds[0].hasChanges(A)) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 3) self.assertEqual(self.builds[0].name, 'project-test1') @@ -133,7 +133,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[2].name, 'project-merge') self.assertTrue(self.builds[2].hasChanges(A, B)) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 5) self.assertEqual(self.builds[0].name, 'project-test1') @@ -149,7 +149,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[4].name, 'project-merge') self.assertTrue(self.builds[4].hasChanges(A, B, C)) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 6) self.assertEqual(self.builds[0].name, 'project-test1') @@ -167,8 +167,8 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[5].name, 'project-test2') self.assertTrue(self.builds[5].hasChanges(A, B, C)) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.builds), 0) @@ -182,21 +182,21 @@ class TestScheduler(ZuulTestCase): def test_failed_changes(self): "Test that a change behind a failed change is retested" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') A.addApproval('code-review', 2) B.addApproval('code-review', 2) - self.launch_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', A) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) self.waitUntilSettled() self.assertBuilds([dict(name='project-merge', changes='1,1')]) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # A/project-merge is complete self.assertBuilds([ @@ -205,7 +205,7 @@ class TestScheduler(ZuulTestCase): dict(name='project-merge', changes='1,1 2,1'), ]) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # A/project-merge is complete # B/project-merge is complete @@ -217,7 +217,7 @@ class TestScheduler(ZuulTestCase): ]) # Release project-test1 for A which will fail. This will - # abort both running B jobs and relaunch project-merge for B. + # abort both running B jobs and reexecute project-merge for B. self.builds[0].release() self.waitUntilSettled() @@ -242,7 +242,7 @@ class TestScheduler(ZuulTestCase): def test_independent_queues(self): "Test that changes end up in the right queues" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project1', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project2', 'master', 'C') @@ -271,7 +271,7 @@ class TestScheduler(ZuulTestCase): self.builds[0].release() self.waitUntilSettled() # Release the merge job for project2 which is behind project1 - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # All the test builds should be running: @@ -316,7 +316,7 @@ class TestScheduler(ZuulTestCase): def test_failed_change_at_head(self): "Test that if a change at the head fails, jobs behind it are canceled" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -324,7 +324,7 @@ class TestScheduler(ZuulTestCase): B.addApproval('code-review', 2) C.addApproval('code-review', 2) - self.launch_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', A) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) @@ -336,11 +336,11 @@ class TestScheduler(ZuulTestCase): dict(name='project-merge', changes='1,1'), ]) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertBuilds([ @@ -381,9 +381,9 @@ class TestScheduler(ZuulTestCase): changes='1,1 2,1 3,1'), ], ordered=False) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.orderedRelease() @@ -431,7 +431,7 @@ class TestScheduler(ZuulTestCase): def test_failed_change_in_middle(self): "Test a failed change in the middle of the queue" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -439,7 +439,7 @@ class TestScheduler(ZuulTestCase): B.addApproval('code-review', 2) C.addApproval('code-review', 2) - self.launch_server.failJob('project-test1', B) + self.executor_server.failJob('project-test1', B) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) @@ -447,11 +447,11 @@ class TestScheduler(ZuulTestCase): self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 6) @@ -471,7 +471,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(len(self.builds), 4) self.assertEqual(self.countJobResults(self.history, 'ABORTED'), 2) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # project-test1 and project-test2 for A @@ -492,8 +492,8 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.countJobResults(builds, 'SUCCESS'), 1) self.assertEqual(self.countJobResults(builds, None), 2) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.builds), 0) @@ -516,7 +516,7 @@ class TestScheduler(ZuulTestCase): B.addApproval('code-review', 2) C.addApproval('code-review', 2) - self.launch_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', A) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) @@ -526,7 +526,7 @@ class TestScheduler(ZuulTestCase): queue = self.gearman_server.getQueue() self.assertEqual(len(self.builds), 0) self.assertEqual(len(queue), 1) - self.assertEqual(queue[0].name, 'launcher:launch') + self.assertEqual(queue[0].name, 'executor:execute') job_args = json.loads(queue[0].arguments) self.assertEqual(job_args['job'], 'project-merge') self.assertEqual(job_args['items'][0]['number'], '%d' % A.number) @@ -578,7 +578,7 @@ class TestScheduler(ZuulTestCase): @skip("Disabled for early v3 development") def _test_time_database(self, iteration): - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) @@ -607,8 +607,8 @@ class TestScheduler(ZuulTestCase): self.assertTrue(found_job['estimated_time'] >= 2) self.assertIsNotNone(found_job['remaining_time']) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() @skip("Disabled for early v3 development") @@ -621,7 +621,7 @@ class TestScheduler(ZuulTestCase): def test_two_failed_changes_at_head(self): "Test that changes are reparented correctly if 2 fail at head" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -629,19 +629,19 @@ class TestScheduler(ZuulTestCase): B.addApproval('code-review', 2) C.addApproval('code-review', 2) - self.launch_server.failJob('project-test1', A) - self.launch_server.failJob('project-test1', B) + self.executor_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', B) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) self.fake_gerrit.addEvent(C.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 6) @@ -664,7 +664,7 @@ class TestScheduler(ZuulTestCase): self.waitUntilSettled() # restart of C after B failure - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 5) @@ -689,9 +689,9 @@ class TestScheduler(ZuulTestCase): self.waitUntilSettled() # restart of B,C after A failure - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 4) @@ -710,8 +710,8 @@ class TestScheduler(ZuulTestCase): self.assertTrue(self.builds[2].hasChanges(B)) self.assertTrue(self.builds[2].hasChanges(C)) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.builds), 0) @@ -819,7 +819,7 @@ class TestScheduler(ZuulTestCase): for connection in self.connections.connections.values(): connection.maintainCache([]) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A.addApproval('approved', 1) B.addApproval('approved', 1) D.addApproval('approved', 1) @@ -829,10 +829,10 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(C.addApproval('approved', 1)) for x in range(8): - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -854,7 +854,7 @@ class TestScheduler(ZuulTestCase): def test_source_cache(self): "Test that the source cache operates correctly" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') @@ -889,8 +889,8 @@ class TestScheduler(ZuulTestCase): # there should still be changes in the cache self.assertNotEqual(len(self.fake_gerrit._change_cache.keys()), 0) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -1069,7 +1069,7 @@ class TestScheduler(ZuulTestCase): B.setDependsOn(A, 1) A.setDependsOn(M1, 1) - self.launch_server.failJob('project-merge', A) + self.executor_server.failJob('project-merge', A) self.fake_gerrit.addEvent(C.addApproval('approved', 1)) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) @@ -1087,7 +1087,7 @@ class TestScheduler(ZuulTestCase): def test_failing_dependent_changes(self): "Test that failing dependent patches are taken out of stream" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -1104,7 +1104,7 @@ class TestScheduler(ZuulTestCase): D.setDependsOn(C, 1) C.setDependsOn(B, 1) - self.launch_server.failJob('project-test1', B) + self.executor_server.failJob('project-test1', B) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.fake_gerrit.addEvent(D.addApproval('approved', 1)) @@ -1113,24 +1113,24 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(E.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False + self.executor_server.hold_jobs_in_build = False for build in self.builds: if build.parameters['ZUUL_CHANGE'] != '1': build.release() self.waitUntilSettled() - self.launch_server.release() + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -1155,7 +1155,7 @@ class TestScheduler(ZuulTestCase): # If it's dequeued more than once, we should see extra # aborted jobs. - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project1', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project1', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project1', 'master', 'C') @@ -1163,9 +1163,9 @@ class TestScheduler(ZuulTestCase): B.addApproval('code-review', 2) C.addApproval('code-review', 2) - self.launch_server.failJob('project-test1', A) - self.launch_server.failJob('project-test2', A) - self.launch_server.failJob('project1-project2-integration', A) + self.executor_server.failJob('project-test1', A) + self.executor_server.failJob('project-test2', A) + self.executor_server.failJob('project1-project2-integration', A) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) @@ -1177,11 +1177,11 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[0].name, 'project-merge') self.assertTrue(self.builds[0].hasChanges(A)) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 9) @@ -1201,8 +1201,8 @@ class TestScheduler(ZuulTestCase): self.assertEqual(len(self.builds), 3) # test2,integration, merge for B self.assertEqual(self.countJobResults(self.history, 'ABORTED'), 6) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.builds), 0) @@ -1221,7 +1221,7 @@ class TestScheduler(ZuulTestCase): A = self.fake_gerrit.addFakeChange('org/nonvoting-project', 'master', 'A') A.addApproval('code-review', 2) - self.launch_server.failJob('nonvoting-project-test2', A) + self.executor_server.failJob('nonvoting-project-test2', A) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() @@ -1262,7 +1262,7 @@ class TestScheduler(ZuulTestCase): "Test failed check queue jobs." A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') - self.launch_server.failJob('project-test2', A) + self.executor_server.failJob('project-test2', A) self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) self.waitUntilSettled() @@ -1284,7 +1284,7 @@ class TestScheduler(ZuulTestCase): # This complicated test is a reproduction of a real life bug self.sched.reconfigure(self.config) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project1', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project1', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project2', 'master', 'C') @@ -1310,9 +1310,9 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(B.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.fake_gerrit.addEvent(C.addApproval('approved', 1)) @@ -1324,13 +1324,13 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(F.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # all jobs running @@ -1344,8 +1344,8 @@ class TestScheduler(ZuulTestCase): c.release() self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'NEW') @@ -1386,7 +1386,7 @@ class TestScheduler(ZuulTestCase): path = os.path.join(self.merger_src_root, "org/project") if os.path.exists(path): repack_repo(path) - path = os.path.join(self.launcher_src_root, "org/project") + path = os.path.join(self.executor_src_root, "org/project") if os.path.exists(path): repack_repo(path) @@ -1405,7 +1405,7 @@ class TestScheduler(ZuulTestCase): def test_merger_repack_large_change(self): "Test that the merger works with large changes after a repack" - # https://bugs.launchpad.net/zuul/+bug/1078946 + # https://bugs.executepad.net/zuul/+bug/1078946 # This test assumes the repo is already cloned; make sure it is tenant = self.sched.abide.tenants.get('tenant-one') url = self.fake_gerrit.getGitUrl( @@ -1418,7 +1418,7 @@ class TestScheduler(ZuulTestCase): path = os.path.join(self.merger_src_root, "org/project1") if os.path.exists(path): repack_repo(path) - path = os.path.join(self.launcher_src_root, "org/project1") + path = os.path.join(self.executor_src_root, "org/project1") if os.path.exists(path): repack_repo(path) @@ -1437,7 +1437,7 @@ class TestScheduler(ZuulTestCase): def test_new_patchset_dequeues_old(self): "Test that a new patchset causes the old to be dequeued" # D -> C (depends on B) -> B (depends on A) -> A -> M - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True M = self.fake_gerrit.addFakeChange('org/project', 'master', 'M') M.setMerged() @@ -1464,8 +1464,8 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(B.getPatchsetCreatedEvent(2)) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -1481,7 +1481,7 @@ class TestScheduler(ZuulTestCase): def test_new_patchset_check(self): "Test a new patchset in check" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') @@ -1559,8 +1559,8 @@ class TestScheduler(ZuulTestCase): self.waitUntilSettled() self.builds[0].release() self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.reported, 1) @@ -1577,7 +1577,7 @@ class TestScheduler(ZuulTestCase): def test_abandoned_gate(self): "Test that an abandoned change is dequeued from gate" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) @@ -1589,7 +1589,7 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(A.getChangeAbandonedEvent()) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertBuilds([]) @@ -1602,7 +1602,7 @@ class TestScheduler(ZuulTestCase): def test_abandoned_check(self): "Test that an abandoned change is dequeued from check" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') @@ -1643,8 +1643,8 @@ class TestScheduler(ZuulTestCase): self.assertEqual(items[1].change.number, '2') self.assertTrue(items[1].live) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.history), 4) @@ -1656,7 +1656,7 @@ class TestScheduler(ZuulTestCase): def test_abandoned_not_timer(self): "Test that an abandoned change does not cancel timer jobs" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True # Start timer trigger - also org/project self.updateConfigLayout('layout-idle') @@ -1683,13 +1683,13 @@ class TestScheduler(ZuulTestCase): self.assertEqual(len(self.builds), 2, "Two timer jobs remain") - self.launch_server.release() + self.executor_server.release() self.waitUntilSettled() def test_zuul_url_return(self): "Test if ZUUL_URL is returning when zuul_url is set in zuul.conf" self.assertTrue(self.sched.config.has_option('merger', 'zuul_url')) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) @@ -1700,14 +1700,14 @@ class TestScheduler(ZuulTestCase): for build in self.builds: self.assertTrue('ZUUL_URL' in build.parameters) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() def test_new_patchset_dequeues_old_on_head(self): "Test that a new patchset causes the old to be dequeued (at head)" # D -> C (depends on B) -> B (depends on A) -> A -> M - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True M = self.fake_gerrit.addFakeChange('org/project', 'master', 'M') M.setMerged() A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') @@ -1733,8 +1733,8 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(2)) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'NEW') @@ -1749,7 +1749,7 @@ class TestScheduler(ZuulTestCase): def test_new_patchset_dequeues_old_without_dependents(self): "Test that a new patchset causes only the old to be dequeued" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -1766,8 +1766,8 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(B.getPatchsetCreatedEvent(2)) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -1780,7 +1780,7 @@ class TestScheduler(ZuulTestCase): def test_new_patchset_dequeues_old_independent_queue(self): "Test that a new patchset causes the old to be dequeued (independent)" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -1793,8 +1793,8 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(B.getPatchsetCreatedEvent(2)) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'NEW') @@ -1839,7 +1839,7 @@ class TestScheduler(ZuulTestCase): def test_zuul_refs(self): "Test that zuul refs exist and have the right changes" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True M1 = self.fake_gerrit.addFakeChange('org/project1', 'master', 'M1') M1.setMerged() M2 = self.fake_gerrit.addFakeChange('org/project2', 'master', 'M2') @@ -1859,13 +1859,13 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(D.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() a_zref = b_zref = c_zref = d_zref = None @@ -1911,8 +1911,8 @@ class TestScheduler(ZuulTestCase): # should have a and b in 1, c and d in 2 self.assertTrue(d_build.hasChanges(A, B, C, D)) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -1926,15 +1926,15 @@ class TestScheduler(ZuulTestCase): def test_rerun_on_error(self): "Test that if a worker fails to run a job, it is run again" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() self.builds[0].requeue = True - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(self.countJobResults(self.history, None), 1) self.assertEqual(self.countJobResults(self.history, 'SUCCESS'), 3) @@ -2094,7 +2094,7 @@ class TestScheduler(ZuulTestCase): "Test that queue precedence works" self.gearman_server.hold_jobs_in_queue = True - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) A.addApproval('code-review', 2) @@ -2107,7 +2107,7 @@ class TestScheduler(ZuulTestCase): # Run one build at a time to ensure non-race order: self.orderedRelease() - self.launch_server.hold_jobs_in_build = False + self.executor_server.hold_jobs_in_build = False self.waitUntilSettled() self.log.debug(self.history) @@ -2120,13 +2120,13 @@ class TestScheduler(ZuulTestCase): def test_json_status(self): "Test that we can retrieve JSON status info" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('project-merge') + self.executor_server.release('project-merge') self.waitUntilSettled() port = self.webapp.server.socket.getsockname()[1] @@ -2145,8 +2145,8 @@ class TestScheduler(ZuulTestCase): self.assertIn('Expires', headers) data = f.read() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() data = json.loads(data) @@ -2193,7 +2193,7 @@ class TestScheduler(ZuulTestCase): self.updateConfigLayout('layout-mutex') self.sched.reconfigure(self.config) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') self.assertFalse('test-mutex' in self.sched.mutex.mutexes) @@ -2206,7 +2206,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[1].name, 'mutex-one') self.assertEqual(self.builds[2].name, 'project-test1') - self.launch_server.release('mutex-one') + self.executor_server.release('mutex-one') self.waitUntilSettled() self.assertEqual(len(self.builds), 3) @@ -2215,7 +2215,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[2].name, 'mutex-two') self.assertTrue('test-mutex' in self.sched.mutex.mutexes) - self.launch_server.release('mutex-two') + self.executor_server.release('mutex-two') self.waitUntilSettled() self.assertEqual(len(self.builds), 3) @@ -2224,7 +2224,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[2].name, 'mutex-one') self.assertTrue('test-mutex' in self.sched.mutex.mutexes) - self.launch_server.release('mutex-one') + self.executor_server.release('mutex-one') self.waitUntilSettled() self.assertEqual(len(self.builds), 3) @@ -2233,7 +2233,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[2].name, 'mutex-two') self.assertTrue('test-mutex' in self.sched.mutex.mutexes) - self.launch_server.release('mutex-two') + self.executor_server.release('mutex-two') self.waitUntilSettled() self.assertEqual(len(self.builds), 2) @@ -2241,8 +2241,8 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[1].name, 'project-test1') self.assertFalse('test-mutex' in self.sched.mutex.mutexes) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.builds), 0) @@ -2256,7 +2256,7 @@ class TestScheduler(ZuulTestCase): self.updateConfigLayout('layout-mutex') self.sched.reconfigure(self.config) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True tenant = self.sched.abide.tenants.get('openstack') check_pipeline = tenant.layout.pipelines['check'] @@ -2279,8 +2279,8 @@ class TestScheduler(ZuulTestCase): # The mutex should be released self.assertFalse('test-mutex' in self.sched.mutex.mutexes) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() def test_mutex_reconfigure(self): @@ -2288,7 +2288,7 @@ class TestScheduler(ZuulTestCase): self.updateConfigLayout('layout-mutex') self.sched.reconfigure(self.config) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') self.assertFalse('test-mutex' in self.sched.mutex.mutexes) @@ -2302,7 +2302,7 @@ class TestScheduler(ZuulTestCase): self.sched.reconfigure(self.config) self.waitUntilSettled() - self.launch_server.release('project-test1') + self.executor_server.release('project-test1') self.waitUntilSettled() # There should be no builds anymore @@ -2313,7 +2313,7 @@ class TestScheduler(ZuulTestCase): def test_live_reconfiguration(self): "Test that live reconfiguration works" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) @@ -2322,8 +2322,8 @@ class TestScheduler(ZuulTestCase): self.sched.reconfigure(self.config) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(self.getJobFromHistory('project-merge').result, 'SUCCESS') @@ -2341,7 +2341,7 @@ class TestScheduler(ZuulTestCase): # sitting in the queue. The job gets added to the change and # enqueued and the change gets stuck. self.worker.registerFunction('build:project-test3') - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True # This change is fine. It's here to stop the queue long # enough for the next change to be subject to the @@ -2376,8 +2376,8 @@ class TestScheduler(ZuulTestCase): self.sched.reconfigure(self.config) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -2400,7 +2400,7 @@ class TestScheduler(ZuulTestCase): # that tests a job added to a job tree with a failed root does # not run. self.worker.registerFunction('build:project-test3') - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True # This change is fine. It's here to stop the queue long # enough for the next change to be subject to the @@ -2410,16 +2410,16 @@ class TestScheduler(ZuulTestCase): A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') - self.launch_server.failJob('project-merge', B) + self.executor_server.failJob('project-merge', B) B.addApproval('code-review', 2) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # Both -merge jobs have run, but no others. @@ -2439,8 +2439,8 @@ class TestScheduler(ZuulTestCase): self.sched.reconfigure(self.config) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -2464,18 +2464,18 @@ class TestScheduler(ZuulTestCase): # bug where the code to re-set build statuses would run on # that build and raise an exception because the job no longer # existed. - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') # This change will fail and later be removed by the reconfiguration. - self.launch_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', A) self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('project-test1') + self.executor_server.release('project-test1') self.waitUntilSettled() self.assertEqual(A.data['status'], 'NEW') @@ -2493,8 +2493,8 @@ class TestScheduler(ZuulTestCase): self.sched.reconfigure(self.config) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(self.getJobFromHistory('project-test2').result, @@ -2515,17 +2515,17 @@ class TestScheduler(ZuulTestCase): # this project but otherwise still exists in the system does # not disrupt reconfiguration. - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project1', 'master', 'A') - self.launch_server.failJob('project1-project2-integration', A) + self.executor_server.failJob('project1-project2-integration', A) self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('project1-project2-integration') + self.executor_server.release('project1-project2-integration') self.waitUntilSettled() self.assertEqual(A.data['status'], 'NEW') @@ -2543,8 +2543,8 @@ class TestScheduler(ZuulTestCase): self.sched.reconfigure(self.config) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(self.getJobFromHistory('project1-merge').result, @@ -2572,7 +2572,7 @@ class TestScheduler(ZuulTestCase): # A failure may indicate incorrect caching or cleaning up of # references during a reconfiguration. - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project1', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project1', 'master', 'B') @@ -2583,7 +2583,7 @@ class TestScheduler(ZuulTestCase): # Add the parent change. self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # Reconfigure (with only one change in the pipeline). @@ -2593,15 +2593,15 @@ class TestScheduler(ZuulTestCase): # Add the child change. self.fake_gerrit.addEvent(B.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # Reconfigure (with both in the pipeline). self.sched.reconfigure(self.config) self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.history), 8) @@ -2616,7 +2616,7 @@ class TestScheduler(ZuulTestCase): # Test project deletion from layout # while changes are enqueued - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project1', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project1', 'master', 'C') @@ -2629,7 +2629,7 @@ class TestScheduler(ZuulTestCase): self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) self.fake_gerrit.addEvent(C.getPatchsetCreatedEvent(1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 5) @@ -2645,8 +2645,8 @@ class TestScheduler(ZuulTestCase): self.assertEqual(job_c.changes, '3,1') self.assertEqual(job_c.result, 'ABORTED') - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(self.getJobFromHistory('project-test1').changes, @@ -2736,8 +2736,8 @@ class TestScheduler(ZuulTestCase): p = 'org/delete-project' if os.path.exists(os.path.join(self.merger_src_root, p)): shutil.rmtree(os.path.join(self.merger_src_root, p)) - if os.path.exists(os.path.join(self.launcher_src_root, p)): - shutil.rmtree(os.path.join(self.launcher_src_root, p)) + if os.path.exists(os.path.join(self.executor_src_root, p)): + shutil.rmtree(os.path.join(self.executor_src_root, p)) B = self.fake_gerrit.addFakeChange('org/delete-project', 'master', 'B') @@ -2777,7 +2777,7 @@ class TestScheduler(ZuulTestCase): def test_timer(self): "Test that a periodic job is triggered" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True self.updateConfigLayout('layout-timer') self.sched.reconfigure(self.config) @@ -2795,12 +2795,12 @@ class TestScheduler(ZuulTestCase): f = urllib.request.urlopen(req) data = f.read() - self.launch_server.hold_jobs_in_build = False + self.executor_server.hold_jobs_in_build = False # Stop queuing timer triggered jobs so that the assertions # below don't race against more jobs being queued. self.commitLayoutUpdate('layout-timer', 'layout-no-timer') self.sched.reconfigure(self.config) - self.launch_server.release() + self.executor_server.release() self.waitUntilSettled() self.assertEqual(self.getJobFromHistory( @@ -2822,7 +2822,7 @@ class TestScheduler(ZuulTestCase): def test_idle(self): "Test that frequent periodic jobs work" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True self.updateConfigLayout('layout-idle') for x in range(1, 3): @@ -2843,7 +2843,7 @@ class TestScheduler(ZuulTestCase): self.waitUntilSettled() self.assertEqual(len(self.builds), 2, 'Timer builds iteration #%d' % x) - self.launch_server.release('.*') + self.executor_server.release('.*') self.waitUntilSettled() self.assertEqual(len(self.builds), 0) self.assertEqual(len(self.history), x * 2) @@ -2885,7 +2885,7 @@ class TestScheduler(ZuulTestCase): def test_timer_smtp(self): "Test that a periodic job is triggered" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True self.updateConfigLayout('layout-timer-smtp') self.sched.reconfigure(self.config) @@ -2895,7 +2895,7 @@ class TestScheduler(ZuulTestCase): self.waitUntilSettled() self.assertEqual(len(self.builds), 2) - self.launch_server.release('.*') + self.executor_server.release('.*') self.waitUntilSettled() self.assertEqual(len(self.history), 2) @@ -2922,7 +2922,7 @@ class TestScheduler(ZuulTestCase): self.commitLayoutUpdate('layout-timer-smtp', 'layout-no-timer') self.sched.reconfigure(self.config) self.waitUntilSettled() - self.launch_server.release('.*') + self.executor_server.release('.*') self.waitUntilSettled() @skip("Disabled for early v3 development") @@ -3072,7 +3072,7 @@ class TestScheduler(ZuulTestCase): def test_client_promote(self): "Test that the RPC client can promote a change" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -3105,11 +3105,11 @@ class TestScheduler(ZuulTestCase): enqueue_times[str(item.change)], item.enqueue_time) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 6) @@ -3132,7 +3132,7 @@ class TestScheduler(ZuulTestCase): self.assertTrue(self.builds[4].hasChanges(C)) self.assertTrue(self.builds[4].hasChanges(A)) - self.launch_server.release() + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -3149,7 +3149,7 @@ class TestScheduler(ZuulTestCase): "Test that the RPC client can promote a dependent change" # C (depends on B) -> B -> A ; then promote C to get: # A -> C (depends on B) -> B - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') @@ -3173,11 +3173,11 @@ class TestScheduler(ZuulTestCase): change_ids=['3,1']) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 6) @@ -3200,7 +3200,7 @@ class TestScheduler(ZuulTestCase): self.assertTrue(self.builds[4].hasChanges(C)) self.assertTrue(self.builds[4].hasChanges(A)) - self.launch_server.release() + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -3215,7 +3215,7 @@ class TestScheduler(ZuulTestCase): def test_client_promote_negative(self): "Test that the RPC client returns errors for promotion" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) @@ -3238,8 +3238,8 @@ class TestScheduler(ZuulTestCase): client.shutdown() self.assertEqual(r, False) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() @skip("Disabled for early v3 development") @@ -3248,13 +3248,13 @@ class TestScheduler(ZuulTestCase): self.updateConfigLayout( 'tests/fixtures/layout-rate-limit.yaml') self.sched.reconfigure(self.config) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') C.setDependsOn(B, 1) - self.launch_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', A) A.addApproval('code-review', 2) B.addApproval('code-review', 2) @@ -3271,9 +3271,9 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[0].name, 'project-merge') self.assertEqual(self.builds[1].name, 'project-merge') - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # Only A and B will have their test jobs queued because @@ -3284,7 +3284,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[2].name, 'project-test1') self.assertEqual(self.builds[3].name, 'project-test2') - self.launch_server.release('project-.*') + self.executor_server.release('project-.*') self.waitUntilSettled() queue = self.sched.layout.pipelines['gate'].queues[0] @@ -3298,7 +3298,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(len(self.builds), 1) self.assertEqual(self.builds[0].name, 'project-merge') - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # Only B's test jobs are queued because window is still 1. @@ -3306,7 +3306,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[0].name, 'project-test1') self.assertEqual(self.builds[1].name, 'project-test2') - self.launch_server.release('project-.*') + self.executor_server.release('project-.*') self.waitUntilSettled() # B was successfully merged so window is increased to 2. @@ -3318,7 +3318,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(len(self.builds), 1) self.assertEqual(self.builds[0].name, 'project-merge') - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # After successful merge job the test jobs for C are queued. @@ -3326,7 +3326,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[0].name, 'project-test1') self.assertEqual(self.builds[1].name, 'project-test2') - self.launch_server.release('project-.*') + self.executor_server.release('project-.*') self.waitUntilSettled() # C successfully merged so window is bumped to 3. @@ -3340,14 +3340,14 @@ class TestScheduler(ZuulTestCase): self.updateConfigLayout( 'tests/fixtures/layout-rate-limit.yaml') self.sched.reconfigure(self.config) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B') C = self.fake_gerrit.addFakeChange('org/project', 'master', 'C') B.setDependsOn(A, 1) - self.launch_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', A) A.addApproval('code-review', 2) B.addApproval('code-review', 2) @@ -3364,9 +3364,9 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[0].name, 'project-merge') self.assertEqual(self.builds[1].name, 'project-merge') - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # Only A and B will have their test jobs queued because @@ -3377,7 +3377,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[2].name, 'project-test1') self.assertEqual(self.builds[3].name, 'project-test2') - self.launch_server.release('project-.*') + self.executor_server.release('project-.*') self.waitUntilSettled() queue = self.sched.layout.pipelines['gate'].queues[0] @@ -3392,7 +3392,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(len(self.builds), 1) self.assertEqual(self.builds[0].name, 'project-merge') - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() # Only C's test jobs are queued because window is still 1. @@ -3400,7 +3400,7 @@ class TestScheduler(ZuulTestCase): self.assertEqual(self.builds[0].name, 'project-test1') self.assertEqual(self.builds[1].name, 'project-test2') - self.launch_server.release('project-.*') + self.executor_server.release('project-.*') self.waitUntilSettled() # C was successfully merged so window is increased to 2. @@ -3411,24 +3411,24 @@ class TestScheduler(ZuulTestCase): @skip("Disabled for early v3 development") def test_worker_update_metadata(self): "Test if a worker can send back metadata about itself" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() - self.assertEqual(len(self.launcher.builds), 1) + self.assertEqual(len(self.executor.builds), 1) self.log.debug('Current builds:') - self.log.debug(self.launcher.builds) + self.log.debug(self.executor.builds) start = time.time() while True: if time.time() - start > 10: raise Exception("Timeout waiting for gearman server to report " + "back to the client") - build = self.launcher.builds.values()[0] + build = self.executor.builds.values()[0] if build.worker.name == "My Worker": break else: @@ -3443,8 +3443,8 @@ class TestScheduler(ZuulTestCase): self.assertEqual("v1.1", build.worker.version) self.assertEqual({'something': 'else'}, build.worker.extra) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() def test_footer_message(self): @@ -3454,7 +3454,7 @@ class TestScheduler(ZuulTestCase): A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) - self.launch_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', A) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() @@ -3535,7 +3535,7 @@ For CI problems and help debugging, contact ci@example.org""" # Check a test failure isn't reported to SMTP A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) - self.launch_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', A) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() @@ -3587,7 +3587,7 @@ For CI problems and help debugging, contact ci@example.org""" def test_client_get_running_jobs(self): "Test that the RPC client can get a list of running jobs" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) @@ -3602,7 +3602,7 @@ For CI problems and help debugging, contact ci@example.org""" if time.time() - start > 10: raise Exception("Timeout waiting for gearman server to report " + "back to the client") - build = self.launch_client.builds.values()[0] + build = self.executor_client.builds.values()[0] if build.worker.name == "My Worker": break else: @@ -3636,8 +3636,8 @@ For CI problems and help debugging, contact ci@example.org""" self.assertEqual('gate', job['pipeline']) break - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() running_items = client.get_running_jobs() @@ -3700,17 +3700,17 @@ For CI problems and help debugging, contact ci@example.org""" for connection in self.connections.connections.values(): connection.maintainCache([]) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True B.addApproval('approved', 1) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(AM2.queried, 0) @@ -3740,20 +3740,20 @@ For CI problems and help debugging, contact ci@example.org""" A.data['commitMessage'] = '%s\n\nDepends-On: %s\n' % ( A.subject, B.data['id']) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True B.addApproval('approved', 1) C.addApproval('approved', 1) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -3780,20 +3780,20 @@ For CI problems and help debugging, contact ci@example.org""" A.data['commitMessage'] = '%s\n\nDepends-On: %s\nDepends-On: %s\n' % ( A.subject, B.data['id'], C.data['id']) - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True B.addApproval('approved', 1) C.addApproval('approved', 1) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -3863,17 +3863,17 @@ For CI problems and help debugging, contact ci@example.org""" self.assertEqual(A.data['status'], 'NEW') self.assertEqual(B.data['status'], 'NEW') - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A.addApproval('approved', 1) self.fake_gerrit.addEvent(B.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'MERGED') @@ -3952,7 +3952,7 @@ For CI problems and help debugging, contact ci@example.org""" def test_crd_check(self): "Test cross-repo dependencies in independent pipelines" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True self.gearman_server.hold_jobs_in_queue = True A = self.fake_gerrit.addFakeChange('org/project1', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project2', 'master', 'B') @@ -3970,7 +3970,7 @@ For CI problems and help debugging, contact ci@example.org""" self.gearman_server.release() self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() path = os.path.join(self.builds[0].jobdir.src_root, "org/project1") @@ -3989,8 +3989,8 @@ For CI problems and help debugging, contact ci@example.org""" 'initial commit', 'add content from fixture', 'B-1'] self.assertEqual(repo_messages, correct_messages) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'NEW') @@ -4035,7 +4035,7 @@ For CI problems and help debugging, contact ci@example.org""" def test_crd_check_duplicate(self): "Test duplicate check in independent pipelines" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project1', 'master', 'A') B = self.fake_gerrit.addFakeChange('org/project1', 'master', 'B') tenant = self.sched.abide.tenants.get('tenant-one') @@ -4060,8 +4060,8 @@ For CI problems and help debugging, contact ci@example.org""" # Release jobs in order to avoid races with change A jobs # finishing before change B jobs. self.orderedRelease() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(A.data['status'], 'NEW') @@ -4305,17 +4305,17 @@ For CI problems and help debugging, contact ci@example.org""" J = self.fake_gerrit.addFakeChange('org/project', 'master', 'J') K = self.fake_gerrit.addFakeChange('org/project', 'master', 'K') - self.launch_server.failJob('project-test1', A) - self.launch_server.failJob('project-test1', B) + self.executor_server.failJob('project-test1', A) + self.executor_server.failJob('project-test1', B) # Let C pass, resetting the counter - self.launch_server.failJob('project-test1', D) - self.launch_server.failJob('project-test1', E) - self.launch_server.failJob('project-test1', F) - self.launch_server.failJob('project-test1', G) - self.launch_server.failJob('project-test1', H) + self.executor_server.failJob('project-test1', D) + self.executor_server.failJob('project-test1', E) + self.executor_server.failJob('project-test1', F) + self.executor_server.failJob('project-test1', G) + self.executor_server.failJob('project-test1', H) # I also passes but should only report to the disabled reporters - self.launch_server.failJob('project-test1', J) - self.launch_server.failJob('project-test1', K) + self.executor_server.failJob('project-test1', J) + self.executor_server.failJob('project-test1', K) self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) self.fake_gerrit.addEvent(B.getPatchsetCreatedEvent(1)) @@ -4409,30 +4409,30 @@ For CI problems and help debugging, contact ci@example.org""" self.assertEqual(3, len(self.smtp_messages)) def test_rerun_on_abort(self): - "Test that if a launch server fails to run a job, it is run again" + "Test that if a execute server fails to run a job, it is run again" - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() self.assertEqual(len(self.builds), 2) self.builds[0].requeue = True - self.launch_server.release('.*-test*') + self.executor_server.release('.*-test*') self.waitUntilSettled() for x in range(3): self.assertEqual(len(self.builds), 1, 'len of builds at x=%d is wrong' % x) self.builds[0].requeue = True - self.launch_server.release('.*-test1') + self.executor_server.release('.*-test1') self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.history), 6) self.assertEqual(self.countJobResults(self.history, 'SUCCESS'), 2) @@ -4440,7 +4440,7 @@ For CI problems and help debugging, contact ci@example.org""" self.assertIn('RETRY_LIMIT', A.messages[0]) def test_zookeeper_disconnect(self): - "Test that jobs are launched after a zookeeper disconnect" + "Test that jobs are executed after a zookeeper disconnect" self.fake_nodepool.paused = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') @@ -4537,8 +4537,8 @@ class TestSchedulerOneJobProject(ZuulTestCase): class TestSchedulerTemplatedProject(ZuulTestCase): tenant_config_file = 'config/templated-project/main.yaml' - def test_job_from_templates_launched(self): - "Test whether a job generated via a template can be launched" + def test_job_from_templates_executed(self): + "Test whether a job generated via a template can be executed" A = self.fake_gerrit.addFakeChange( 'org/templated-project', 'master', 'A') @@ -4551,7 +4551,7 @@ class TestSchedulerTemplatedProject(ZuulTestCase): 'SUCCESS') def test_layered_templates(self): - "Test whether a job generated via a template can be launched" + "Test whether a job generated via a template can be executed" A = self.fake_gerrit.addFakeChange( 'org/layered-project', 'master', 'A') @@ -4603,7 +4603,7 @@ class TestSchedulerSuccessURL(ZuulTestCase): 'docs-draft-test/%s/publish-docs/' % uuid, body[2]) - # NOTE: This default URL is currently hard-coded in launcher/server.py + # NOTE: This default URL is currently hard-coded in executor/server.py self.assertIn( '- docs-draft-test2 https://server/job', body[3]) @@ -4613,9 +4613,9 @@ class TestSchedulerMerges(ZuulTestCase): tenant_config_file = 'config/merges/main.yaml' def _test_project_merge_mode(self, mode): - self.launch_server.keep_jobdir = False + self.executor_server.keep_jobdir = False project = 'org/project-%s' % mode - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange(project, 'master', 'A') B = self.fake_gerrit.addFakeChange(project, 'master', 'B') C = self.fake_gerrit.addFakeChange(project, 'master', 'C') @@ -4635,8 +4635,8 @@ class TestSchedulerMerges(ZuulTestCase): repo_messages = [c.message.strip() for c in repo.iter_commits(ref)] repo_messages.reverse() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() return repo_messages @@ -4675,7 +4675,7 @@ class TestSchedulerMerges(ZuulTestCase): "Test that the right commits are on alternate branches" self.create_branch('org/project-merge-branches', 'mp') - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange( 'org/project-merge-branches', 'mp', 'A') B = self.fake_gerrit.addFakeChange( @@ -4690,11 +4690,11 @@ class TestSchedulerMerges(ZuulTestCase): self.fake_gerrit.addEvent(C.addApproval('approved', 1)) self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() build = self.builds[-1] @@ -4713,15 +4713,15 @@ class TestSchedulerMerges(ZuulTestCase): 'A-1', 'B-1', 'C-1'] self.assertEqual(repo_messages, correct_messages) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() def test_merge_multi_branch(self): "Test that dependent changes on multiple branches are merged" self.create_branch('org/project-merge-branches', 'mp') - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange( 'org/project-merge-branches', 'master', 'A') B = self.fake_gerrit.addFakeChange( @@ -4755,7 +4755,7 @@ class TestSchedulerMerges(ZuulTestCase): 'initial commit', 'add content from fixture', 'A-1'] self.assertEqual(repo_messages, correct_messages) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() job_B = None @@ -4777,7 +4777,7 @@ class TestSchedulerMerges(ZuulTestCase): 'initial commit', 'add content from fixture', 'mp commit', 'B-1'] self.assertEqual(repo_messages, correct_messages) - self.launch_server.release('.*-merge') + self.executor_server.release('.*-merge') self.waitUntilSettled() job_C = None @@ -4801,6 +4801,6 @@ class TestSchedulerMerges(ZuulTestCase): # Ensure the right commits are in the history for this ref self.assertEqual(repo_messages, correct_messages) - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() diff --git a/tests/unit/test_webapp.py b/tests/unit/test_webapp.py index 2211d1be4..acff09a9b 100644 --- a/tests/unit/test_webapp.py +++ b/tests/unit/test_webapp.py @@ -27,7 +27,7 @@ class TestWebapp(ZuulTestCase): def setUp(self): super(TestWebapp, self).setUp() - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A.addApproval('code-review', 2) self.fake_gerrit.addEvent(A.addApproval('approved', 1)) @@ -38,8 +38,8 @@ class TestWebapp(ZuulTestCase): self.port = self.webapp.server.socket.getsockname()[1] def tearDown(self): - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() super(TestWebapp, self).tearDown() diff --git a/tests/unit/test_zuultrigger.py b/tests/unit/test_zuultrigger.py index 5d9c6e073..476cb742a 100644 --- a/tests/unit/test_zuultrigger.py +++ b/tests/unit/test_zuultrigger.py @@ -27,7 +27,7 @@ class TestZuulTriggerParentChangeEnqueued(ZuulTestCase): # When A is enqueued in the gate, B1 and B2 should both attempt # to be enqueued in both pipelines. B1 should end up in check # and B2 in gate because of differing pipeline requirements. - self.launch_server.hold_jobs_in_build = True + self.executor_server.hold_jobs_in_build = True A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') B1 = self.fake_gerrit.addFakeChange('org/project', 'master', 'B1') B2 = self.fake_gerrit.addFakeChange('org/project', 'master', 'B2') @@ -46,8 +46,8 @@ class TestZuulTriggerParentChangeEnqueued(ZuulTestCase): # to enqueue behind 1,1 so that the test is more # deterministic. self.waitUntilSettled() - self.launch_server.hold_jobs_in_build = False - self.launch_server.release() + self.executor_server.hold_jobs_in_build = False + self.executor_server.release() self.waitUntilSettled() self.assertEqual(len(self.history), 3) diff --git a/zuul/cmd/client.py b/zuul/cmd/client.py index cc8edaa28..487096f84 100644 --- a/zuul/cmd/client.py +++ b/zuul/cmd/client.py @@ -239,8 +239,8 @@ class Client(zuul.cmd.ZuulApp): 'uuid': { 'title': 'UUID' }, - 'launch_time': { - 'title': 'Launch Time', + 'execute_time': { + 'title': 'Execute Time', 'transform': self._epoch_to_relative_time, 'append': ' ago' }, diff --git a/zuul/cmd/launcher.py b/zuul/cmd/executor.py index 596fd1ac7..13d24b4be 100644 --- a/zuul/cmd/launcher.py +++ b/zuul/cmd/executor.py @@ -29,7 +29,7 @@ import sys import signal import zuul.cmd -import zuul.launcher.server +import zuul.executor.server # No zuul imports that pull in paramiko here; it must not be # imported until after the daemonization. @@ -37,10 +37,10 @@ import zuul.launcher.server # Similar situation with gear and statsd. -class Launcher(zuul.cmd.ZuulApp): +class Executor(zuul.cmd.ZuulApp): def parse_arguments(self): - parser = argparse.ArgumentParser(description='Zuul launch worker.') + parser = argparse.ArgumentParser(description='Zuul executor.') parser.add_argument('-c', dest='config', help='specify the config file') parser.add_argument('-d', dest='nodaemon', action='store_true', @@ -52,7 +52,7 @@ class Launcher(zuul.cmd.ZuulApp): action='store_true', help='keep local jobdirs after run completes') parser.add_argument('command', - choices=zuul.launcher.server.COMMANDS, + choices=zuul.executor.server.COMMANDS, nargs='?') self.args = parser.parse_args() @@ -63,55 +63,55 @@ class Launcher(zuul.cmd.ZuulApp): self.config.get('zuul', 'state_dir')) else: state_dir = '/var/lib/zuul' - path = os.path.join(state_dir, 'launcher.socket') + path = os.path.join(state_dir, 'executor.socket') s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) s.connect(path) s.sendall('%s\n' % cmd) def exit_handler(self): - self.launcher.stop() - self.launcher.join() + self.executor.stop() + self.executor.join() def main(self, daemon=True): # See comment at top of file about zuul imports - self.setup_logging('launcher', 'log_config') + self.setup_logging('executor', 'log_config') - self.log = logging.getLogger("zuul.Launcher") + self.log = logging.getLogger("zuul.Executor") - LaunchServer = zuul.launcher.server.LaunchServer - self.launcher = LaunchServer(self.config, self.connections, + LaunchServer = zuul.executor.server.LaunchServer + self.executor = LaunchServer(self.config, self.connections, keep_jobdir=self.args.keep_jobdir) - self.launcher.start() + self.executor.start() signal.signal(signal.SIGUSR2, zuul.cmd.stack_dump_handler) if daemon: - self.launcher.join() + self.executor.join() else: while True: try: signal.pause() except KeyboardInterrupt: - print("Ctrl + C: asking launcher to exit nicely...\n") + print("Ctrl + C: asking executor to exit nicely...\n") self.exit_handler() sys.exit(0) def main(): - server = Launcher() + server = Executor() server.parse_arguments() server.read_config() - if server.args.command in zuul.launcher.server.COMMANDS: + if server.args.command in zuul.executor.server.COMMANDS: server.send_command(server.args.command) sys.exit(0) server.configure_connections() - if server.config.has_option('launcher', 'pidfile'): - pid_fn = os.path.expanduser(server.config.get('launcher', 'pidfile')) + if server.config.has_option('executor', 'pidfile'): + pid_fn = os.path.expanduser(server.config.get('executor', 'pidfile')) else: - pid_fn = '/var/run/zuul-launcher/zuul-launcher.pid' + pid_fn = '/var/run/zuul-executor/zuul-executor.pid' pid = pid_file_module.TimeoutPIDLockFile(pid_fn, 10) if server.args.nodaemon: diff --git a/zuul/cmd/scheduler.py b/zuul/cmd/scheduler.py index 8b8fb57fc..889ad2b22 100755 --- a/zuul/cmd/scheduler.py +++ b/zuul/cmd/scheduler.py @@ -78,7 +78,7 @@ class Scheduler(zuul.cmd.ZuulApp): def test_config(self): # See comment at top of file about zuul imports import zuul.scheduler - import zuul.launcher.client + import zuul.executor.client logging.basicConfig(level=logging.DEBUG) try: @@ -124,7 +124,7 @@ class Scheduler(zuul.cmd.ZuulApp): def main(self): # See comment at top of file about zuul imports import zuul.scheduler - import zuul.launcher.client + import zuul.executor.client import zuul.merger.client import zuul.nodepool import zuul.webapp @@ -141,7 +141,7 @@ class Scheduler(zuul.cmd.ZuulApp): self.sched = zuul.scheduler.Scheduler(self.config) - gearman = zuul.launcher.client.LaunchClient(self.config, self.sched) + gearman = zuul.executor.client.LaunchClient(self.config, self.sched) merger = zuul.merger.client.MergeClient(self.config, self.sched) nodepool = zuul.nodepool.Nodepool(self.sched) @@ -174,7 +174,7 @@ class Scheduler(zuul.cmd.ZuulApp): rpc = zuul.rpclistener.RPCListener(self.config, self.sched) self.configure_connections() - self.sched.setLauncher(gearman) + self.sched.setExecutor(gearman) self.sched.setMerger(merger) self.sched.setNodepool(nodepool) self.sched.setZooKeeper(zookeeper) diff --git a/zuul/configloader.py b/zuul/configloader.py index b788237ac..3f722125b 100644 --- a/zuul/configloader.py +++ b/zuul/configloader.py @@ -782,7 +782,7 @@ class TenantParser(object): for job in jobs: # Note: this is an ordered list -- we wait for cat jobs to - # complete in the order they were launched which is the + # complete in the order they were executed which is the # same order they were defined in the main config file. # This is important for correct inheritance. TenantParser.log.debug("Waiting for cat job %s" % (job,)) diff --git a/zuul/launcher/__init__.py b/zuul/executor/__init__.py index e69de29bb..e69de29bb 100644 --- a/zuul/launcher/__init__.py +++ b/zuul/executor/__init__.py diff --git a/zuul/launcher/ansiblelaunchserver.py b/zuul/executor/ansiblelaunchserver.py index 875cf2be2..875cf2be2 100644 --- a/zuul/launcher/ansiblelaunchserver.py +++ b/zuul/executor/ansiblelaunchserver.py diff --git a/zuul/launcher/client.py b/zuul/executor/client.py index 52e439774..aa32517a0 100644 --- a/zuul/launcher/client.py +++ b/zuul/executor/client.py @@ -102,7 +102,7 @@ def getJobData(job): class ZuulGearmanClient(gear.Client): def __init__(self, zuul_gearman): - super(ZuulGearmanClient, self).__init__('Zuul Launch Client') + super(ZuulGearmanClient, self).__init__('Zuul Executor Client') self.__zuul_gearman = zuul_gearman def handleWorkComplete(self, packet): @@ -144,8 +144,8 @@ class ZuulGearmanClient(gear.Client): self.__zuul_gearman.onUnknownJob(job) -class LaunchClient(object): - log = logging.getLogger("zuul.LaunchClient") +class ExecutorClient(object): + log = logging.getLogger("zuul.ExecutorClient") negative_function_cache_ttl = 5 def __init__(self, config, sched): @@ -209,10 +209,10 @@ class LaunchClient(object): self.log.debug("Function %s is not registered" % name) return False - def launch(self, job, item, pipeline, dependent_items=[]): + def execute(self, job, item, pipeline, dependent_items=[]): uuid = str(uuid4().hex) self.log.info( - "Launch job %s (uuid: %s) on nodes %s for change %s " + "Execute job %s (uuid: %s) on nodes %s for change %s " "with dependent changes %s" % ( job, uuid, item.current_build_set.getJobNodeSet(job.name), @@ -339,7 +339,7 @@ class LaunchClient(object): self.sched.onBuildCompleted(build, 'SUCCESS') return build - gearman_job = gear.Job('launcher:launch', json.dumps(params), + gearman_job = gear.Job('executor:execute', json.dumps(params), unique=uuid) build.__gearman_job = gearman_job build.__gearman_manager = None @@ -433,7 +433,7 @@ class LaunchClient(object): # internal dict after it's added to the report queue. del self.builds[job.unique] else: - if not job.name.startswith("launcher:stop:"): + if not job.name.startswith("executor:stop:"): self.log.error("Unable to find build %s" % job.unique) def onWorkStatus(self, job): @@ -483,7 +483,7 @@ class LaunchClient(object): (build,)) stop_uuid = str(uuid4().hex) data = dict(uuid=build.__gearman_job.unique) - stop_job = gear.Job("launcher:stop:%s" % build.__gearman_manager, + stop_job = gear.Job("executor:stop:%s" % build.__gearman_manager, json.dumps(data), unique=stop_uuid) self.meta_jobs[stop_uuid] = stop_job self.log.debug("Submitting stop job: %s", stop_job) diff --git a/zuul/launcher/server.py b/zuul/executor/server.py index 1b8d2c63e..3995b1a5a 100644 --- a/zuul/launcher/server.py +++ b/zuul/executor/server.py @@ -211,15 +211,15 @@ class DeduplicateQueue(object): self.condition.release() -class LaunchServer(object): - log = logging.getLogger("zuul.LaunchServer") +class ExecutorServer(object): + log = logging.getLogger("zuul.ExecutorServer") def __init__(self, config, connections={}, jobdir_root=None, keep_jobdir=False): self.config = config self.keep_jobdir = keep_jobdir self.jobdir_root = jobdir_root - # TODOv3(mordred): make the launcher name more unique -- + # TODOv3(mordred): make the executor name more unique -- # perhaps hostname+pid. self.hostname = socket.gethostname() self.zuul_url = config.get('merger', 'zuul_url') @@ -232,10 +232,10 @@ class LaunchServer(object): unverbose=self.verboseOff, ) - if self.config.has_option('launcher', 'git_dir'): - self.merge_root = self.config.get('launcher', 'git_dir') + if self.config.has_option('executor', 'git_dir'): + self.merge_root = self.config.get('executor', 'git_dir') else: - self.merge_root = '/var/lib/zuul/launcher-git' + self.merge_root = '/var/lib/zuul/executor-git' if self.config.has_option('merger', 'git_user_email'): self.merge_email = self.config.get('merger', 'git_user_email') @@ -260,7 +260,7 @@ class LaunchServer(object): self.config.get('zuul', 'state_dir')) else: state_dir = '/var/lib/zuul' - path = os.path.join(state_dir, 'launcher.socket') + path = os.path.join(state_dir, 'executor.socket') self.command_socket = commandsocket.CommandSocket(path) ansible_dir = os.path.join(state_dir, 'ansible') self.library_dir = os.path.join(ansible_dir, 'library') @@ -303,7 +303,7 @@ class LaunchServer(object): port = self.config.get('gearman', 'port') else: port = 4730 - self.worker = gear.Worker('Zuul Launch Server') + self.worker = gear.Worker('Zuul Executor Server') self.worker.addServer(server, port) self.log.debug("Waiting for server") self.worker.waitForServer() @@ -325,8 +325,8 @@ class LaunchServer(object): self.thread.start() def register(self): - self.worker.registerFunction("launcher:launch") - self.worker.registerFunction("launcher:stop:%s" % self.hostname) + self.worker.registerFunction("executor:execute") + self.worker.registerFunction("executor:stop:%s" % self.hostname) self.worker.registerFunction("merger:merge") self.worker.registerFunction("merger:cat") @@ -398,15 +398,15 @@ class LaunchServer(object): return task def run(self): - self.log.debug("Starting launch listener") + self.log.debug("Starting executor listener") while self._running: try: job = self.worker.getJob() try: - if job.name == 'launcher:launch': - self.log.debug("Got launch job: %s" % job.unique) - self.launchJob(job) - elif job.name.startswith('launcher:stop'): + if job.name == 'executor:execute': + self.log.debug("Got execute job: %s" % job.unique) + self.executeJob(job) + elif job.name.startswith('executor:stop'): self.log.debug("Got stop job: %s" % job.unique) self.stopJob(job) elif job.name == 'merger:cat': @@ -426,7 +426,7 @@ class LaunchServer(object): except Exception: self.log.exception("Exception while getting job") - def launchJob(self, job): + def executeJob(self, job): self.job_workers[job.unique] = AnsibleJob(self, job) self.job_workers[job.unique].run() @@ -481,8 +481,8 @@ class AnsibleJob(object): RESULT_UNREACHABLE = 3 RESULT_ABORTED = 4 - def __init__(self, launcher_server, job): - self.launcher_server = launcher_server + def __init__(self, executor_server, job): + self.executor_server = executor_server self.job = job self.jobdir = None self.proc = None @@ -490,16 +490,16 @@ class AnsibleJob(object): self.running = False self.aborted = False - if self.launcher_server.config.has_option( - 'launcher', 'private_key_file'): - self.private_key_file = self.launcher_server.config.get( - 'launcher', 'private_key_file') + if self.executor_server.config.has_option( + 'executor', 'private_key_file'): + self.private_key_file = self.executor_server.config.get( + 'executor', 'private_key_file') else: self.private_key_file = '~/.ssh/id_rsa' def run(self): self.running = True - self.thread = threading.Thread(target=self.launch) + self.thread = threading.Thread(target=self.execute) self.thread.start() def stop(self): @@ -507,13 +507,13 @@ class AnsibleJob(object): self.abortRunningProc() self.thread.join() - def launch(self): + def execute(self): try: - self.jobdir = JobDir(root=self.launcher_server.jobdir_root, - keep=self.launcher_server.keep_jobdir) - self._launch() + self.jobdir = JobDir(root=self.executor_server.jobdir_root, + keep=self.executor_server.keep_jobdir) + self._execute() except Exception: - self.log.exception("Exception while launching job") + self.log.exception("Exception while executing job") self.job.sendWorkException(traceback.format_exc()) finally: self.running = False @@ -522,11 +522,11 @@ class AnsibleJob(object): except Exception: self.log.exception("Error cleaning up jobdir:") try: - self.launcher_server.finishJob(self.job.unique) + self.executor_server.finishJob(self.job.unique) except Exception: self.log.exception("Error finalizing job thread:") - def _launch(self): + def _execute(self): self.log.debug("Job %s: beginning" % (self.job.unique,)) self.log.debug("Job %s: args: %s" % (self.job.unique, self.job.arguments,)) @@ -537,7 +537,7 @@ class AnsibleJob(object): for project in args['projects']: self.log.debug("Job %s: updating project %s" % (self.job.unique, project['name'])) - tasks.append(self.launcher_server.update( + tasks.append(self.executor_server.update( project['name'], project['url'])) for task in tasks: task.wait() @@ -546,14 +546,14 @@ class AnsibleJob(object): for project in args['projects']: self.log.debug("Cloning %s" % (project['name'],)) repo = git.Repo.clone_from( - os.path.join(self.launcher_server.merge_root, + os.path.join(self.executor_server.merge_root, project['name']), os.path.join(self.jobdir.src_root, project['name'])) repo.remotes.origin.config_writer.set('url', project['url']) # Get a merger in order to update the repos involved in this job. - merger = self.launcher_server._getMerger(self.jobdir.src_root) + merger = self.executor_server._getMerger(self.jobdir.src_root) merge_items = [i for i in args['items'] if i.get('refspec')] if merge_items: commit = merger.mergeChanges(merge_items) # noqa @@ -569,14 +569,14 @@ class AnsibleJob(object): self.prepareAnsibleFiles(args) data = { - 'manager': self.launcher_server.hostname, + 'manager': self.executor_server.hostname, 'url': 'https://server/job/{}/0/'.format(args['job']), 'worker_name': 'My Worker', } # TODOv3: # 'name': self.name, - # 'manager': self.launch_server.hostname, + # 'manager': self.executor_server.hostname, # 'worker_name': 'My Worker', # 'worker_hostname': 'localhost', # 'worker_ips': ['127.0.0.1', '192.168.1.1'], @@ -696,7 +696,7 @@ class AnsibleJob(object): # Check out the playbook repo if needed and set the path to # the playbook that should be run. jobdir_playbook.trusted = playbook['trusted'] - source = self.launcher_server.connections.getSource( + source = self.executor_server.connections.getSource( playbook['connection']) project = source.getProject(playbook['project']) # TODO(jeblair): construct the url in the merger itself @@ -721,7 +721,7 @@ class AnsibleJob(object): # the stack of changes we are testing, so check out the branch # tip into a dedicated space. - merger = self.launcher_server._getMerger(jobdir_playbook.root) + merger = self.executor_server._getMerger(jobdir_playbook.root) merger.checkoutBranch(project.name, url, playbook['branch']) path = os.path.join(jobdir_playbook.root, @@ -762,7 +762,7 @@ class AnsibleJob(object): def prepareZuulRole(self, args, role, root): self.log.debug("Prepare zuul role for %s" % (role,)) # Check out the role repo if needed - source = self.launcher_server.connections.getSource( + source = self.executor_server.connections.getSource( role['connection']) project = source.getProject(role['project']) # TODO(jeblair): construct the url in the merger itself @@ -791,7 +791,7 @@ class AnsibleJob(object): # tip into a dedicated space. if not role_repo: - merger = self.launcher_server._getMerger(root) + merger = self.executor_server._getMerger(root) merger.checkoutBranch(project.name, url, 'master') role_repo = os.path.join(root, project.name) @@ -816,7 +816,7 @@ class AnsibleJob(object): with open(self.jobdir.vars, 'w') as vars_yaml: zuul_vars = dict(args['vars']) - zuul_vars['zuul']['launcher'] = dict(src_root=self.jobdir.src_root, + zuul_vars['zuul']['executor'] = dict(src_root=self.jobdir.src_root, log_root=self.jobdir.log_root) vars_yaml.write( yaml.safe_dump(zuul_vars, default_flow_style=False)) @@ -836,19 +836,19 @@ class AnsibleJob(object): config.write('log_path = %s\n' % self.jobdir.ansible_log) config.write('gathering = explicit\n') config.write('library = %s\n' - % self.launcher_server.library_dir) + % self.executor_server.library_dir) if self.jobdir.roles_path: config.write('roles_path = %s\n' % ':'.join(self.jobdir.roles_path)) config.write('callback_plugins = %s\n' - % self.launcher_server.callback_dir) + % self.executor_server.callback_dir) config.write('stdout_callback = zuul_stream\n') # bump the timeout because busy nodes may take more than # 10s to respond config.write('timeout = 30\n') if not trusted: config.write('action_plugins = %s\n' - % self.launcher_server.action_dir) + % self.executor_server.action_dir) # On trusted jobs, we want to prevent the printing of args, # since trusted jobs might have access to secrets that they may diff --git a/zuul/manager/__init__.py b/zuul/manager/__init__.py index 85b972355..e7508b688 100644 --- a/zuul/manager/__init__.py +++ b/zuul/manager/__init__.py @@ -365,33 +365,33 @@ class PipelineManager(object): build_set.setJobNodeRequest(job.name, req) return True - def _launchJobs(self, item, jobs): - self.log.debug("Launching jobs for change %s" % item.change) + def _executeJobs(self, item, jobs): + self.log.debug("Executing jobs for change %s" % item.change) dependent_items = self.getDependentItems(item) for job in jobs: self.log.debug("Found job %s for change %s" % (job, item.change)) try: nodeset = item.current_build_set.getJobNodeSet(job.name) self.sched.nodepool.useNodeSet(nodeset) - build = self.sched.launcher.launch(job, item, - self.pipeline, - dependent_items) + build = self.sched.executor.execute(job, item, + self.pipeline, + dependent_items) self.log.debug("Adding build %s of job %s to item %s" % (build, job, item)) item.addBuild(build) except: - self.log.exception("Exception while launching job %s " + self.log.exception("Exception while executing job %s " "for change %s:" % (job, item.change)) - def launchJobs(self, item): + def executeJobs(self, item): # TODO(jeblair): This should return a value indicating a job - # was launched. Appears to be a longstanding bug. + # was executed. Appears to be a longstanding bug. if not item.current_build_set.layout: return False jobs = item.findJobsToRun(self.sched.mutex) if jobs: - self._launchJobs(item, jobs) + self._executeJobs(item, jobs) def cancelJobs(self, item, prime=True): self.log.debug("Cancel jobs for change %s" % item.change) @@ -409,7 +409,7 @@ class PipelineManager(object): continue was_running = False try: - was_running = self.sched.launcher.cancel(build) + was_running = self.sched.executor.cancel(build) except: self.log.exception("Exception while canceling build %s " "for change %s" % (build, item.change)) @@ -594,7 +594,7 @@ class PipelineManager(object): failing_reasons.append("it has an invalid configuration") if ready and self.provisionNodes(item): changed = True - if actionable and ready and self.launchJobs(item): + if actionable and ready and self.executeJobs(item): changed = True if item.didAnyJobFail(): failing_reasons.append("at least one job failed") diff --git a/zuul/model.py b/zuul/model.py index b4ee72f12..136a8309a 100644 --- a/zuul/model.py +++ b/zuul/model.py @@ -576,7 +576,7 @@ class PlaybookContext(object): self.path == other.path) def toDict(self): - # Render to a dict to use in passing json to the launcher + # Render to a dict to use in passing json to the executor return dict( connection=self.source_context.project.connection_name, project=self.source_context.project.name, @@ -607,7 +607,7 @@ class Role(object): @abc.abstractmethod def toDict(self): - # Render to a dict to use in passing json to the launcher + # Render to a dict to use in passing json to the executor return dict(target_name=self.target_name) @@ -632,7 +632,7 @@ class ZuulRole(Role): self.trusted == other.trusted) def toDict(self): - # Render to a dict to use in passing json to the launcher + # Render to a dict to use in passing json to the executor d = super(ZuulRole, self).toDict() d['type'] = 'zuul' d['connection'] = self.connection_name @@ -918,7 +918,7 @@ class Build(object): self.url = None self.result = None self.build_set = None - self.launch_time = time.time() + self.execute_time = time.time() self.start_time = None self.end_time = None self.estimated_time = None @@ -994,9 +994,9 @@ class BuildSet(object): """A collection of Builds for one specific potential future repository state. - When Zuul launches Builds for a change, it creates a Build to + When Zuul executes Builds for a change, it creates a Build to represent each execution of each job and a BuildSet to keep track - of all the Builds running for that Change. When Zuul re-launches + of all the Builds running for that Change. When Zuul re-executes Builds for a Change with a different configuration, all of the running Builds in the BuildSet for that change are aborted, and a new BuildSet is created to hold the Builds for the Jobs being @@ -1459,7 +1459,7 @@ class QueueItem(object): 'result': result, 'voting': job.voting, 'uuid': build.uuid if build else None, - 'launch_time': build.launch_time if build else None, + 'execute_time': build.execute_time if build else None, 'start_time': build.start_time if build else None, 'end_time': build.end_time if build else None, 'estimated_time': build.estimated_time if build else None, diff --git a/zuul/scheduler.py b/zuul/scheduler.py index 8eab54527..131378e7a 100644 --- a/zuul/scheduler.py +++ b/zuul/scheduler.py @@ -236,7 +236,7 @@ class Scheduler(threading.Thread): The Scheduler is reponsible for recieving events and dispatching them to appropriate components (including pipeline managers, - mergers and launchers). + mergers and executors). It runs a single threaded main loop which processes events received one at a time and takes action as appropriate. Other @@ -264,7 +264,7 @@ class Scheduler(threading.Thread): self._pause = False self._exit = False self._stopped = False - self.launcher = None + self.executor = None self.merger = None self.connections = None self.statsd = extras.try_import('statsd.statsd') @@ -304,8 +304,8 @@ class Scheduler(threading.Thread): def stopConnections(self): self.connections.stop() - def setLauncher(self, launcher): - self.launcher = launcher + def setExecutor(self, executor): + self.executor = executor def setMerger(self, merger): self.merger = merger @@ -355,7 +355,7 @@ class Scheduler(threading.Thread): # interesting. if label == build.node_name: continue - dt = int((build.start_time - build.launch_time) * 1000) + dt = int((build.start_time - build.execute_time) * 1000) key = 'zuul.pipeline.%s.label.%s.wait_time' % ( build.pipeline.name, label) self.statsd.timing(key, dt) @@ -368,7 +368,7 @@ class Scheduler(threading.Thread): key = 'zuul.pipeline.%s.job.%s.wait_time' % ( build.pipeline.name, jobname) - dt = int((build.start_time - build.launch_time) * 1000) + dt = int((build.start_time - build.execute_time) * 1000) self.statsd.timing(key, dt) except: self.log.exception("Exception reporting runtime stats") @@ -572,7 +572,7 @@ class Scheduler(threading.Thread): self.log.warning( "Canceling build %s during reconfiguration" % (build,)) try: - self.launcher.cancel(build) + self.executor.cancel(build) except Exception: self.log.exception( "Exception while canceling build %s " @@ -695,7 +695,7 @@ class Scheduler(threading.Thread): self.process_management_queue() # Give result events priority -- they let us stop builds, - # whereas trigger events cause us to launch builds. + # whereas trigger events cause us to execute builds. while not self.result_event_queue.empty(): self.process_result_queue() |