Set the default python to python3 except for the py27 environment. We
have to set that explicitly to override the new default.

Have py27 and py3* jobs run same commands by removing the commands entry
for py27.

The pep8 rules under python 3 are different, so we need to fix a
couple of issues with blank space. The E721 rule is ignored entirely
because we are not yet using a version of flake8 that supports
ignoring individual rules on specific lines, although the instructions
for that are added so that when we upgrade flake8 to a newer version
we can remove the global rule.

Change-Id: Id236b0632a83a188958130d04f1e9c476c5ace65
Signed-off-by: Doug Hellmann <doug@doughellmann.com>

When a worker hasn't responded to a notification
request for a given amount of time remove it from
being a useable worker that we can match task submissions
to.

Change-Id: I596bccc1c42f83ee79136dd27bc87039154ff7b1

Instead of having the dispatcher target be a tuple
or a single callback, have it be an actual type with
comments as to what the types fields are and how they
are used.

This makes it more obvious as to what those fields are
and how they are used so that it is easier to understand
how the WBE engine and components work.

Change-Id: I5541ccd5a7aa6ae73ed9adceeac2c0524e2a1dc9

Instead of having a nominally useful single-threaded storage
unit that uses a dummy r/w lock and a multi-threaded storage
unit just have the storage unit by default protect itself from
multi-threading calls being used on it (with the appropriate
reader/writer locks being activated to make this work
correctly).

Change-Id: Ib6879edb465156a8e54fd5b4002550d1cec49137

Link the engine/worker 'retry_options' to the proxy which has
linkage to what the defaults are and to where more information
can be gathered about the option (which really gets sent in to
kombu) so that users can more easily understand it.

Also removes showing the docs about the following as its
more of an internal API/class not meant for public consumption:

* Module: taskflow.engines.worker_based.executor

Change-Id: Ib16ee10765ec6b9a0af320bd6818d9649c2485b1

Use the kombu provided ensure() decorator/wrapper along
with sensible default settings to ensure that retries are
attempted when kombu detects recoverable connection or
recoverable channel errors have occurred.

Change-Id: If47f72d02561d0b5d556ac386869a6122c8b871d

To enable a parallel process executor to be used without
having to pass in a futures executor allow for the executor
option to be a string 'processes' (or similar) that will cause
the default parallel process executor to be used automatically.

Also allow for a 'threads' string that ensure a parallel thread
executor is used to match the ability to uses processes.

This also adjusts the WBE engine to have a similar executor fetching
function (which in the WBE case now validates a provided executor to
be of the desired type).

Change-Id: I54a82584c32c697922507b4f6e01ea7b8acc73c6

Since we support various executors (threaded and distributed)
the next best executor when a threaded executor will not perform
and a distributed one requires to much setup is a local process
based one so it would be great to support this where we can.

Things that are currently (likely never) not going to work:

 * Non-pickleable/non-copyable tasks
 * Tasks that return non-pickleable/non-copyable results
 * Tasks that use non-pickleable/non-copyable args/kwargs

Part of blueprint process-executor

Change-Id: I966ae01d390c7217b858db3feb2db949ce5c08d1

To avoid having one set of options coming from `engine_conf`
and another set of options coming from `kwargs` and another set
coming from `engine_conf` if it is a URI just start to shift
toward `engine_conf` being deprecated and `engine` being a string
type only (or a URI with additional query parameters) and having
any additional **kwargs that are provided just get merged into the
final engine options.

This adds a new helper function that handles all these various
options and adds in a keyword argument `engine` that will be shifted
to in a future version (in that future version we can also then
remove the `engine_conf` and just stick to a smaller set of option
mechanisms).

It also adjusts all examples to use this new and more easier to
understand format and adjusts tests, conductor interface to use
this new more easily understandable style of getting an engine.

Change-Id: Ic7617057338e0c63775cf38a24643cff6e454950

To enable longer (or shorter) timeouts for a WBE submitted
request to transition out of the (PENDING, WAITING) states
allow the transition timeout that was previously set to 60
seconds to be provided as a WBE configuration option (it
still defaults to the previously fixed 60 seconds when it
is not provided).

Fixes bug 1356002

Change-Id: Idf384217004a334df03e2fff9150309fdfe08005