delta/openstack/python-openstackclient.git, branch stable/stein opendev.org: openstack/python-openstackclient Merge "Fix openstack server list --deleted --marker option" into stable/stein 2020-08-07T19:41:01+00:00 Zuul zuul@review.opendev.org 2020-08-07T19:41:01+00:00 b8ff69eacac91342d006ce2b5f10bce65c8153d3 






Stop silently ignoring invalid 'server create --hint' options
2020-02-19T13:28:49+00:00

Stephen Finucane
sfinucan@redhat.com

2019-09-27T11:19:29+00:00

38e1274f478219bee8899b6c9eaa09dbfd03bbcd

The '--hint' option for 'server create' expects a key-value pair like so:

  openstack server create --hint group=245e1dfe-2d0e-4139-80a9-fce124948896 ...

However, the command doesn't complain if this isn't the case, meaning
typos like the below aren't indicated to the user:

  openstack server create --hint 245e1dfe-2d0e-4139-80a9-fce124948896

Due to how we'd implemented this here, this ultimately results in us
POSTing the following as part of the body to 'os-servers':

  {
    ...
    "OS-SCH-HNT:scheduler_hints": {
      "245e1dfe-2d0e-4139-80a9-fce124948896": null
    }
    ...
  }

Which is unfortunately allowed and ignored by nova due to the use of
'additionalProperties' in the schema [1]

Do what we do for loads of other options and explicitly fail on invalid
values. This involves adding a new argparse action since none of those
defined in osc-lib work for us. This is included here to ease
backporting of the fix but will be moved to osc-lib in a future patch.

Conflicts:
	openstackclient/tests/unit/compute/v2/test_server.py

NOTE(stephenfin): Conflicts are due to the absence of tests for changes
Ic06d97b29e51828b29d7ac5172645c288e4ada9e ("Compute: Add description
support for server") and If188c3d96fa506dbe62ef256418f2f9bca1520c2 ("Add
host and hypervisor_hostname to create servers"), neither of which we
want to backport.

[1] https://github.com/openstack/nova/blob/19.0.0/nova/api/openstack/compute/schemas/servers.py#L142-L146

Change-Id: I9e96d2978912c8dfeadae4a782c481a17cd7e348
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Story: #2006628
Task: #36840
Related-Bug: #1845322
(cherry picked from commit ea27ebb0f918db9eab2f5751a1b065818faa0e6d)
(cherry picked from commit 576d9ecfd0ac27611ec89af6d34e7378b7a39fb3)





The '--hint' option for 'server create' expects a key-value pair like so:

  openstack server create --hint group=245e1dfe-2d0e-4139-80a9-fce124948896 ...

However, the command doesn't complain if this isn't the case, meaning
typos like the below aren't indicated to the user:

  openstack server create --hint 245e1dfe-2d0e-4139-80a9-fce124948896

Due to how we'd implemented this here, this ultimately results in us
POSTing the following as part of the body to 'os-servers':

  {
    ...
    "OS-SCH-HNT:scheduler_hints": {
      "245e1dfe-2d0e-4139-80a9-fce124948896": null
    }
    ...
  }

Which is unfortunately allowed and ignored by nova due to the use of
'additionalProperties' in the schema [1]

Do what we do for loads of other options and explicitly fail on invalid
values. This involves adding a new argparse action since none of those
defined in osc-lib work for us. This is included here to ease
backporting of the fix but will be moved to osc-lib in a future patch.

Conflicts:
	openstackclient/tests/unit/compute/v2/test_server.py

NOTE(stephenfin): Conflicts are due to the absence of tests for changes
Ic06d97b29e51828b29d7ac5172645c288e4ada9e ("Compute: Add description
support for server") and If188c3d96fa506dbe62ef256418f2f9bca1520c2 ("Add
host and hypervisor_hostname to create servers"), neither of which we
want to backport.

[1] https://github.com/openstack/nova/blob/19.0.0/nova/api/openstack/compute/schemas/servers.py#L142-L146

Change-Id: I9e96d2978912c8dfeadae4a782c481a17cd7e348
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Story: #2006628
Task: #36840
Related-Bug: #1845322
(cherry picked from commit ea27ebb0f918db9eab2f5751a1b065818faa0e6d)
(cherry picked from commit 576d9ecfd0ac27611ec89af6d34e7378b7a39fb3)







Fix openstack server list --deleted --marker option
2020-01-21T21:56:03+00:00

KeithMnemonic
keith.berger@suse.com

2019-10-24T18:39:50+00:00

3b48ef54c8ad205dd086520cc17e4f96583e88ec

This patch removes using the "name" option for a marker when
--deleted is also used. The find_resource() function
that is being called does not correctly handle using the marker
as the "name" in the search when also using deleted=True.
One simple way to fix this is force the marker to only be an ID
when --deleted is used. This is how the nova client works.

Using the --deleted option is available to users with the admin
role by default. If you're an admin listing --deleted servers
with a marker by name, find_resource() is going to fail to find
it since it doesn't apply the --deleted filter to find_resource().

The find_resource() function is trying to find the marker server
by name if it's not found by id, and to find it by name it's
listing servers with the given marker as the name, but not
applying the --deleted filter so it doesn't get back any results.

In the story it was suggested modifying find_resource to include
the deleted query param when it's specified on the command line but
that didn't work because it still results in something like this:

http://192.168.1.123/compute/v2.1/servers?deleted=True&name=4cecd49f-bc25-4a7e-826e-4aea6f9267d9

It seems like there are bugs in find_resource().

Restricting the marker to be the server ID when listing deleted servers
is probably OK since if you're using --deleted you're an admin and you could
be listing across all projects and if you're filtering by a server across all
projects anyway (not that you have to, I'm just saying if you are), or even
showing a server in another project, you have to do it by id rather than name
because find_resource() won't find the server in another project by name, only ID.

story: 2006761
Task: 37258

The server test case was removed from the cherry-pick as it used
features of the openstack client that are not in stable/stein

Change-Id: Ib878982b1d469212ca3483dcfaf407a8e1d2b417
(cherry picked from commit c42c27aa926e93a2507e686eeaf0c5510a29d245)





This patch removes using the "name" option for a marker when
--deleted is also used. The find_resource() function
that is being called does not correctly handle using the marker
as the "name" in the search when also using deleted=True.
One simple way to fix this is force the marker to only be an ID
when --deleted is used. This is how the nova client works.

Using the --deleted option is available to users with the admin
role by default. If you're an admin listing --deleted servers
with a marker by name, find_resource() is going to fail to find
it since it doesn't apply the --deleted filter to find_resource().

The find_resource() function is trying to find the marker server
by name if it's not found by id, and to find it by name it's
listing servers with the given marker as the name, but not
applying the --deleted filter so it doesn't get back any results.

In the story it was suggested modifying find_resource to include
the deleted query param when it's specified on the command line but
that didn't work because it still results in something like this:

http://192.168.1.123/compute/v2.1/servers?deleted=True&name=4cecd49f-bc25-4a7e-826e-4aea6f9267d9

It seems like there are bugs in find_resource().

Restricting the marker to be the server ID when listing deleted servers
is probably OK since if you're using --deleted you're an admin and you could
be listing across all projects and if you're filtering by a server across all
projects anyway (not that you have to, I'm just saying if you are), or even
showing a server in another project, you have to do it by id rather than name
because find_resource() won't find the server in another project by name, only ID.

story: 2006761
Task: 37258

The server test case was removed from the cherry-pick as it used
features of the openstack client that are not in stable/stein

Change-Id: Ib878982b1d469212ca3483dcfaf407a8e1d2b417
(cherry picked from commit c42c27aa926e93a2507e686eeaf0c5510a29d245)







Fix functional tests for py3
2020-01-12T07:51:11+00:00

Eric Fried
openstack@fried.cc

2019-10-29T20:55:11+00:00

29ec8c8134072b9b3eb6ff89320506bef9ec358a

Fix various things so the functional tests will work under python3:

- A hashlib.md5() can only be update()d with an encoded string in py3.
- There's no dict.iteritems(), change to dict.items() (which is already
  an iterator).
- Open temp files with 'w+' mode rather than the default 'w+b' (as an
  alternative to encoding all the write and expected-read payloads as
  bytes).
- (This is a weird one) Explicitly raise SkipTest from unittest (rather
  than unittest2, which is where cls.skipException landed). Not sure why
  this is busted, but this moves the ball.

Conflict/issue with raising SkipTest on this branch.

(cherry picked from commit f1d742f32adeb662a3fdf8fa3ef3bc391e71ed81)
(cherry picked from commit b866202f54afddca66a77cd989b082e193a96956)

Includes squash of:
Before writing object data to stdout, re-open it in binary mode

Otherwise, you can hit TypeErrors on Python3.

Closes-Bug: 1775482
(cherry picked from commit 415b48056d9d021e04ec972029040a89a6b13928)

Change-Id: Ic9b2b47848a600e87a3674289ae7ae8c3e091fee





Fix various things so the functional tests will work under python3:

- A hashlib.md5() can only be update()d with an encoded string in py3.
- There's no dict.iteritems(), change to dict.items() (which is already
  an iterator).
- Open temp files with 'w+' mode rather than the default 'w+b' (as an
  alternative to encoding all the write and expected-read payloads as
  bytes).
- (This is a weird one) Explicitly raise SkipTest from unittest (rather
  than unittest2, which is where cls.skipException landed). Not sure why
  this is busted, but this moves the ball.

Conflict/issue with raising SkipTest on this branch.

(cherry picked from commit f1d742f32adeb662a3fdf8fa3ef3bc391e71ed81)
(cherry picked from commit b866202f54afddca66a77cd989b082e193a96956)

Includes squash of:
Before writing object data to stdout, re-open it in binary mode

Otherwise, you can hit TypeErrors on Python3.

Closes-Bug: 1775482
(cherry picked from commit 415b48056d9d021e04ec972029040a89a6b13928)

Change-Id: Ic9b2b47848a600e87a3674289ae7ae8c3e091fee







Document 2.53 behavior for compute service list/delete
2019-09-20T14:42:11+00:00

Matt Riedemann
mriedem.os@gmail.com

2019-07-25T18:28:43+00:00

111d6a9b23d8c6b9bcce437039ff49536f0b5dd1

With compute API microversion 2.53, nova-compute services
can only be deleted with the ID as a UUID to uniquely identify
the service in a multi-cell deployment. This documents that for
the "compute service delete <service>" argument. The description
of the "compute service list" command is also updated to mention
that the ID can be retrieved as a UUID using 2.53 or greater.

Change-Id: If7d4a27c0aaef588bcd77dd9edddec1e535fbf31
Story: 2005349
Task: 30302
(cherry picked from commit 1557afb554e908e097abd39081891ea78083e20e)





With compute API microversion 2.53, nova-compute services
can only be deleted with the ID as a UUID to uniquely identify
the service in a multi-cell deployment. This documents that for
the "compute service delete <service>" argument. The description
of the "compute service list" command is also updated to mention
that the ID can be retrieved as a UUID using 2.53 or greater.

Change-Id: If7d4a27c0aaef588bcd77dd9edddec1e535fbf31
Story: 2005349
Task: 30302
(cherry picked from commit 1557afb554e908e097abd39081891ea78083e20e)







Fix compute service set handling for 2.53+
2019-09-20T14:10:04+00:00

Matt Riedemann
mriedem.os@gmail.com

2019-07-24T18:39:07+00:00

100d34c54ecdfedf6fb40a2686e1aae1f54df97e

With compute API microversion 2.53 there is a single
PUT /os-services/{service_id} API which takes the service
id as a UUID. Since the openstack compute service set
command only takes --host and --service (binary) to identify
the service, this change checks if 2.53 or greater is being
used and if so, looks up the service by host and binary and
calls the appropriate methods in novaclient.

If the command cannot uniquely identify a compute service
with the given host and binary, an error is raised. A future
change could add an --id option to be used with 2.53+ to
pass the service id (as UUID) directly to avoid the host/binary
filtering.

Change-Id: I868e0868e8eb17e7e34eef3d2d58dceedd29c2b0
Story: 2005349
Task: 30302
(cherry picked from commit 4bd53dc1090fda86f6ce25b76a079e250c9206d8)





With compute API microversion 2.53 there is a single
PUT /os-services/{service_id} API which takes the service
id as a UUID. Since the openstack compute service set
command only takes --host and --service (binary) to identify
the service, this change checks if 2.53 or greater is being
used and if so, looks up the service by host and binary and
calls the appropriate methods in novaclient.

If the command cannot uniquely identify a compute service
with the given host and binary, an error is raised. A future
change could add an --id option to be used with 2.53+ to
pass the service id (as UUID) directly to avoid the host/binary
filtering.

Change-Id: I868e0868e8eb17e7e34eef3d2d58dceedd29c2b0
Story: 2005349
Task: 30302
(cherry picked from commit 4bd53dc1090fda86f6ce25b76a079e250c9206d8)







Fix BFV server list handling with --name-lookup-one-by-one
2019-07-22T13:23:51+00:00

melanie witt
melwittt@gmail.com

2019-06-28T18:17:10+00:00

83359fbe4fd7e5850abd45a467bf197c284519b1

When the --name-lookup-one-by-one option passed to the 'server list'
command, the image and flavor names will be looked up for each
server being listed instead of fetching all image/flavor names.

The current code assumes all servers have an image attribute, but
servers booted from volumes have no image, so the following error is
raised when listing BFV servers with --name-lookup-one-by-one:

  AttributeError: ('unicode'|'str') object has no attribute 'get'

The error occurs when the code attempts server.image.get('id').

This fixes the --name-lookup-one-by-one code not to assume an image
for a server. The unit tests for 'server list' have also been
robustified to feature one BFV server to enhance our test coverage.

Story: #2006063
Task: #34777

Change-Id: I312c971346c7ded93f6fcaa515098554b8580295
(cherry picked from commit bfc34e11b3437506508b3e120accc0e212268ac6)





When the --name-lookup-one-by-one option passed to the 'server list'
command, the image and flavor names will be looked up for each
server being listed instead of fetching all image/flavor names.

The current code assumes all servers have an image attribute, but
servers booted from volumes have no image, so the following error is
raised when listing BFV servers with --name-lookup-one-by-one:

  AttributeError: ('unicode'|'str') object has no attribute 'get'

The error occurs when the code attempts server.image.get('id').

This fixes the --name-lookup-one-by-one code not to assume an image
for a server. The unit tests for 'server list' have also been
robustified to feature one BFV server to enhance our test coverage.

Story: #2006063
Task: #34777

Change-Id: I312c971346c7ded93f6fcaa515098554b8580295
(cherry picked from commit bfc34e11b3437506508b3e120accc0e212268ac6)







Fix bug in endpoint group deletion
2019-06-27T18:36:16+00:00

Jose Castro Leon
jose.castro.leon@cern.ch

2019-03-14T12:33:51+00:00

fd3a94d16be8e7806ff4f3aa7417d8a0d36375fb

There is a typo in the endpoint group deletion, due to this you
can't remove endpoint groups once assigned. I am adding also the
unit tests to avoid this kind of issues in the future

Task: 30640
Story: 2005521
Change-Id: Ie938f2c9894bb39b4c0ed1f7aa3a6a751a303058
(cherry picked from commit 04e03b2a1fe34046e2148d3c1d17b9053010c8af)





There is a typo in the endpoint group deletion, due to this you
can't remove endpoint groups once assigned. I am adding also the
unit tests to avoid this kind of issues in the future

Task: 30640
Story: 2005521
Change-Id: Ie938f2c9894bb39b4c0ed1f7aa3a6a751a303058
(cherry picked from commit 04e03b2a1fe34046e2148d3c1d17b9053010c8af)







Stable branch combination fix
2019-05-16T19:07:09+00:00

Dean Troyer
dtroyer@gmail.com

2019-05-09T21:47:41+00:00

b507f605e46b2df3253de528ff8551ee5706689d

We have two dueling problems in the stable branches that have to be
fixed at the same time:

* bandit 1.6.0
* sphinx 2.0

This is a squash of the two cherry-picks from master:

----------
Blacklist Bandit 1.6.0 due to directory exclusion bug

Bandit 1.6.0 introduces a regression[0] with the -x option, a fix
is expected to be included in 1.6.1 soon.

[0] https://github.com/PyCQA/bandit/issues/488
[1] https://github.com/PyCQA/bandit/pull/489

Signed-off-by: Dean Troyer <dtroyer@gmail.com>
(cherry picked from commit 6385d64237c9973dd4c7dd53efb6664ea2c719da)

----------

Update sphinx requirement.

Sphinx 2.0 no longer works on python 2.7, start cappingit there as well.

(cherry picked from commit f1791179768115b6d074f70f9a8695f9c1e0b9f5)

Change-Id: I0076645d9e1a2429efce39f51ceea679fa6c13cb





We have two dueling problems in the stable branches that have to be
fixed at the same time:

* bandit 1.6.0
* sphinx 2.0

This is a squash of the two cherry-picks from master:

----------
Blacklist Bandit 1.6.0 due to directory exclusion bug

Bandit 1.6.0 introduces a regression[0] with the -x option, a fix
is expected to be included in 1.6.1 soon.

[0] https://github.com/PyCQA/bandit/issues/488
[1] https://github.com/PyCQA/bandit/pull/489

Signed-off-by: Dean Troyer <dtroyer@gmail.com>
(cherry picked from commit 6385d64237c9973dd4c7dd53efb6664ea2c719da)

----------

Update sphinx requirement.

Sphinx 2.0 no longer works on python 2.7, start cappingit there as well.

(cherry picked from commit f1791179768115b6d074f70f9a8695f9c1e0b9f5)

Change-Id: I0076645d9e1a2429efce39f51ceea679fa6c13cb







OpenDev Migration Patch
2019-04-19T19:45:11+00:00

OpenDev Sysadmins
openstack-infra@lists.openstack.org

2019-04-19T19:45:11+00:00

97da13dbc3c8447c93a0f0e1749fb4b840837e90

This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.





This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.