| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
Unused by Nova and unlike memory_mb/local_gb also by Ironic (actually,
our usage of local_gb is worth double-checking as well, but at the very
least it's referenced by inspection implementations).
Change-Id: Ie8b0d9f58f4dcd102c183c30ae7f5acf68a5e4c3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
... And tags, but nobody uses tags since it is not available
via the API.
Anyhow, the online upgrade code was written under the assumption
that *all* tables had an "id" column. This is not always true
in the ironic data model for tables which started as pure extensions
of the Nodes table, and fails in particular when:
1) A database row has data stored in an ealier version of the object
2) That same object gets a version upgrade.
In the case which discovered this, BIOSSetting was added at version
1.0, and later updated to include additional fields which incremented
the version to 1.1. When the upgrade went to evaluate and iterate
through the fields, the command failed because the table was designed
around "node_id" instead of "id".
Story: 2010632
Task: 47590
Change-Id: I7bec6cfacb9d1558bc514c07386583436759f4df
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The tl;dr is that we changed ``inspecting`` to include a
``inspect wait`` state. Unfortunately we never spotted the logic
inside of the db API. We never spotted it because our testing in
inspection code uses a mocked task manager... and we *really* don't
have intense db testing because we expect the objects and higher
level interactions to validate the lowest db level.
Unfortunately, because of the out of band inspection workflow,
we have to cover both cases in terms of what the starting state
and ending state could be, but we've added tests to
validate this is handled as we expect.
Change-Id: Icccbc6d65531e460c55555e021bf81d362f5fc8b
|
| |\ \ |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This change adds 'node_uuid' to:
ironic.objects.portgroup.Portgroup
'node_uuid' is a relationship using association_proxy in
models.Portgroup. Using the association_proxy removes the
need to do the node lookup to populate node uuid for
portgroups in the api controller.
NOTE:
On portgroup create a read is added to read the port from
the database, this ensures node_uuid is loaded and solves
the DetachedInstanceError which is otherwise raised.
The test test_list_with_deleted_port_group was deleted, if
the portgroup does not exist porgroup_uuid on the port will
be None, no need for extra handling of that case.
Bumps Portgroup object version to 1.5
Change-Id: I4317d034b6661da4248935cb0b9cb095982cc052
|
| | |/
|/|
| |
| |
| |
| |
| |
| |
| |
| | |
Follow-up to I6b830e5cc30f1fa1f1900e7c45e6f246fa1ec51c
Original changa introduced some errors such as mismatched
arguments for exceptions
Story: 2010275
Task: 46204
Change-Id: I550e048ab22a6cd25502b41d1c579819df369249
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Adds ability to query ports by shard.
Example usage: /v1/ports?shard=lol,cats
Change-Id: Icacef7f71414d30f492a6d144bcc89dff4891784
|
| |/
|
|
|
|
|
|
| |
DB and object implementations for new node.shard key.
Story: 2010768
Task: 46624
Change-Id: Ia7ef3cffc321c93501b1cc5185972a4ac1dcb212
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change adds 'node_uuid' to ironic.objects.port.Port
and adds a relationship using association_proxy in
models.Port. Using the association_proxy removes the need
to do the node lookup to populate node uuid for ports in
the api controller.
NOTE:
On port create a read is added to read the port from the
database, this ensures node_uuid is loaded and solves the
DetachedInstanceError which is otherwise raised.
Bumps Port object version to 1.11
With patch:
1. Returned 20000 ports in python 2.7768702507019043
seconds from the DB.
2. Took 0.433107852935791 seconds to iterate through
20000 port objects.
Ports table is roughly 12800000 bytes of JSON.
3. Took 5.662816762924194 seconds to return all 20000
ports via ports API call pattern.
Without patch:
1. Returned 20000 ports in python 1.0273635387420654
seconds from the DB.
2. Took 0.4772777557373047 seconds to iterate through
20000 port objects.
Ports table is roughly 12800000 bytes of JSON.
3. Took 147.8800814151764 seconds to return all 20000
ports via ports API call pattern.
Conclusion:
Test #1 plain dbapi.get_port_list() test is ~3 times
slower, but Test #3 doing the API call pattern test
is ~2500% better.
Story: 2007789
Task: 40035
Change-Id: Iff204b3056f3058f795f05dc1d240f494d60672a
|
| |
|
|
|
|
|
|
|
| |
Prepare the ironic database to accommodate node inventory received from
the inspector once the API is implemented.
Story: 2010275
Task: 46204
Change-Id: I6b830e5cc30f1fa1f1900e7c45e6f246fa1ec51c
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The model_query call results in a nested read transaction, that does not
seem to play well with SQLite support. Since it's inherently relying on
the query style deprecated in SQLAlchemy 2.0, we need to migrate away
from this call. As an intermediate step, change instances of model_query
to session.query, making sure every call creates a session that lives
as long as is needed to fetch the results.
Removes a unit test which was built around creating
a fake deadlock condition to test that oslo_db was working
as expected. It's interaction was totally mocked, and in
retooling the base method there was no easy to keep
the same test logic around.
Co-Authored-By: Dmitry Tantsur <dtantsur@protonmail.com>
Change-Id: Ic8b1d964f7be5784e01c89bfb6c0277ea82eec2d
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch rewrites portions of the database migration
testing to the style required to not break when SQLAlchemy 2.0
is released.
The *Major* difference is a transition from using Dictionary key
styles towards using object names. This is because to retrieve
a dictionary form, or access a row object as a dictionary, requires
it to be cast as a dictonary, but in SQLAlchemy 2.0 row result
attribute .keys is no longer present, which ultimately prevents
casting and loading as such. Ultimately this just meant change
the tests to use the object model field labels.
One other change is we now query just the columns needed to get
an ORM object. This is a result of the unification of the select
interface and us being unable to instantiate a current full DB
object (as in models.Node in current code) against an older
database version in order to perform migration validation.
One last item, there appears to be a minor trivial difference
in the behavior in the return of a dictionary/json response
object with Postgres. Ultimately, it seems trivial, we just
needed the test to be aware of the difference as it is a
very low level test.
Change-Id: I4d7213488ce90176893459087fe2f0491a6a61fc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
One of the major changes in SQLAlchemy 2.0 is the removal
of autocommit support. It turns out Ironic was using this quite
aggressively without even really being aware of it.
* Moved the declaritive_base to ORM, as noted in the SQLAlchemy 2.0
changes[0].
* Console testing caused us to become aware of issues around locking
where session synchronization, when autocommit was enabled, was
defaulted to False. The result of this is that you could have two
sessions have different results, which could results on different
threads, and where one could still attempt to lock based upon prior
information. Inherently, while this basically worked, it was
also sort of broken behavior. This resulted in locking being
rewritten to use the style mandated in SQLAlchemy 2.0 migration
documentation. This ultimately is due to locking, which is *heavily*
relied upon in Ironic, and in unit testing with sqlite, there are
no transactions, which means we can get some data inconsistency
in unit testing as well if we're reliant upon the database to
precisely and exactly return what we committed.[1]
* Begins changing the query.one()/query.all() style to use explicit
select statements as part of the new style mandated for migration
to SQLAlchemy 2.0.
* Instead of using field label strings for joined queries, use the
object format, which makes much more sense now, and is part of
the items required for eventual migration to 2.0.
* DB queries involving Traits are now loaded using SelectInLoad
as opposed to Joins. The now deprecated ORM queries were quietly
and silently de-duplicating rows and providing consistent sets
from the resulting joined table responses, however putting much
higher CPU load on the processing of results on the client.
Prior performance testing has informed us this should be a minimal
overhead impact, however these queries should no longer be in
transactions with the Database Servers which should offset the
shift in load pattern. The reason we cannot continue to deduplicate
locally in our code is because we carry Dict data sets which cannot
be hashed for deduplication. Most projects have handled this by
treating them as Text and then converting, but without a massive
rewrite, this seems to be the viable middle ground.
* Adds an explict mapping for traits and tags on the Node object
to point directly to the NodeTrait and NodeTag classes. This
superceeds the prior usage of a backref to make the association.
* Splits SQLAlchemy class model Node into Node and NodeBase, which
allows for high performance queries to skip querying for ``tags``
and ``traits``. Otherwise with the afrormentioned lookups would
always execute as they are now properties as well on the Node
class. This more common of a SQLAlchemy model, but Ironic's model
has been a bit more rigid to date.
* Adds a ``start_consoles`` and ``start_allocations`` option to the
conductor ``init_host`` method. This allows unit tests to be
executed and launched with the service context, while *not* also
creating race conditions which resulted in failed tests.
* The db API ``_paginate_query`` wrapper now contains additional
logic to handle traditional ORM query responses and the newer style
of unified query responses. Due to differences in queries and handling,
which also was part of the driver for the creation of ``NodeBase``,
as SQLAlchemy will only create an object if a base object is referenced.
Also, by default, everything returned is a tuple in 1.4 with the
unified interface.
* Also modified one unit test which counted time.sleep calls, which is
a known pattern which can create failures which are ultimately noise.
Ultimately, I have labelled the remaining places which SQLAlchemy
warnings are raised at for deprecation/removal of functionality,
which needs to be addressed.
[0] https://docs.sqlalchemy.org/en/14/changelog/migration_20.html
[1] https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#transaction-isolation-level-autocommit
Change-Id: Ie0f4b8a814eaef1e852088d12d33ce1eab408e23
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Provide the ability to limit resource intensive or potentially
wide scale operations which could be a symptom of a highly
distructive and unplanned operation in progress.
The idea behind this change is to help guard the overall deployment
to prevent an overall resource exhaustion situation, or prevent an
attacker with valid credentials from putting an entire deployment
into a potentially disasterous cleaning situation since ironic only
other wise limits concurrency based upon running tasks by conductor.
Story: 2010007
Task: 45140
Change-Id: I642452cd480e7674ff720b65ca32bce59a4a834a
|
| |
|
|
|
|
|
|
| |
For backward compatibility reasons, they're off by default.
Fix all unit tests that rely on dangling foreign keys.
Change-Id: I54a45fab1aefc03b2252c655e14b375dea3d9f12
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When using the node managed by the `irmc` hardware type,
if the port number of the bmc address is manually specified,
the following error will occur:
```
Value '443' is not supported for 'irmc_port'
```
However iRMC supports 80 and 443 ports, and default to 443.
The reason for this error is that the type of irmc_port is not converted to int.
Story: #2009671
Task: #43915
Signed-off-by: Zhou Hao <zhouhao@fujitsu.com>
Change-Id: I4e9274de09758fdb468382a7f88298a279f43e92
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This patch provides basic data model change to support node history.
Batch removal is not included in this patch.
Change-Id: I5c7cebd585ee84b5b57bd4690d4074baf0d05699
Story: 2002980
Task: 22989
|
| |\ \
| |/
|/| |
|
| | |
| |
| |
| | |
Change-Id: If21760f21ad241611e2f6c95879cb44d8df90c94
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The database schema upgrade check had support for an
explicit list of known versions to handle the upgrade.
who knew!
Anyhow, we haven't used it in three years since the addition,
and it seems to make more sense to just be able to indicate
"we know initial field versions may not be able to be retrieved
and act accordingly". As such, when no table is found, the
pre-upgrade version check execution will continue onward fearlessly!
Call it a spiritual successor to Ibcf0728bc5d1b0cbdd78796526f9c93fc99e8c08
Change-Id: Icae5162c2501b0d1217ad0e6ee34ebef40e95204
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
* add fields to Node object
* expose them at endpoint `/v1/nodes/{node_ident}/states`
* update states on powersync / entering managed state.
* tests
* update api endpoint info in api-ref
Story: 2008567
Task: 41709
Change-Id: Iddd1421a6fa37d69da56658a2fefa5bc8cfd15e4
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A node object has many fields, and a huge opportunity
for performance improvement is to reduce the amount of work
performed at the lower layers when it is not necessary.
In particular, the easiest case to identify and handle is
when a list of fields to be fulfilled is explicitly supplied.
This is particularlly noteworthy when we collecting a list of
nodes for synconization with Nova, where only 9 columns are
really needed to supply Nova with the information required,
and thus numerous fields are discarded.
This is all particularlly compounded when traits are used,
which presently uses a joined load pattern from SQL.
With this patch, we explicitly load and convert only the fields
requested at the lowest level, and then perform a different style
of loading node traits which requires less overhead by SQLAlchemy
to de-duplicate the returned result set to the application.
This turns out to be far more efficent as we're reducing the amount
of data/object conversion work by 360%, even before we consider
populating traits, which is performed as a separate query when
traits are requested.
Story: 2008885
Task: 42434
Change-Id: Iac703d2a9b7b240a47477be8a64c7c33e28f692f
|
| |
|
|
|
|
|
|
|
|
| |
Provide the fields in the BIOS setting API -
``/v1/nodes/{node}/bios/{setting}``, and in the BIOS setting list API
when details are requested - ``/v1/nodes/<node>/bios?detail=True``.
Story: #2008571
Task: #42483
Change-Id: Ie86ec57e428e2bb2efd099a839105e51a94824ab
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The iSCSI deploy was very easy to start with, but it has since become
apparently that it suffers from scalability and maintenance issues.
It was deprecated in the Victoria cycle and can now be removed.
Hide the guide to upgrade to hardware types since it's very outdated.
I had to remove the iBMC diagram since my SVG-fu is not enough to fix it.
Change-Id: I2cd6bf7b27fe0be2c08104b0cc37654b506b2e62
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds MVP support for idrac-redfish to RAID interface. Based on
generic redfish implementation, but requires OEM extension
to check when `Immediate` time becomes available shortly
after IPA starts executing steps.
Does not support foreign disks, convert from non-RAID mode.
Story: 2008602
Task: 41778
Depends-On: https://review.opendev.org/c/x/sushy-oem-idrac/+/776224
Change-Id: Iefb7f882c97e33a176962e4e907163d9e4809445
|
| |
|
|
|
|
|
|
|
|
| |
Prevent each driver comming online one at a time. So that
/driver returns nothign until all interfaces are registered
Story: #2008423
Task: #41368
Change-Id: I6ef3e6e36b96106faf4581509d9219e5c535a6d8
|
| |
|
|
|
|
|
|
|
|
| |
MAC address is not user friendly for port management, having
a name field is also a feature parity with other resources.
This patch implements db related change.
Change-Id: Ibad9a1b6bbfddc0af1950def4e27db3757904cb1
Story: 2003091
Task: 23180
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We hit an issue in Kayobe CI where we have the following config:
enabled_deploy_interfaces: iscsi,direct
default_deploy_interface: iscsi
In practice, direct does not work out of the box in Kayobe currently.
The online migration added recently to move nodes from iscsi to direct
caused issues in our upgrade jobs, since it migrated nodes to direct,
without the config in place to make this work.
This change avoids migrating nodes when iscsi is the default deploy
interface.
Story: 2008114
Task: 40996
Change-Id: I5cf0684dc53a2d89fac12e578b28e73e7697a97e
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change marks the iscsi deploy interface as deprecated and
stops enabling it by default.
An online data migration is provided for iscsi->direct, provided that:
1) the direct deploy is enabled,
2) image_download_source!=swift.
The CI coverage for iscsi deploy is left only on standalone jobs.
Story: #2008114
Task: #40830
Change-Id: I4a66401b24c49c705861e0745867b7fc706a7509
|
| |
|
|
|
|
|
| |
Update the port node's filter to allow both owner and lessee to
see a node's ports. This filter is only used when listing ports.
Change-Id: I568e8d23375239d9c044df95b4bc24d5174c145b
|
| |
|
|
|
|
|
|
|
|
| |
We allow deleting allocated nodes if maintenance is True, but it
fails with HTTP 500 because of a foreign constraint on the nodes
table.
Story: #2007823
Task: #40102
Change-Id: I7a689c8c403b613fca412fa70e31df40cce8ad95
|
| |
|
|
|
|
|
|
|
|
|
| |
The assertItemsEqual method has been removed in Python 3.3 [1] but
it was kept alive by unittest2, imported by testtools.
To prevent mayhem and despair caused by testtools removing unittest2
from its requirements in the near future, we switch to assertCountEqual.
[1] https://bugs.python.org/issue17866
Change-Id: I614bcedc9981370fb54cfe3da997a3d52484c07f
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
A new dictionary field `network_data` is added to the node object.
This new field can be populated by the operator with node static
network configuration.
Ironic API now performs formal JSON document validation of node
`network_data` field content against user-supplied JSON schema at
driver validation step.
As of this commit, the new `network_data` field is not actually
used by ironic, otherwise it should be perfectly functional. In
the following commits, network static configuration will be taken
from this field and handed over to ironic ramdisk bootstrapping
utilities.
Change-Id: I868b3b56a17f59e5aa1494b2e0ebc9c4e34ef173
Story: 2006691
Task: 36991
|
| |/
|
|
|
|
|
|
| |
This should help slowly enabling H210 test everywhere.
Also fixing H210 enforced autospec in some places.
Change-Id: Ibbc00b528265f8b86bc2fe45e078d56d36381c5d
|
| |
|
|
|
|
|
| |
Python3 have a standard library for mock in the unittest module,
let's drop the mock requirement and switch tests to unittest mock.
Change-Id: I4f1b3e25c8adbc24cdda51c73da3b66967f7ef23
|
| |
|
|
|
|
|
| |
This commit updates the mailmap file and changes my alias
in a few places within old comments.
Change-Id: Ica0e184109d794b8e129d567b5606d7fe84ff384
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This change adds a `lessee` field to nodes, and exposes it
to policy. It also updates the non-admin node list API
to match for both owner and lessee; and updates the
allocation conductor to match owner allocations with nodes
with the appropriate lessee.
Change-Id: Ib31b49c7143ec8fd6cb486fc24038215b197c418
Story: 2006506
Task: 37930
|
| |
|
|
|
|
|
|
|
|
| |
Fixes W504 and E117, resulting in some indentation changes.
Also fixes code that exceeds the complexity requirement, that is bumped
to 20 (mostly to avoid refactoring the agent heartbeat call, resulting
in conflicts for the deploy steps work).
Change-Id: I8e49f2c039b0ddfca9138f8e148708b7e8b5df7e
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a conductor hostname is changed while reservations are
issued to a conductor with one hostname, such as 'hostName'
and then the process is restarted with 'hostname', then the
queries would not match the node and effectively the nodes
would become inaccessible until the reservation is cleared.
This patch changes the queries to use a case insenitive
like search, better known as ilike.
Special thanks to the nova team for finding a bug in the hash
ring calculation in the ironic virt driver where a customer
changed their nova-compute hostname, which inspired this
patch.
Change-Id: Ib7da925ba5ca6a82ba542e0f4ae4cf7f0d070835
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Expose a port's node's owner to policy, giving Ironic admins
the option of modifying the policy file to allow users
specified by a node's owner field to perform API actions on
that node's associated ports.
Change-Id: I47ecbad9729f18dacc703e2a625ef3e92bfb00ed
Story: #2006506
Task: #37539
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change adds support for node retirement: nodes can
have additional properties 'retired' and 'retired_reason'
which change the way the nodes (can) traverse the FSM
and which operations are allowed. In particular:
- retired nodes cannot move from manageable to available;
- upon instance deletion, retired nodes move to manageable
(rather than available).
Story: #2005425
Task: #38142
Change-Id: I8113a44c28f62bf83f8e213aeb6704f96055d52b
|
| |
|
|
|
|
|
|
|
|
| |
Since we've dropped support for Python 2.7, it's time to look at
the bright future that Python 3.x will bring and stop forcing
compatibility with older versions.
This patch removes the six library from requirements, not
looking back.
Change-Id: Ib546f16965475c32b2f8caabd560e2c7d382ac5a
|
| |
|
|
|
|
|
|
|
|
| |
Add an owner to allocations. Depending on policy, a non-admin
can then create an allocation and have the owner set to their
project. Allocation processing then respects the owner.
Change-Id: I2965a4a601b9fa2c0212097da37b104a3e5514df
Story: #2006506
Task: #37540
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
PXE is inherently unreliable and sometimes times out without an
obvious reason. It happens particularly often in resource constrained
environments, such as the CI. This change allows an operator to
set a timeout, after which the boot is retried again.
The _add_node_filters call had to be refactored to avoid hitting
the complexity limit.
Change-Id: I34a11f52e8e98e5b64f2d21f7190468a9e4b030d
Story: #2005167
Task: #29901
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cisco's Third-Party CI was taken down as a result of the
CTO's office being restructured. Numerous attempts to
re-engage with Cisco directly and address the various
known issues in their drivers have not proven to be
fruitful.
Additionally, the drivers are not Python3 compatible,
and some reports have indicated that the CIMC driver is
no longer compatible with newer versions.
As such, the ironic community has little choice but to
to remove the Cisco UCS/CIMC hardware types and driver
interface code.
Story: 2005033
Task: 29522
Change-Id: Ie12eaf7572ce4d66f6a68025b7fe2d294185ce28
|
| |
|
|
|
|
|
| |
Thorough replacement of git.openstack.org URLs with their opendev.org
counterparts.
Change-Id: Ic19e45986e9f650c1deb297f463e679fd268ad97
|
