summaryrefslogtreecommitdiff
path: root/ceilometer
Commit message (Collapse)AuthorAgeFilesLines
* Imported Translations from ZanataOpenStack Proposal Bot2023-05-091-2/+5
| | | | | | | For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I41671793e5c3249c8e5fd222bf37f83392c7c92a
* Merge "Make TCP publisher log warning instead of failing."Zuul2023-04-241-3/+7
|\
| * Make TCP publisher log warning instead of failing.Jaromír Wysoglad2023-04-241-3/+7
| | | | | | | | Change-Id: I56ca759c2d43830f4e342130f27b75bdba65c469
* | Add vanity names to notification samplesYadnesh Kulkarni2023-04-045-32/+74
|/ | | | | | | | | | This change adds "project_name" and "user_name" fields to the polling samples created from notifications of "event_type". Also move caching helper functions into "ceilometer/cache_utils.py" to make them accessible throughout the project. Change-Id: I68bd4ee096b28a2fd952e749d56a6b3eed9bfb94
* Merge "Add vanity names to telemetry polling notifications"Zuul2023-03-241-0/+2
|\
| * Add vanity names to telemetry polling notificationsYadnesh Kulkarni2023-03-021-0/+2
| | | | | | | | | | | | | | | | This change adds "project_name" and "user_name" fields to polling samples which is related to the identification of vanity names change 79454d6b22787627ae6239aa7b2707101ba30212 Change-Id: I5fbe97439e7fadbdd8fd2641c49f1c88fbc416fc
* | [coordination] backend_url should be secretTakashi Kajinami2023-03-201-0/+1
|/ | | | | | | | | | | | | The backend_url option can sometimes contain secrets. For example when redis coordination backend is used and authentication is enabled in redis, the plain redis password is put as an URL element. [coordination] backend_url=redis://:password@127.0.0.1:6379 Closes-Bug: #2012246 Change-Id: I9b61dd459445e0d32a305835fb05a072c32848a4
* Merge "Post individual JSONs instead of the whole list"Zuul2023-02-211-1/+1
|\
| * Post individual JSONs instead of the whole listMariusz Karpiarz2022-10-281-1/+1
| | | | | | | | Change-Id: I5114222c1ead7858c0ed9ddafe7c23496608a6c6
* | Imported Translations from ZanataOpenStack Proposal Bot2023-01-271-2/+11
| | | | | | | | | | | | | | For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I811d00ac0995980b4f6ab17b580f164d956d0bfe
* | Merge "Add TCP publisher"Zuul2023-01-203-1/+320
|\ \
| * | Add TCP publisherjwysogla2023-01-133-1/+320
| | | | | | | | | | | | | | | | | | | | | This commit adds a TCP publisher. The publisher works similarly to the UDP publisher. Change-Id: Iac662018039e74ad59ac9c7fa4db994da540ef2f
* | | Merge "NoUniqueMatch: ClientException on Gnocchi publisher"Zuul2022-12-212-12/+42
|\ \ \
| * | | NoUniqueMatch: ClientException on Gnocchi publisherRafael Weingärtner2022-12-082-12/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ceilometer can ignore/discard measurements that come from possible VMs that can be used to host Gnocchi. The assumption is that one can use OpenStack itself to host the VMs that are used to run Gnocchi. The configuration is called `filter_project`. This configuration can be used in `event_pipeline.yaml` and `pipeline.yaml` configuration files. However, when this config is not used, it has a default project name for Gnocchi as `service`, as we can see in the following code snippet: ``` def __init__(self, conf, parsed_url): super(GnocchiPublisher, self).__init__(conf, parsed_url) # TODO(jd) allow to override Gnocchi endpoint via the host in the URL options = urlparse.parse_qs(parsed_url.query) self.filter_project = options.get('filter_project', ['service'])[-1] ``` Which means that if somebody creates a project called `service`, this project would not push measurements to Gnocchi. This configuration is then used by the following code: ``` def gnocchi_project_id(self): if self._gnocchi_project_id is not None: return self._gnocchi_project_id with self._gnocchi_project_id_lock: if self._gnocchi_project_id is None: try: project = self._ks_client.projects.find( name=self.filter_project, domain=self.filter_domain) except ka_exceptions.NotFound: LOG.warning('filtered project not found in keystone,' ' ignoring the filter_project ' 'option') self.filter_project = None return None except Exception: LOG.exception('fail to retrieve filtered project ') raise self._gnocchi_project_id = project.id LOG.debug("filtered project found: %s", self._gnocchi_project_id) return self._gnocchi_project_id ``` Basically, this method will look for the project ID of the project name that is configured in `filter_project` option. If it does not find any project, it returns None, and it changes the value of `filter_project` to None as well. Before this `gnocchi_project_id` method/property is called, there is a verification if `filter_project` is None. Therefore, it is assumed that when we set the value of `filter_project` to None, this method (`gnocchi_project_id`) would not be called anymore. However, that is not taking into account concurrency parallel executions. In the code, we can see `with self._gnocchi_project_id_lock:` statement, which seems to execute locking in the execution flow. However, that will not always be the case because multiple concurrent calls can be queued in that part of the code, and when the first one finishes setting the `filter_project` to None, the others will execute with this variable as None, which will cause Keystone command `self._ks_client.projects.find` to find/list all projects. That command was designed to list/find only one project; therefore, when it finds more than one project, it throws an error. That is the cause for the exception we were seeing from time to time in the log files. Change-Id: I3b4ac918015b2fd3fbe24047c3eb13419f580b27
* | | | Merge "Change oslo_cache implementation"Zuul2022-12-214-49/+117
|\ \ \ \
| * | | | Change oslo_cache implementationYadnesh Kulkarni2022-11-214-49/+117
| | |_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As of now to leverage caching, oslo_cache library has to be imported and configured everytime it's needed. This change migrates such implementatio to use `cache_utils.py` which returns a cache client to perform caching operations. This eliminates the purpose of importing oslo_cache everytime when needed. To get a cache client: ``` from ceilometer import cache_utils . cache_client = cache_utils.get_client(conf) ``` Signed-off-by: Yadnesh Kulkarni <ykulkarn@redhat.com> Change-Id: I14f9e1cbe84a953b092c3a88345d5faa9bcc9fb2
* | | | Merge "Fix barbican credentials retrieval"Zuul2022-12-202-5/+7
|\ \ \ \ | |_|/ / |/| | |
| * | | Fix barbican credentials retrievalRafael Weingärtner2022-12-062-5/+7
| | | | | | | | | | | | | | | | Change-Id: Iec0d885e1fdd42edb2da37d4a71a80168c690c98
* | | | Merge "Exclude metrics from "service" project"Zuul2022-12-082-4/+4
|\ \ \ \
| * | | | Exclude metrics from "service" projectYadnesh Kulkarni2022-09-072-4/+4
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Upon installing ceilometer and gnocchi using devstack, "filter_project" in "pipeline.yaml" is set to "gnocchi_swift" project. However, no such project exists in keystone. ``` publishers: - gnocchi://?archive_policy=ceilometer-low&filter_project=gnocchi_swift ``` This change, will set "filter_project" to "service" project which will exclude all metrics from it. Also add the name of the project to the logs which couldn't be found in keystone. Closes-Bug: #1988923 Signed-off-by: Yadnesh Kulkarni <ykulkarn@redhat.com> Change-Id: Ifc215f86134729665c142bb9bf64f66ca8de2a29
* | | | Tests the sample nameArnaud Morin2022-11-291-4/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Patching the unit test to verify the name of the samples. Change-Id: Ifcea7a5d346e9b736b93b1a76677a0f3730541a2 Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
* | | | Fix OutgoingBytesDeltaPollster sample nameArnaud Morin2022-11-291-1/+1
| |_|/ |/| | | | | | | | | | | | | | | | | The good name is network.outgoing.bytes.delta. Change-Id: I78e9fbe9d60b3e83761f6490d25e85ad54fcc7c4 Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
* | | Improve logging for Gnocchi publisherRafael Weingärtner2022-11-062-8/+46
| |/ |/| | | | | Change-Id: I2f26746225a76df66e999490c0055101d325b114
* | Merge "Fix the handle of plain text outputs"Zuul2022-10-261-2/+4
|\ \
| * | Fix the handle of plain text outputsRafael Weingärtner2022-10-121-2/+4
| | | | | | | | | | | | | | | | | | The handler of Plaintext output was using the wrong data structure to store the output in plaintext. The output needs to be enclosed as a dictionary in a list. Change-Id: Ib134a5a0de135eb1e00073c4dfc6654ad2d93709
* | | Merge "Properly handle 'resource_id' as None for Gnocchi publisher"Zuul2022-10-182-13/+36
|\ \ \ | |/ / |/| |
| * | Properly handle 'resource_id' as None for Gnocchi publisherRafael Weingärtner2022-09-212-13/+36
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This issue is related to [1]. In Python 3 (I did not check exactly in which version), the following code was throwing an error `data.sort(key=operator.attrgetter('resource_id'))`. Therefore, this patch is proposing a solution for such situations to properly handle samples where the `resource_id` is None. Of course, `resource_id` cannot be None in Gnocchi. However, that can be properly handled by removing the metric to be pushed from the `gnocchi_resource.yml` file. Which is what we were doing when we proposed [1]. The problem is that the sort is happening before that part of the code is executed, to decide if the sample should be ignored or not. [1] https://review.opendev.org/c/openstack/ceilometer/+/746717 Change-Id: Ie8eb42df3d5b9505160c9e9d6b86bdaa9a02d16a
* | Merge "Add extra metadata fields skip"Zuul2022-10-112-0/+251
|\ \
| * | Add extra metadata fields skipPedro Henrique2022-09-132-0/+251
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem description =================== Some OpenStack APIs do not always return exactly the same metadata for all resources. As an example, we have the server API, which might not return the 'OS-EXT-SRV-ATTR:host' attribute; it depends on the virtual machine status. Therefore, if the operator configures to retrieve the value of the 'OS-EXT-SRV-ATTR:host' attribute in the response, when the VM is in the 'RUNNING' state, it will work properly; however, if it is in 'ERROR' or other state, it will throw a 'key not found' error when we are collecting the metadata. Proposal ======== To allow operators to skip the extra_metadata_fields gathering based on the collected sample attributes. We propose to add a new 'extra_metadata_fields_skip' in the dynamic pollster YAML definition where operators can define some rules to skip gathering extra_metadata for some samples based on their attributes. Change-Id: I40176328e1863283890870098418c0944a75bad9 Depends-On: https://review.opendev.org/c/openstack/ceilometer/+/852021
* | | Merge "Add support to host command dynamic pollster definitions"Zuul2022-10-112-128/+654
|\ \ \ | |/ /
| * | Add support to host command dynamic pollster definitionsPedro Henrique2022-09-132-128/+654
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem description =================== Today we have some hardcoded pollsters that are gathering data from running virtual machines through libvirt or different programs running in the compute nodes. However, the Dynamic pollster definition does not support this kind of operations to gather data, it only supports HTTP Rest requests to collect data. Therefore, it is not possible to use the dynamic pollster definition to create a YML based pollster that runs and collects data from Libvirt in the compute nodes. Proposal ======== To allow host commands/scripts in the Dynamic pollsters, we propose to add a new pollster definition using the `os.subprocess` lib to run host commands to collect Host/VMs data and store them in the configured backend. This will provide more flexibility and make the Dynamic pollsters able to be used in Ceilometer compute instances as well. Change-Id: I50b8dc341ce457780416b41d138e35f5a0d083b6 Depends-On: https://review.opendev.org/c/openstack/ceilometer/+/850253
* | | Merge "Add response handlers to support different response types"Zuul2022-10-114-12/+257
|\ \ \ | |/ /
| * | Add response handlers to support different response typesPedro Henrique2022-09-014-12/+257
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem description =================== The dynamic pollsters only support APIs that produce JSON responses. Therefore the dynamic pollsters do not support APIs where the response is an XML or not Restful compliant APIs with HTTP 200 within a plain text message on errors. Proposal ======== To allow the dynamic pollsters to support other APIs response formats, we propose to add a response handling that supports multiple response types. It must be configurable in the dynamic pollsters YAML. The default continues to be JSON. Change-Id: I4886cefe06eccac2dc24adbc2fad2166bcbfdd2c
* | | Merge "Add user/project names to polled samples"Zuul2022-09-305-2/+132
|\ \ \
| * | | Add user/project names to polled samplesYadnesh Kulkarni2022-09-135-2/+132
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Project and user names would be first fetched from cache, if not found, they will be requested from keystone and then cached. Using cache will significanlty reduce the number of calls made to keystone. If ceilometer is configured with no caching backend, the results will always be fetched by querying request to keystone. A new config option, `tenant_name_discovery` is introduced to operate this feature. This feature is optional and is disabled by default. No attempts to identify names will be made if uuids are found to be `None`. Signed-off-by: Yadnesh Kulkarni <ykulkarn@redhat.com> Change-Id: Iee5dbf09a1fd3ac571746fc66d2683eb8e6a1b27
* | | | Merge "Add support to namespaces on dynamic pollsters"Zuul2022-09-292-2/+91
|\ \ \ \
| * | | | Add support to namespaces on dynamic pollstersPedro Henrique2022-09-162-2/+91
| | |_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem description =================== The hardcoded pollsters are defined by namespaces, so they are instantied based on the namespaces provided to the 'AgentManager'. The dynamic pollsters, on the other hand, are always instantied, independent of the provided namespaces. Proposal ======== To allow operators to define in which namespaces the dynamic pollster will be deployed, we propose to add a new configuration 'namespaces' in the dynamic pollsters yaml configuration. This configuration will support a single entry or a list of the namespaces that the pollster must be instantiated. Change-Id: I39ba0c3dd312a0601e02f8cfcab7a44e585a8a7f
* | | | Merge "Remove unnecessary logic for missing oslo.cache"Zuul2022-09-291-3/+1
|\ \ \ \ | |_|/ / |/| | |
| * | | Remove unnecessary logic for missing oslo.cacheTakashi Kajinami2022-09-051-3/+1
| | |/ | |/| | | | | | | | | | | | | | | | | | | The oslo.cache library is now included in the requirements thus should be always installed. This change removes the logic to handle missing oslo.cache. Change-Id: I1cc09aa0db5f19e913030b528c51052fd4ef331d
* | | Fix Non-OpenStack dynamic pollster credentials handlingRafael Weingärtner2022-09-071-0/+14
| |/ |/| | | | | | | | | | | | | | | | | There was a problem with the barbican credentials processing for Non-OpenStack dynamic pollster. The credentials variables would be retrieved as a byte string, which would break the processing. Therefore, we needed to introduce a method to check if the credentials are a String object. if they are not, we convert them to String. Change-Id: I2084061eb8f543d4c557963599732e35cfa22996
* | Imported Translations from ZanataOpenStack Proposal Bot2022-08-2311-325/+11
| | | | | | | | | | | | | | For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I48d1633a24b0ccf1e6a7517956ca7f2dbe2fdb15
* | Merge "Remove unicode prefixes"Zuul2022-08-2212-495/+495
|\ \
| * | Remove unicode prefixesTakashi Kajinami2022-08-0712-495/+495
| |/ | | | | | | | | | | | | A unicode prefix is meaningless in Python 3. Because now ceilometer supports only Python 3, we can remove the prefix. Change-Id: I7bc91be21df646d8bbc7793eec28a93179a3eefa
* | Merge "Replace abc.abstractproperty with property and abc.abstractmethod"Zuul2022-08-225-17/+34
|\ \
| * | Replace abc.abstractproperty with property and abc.abstractmethodljhuang2022-08-035-17/+34
| |/ | | | | | | | | | | | | | | | | Replace abc.abstractproperty with property and abc.abstractmethod, as abc.abstractproperty has been deprecated since python3.3[1] [1]https://docs.python.org/3.8/whatsnew/3.3.html?highlight=deprecated#abc Change-Id: Ie96896399304c6ac1a3f38900aa6695d56f2c5ca
* | Merge "Remove GenericHardwareDeclarativePollster"Zuul2022-08-1220-1845/+0
|\ \ | |/ |/|
| * Remove GenericHardwareDeclarativePollsterTakashi Kajinami2022-07-0120-1845/+0
| | | | | | | | | | | | | | | | | | ... and discovery/inspector plugins depending on the pollster. These were implemented to gather metrics via SNMP daemon in TripleO-managed deployment but these are no longer valid since Telemetry services and Nova were removed from undercloud. Change-Id: If9a6b695ba799c766314a88328ea8a779407acc0
* | Merge "Remove support for neutron-lbaas"Zuul2022-07-0410-2190/+1
|\ \
| * | Remove support for neutron-lbaasTakashi Kajinami2022-06-0810-2190/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... because it is of no use since neutron-lbaas was retired and was replaced by Octavia. This feature was officially deprecated during Yoga cycle[1] so we are ready to remove it. [1] 8917c73964eb764c3c4beb65d3713b36938181dd Change-Id: Ic145c23cc0b0372ef78f4a45ffb084bec24936c3
* | | Merge "Remove [coordination] check_watchers"Zuul2022-07-041-8/+1
|\ \ \ | |_|/ |/| |
View Lorry Controller Status