Dynamic pollsters: enable operation on attributes

This PR enables the use of python operation to transform the attributes that
 are extracted in the JSON response that the Dynamic pollster handle. By
 enabling operators to define transformations on the fly, we can provide even
 more flexibility for the dynamic pollsters' system.

One example of use case is the RadosGW that uses <project_id$project_id> as the
 username. With this implementation, one can create configurations in the
 dynamic pollster to clean that variable. It is as simple as defining
 `resource_id_attribute: "user | value.split('$')[0].strip()"`

The operations are separated by `|` symbol. The first element of the expression
 is the key to be retrieved from the JSON object. The other elements are
 operations that can be applied to the `value` variable. The value variable is
 the variable we use to hold the data.

Depends-On: https://review.opendev.org/#/c/693088/
Change-Id: I9ee209410491b3f04259e1a5c62ac20461070ae1
Signed-off-by: Rafael Weingärtner <rafael@apache.org>

1 files changed, 50 insertions, 3 deletions

diff --git a/doc/source/admin/telemetry-dynamic-pollster.rst b/doc/source/admin/telemetry-dynamic-pollster.rst
index 72421f78..2c829836 100644
--- a/doc/source/admin/telemetry-dynamic-pollster.rst
+++ b/doc/source/admin/telemetry-dynamic-pollster.rst
@@ -24,9 +24,6 @@ dynamic pollster system:
    fashion. This feature is "a nice" to have, but is currently not
    implemented.
 
-*  APIs that return a list of entries directly, without a first key for the
-   list. An example is Aodh alarm list.
-
 
 The dynamic pollsters system configuration (for OpenStack APIs)
 ---------------------------------------------------------------
@@ -329,3 +326,53 @@ ones), we can use the `successful_ops`.
   project_id_attribute: "user"
   resource_id_attribute: "user"
   response_entries_key: "summary"
+
+Operations on extracted attributes
+----------------------------------
+
+The dynamic pollster system can execute Python operations to transform the
+attributes that are extracted from the JSON response that the system handles.
+
+One example of use case is the RadosGW that uses <project_id$project_id> as the
+username (which is normally mapped to the Gnocchi resource_id). With this
+feature (operations on extracted attributes), one can create configurations in
+the dynamic pollster to clean/normalize that variable. It is as simple as
+defining `resource_id_attribute: "user | value.split('$')[0].strip()"`
+
+The operations are separated by `|` symbol. The first element of the expression
+is the key to be retrieved from the JSON object. The other elements are
+operations that can be applied to the `value` variable. The value variable
+is the variable we use to hold the data being extracted. The previous
+example can be rewritten as:
+`resource_id_attribute: "user | value.split ('$') | value[0] | value.strip()"`
+
+As follows we present a complete configuration for a RadosGW dynamic
+pollster that is removing the `$` symbol, and getting the first part of the
+String.
+
+.. code-block:: yaml
+
+  ---
+
+  - name: "dynamic.radosgw.api.request.successful_ops"
+  sample_type: "gauge"
+  unit: "request"
+  value_attribute: "total.successful_ops"
+  url_path: "http://rgw.service.stage.i.ewcs.ch/admin/usage"
+  module: "awsauth"
+  authentication_object: "S3Auth"
+  authentication_parameters: "<access_key>, <secret_key>,
+  <rados_gateway_server>"
+  user_id_attribute: "user | value.split ('$') | value[0]"
+  project_id_attribute: "user | value.split ('$') | value[0]"
+  resource_id_attribute: "user | value.split ('$') | value[0]"
+  response_entries_key: "summary"
+
+The Dynamic pollster configuration options that support this feature are the
+following:
+
+* value_attribute
+* response_entries_key
+* user_id_attribute
+* project_id_attribute
+* resource_id_attribute