Ian Wienand 2395651429 Allow for override of statsd/influxdb settings per cloud

There is currently no way to separate out statsd settings per cloud.
While you can set a prefix other than 'openstack.api', this applies to
all clouds and groups all the stats together.

While this may be what you want, it may also be not what you want :)
For example in nodepool, we have multiple cloud providers who each
have their own grafana page, so we'd like them all to log themselves
to different stats buckets (i.e. set individual prefixes).

This allows setting a "metrics" field in each individual cloud entry
that overrides the global settings.  TBH I don't know if this is
important for InfluxDB as well, but I've implemented it for that too
for consistency.

Test cases are added for the global and merged settings.

I've also updated the documentation page a bit.  I've separated it
into subsections for the three types of stats available.  I removed
some of the in-depth stuff about logging types which wasn't that clear
(I think we can just present what we support and let people decide).

Change-Id: I9c3342161a257603f3cfd26bd03f6b71ffbfdd0d

2021-04-20 08:36:10 +10:00

2.4 KiB

Raw Blame History

Statistics reporting

openstacksdk can report statistics on individual API requests/responses in several different formats.

Note that metrics will be reported only when corresponding client libraries (statsd for 'statsd' reporting, influxdb for influxdb, etc.). If libraries are not available reporting will be silently ignored.

statsd

statsd can be configured via configuration entries or environment variables.

A global metrics entry defines defaults for all clouds. Each cloud can specify a metrics section to override variables; this may be useful to separate results reported for each cloud.

metrics:
  statsd:
    host: __statsd_server_host__
    port: __statsd_server_port__
    prefix: __statsd_prefix__ (default 'openstack.api')
clouds:
  a-cloud:
    auth:
     ...
    metrics:
      statsd:
        prefix: 'openstack.api.a-cloud'

If the STATSD_HOST or STATSD_PORT environment variables are set, they will be taken as the default values (and enable statsd reporting if no other configuration is specified).

InfluxDB

InfluxDB is supported via configuration in the metrics field. Similar to statsd, each cloud can provide it's own metrics section to override any global defaults.

metrics:
  influxdb:
    host: __influxdb_server_host__
    port: __influxdb_server_port__
    use_udp: __True|False__
    username: __influxdb_auth_username__
    password: __influxdb_auth_password__
    database: __influxdb_db_name__
    measurement: __influxdb_measurement_name__
    timeout: __infludb_requests_timeout__
clouds:
  ..

InfluxDB reporting allows setting additional tags into the metrics based on the selected cloud.

clouds:
  my_cloud:
    profile: some_profile
    ...
    additional_metric_tags:
      environment: production

prometheus

The prometheus support does not read from config, and does not run an http service since OpenstackSDK is a library. It is expected that an application that uses OpenstackSDK and wants request stats be collected will pass a prometheus_client.CollectorRegistry to collector_registry.

2.4 KiB Raw Blame History

Statistics reporting

statsd

InfluxDB

prometheus

2.4 KiB

Raw Blame History