Surfacers

One of the biggest strengths of cloudprober is that it can export data to multiple monitoring systems, even simultaneously, just based on simple configuration. Cloudprober does that using a built-in mechanism, called surfacers. Each surfacer type implements interface for a specific monitoring system, for example, pubsub surfacer publishes data to Google Pub/Sub. You can configure multiple surfacers at the same time. If you don’t specify any surfacer, prometheus and file surfacers are enabled automatically.

Why other monitoring systems? Cloudprober’s main purpose is to run probes and build standard, usable metrics based on the results of those probes. It doesn’t take any action on the generated data. Instead, it provides an easy interface to make that probe data available to systems that provide ways to consume monitoring data, for example for graphing and alerting.

Cloudprober currently supports following surfacer types:

• Prometheus (config)
• Stackdriver (Google Cloud Monitoring)
• Google Pub/Sub (config)
• Postgres (config)
• File (config)
• Cloudwatch (AWS Cloud Monitoring)

Source: surfacers config.

It’s easy to add more surfacers without having to understand the internals of cloudprober. You only need to implement the Surfacer interface.

Configuration #

Adding surfacers to cloudprober is as easy as adding “surfacer” config stanzas to your config, like the following:

# Enable prometheus and stackdriver surfacers.

# Make probe metrics available at the URL :<cloudprober_port>/metrics, for
# scraping by prometheus.
surfacer {
  type: PROMETHEUS

  prometheus_surfacer {
    # Following option adds a prefix to exported metrics, for example,
    # "total" metric is exported as "cloudprober_total".
    metrics_prefix: "cloudprober_"
  }
}

# Stackdriver (Google Cloud Monitoring) surfacer. No other configuration
# is necessary if running on GCP.
surfacer {
  type: STACKDRIVER
}

Filtering Metrics #

It is possible to filter the metrics that the surfacers receive.

Filtering by Label #

Cloudprober can filter the metrics that are published to surfacers. To filter metrics by labels, reference one of the following keys in the surfacer configuration:

• allow_metrics_with_label
• ignore_metrics_with_label

Note: ignore_metrics_with_label takes precedence over allow_metrics_with_label.

For example, to ignore all sysvar metrics:

surfacer {
  type: PROMETHEUS

  ignore_metrics_with_label {
    key: "probe",
    value: "sysvars",
  }
}

Or to only allow metrics from http probes:

surfacer {
  type: PROMETHEUS

  allow_metrics_with_label {
    key: "ptype",
    value: "http",
  }
}

Filtering by Metric Name #

For certain surfacers, cloudprober can filter the metrics that are published by name. The surfacers that support this functionality are:

• Cloudwatch
• Prometheus
• Stackdriver

Within the surfacer configuration, the following options are defined:

• allow_metrics_with_name
• ignore_metrics_with_name

Note: ignore_metrics_with_name takes precedence over allow_metrics_with_name.

To filter out all validation_failure metrics by name:

surfacer {
  type: PROMETHEUS

  ignore_metrics_with_name: "validation_failure"
}

(Source: https://github.com/cloudprober/cloudprober/blob/master/surfacers/proto/config.proto)