action.skip

Data Access data, datasets, and views

All the time-series data that Zenoss Cloud gathers or is sent from your environment is stored in BiqQuery tables that are partitioned by timestamp.

  • The metrics table contains all metric data points.
  • The event occurrences table contains all event context data.
  • Separate metadata tables contains all entity, metric, event, and anomaly metadata updates.

An additional, regular table contains all metric entries in your Zenoss Cloud dictionary.

Warning

Tables partitioned by timestamp contain millions of records! Always scope your queries by time to avoid unexpected query costs.

You access your data through views. Views enable secure, read-only access to your data tables and encapsulate the domain knowledge required to query Zenoss Cloud data efficiently. Through the views that Zenoss Cloud provides, you can begin using your data quickly.

The easiest way to explore the Data Access views is through Google BigQuery. In BigQuery, each view includes a brief description of its purpose and results and the query that defines the view. And you can use BigQuery to review schemas and perform queries.

The following sections describe the zenoss_view and zenoss_example datasets, which contain your views, and the zenoss_udf dataset, which contains helpful utility functions.

The zenoss_view dataset

The zenoss_view dataset contains the primary, general-purpose views of your data. The views in the zenoss_view_daily, zenoss_view_weekly, and zenoss_view_monthly datasets are modeled on the views in the zenoss_view dataset and so are not documented separately.

Note

The daily datasets (zenoss_view_daily and zenoss_example_daily) and the weekly datasets (zenoss_view_weekly and zenoss_example_weekly) are provided to reduce query development costs. The datasets search through the most recent 31 days of metadata and the most recent day or week of metric data.

Relationships diagram

The following diagram shows the relationships among the views in the zenoss_view dataset and the BigQuery tables in Zenoss Cloud.

View descriptions

The following sections summarize each view in the zenoss_view dataset. For additional details, use BigQuery to display the SQL query that defines a view.

Warning

None of the views in the zenoss_view dataset are constrained by time. Be careful!

metrics

The metrics view returns the contents of the metrics table—the raw data. To make sense of it, you need to join the data with records from a metadata view and the data_dictionary view, which is done for you in the metrics_full view.

data_dictionary

The data_dictionary view returns the contents of the dictionary table, which contains metric dictionary entries.

metadata

The metadata view returns the contents of all metadata tables. The following columns contain values unique to Zenoss Cloud.

Column Value Description
type

ENTITY An object that represents any discrete system, process, or feature for which Zenoss Cloud gathers or is sent time series data. See Entities and devices.
EVENT A structured message that is sent to the event processing service.
METRIC A measurement that quantifies an entity property or phenomenon.
operation DELETE The object no longer exists as of the timestamp on the record.
MERGE Add the values in the record to the object's state, or replace existing values.
REPLACE Replace the object's state with the values in the record.
dimensions (array) The key-value pairs that uniquely identify an entity. See Using entities.
metadata (array)

For entities, all the key-value pairs that are not used as dimensions. See Using entities.

For metrics, the key-value pairs associated with the metric.

For events, the key-value pairs associated with the event.

This view is not used by any other view in the zenoss_view dataset.

metadata_current

Like the metadata view, the metadata_current view returns the contents of all metadata tables, but with duplicate records removed.

This view is not used by any other view in the zenoss_view dataset.

metric_metadata

The metric_metadata view returns only metric metadata, with duplicate records removed.

metric_info

The metric_info view returns a join of the data_dictionary and metric_metadata views. The result is similar to the metric information that is displayed in dashboards and Smart View, less the metrics themselves.

metrics_full

The metrics_full view joins the data in the metric_info view with the data in the metrics view. The result is similar to the metric information that is displayed in dashboards and Smart View.

cz_events

The cz_events view returns data about events, with duplicate records removed. The source column identifies the name of the originating Collection Zone.

cz_entities

The cz_entities view returns data about all the entities in your environment, with duplicate records removed. The source column identifies the name of the originating Collection Zone or streaming data agent.

cz_devices

The cz_devices view refines the data from the cz_entities view to return all the Collection Zone devices in your environment. The source column identifies the name of the originating Collection Zone.

cz_components

The cz_components view refines the data from the cz_entities view to return all the Collection Zone components in your environment. The source column identifies the name of the originating Collection Zone.

cz_occurrences

The cz_occurrences view provides event metadata for events that originate in and are unique to Collection Zones. This includes fields such as ContextUuid, ParentContextUuid, and Fingerprint.

occurrences

The occurrences view provides the event occurrences.

occurrence_full

The occurrence_full view adds the labels from the event fields dictionary to the contents of the occurrences view.

occurrence_metadata

The occurrence_metadata view provides event occurrence metadata.

The zenoss_example dataset

The zenoss_example dataset contains examples of how to create views for specific reporting purposes. The views in the zenoss_example_daily, zenoss_example_weekly, and zenoss_example_monthly datasets differ only by the addition of specific time constraints.

View Description
cz_entity_metrics Combines the zenoss_view/cz_entities and zenoss_view/metrics_full views to return metrics by entity.
cz_entity_events Combines the zenoss_view/cz_entities and zenoss_view/cz_events views to return events by entity.
cz_device_metrics Refines the cz_entity_metrics view to return metrics by device.
cz_device_events Refines the cz_entity_events view to return events by device.
cz_component_metrics Refines the cz_entity_metrics view to return metrics by component.
cz_component_events Refines the cz_entity_events view to return events by component.
cz_occurrences Combines event occurrence with the event to show the cz specific set of data for an event
occurrence_metadata Combines the event occurrence (occurrences dataset) with the event metadata (metadata_event dataset) for each occurrence
occurrences Combines event occurrence with the event to show a basic set of data for any event

The zenoss_udf dataset

The zenoss_udf dataset contains utility functions that simplify the creation and maintenance of queries.

Info

The dedup_keys_aggregate_values_array function is reserved for Zenoss use only.

xtract_kv_struct

Syntax
extract_kv_struct(array,string)
Calling arguments
An array of key-value pairs (dimensions or metadata) and a string.
Return values

An array of key-value pairs in which all keys match the supplied string.

The array may include just one item.

extract_kv_value_array

Syntax
extract_kv_value_array(array,string)
Calling arguments
An array of key-value pairs (dimensions or metadata) and a string.
Return values

An array of strings in which all values in the key-value pairs match the supplied string.

The array may include just one item.

extract_kv_value_string

Syntax
extract_kv_value_string(array,string)
Calling arguments
An array of key-value pairs (dimensions or metadata) and a string.
Return values
The first value in the supplied key-value pairs that matches the supplied string.

extract_last_string

Syntax
extract_last_string(string)
Calling arguments
A string; the path of a group or system.
Return values

The portion of the supplied string that follows the last solidus character (/).

If no solidus is found, returns the supplied string.

is_component

Syntax
is_component(array,string,string)
Calling arguments
An array of key-value pairs and two strings; typically, the metadata, operation, and source columns from zenoss_view/cz_entities.
Return values
True if the arguments represent a Collection Zone component, False if they do not.

is_device

Syntax
is_device(array,string,string)
Calling arguments
An array of key-value pairs and two strings; typically, the metadata, operation, and source columns from zenoss_view/cz_entities.
Return values
True if the arguments represent a Collection Zone device, False if they do not.

split_metric_name

Syntax
split_metric_name(string)
Calling arguments
A string; the name of a metric.
Return values

The portion of the supplied string that follows the last solidus character (/).

If no solidus is found, returns the supplied string.

split_occurrence_id

Syntax
split_occurrence_id(string)
Calling arguments
A string; the ID of an event occurrence.
Return values

The portion of the supplied string that precedes the colon character (:).

If no colon is found, returns the supplied string.