Data Access data, datasets, and views
All the time-series data that Zenoss Cloud gathers or is sent from your environment is stored in BiqQuery tables that are partitioned by timestamp.
- The metrics table contains all metric data points.
- The event occurrences table contains all event context data.
- Separate metadata tables contains all entity, metric, event, and anomaly metadata updates.
An additional, regular table contains all metric entries in your Zenoss Cloud dictionary.
Warning
Tables partitioned by timestamp contain millions of records! Always scope your queries by time to avoid unexpected query costs.
You access your data through views. Views enable secure, read-only access to your data tables and encapsulate the domain knowledge required to query Zenoss Cloud data efficiently. Through the views that Zenoss Cloud provides, you can begin using your data quickly.
The easiest way to explore the Data Access views is through Google BigQuery. In BigQuery, each view includes a brief description of its purpose and results and the query that defines the view. And you can use BigQuery to review schemas and perform queries.
The following sections describe the zenoss_view and zenoss_example datasets, which contain your views, and the zenoss_udf dataset, which contains helpful utility functions.
The zenoss_view dataset
The zenoss_view dataset contains the primary, general-purpose views of your data. The views in the zenoss_view_daily, zenoss_view_weekly, and zenoss_view_monthly datasets are modeled on the views in the zenoss_view dataset and so are not documented separately.
Note
The daily datasets (zenoss_view_daily and zenoss_example_daily) and the weekly datasets (zenoss_view_weekly and zenoss_example_weekly) are provided to reduce query development costs. The datasets search through the most recent 31 days of metadata and the most recent day or week of metric data.
Relationships diagram
The following diagram shows the relationships among the views in the zenoss_view dataset and the BigQuery tables in Zenoss Cloud.
View descriptions
The following sections summarize each view in the zenoss_view dataset. For additional details, use BigQuery to display the SQL query that defines a view.
Warning
None of the views in the zenoss_view dataset are constrained by time. Be careful!
metrics
The metrics view returns the contents of the metrics table—the raw data. To make sense of it, you need to join the data with records from a metadata view and the data_dictionary view, which is done for you in the metrics_full view.
data_dictionary
The data_dictionary view returns the contents of the dictionary table, which contains metric dictionary entries.
metadata
The metadata view returns the contents of all metadata tables. The following columns contain values unique to Zenoss Cloud.
| Column | Value | Description |
|---|---|---|
| type |
ENTITY |
An object that represents any discrete system, process, or feature for which Zenoss Cloud gathers or is sent time series data. See Entities and devices. |
EVENT |
A structured message that is sent to the event processing service. | |
METRIC |
A measurement that quantifies an entity property or phenomenon. | |
| operation | DELETE |
The object no longer exists as of the timestamp on the record. |
MERGE |
Add the values in the record to the object's state, or replace existing values. | |
REPLACE |
Replace the object's state with the values in the record. | |
| dimensions | (array) | The key-value pairs that uniquely identify an entity. See Using entities. |
| metadata | (array) | For entities, all the key-value pairs that are not used as dimensions. See Using entities. For metrics, the key-value pairs associated with the metric. For events, the key-value pairs associated with the event. |
This view is not used by any other view in the zenoss_view dataset.
metadata_current
Like the metadata view, the metadata_current view returns the contents of all metadata tables, but with duplicate records removed.
This view is not used by any other view in the zenoss_view dataset.
metric_metadata
The metric_metadata view returns only metric metadata, with duplicate records removed.
metric_info
The metric_info view returns a join of the data_dictionary and metric_metadata views. The result is similar to the metric information that is displayed in dashboards and Smart View, less the metrics themselves.
metrics_full
The metrics_full view joins the data in the metric_info view with the data in the metrics view. The result is similar to the metric information that is displayed in dashboards and Smart View.
cz_events
The cz_events view returns data about events, with duplicate records removed.
The source column identifies the name of the originating Collection Zone.
cz_entities
The cz_entities view returns data about all the entities in your environment,
with duplicate records removed. The source column identifies the name of
the originating Collection Zone or streaming data agent.
cz_devices
The cz_devices view refines the data from the cz_entities view to return
all the Collection Zone devices in your environment. The
source column identifies the name of the originating Collection Zone.
cz_components
The cz_components view refines the data from the cz_entities view to
return all the Collection Zone components in your environment. The
source column identifies the name of the originating Collection Zone.
cz_occurrences
The cz_occurrences view provides event metadata for events that originate
in and are unique to Collection Zones. This includes fields such as
ContextUuid, ParentContextUuid, and Fingerprint.
occurrences
The occurrences view provides the event occurrences.
occurrence_full
The occurrence_full view adds the labels from the
event fields dictionary
to the contents of the occurrences view.
occurrence_metadata
The occurrence_metadata view provides event occurrence metadata.
The zenoss_example dataset
The zenoss_example dataset contains examples of how to create views for specific reporting purposes. The views in the zenoss_example_daily, zenoss_example_weekly, and zenoss_example_monthly datasets differ only by the addition of specific time constraints.
| View | Description |
|---|---|
| cz_entity_metrics | Combines the zenoss_view/cz_entities and zenoss_view/metrics_full views to return metrics by entity. |
| cz_entity_events | Combines the zenoss_view/cz_entities and zenoss_view/cz_events views to return events by entity. |
| cz_device_metrics | Refines the cz_entity_metrics view to return metrics by device. |
| cz_device_events | Refines the cz_entity_events view to return events by device. |
| cz_component_metrics | Refines the cz_entity_metrics view to return metrics by component. |
| cz_component_events | Refines the cz_entity_events view to return events by component. |
| cz_occurrences | Combines event occurrence with the event to show the cz specific set of data for an event |
| occurrence_metadata | Combines the event occurrence (occurrences dataset) with the event metadata (metadata_event dataset) for each occurrence |
| occurrences | Combines event occurrence with the event to show a basic set of data for any event |
The zenoss_udf dataset
The zenoss_udf dataset contains utility functions that simplify the creation and maintenance of queries.
Info
The dedup_keys_aggregate_values_array function is reserved
for Zenoss use only.
xtract_kv_struct
- Syntax
extract_kv_struct(array,string)- Calling arguments
- An array of key-value pairs (dimensions or metadata) and a string.
- Return values
-
An array of key-value pairs in which all keys match the supplied string.
The array may include just one item.
extract_kv_value_array
- Syntax
extract_kv_value_array(array,string)- Calling arguments
- An array of key-value pairs (dimensions or metadata) and a string.
- Return values
-
An array of strings in which all values in the key-value pairs match the supplied string.
The array may include just one item.
extract_kv_value_string
- Syntax
extract_kv_value_string(array,string)- Calling arguments
- An array of key-value pairs (dimensions or metadata) and a string.
- Return values
- The first value in the supplied key-value pairs that matches the supplied string.
extract_last_string
- Syntax
extract_last_string(string)- Calling arguments
- A string; the path of a group or system.
- Return values
-
The portion of the supplied string that follows the last solidus character (
/).If no solidus is found, returns the supplied string.
is_component
- Syntax
is_component(array,string,string)- Calling arguments
- An array of key-value pairs and two strings; typically, the metadata, operation, and source columns from zenoss_view/cz_entities.
- Return values
Trueif the arguments represent a Collection Zone component,Falseif they do not.
is_device
- Syntax
is_device(array,string,string)- Calling arguments
- An array of key-value pairs and two strings; typically, the metadata, operation, and source columns from zenoss_view/cz_entities.
- Return values
Trueif the arguments represent a Collection Zone device,Falseif they do not.
split_metric_name
- Syntax
split_metric_name(string)- Calling arguments
- A string; the name of a metric.
- Return values
-
The portion of the supplied string that follows the last solidus character (
/).If no solidus is found, returns the supplied string.
split_occurrence_id
- Syntax
split_occurrence_id(string)- Calling arguments
- A string; the ID of an event occurrence.
- Return values
-
The portion of the supplied string that precedes the colon character (
:).If no colon is found, returns the supplied string.