Data Access data, datasets, and views
You can access your data through BigQuery views provided by Zenoss. Views are SQL templates that fetch your data and define it for you in your schema. Views allow for secure, read-only access to your data and contain the domain knowledge that is needed to query Zenoss Cloud data efficiently.
Zenoss provides multiple views in datasets so you can begin using your data quickly, including the default Zenoss View (zenoss_view) dataset and the Zenoss History View (zenoss_history_view) dataset. The views in these datasets consist of the time-series data that Zenoss Cloud gathers from your environment.
The Zenoss View dataset allows you to focus on particular types of data that you may want to analyze, such as events or anomalies. The Zenoss History View dataset is useful to investigate the history of specific data elements, such as how properties have changed over time. For example, you can use the history view to see how long an entity was in maintenance over a specified time range.
About BigQuery and compute costs
BigQuery is a serverless data analytics platform that can automatically allocate computing resources as you need them or reserve compute capacity ahead of time. When you are using data access views, you are processing SQL queries and therefore incurring charges in your Google project. As a best practice, consult with your Google Cloud administrator when using data access views.
The underlying BigQuery tables that store the actual data are partitioned by timestamp. Each table can contain millions or even billions of rows. To help avoid unexpected query costs, use the view with the minimum time range that can meet your goals. Always scope your queries by time. For more information, see BigQuery pricing.
BigQuery and views
Each BigQuery view provided by Zenoss includes a brief description of its purpose, details of columns exposed in the view, and the query that defines the view. You can use BigQuery Studio to review schemas and perform queries.
You can explore various views, including:
- Metrics view: Contains all metric data points.
- Event occurrences view: Contains all event context data.
- Separate metadata views: Contain all entity, metric, event, and anomaly metadata updates.
- A data dictionary view: Contains all metric entries in your Zenoss Cloud dictionary.
To successfully query your data in Data Access, you should have knowledge of BigQuery syntax.
Datasets and views
Let’s take a closer look at the datasets that Zenoss provides:
-
Zenoss view (zenoss_view): Contains primary, general-purpose views of your data.
-
Zenoss history view (zenoss_history_view): Contains historical views of entity and event data not available in the zenoss_view.
-
Zenoss example view (zenoss_example): Contains examples of how to create views for specific reporting purposes.
-
Zenoss UDF view (zenoss_udf): Contains helpful, user-defined utility functions. As a best practice, use these provided functions instead of defining your own.
The Zenoss View and Zenoss History View datasets contain views for daily, weekly, and monthly analysis. Zenoss provides these time-bounded datasets to help reduce your query costs. The datasets search through the most recent 31 days of metadata and the most recent day or week of metric data. The daily, weekly, and monthly views are modeled on their “parent” view, as follows:
| Datasets | Modeled on |
|---|---|
| - zenoss_view_daily - zenoss_view_weekly - zenoss_view_monthly |
zenoss_view |
| - zenoss_history_view_daily - zenoss_history_view_weekly - zenoss_history_view_monthly |
zenoss_history_view |
| - zenoss_example_daily - zenoss_example_weekly - zenoss_example_monthly |
zenoss_example |
Zenoss view dataset
The zenoss_view dataset contains the primary, general-purpose views of your data. The views in the zenoss_view_daily, zenoss_view_weekly, and zenoss_view_monthly datasets differ only by the addition of specific time constraints.
The following diagram shows the relationships among the views in the zenoss_view dataset and the BigQuery tables in Zenoss Cloud.
The following table summarizes each view in the zenoss_view dataset. For additional details, use BigQuery to display the SQL query that defines a view.
Warning
Use caution: None of the views in the zenoss_view dataset are constrained by time.
| View | Description |
|---|---|
| metrics | The metrics view returns the contents of the metric data as data. To make sense of it, you need to join the data with records from a metadata view and the data_dictionary view, which is done for you in the metrics_full view. |
| data_dictionary | The data_dictionary view returns the contents of the dictionary, which contains metric dictionary entries. |
| metadata | The metadata view returns the contents of all metadata. This view is not used by any other view in the zenoss_view dataset. For more information about the columns and values in the metadata view, see Metadata view |
| metadata_current | Like the metadata view, the metadata_current view returns the contents of all metadata, but with duplicate records removed. This view is not used by any other view in the zenoss_view dataset. |
| metric_metadata | The metric_metadata view returns only metric metadata with duplicate records removed. |
| metric_info | The metric_info view returns a join of the data_dictionary and metric_metadata views. The result is similar to the metric information that is displayed in dashboards and Smart View without the metrics themselves. |
| metrics_full | The metrics_full view joins the data in the metric_info view with the data in the metrics view. The result is similar to the metric information that is displayed in dashboards and Smart View. |
| cz_events | The cz_events view returns data about events with duplicate records removed. The source column identifies the name of the originating Collection Zone. |
| cz_entities | The cz_entities view returns data about all the entities in your environment with duplicate records removed. The source column identifies the name of the originating Collection Zone or streaming data agent. |
| cz_devices | The cz_devices view refines the data from the cz_entities view to return all the Collection Zone devices in your environment. The source column identifies the name of the originating Collection Zone. |
| cz_components | The cz_components view refines the data from the cz_entities view to return all the Collection Zone components in your environment. The source column identifies the name of the originating Collection Zone. |
| cz_occurrences | The cz_occurrences view provides event metadata for events that originate in and are unique to Collection Zones. It includes fields such as Summary, Severity, and eventClass. |
| occurrences | The occurrences view provides the event occurrences. |
| occurrence_metadata | The occurrence_metadata view merges any available dictionary labels with the occurrence metadata fields. |
Metadata view
The metadata view returns the contents of all metadata. This view is not used by any other view in the zenoss_view dataset.
The following columns contain values unique to Zenoss Cloud.
| Column | Value | Description |
|---|---|---|
| type |
ENTITY |
An object that represents any discrete system, process, or feature for which Zenoss Cloud gathers or is sent time series data. See Entities and devices. |
EVENT |
A structured message that is sent to the event processing service. | |
METRIC |
A measurement that quantifies an entity property or phenomenon. | |
| operation | DELETE |
The object no longer exists as of the timestamp on the record. |
MERGE |
Add the values in the record to the object's state, or replace existing values. | |
REPLACE |
Replace the object's state with the values in the record. | |
| dimensions | (array) | The key-value pairs that uniquely identify an entity. See Using entities. |
| metadata | (array) | For entities, all the key-value pairs that are not used as dimensions. See Using entities. For metrics, the key-value pairs associated with the metric. For events, the key-value pairs associated with the event. |
Zenoss history view dataset
Provides historical views of entity and event data not available in the zenoss_view. The views in the zenoss_history_view_daily, zenoss_history_view_weekly, and zenoss_history_view_monthly datasets differ only by the addition of specific time constraints.
The following table summarizes each view in the zenoss_history_view dataset.
Warning
Use caution. None of the views in the zenoss_history_view dataset are constrained by time.
| View | Description |
|---|---|
| metadata | The metadata view returns the contents of all metadata. This view is not used by any other view in the zenoss_view dataset. For more information about the columns and values in the metadata view, see Metadata view |
| cz_events | The cz_events view returns data about events. The source column identifies the name of the originating Collection Zone. |
| cz_entities | The cz_entities view returns data about all the entities in your environment. The source column identifies the name of the originating Collection Zone. |
| entities | The entities view returns data about all the entities in your environment. The source column identifies the name of the originating Collection Zone or streaming data agent. |
| cz_devices | The cz_devices view refines the data from the cz_entities view to return all the Collection Zone devices in your environment. The source column identifies the name of the originating Collection Zone. |
| cz_components | The cz_components view refines the data from the cz_entities view to return all the Collection Zone components in your environment. The source column identifies the name of the originating Collection Zone. |
Zenoss example dataset
The zenoss_example dataset contains examples of how to create views for specific reporting purposes. The views in the zenoss_example_daily, zenoss_example_weekly, and zenoss_example_monthly datasets differ only by the addition of specific time constraints. The following table summarizes each view in the zenoss_example_view dataset.
| View | Description |
|---|---|
| cz_component_events | Refines the cz_entity_events view to return events by component. |
| cz_component_metrics | Refines the cz_entity_metrics view to return metrics by component. |
| cz_component_occurrences | Refines the cz_entity_events view to return events by component. |
| cz_device_events | Refines the cz_entity_events view to return events by device. |
| cz_device_metrics | Refines the cz_entity_metrics view to return metrics by device. |
| cz_device_occurrences | Refines the cz_entity_events view to return events by device. |
| cz_entity_events | Combines the zenoss_view/cz_entities and zenoss_view/cz_events views to return events by entity. |
| cz_entity_metrics | Combines the zenoss_view/cz_entities and zenoss_view/metrics_full views to return metrics by entity. |
| cz_entity_occurrences | Combines the cz_entities and the cz_occurrences views to return occurrences by entity. |
| entity_occurrences | Combines the entities and occurrences views to return occurrences by entity. |
Zenoss UDF dataset
The zenoss_udf dataset contains user-defined utility functions (UDF) to simplify the creation and maintenance of queries. They are designed by Zenoss to carry out specific tasks. As a best practice, use these provided functions instead of defining your own logic in queries.
Info
The dedup_keys_aggregate_values_array function is reserved
for Zenoss-use only.
Data formatting functions
The following formatting functions are available in the Zenoss UDF dataset.
extract_kv_struct
| Element | Description |
|---|---|
| Syntax | extract_kv_struct(array,string) |
| Arguments | An array of key-value pairs (dimensions or metadata) and a string. |
| Returns | An array of key-value pairs in which all keys match the supplied string. The array may include just one item. |
extract_kv_value_array
| Element | Description |
|---|---|
| Syntax | extract_kv_value_array(array,string) |
| Arguments | An array of key-value pairs (dimensions or metadata) and a string. |
| Returns | An array of strings in which all values in the key-value pairs match the supplied string. The array may include just one item. |
extract_kv_value_string
| Element | Description |
|---|---|
| Syntax | extract_kv_value_string(array,string) |
| Arguments | An array of key-value pairs (dimensions or metadata) and a string. |
| Returns | The first value in the supplied key-value pairs that matches the supplied string. |
extract_last_string
| Element | Description |
|---|---|
| Syntax | extract_last_string(string) |
| Arguments | A string; the path of a group or system. |
| Returns | The portion of the supplied string that follows the last slash (/). If no slash is found, it returns the supplied string. |
split_metric_name
| Element | Description |
|---|---|
| Syntax | split_metric_name(string) |
| Arguments | A string; the name of a metric. |
| Returns | The portion of the supplied string that follows the last slash (/). If no slash is found, it returns the supplied string. |
split_occurrence_id
| Element | Description |
|---|---|
| Syntax | split_occurrence_id(string) |
| Arguments | A string; the ID of an event occurrence. |
| Returns | The portion of the supplied string that precedes the colon character (:). If no colon is found, it returns the supplied string. |
Logic functions
The following logical functions are available in the Zenoss UDF dataset.
is_component
| Element | Description |
|---|---|
| Syntax | is_component(array,string,string) |
| Arguments | An array of key-value pairs and two strings; typically, the metadata, operation, and source columns from zenoss_view/cz_entities. |
| Returns | True if the arguments represent a Collection Zone component, False if they do not. |
is_device
| Element | Description |
|---|---|
| Syntax | is_device(array,string,string) |
| Arguments | An array of key-value pairs and two strings; typically, the metadata, operation, and source columns from zenoss_view/cz_entities. |
| Returns | True if the arguments represent a Collection Zone device, False if they do not. |