Support stack overview

The NetFoundry support stack gives you observability and troubleshooting capabilities by collecting telemetry and events from across your network. It requires a private access token from NetFoundry and deploys via a proprietary Helm chart. Components are pre-configured to work together, providing telemetry dashboards and searchable logs out of the box.

Stack components

• Elasticsearch: Stores and indexes telemetry and log data, and exposes an API for querying it.
• Logstash: Processes data from RabbitMQ and writes it to Elasticsearch. Can be extended to stream metrics to additional systems.
• Kibana: Web UI for searching and browsing raw log and telemetry data in Elasticsearch. Ships pre-configured with organized data sources.
• Grafana: Industry-standard dashboard tool for viewing and analyzing metrics. Ships pre-configured with all data sources and the dashboards used internally at NetFoundry.
• RabbitMQ: Buffers network metrics and events before Logstash processes them. You can add additional queues to stream metrics to your own systems.
• Beats agents: Micro-containers deployed as a DaemonSet on all Kubernetes nodes. They collect logs and metrics from all pods in the ziti and support namespaces and ship them to Elasticsearch.
• OpenZiti tunnelers (ziti-edge-tunnel, optional): By default, none of the support tools are exposed externally. You control how they're exposed, but we recommend accessing them over the OpenZiti network using an OpenZiti tunneler. This eliminates extra open ports and satisfies most compliance requirements.

Resource requirements

The support stack requires a minimum of 4 CPU cores available in the cluster. The installer checks available cores before installation and skips the support stack if this threshold is not met. All other OpenZiti components still install normally.

Storage and index lifecycle defaults vary based on deployment type:

Setting	K3s (single-node)	Multi-node cluster
Elasticsearch storage	30Gi	100Gi
Index rollover size	1GB	3GB
Index rollover age	1d	3d
Max index age	7d	7d
RabbitMQ storage	3Gi	3Gi

Data flow

Telemetry and event data flows through the support stack as follows:

1. The controller publishes events and metrics via AMQP to RabbitMQ.
2. Logstash consumes messages from RabbitMQ, processes them, and writes to Elasticsearch.
3. Beats agents collect container logs and metrics from all pods in the ziti and support namespaces and ship them to Elasticsearch.
4. Grafana and Kibana read from Elasticsearch to provide dashboards and searchable log views.

Configuration

The installer generates a support-values.yml file with default settings based on your deployment type (K3s or production). If the file already exists, it is left unchanged. Key settings:

elasticsearch:
  node_count: 1
  node_storage: 100Gi        # 30Gi for K3s
  max_index_age: 7d
  rollover_index_age: 3d     # 1d for K3s
  rollover_index_size: 3GB   # 1GB for K3s

rabbitmq:
  storage: 3Gi

• node_count: Number of Elasticsearch nodes. Increase for production high-availability.
• node_storage: Persistent volume size per Elasticsearch node.
• max_index_age: Maximum age before an index is deleted by the ILM policy.
• rollover_index_age / rollover_index_size: Triggers for creating a new index when the current one reaches the specified age or size.
• rabbitmq.storage: Persistent volume size for the RabbitMQ message buffer.

To apply configuration changes after installation:

helm upgrade --install support ./helm-charts/support/ --values support-values.yml -n support

Accessing support tools

By default, none of the support tools are exposed externally. The installer creates Ziti services for each tool, making them accessible over the OpenZiti network at the following intercept addresses:

Tool	Intercept address	Port
Grafana	grafana.ziti	80 (HTTP)
Kibana	kibana.ziti	443 (HTTPS)
Elasticsearch	elasticsearch.ziti	443 (HTTPS)

The installer generates a support-user.jwt enrollment token for client access. To connect:

1. Enroll the support-user.jwt token with an OpenZiti client (Desktop Edge, mobile tunneler, or CLI tunneler).
2. Once connected, access the tools at the intercept addresses above (e.g., http://grafana.ziti in your browser).

This approach eliminates the need for any externally exposed ports and satisfies most compliance requirements.

Default credentials

Tool	Username	Password
Grafana	admin	admin (you'll be prompted to change this on first login)
Elasticsearch / Kibana	elastic	Auto-generated during installation

To retrieve the Elasticsearch / Kibana password:

kubectl get secrets "elasticsearch-es-elastic-user" -n support \
  -o go-template='{{index .data "elastic" | base64decode}}'

Related tools

• Database snapshots: The quickstart automatically deploys a scheduled job that captures the OpenZiti controller database daily. See Back up your installation for configuration and restore instructions.
• Support bundle: Run nf-support-bundle to collect recent logs and stack dumps from the controller and router into a zip file for sending to NetFoundry support. See Collect diagnostics for NetFoundry support for usage and optional flags.