Installing observability tools

Introduction

Prometheus, Alertmanager, Grafana, Loki, and Promtail are available to be installed through the Privacy Dynamics Installer. These tools can provide crucial insight into the functioning of your cluster. Prometheus gathers metric data from Kubernetes workloads into a time-series database, and provides the ability to query that database. Certain queries can be defined as "Rules" which, when they evaluate as true, can trigger an alert. Alertmanager manages the notification process for triggered alerts. Loki gathers log data from workloads in a similar way to Prometheus gathering metrics. Finally, Grafana provides sophisticated, unified user interface for all of these tolls.

This guide covers manual setup of observability tools. It is intended as a point of reference from which customers can adapt according to their organization's needs. Strictly following this guide as written will result in the same cluster configuration as is available from the Privacy Dynamics Installer. If no variance is needed, we recommend deploying using our Installer.

Install Loki and Promtail

Promtail exports logs in a manner than Loki can collect them. Loki aggregates the logs together and makes them searchable. Loki has two main deployment methods. We currently recommend the monolithic deployment method.

1. Add the repository to Helm on your local machine

   helm repo add grafana https://grafana.github.io/helm-charts
   helm repo update
   

3. Install Loki and Promtail

   helm install --create-namespace loki grafana/loki \
   --namespace loki \
   --values loki-values.yaml
   

   helm install promtail grafana/promtail \
   --namespace loki
   

Install Prometheus

1. Add the repository to Helm on your local machine

   helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
   helm repo update
   

helm install prometheus-stack \
prometheus-community/kube-prometheus-stack \
--version 50.3.1 -n prometheus \
--values prometheus-stack-values.yaml

Access Grafana

After this is completed, you will be able create an Ingress for Grafana (as well as Prometheus and Alertmanager). See Ingresses for details. Once the Ingress is properly installed, access the Grafana dashboard at grafana.pvcy.customer.com with the username admin and the password defined in prometheus-stack-values.yaml.

Application logs can also be accessed via Grafana in the "Explore" section.