Grafana dashboards created in UI wont persist on restarts/updates. To make sure your dashboards are persistent you need to update the grafana.yaml file that you used to install grafana with istio. First make your dashboard in grafana and export it as json. Then make a copy of grafana.yaml file so you have a backup. Create a config map as follows with the json data of your dashboard as follows apiVersion: v1 kind: ConfigMap metadata: name: namespace: istio-system data: .json: |- Then add the config map to the grafana deployment under spec.template.spec.volumes, below is an example: spec: ... template: ... spec: ... volumes: ... - name: configMap: name: Then add the dashboard provider in grafana's serviceaccount under data.dashboardproviders.yaml.providers, below is an example dashboardproviders.yaml: | apiVersion: 1 providers: ... - disableDeletion: false folder: name: options: path: /var/lib/grafana/dashboards/ orgId: 1 type: file Lastly, update the volumeMounts in grafana container to use the above volume, below is an example containers: - name: grafana ... volumeMounts: .... - name: mountPath: "/var/lib/grafana/dashboards/" Apply the yaml file and you should have your custom dashboard in the grafana accessible by istioctl dashboard grafana this is one of the most reliable way to add persistent dashboards with grafan in Istio
Nice bro, saved me hours
Nice demo
how could make it persistent? I import dashboard, but when grafana pod restart, it remove all dashboard and remains istio dashboard.
Grafana dashboards created in UI wont persist on restarts/updates. To make sure your dashboards are persistent you need to update the grafana.yaml file that you used to install grafana with istio.
First make your dashboard in grafana and export it as json.
Then make a copy of grafana.yaml file so you have a backup.
Create a config map as follows with the json data of your dashboard as follows
apiVersion: v1
kind: ConfigMap
metadata:
name:
namespace: istio-system
data:
.json: |-
Then add the config map to the grafana deployment under spec.template.spec.volumes, below is an example:
spec:
...
template:
...
spec:
...
volumes:
...
- name:
configMap:
name:
Then add the dashboard provider in grafana's serviceaccount under data.dashboardproviders.yaml.providers, below is an example
dashboardproviders.yaml: |
apiVersion: 1
providers:
...
- disableDeletion: false
folder:
name:
options:
path: /var/lib/grafana/dashboards/
orgId: 1
type: file
Lastly, update the volumeMounts in grafana container to use the above volume, below is an example
containers:
- name: grafana
...
volumeMounts:
....
- name:
mountPath: "/var/lib/grafana/dashboards/"
Apply the yaml file and you should have your custom dashboard in the grafana accessible by istioctl dashboard grafana
this is one of the most reliable way to add persistent dashboards with grafan in Istio
Nice
what is the best storage to use prometheus like s3,ELK etc
can you create tutorial istio with Grafana Mimir with storage GCS/S3 ?
There is a video on Grafana Loki in the pipeline. We'll think about Grafana Mimir. Stay tuned!
It seems the repo does not contrains this prometheus example
Check please the code is there in repo