Excellent video Sam!... I would like ask you some quick questions; 1. Which can be the difference between init and sidecar Vault agents?. 2. The API Pod; how its to grab the credentials injected, its must to read the local volume for get them?. 3. If the credentials was changed into the vault then how the injector to can update this values automatically into the API pod or which is the alternative?
Great questions...here's my attempt at answering them: 1. Difference Between Init and Sidecar Vault Agents: - Init Agent: This agent runs at the start of a Kubernetes pod's lifecycle. Its job is to pre-populate the pod's designated volumes with secrets fetched from Vault. Once it does this, it exits. The init agent is great for secrets that don't need to be updated while the pod is running. - Sidecar Agent: Unlike the init agent, the sidecar agent runs alongside the application in the same pod throughout its lifecycle. It continuously checks for updates to the secrets and can dynamically update them as needed. This is ideal for scenarios where secrets might change during the pod’s runtime. 2. API Pod Accessing Credentials: - Yes, the API pod accesses the credentials injected by the Vault agent from a local volume. When the Vault agent (either init or sidecar) fetches secrets from Vault, it writes them to a shared memory volume in the pod. Your application, running in the API pod, can then read these secrets directly from this volume. 3. Updating Credentials in the API Pod: - If the credentials in Vault are updated, the sidecar agent can automatically fetch these new values and update the shared volume. Your application can be configured to periodically check this volume for any changes to the secrets and use the updated values accordingly. - For an init agent, since it doesn’t run continuously, you would need to restart the pod to fetch the updated credentials. This is less dynamic compared to the sidecar approach. An alternative approach, if you're not using a sidecar agent, is to implement a mechanism within your application to periodically check for updated credentials directly from Vault. However, this requires your application to be Vault-aware and handle the authentication and fetching process, which adds complexity.
What version of Kubernetes have you been using? I tried to setup this vault-k8s auth with vault-agent to auto populate the secrets inside the pod. I see 'Permission Denied' error in the vault-agent-init container when I try this setup. I've tried this in clusters K8S 1.24 and K8S 1.28 same result in both. While the same setup has been working fine in k8s 1.23!! vault was v1.15.0
Hi, is it possible in any way to also have vault itself synced via argocd? Seems like a chicken/egg conundrum as of yet. The context is: having multiple vault instances/environments and managing those vault secret engines/settings via Kustomize generated manifests instead of terraform (not sure if that is possible)
That’s an excellent question. I don’t see why we can’t use ArgoCD to sync Vault using Vault’s helm chart as long as Vault is running in K8s. The question is what to do with the Vault configuration. Most people use the Terraform provider for vault. I suppose we could look into crossplane so it’s K8s native. I haven’t looked to see if there is a Vault provider in Crossplane. I also believe crossplane can use Terraform providers so that’s another thing to look at. Finally, HashiCorp had a project where you can configure Terraform from within K8s, I can’t remember what that was, but also worth exploring.
@@TeKanAid Crossplane does seem to have Vault support, but it seems it may not be 'quite there' yet. Thanks for pointing me into that direction anyway, I can see some usage out of it still :)
Excellent video Sam!... I would like ask you some quick questions; 1. Which can be the difference between init and sidecar Vault agents?. 2. The API Pod; how its to grab the credentials injected, its must to read the local volume for get them?. 3. If the credentials was changed into the vault then how the injector to can update this values automatically into the API pod or which is the alternative?
Great questions...here's my attempt at answering them:
1. Difference Between Init and Sidecar Vault Agents:
- Init Agent: This agent runs at the start of a Kubernetes pod's lifecycle. Its job is to pre-populate the pod's designated volumes with secrets fetched from Vault. Once it does this, it exits. The init agent is great for secrets that don't need to be updated while the pod is running.
- Sidecar Agent: Unlike the init agent, the sidecar agent runs alongside the application in the same pod throughout its lifecycle. It continuously checks for updates to the secrets and can dynamically update them as needed. This is ideal for scenarios where secrets might change during the pod’s runtime.
2. API Pod Accessing Credentials:
- Yes, the API pod accesses the credentials injected by the Vault agent from a local volume. When the Vault agent (either init or sidecar) fetches secrets from Vault, it writes them to a shared memory volume in the pod. Your application, running in the API pod, can then read these secrets directly from this volume.
3. Updating Credentials in the API Pod:
- If the credentials in Vault are updated, the sidecar agent can automatically fetch these new values and update the shared volume. Your application can be configured to periodically check this volume for any changes to the secrets and use the updated values accordingly.
- For an init agent, since it doesn’t run continuously, you would need to restart the pod to fetch the updated credentials. This is less dynamic compared to the sidecar approach.
An alternative approach, if you're not using a sidecar agent, is to implement a mechanism within your application to periodically check for updated credentials directly from Vault. However, this requires your application to be Vault-aware and handle the authentication and fetching process, which adds complexity.
Brilliant@@TeKanAid, thank you!
You're welcome, @@samys288
I learnt so much from Sam! Thank you ❤
Great to hear! Thank you!
What version of Kubernetes have you been using? I tried to setup this vault-k8s auth with vault-agent to auto populate the secrets inside the pod. I see 'Permission Denied' error in the vault-agent-init container when I try this setup. I've tried this in clusters K8S 1.24 and K8S 1.28 same result in both. While the same setup has been working fine in k8s 1.23!! vault was v1.15.0
Yes, something changed between kubernetes versions. I remember running into this a while ago. Take a look at the documentation.
Hi, is it possible in any way to also have vault itself synced via argocd? Seems like a chicken/egg conundrum as of yet. The context is: having multiple vault instances/environments and managing those vault secret engines/settings via Kustomize generated manifests instead of terraform (not sure if that is possible)
That’s an excellent question. I don’t see why we can’t use ArgoCD to sync Vault using Vault’s helm chart as long as Vault is running in K8s. The question is what to do with the Vault configuration. Most people use the Terraform provider for vault. I suppose we could look into crossplane so it’s K8s native. I haven’t looked to see if there is a Vault provider in Crossplane. I also believe crossplane can use Terraform providers so that’s another thing to look at. Finally, HashiCorp had a project where you can configure Terraform from within K8s, I can’t remember what that was, but also worth exploring.
@@TeKanAid Crossplane does seem to have Vault support, but it seems it may not be 'quite there' yet. Thanks for pointing me into that direction anyway, I can see some usage out of it still :)
@@pixeldrama-yt good to hear. Yeah, I really like crossplane