{"id":990,"date":"2020-05-11T21:26:42","date_gmt":"2020-05-12T04:26:42","guid":{"rendered":"http:\/\/blog.nillsf.com\/?p=990"},"modified":"2021-02-04T09:56:44","modified_gmt":"2021-02-04T17:56:44","slug":"accessing-key-vault-secrets-in-kubernetes-using-the-key-vault-csi-driver","status":"publish","type":"post","link":"https:\/\/nillsf.com\/index.php\/2020\/05\/11\/accessing-key-vault-secrets-in-kubernetes-using-the-key-vault-csi-driver\/","title":{"rendered":"Accessing Key Vault Secrets in Kubernetes using the Key Vault CSI driver"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><em>Note: There&#8217;s a new post available combining <a href=\"https:\/\/blog.nillsf.com\/index.php\/2021\/01\/31\/key-vault-csi-driver-integrated-with-aad-managed-pod-identities\/\">CSI driver + AAD pod identity<\/a>.<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When you store secrets in a Kubernetes cluster, by default those are stored in the etcd database within the master nodes. The same is true for secrets stored in an AKS cluster on Azure. The best practice for storing secrets is to store your secrets in KeyVault. Up to now, there was an opensource project called <a href=\"https:\/\/github.com\/Azure\/kubernetes-keyvault-flexvol\">KeyVault flexvolume<\/a>, which was recently deprecated in favor of a <a href=\"https:\/\/github.com\/Azure\/secrets-store-csi-driver-provider-azure\">CSI driver for secrets<\/a>. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In this blog post, we&#8217;ll explore the CSI driver for Azure KeyVault, and build a quick demo for how to run it. If you want to follow along, I uploaded the source code for this to the <a href=\"https:\/\/github.com\/NillsF\/blog\/tree\/master\/key-vault-csi\">blog&#8217;s git repo<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Exploring the Azure Key Vault Provider for Secret Store CSI Driver<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">There is a Kubernetes SIG that works on the <a href=\"https:\/\/github.com\/kubernetes-sigs\/secrets-store-csi-driver\">Kubernetes Secrets Store CSI Driver<\/a>. The work from that SIG had led to two implementation thus far, one for Azure Key Vault and one for Hashicorp Vault. We&#8217;ll be focusing today on the Azure Key Vault implementation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What the CSI driver allows you to do is mount secrets stored in a vault to your pods. It is not a replacement for the default secrets store in Kubernetes. This means you cannot store actual Kubernetes secrets in Key Vault, but you access secrets in Key Vault through the CSI driver. The CSI driver mounts any secrets you need as a file in your pods.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To get this to work, you have to install a <code>SecretProviderClass<\/code> in your Kubernetes cluster. With that provider installed in your cluster, you can then mount volumes using a <code>csi<\/code> reference in the volume reference in your pod\/deployment. We&#8217;ll look at what this looks like later on.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">From a security perspective, the Azure Secret Store CSI driver has three ways to access your secrets in Key Vault:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Using a Service Principal<\/li><li>Using Pod Identity<\/li><li>Using VMSS managed identity (system assigned is the only supported version for now)<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">And with that knowledge, let&#8217;s have a look at deploying this onto a cluster.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Setting up the prerequisites<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To get this up and running, we&#8217;ll need:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>A Kubernetes cluster running 1.16 or higher.<\/li><li>A Key Vault, and a way to authorize against it. For this demo, I&#8217;ll use the way using a Service Principal for the authorization.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s start with the Kubernetes cluster. You can create one using your preferred tool, or use any cluster that runs Kubernetes 1.16 or higher. I created a new cluster for this demo called <code>aks-secret<\/code>. I will use Cloud Shell in the Azure portal to run commands against this cluster. The first one being getting the Kubernetes credentials loaded:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>az aks get-credentials -n aks-secret -g aks-secret<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">To confirm that we are connected successfully, let&#8217;s run the following command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl get nodes<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Which should return the nodes in your cluster:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"823\" height=\"115\" src=\"\/wp-content\/uploads\/2020\/05\/image-20.png\" alt=\"\" class=\"wp-image-992\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-20.png 823w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-20-300x42.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-20-768x107.png 768w\" sizes=\"auto, (max-width: 823px) 100vw, 823px\" \/><figcaption>Verifying connectivity to the Kubernetes Cluster<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Next up, we&#8217;ll create a Key Vault (or you can re-use an existing one). As I&#8217;m only going to be using this Key Vault for this demo, I&#8217;ll quickly create a new using the CLI:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>az keyvault create -n aks-secret-nf -g aks-secret<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">This took about 30 seconds for me. Next up, we&#8217;ll create a service principal and give that service principal access to our Key Vault.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>SP=`az ad sp create-for-rbac --skip-assignment`\n\nAPPID=`echo $SP | jq .appId`\nAPPID=`echo \"${APPID\/\/\\\"}\"`\n\nSECRET=`echo $SP | jq .password`\nSECRET=`echo \"${SECRET\/\/\\\"}\"`\n\naz keyvault set-policy -n aks-secret-nf \\\n    --spn $APPID --secret-permissions get list<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">We&#8217;ll create a &#8211; regular &#8211; secret in Kubernetes to store the credentials for our service principal:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl create secret generic secrets-store-creds \\\n    --from-literal clientid=$APPID \\\n    --from-literal clientsecret=$SECRET<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Now, we can create a couple of secrets in our Key Vault:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>az keyvault secret set --vault-name aks-secret-nf \\\n    --name secret1 --value superSecret1\naz keyvault secret set --vault-name aks-secret-nf \\\n    --name secret2 --value verySuperSecret2<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And that should do it for the prerequisites. Let&#8217;s now setup the CSI driver.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Setting up and using the CSI driver<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">First, we need to install the CSI driver. This can be done using helmv3, using the following commands:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>helm repo add csi-secrets-store-provider-azure https:\/\/raw.githubusercontent.com\/Azure\/secrets-store-csi-driver-provider-azure\/master\/charts\nhelm install csi-secrets csi-secrets-store-provider-azure\/csi-secrets-store-provider-azure<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">This will create 2 daemonsets and a CRD (custom resource definition) for you:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"456\" src=\"\/wp-content\/uploads\/2020\/05\/image-21-1024x456.png\" alt=\"\" class=\"wp-image-993\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-21-1024x456.png 1024w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-21-300x134.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-21-768x342.png 768w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-21.png 1080w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Installing the driver will install two daemonsets and a CRD.<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Next, we&#8217;ll need to configure the driver to give it access to Key Vault and point it to the secrets in Key Vault. We&#8217;ll create a YAML file for this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>apiVersion: secrets-store.csi.x-k8s.io\/v1alpha1\nkind: SecretProviderClass\nmetadata:\n  name: aks-secret-nf\nspec:\n  provider: azure\n  parameters:\n    keyvaultName: \"aks-secret-nf\"\n    objects:  |\n      array:\n        - |\n          objectName: secret1\n          objectType: secret\n        - |\n          objectName: secret2\n          objectType: secret\n    tenantId: \"72f988bf-86f1-41af-91ab-2d7cd011db47\"<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And we can deploy this using:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl apply -f secretProviderClass.yaml<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And then, we should be able to reference those secrets in a regular deployment. Let&#8217;s create this sample deployment:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>apiVersion: apps\/v1\nkind: Deployment\nmetadata:\n  name: nginx-secret\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n        app: web-server\n  template:\n    metadata:\n      labels:\n        app: web-server\n    spec:\n      containers:\n        - name: nginx\n          image: nginx\n          volumeMounts:\n          - name: secrets\n            mountPath: \"\/mnt\/secrets\"\n            readOnly: true\n      volumes:\n        - name: secrets\n          csi:\n            driver: secrets-store.csi.k8s.io \n            readOnly: true\n            volumeAttributes:\n              secretProviderClass: \"aks-secret-nf\"\n            nodePublishSecretRef:\n              name: secrets-store-creds<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Which we can deploy using:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl apply -f deployment.yaml<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">After a couple of seconds, we should then see a running pod (<code>kubectl get pods<\/code>):<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"195\" src=\"\/wp-content\/uploads\/2020\/05\/image-22-1024x195.png\" alt=\"\" class=\"wp-image-996\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-22-1024x195.png 1024w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-22-300x57.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-22-768x146.png 768w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-22.png 1060w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">And we can get a list of the secrets mounted in our pod using the following command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl exec -it nginx-secret-59d7747b84-qwq2w ls \/mnt\/secrets<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Which shows us our two secrets:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"940\" height=\"62\" src=\"\/wp-content\/uploads\/2020\/05\/image-23.png\" alt=\"\" class=\"wp-image-997\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-23.png 940w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-23-300x20.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-23-768x51.png 768w\" sizes=\"auto, (max-width: 940px) 100vw, 940px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">And we can see the secrets themselves as well, using<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl exec -it nginx-secret-59d7747b84-qwq2w cat \/mnt\/secrets\/secret1<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"63\" src=\"\/wp-content\/uploads\/2020\/05\/image-24-1024x63.png\" alt=\"\" class=\"wp-image-998\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-24-1024x63.png 1024w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-24-300x18.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-24-768x47.png 768w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-24.png 1044w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">That&#8217;s it for basic functionality. This had  me intrigued a bit, and I wanted to check two final things:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>What happens if we change a secret in Key Vault?<\/li><li>What happens if we have more secrets in Key Vault than we specify in our SecretProviderClass?<\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s find out:<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What happens if we change a secret in Key Vault?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This one we can quickly find out. Let&#8217;s first change our first secret in Key Vault:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>az keyvault secret set --vault-name aks-secret-nf \\\n    --name secret1 --value newSecret<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And then list the secret in the pod:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"58\" src=\"\/wp-content\/uploads\/2020\/05\/image-25-1024x58.png\" alt=\"\" class=\"wp-image-999\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-25-1024x58.png 1024w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-25-300x17.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-25-768x43.png 768w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-25.png 1044w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This shows us that the secret in the file does not change. Which is actually not what I was expecting. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s try to see if restarting the pod makes a new secret appear. To do this, I&#8217;ll delete the pod (<code>kubectl delete pod nginx-secret-59d7747b84-qwq2w<\/code>). This will cause the replicaSet to create a new one. I&#8217;ll then exec into the new one to see the secret. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"62\" src=\"\/wp-content\/uploads\/2020\/05\/image-26-1024x62.png\" alt=\"\" class=\"wp-image-1000\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-26-1024x62.png 1024w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-26-300x18.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-26-768x46.png 768w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-26.png 1043w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This has mounted the new version of the secret. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This last step was as expected, but the overall behavior was not as expected. With typical Kubernetes secrets, <a href=\"https:\/\/kubernetes.io\/docs\/concepts\/configuration\/secret\/#mounted-secrets-are-updated-automatically\">mounted secrets update automatically.<\/a> This appears not to be the case with the CSI-driver. I opened <a href=\"https:\/\/github.com\/Azure\/secrets-store-csi-driver-provider-azure\/issues\/93\">an issue<\/a> to get more clarity about this. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What happens if we have more secrets in Key Vault than we specify in our SecretProviderClass?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">My expectation for this question is that only the secrets you provide in the SecretProviderClass are loaded and mounted in the file system. Let&#8217;s try this out. To do this, let&#8217;s first delete the SecretProviderClass and the deployment:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl delete -f secretProviderClass.yaml\nkubectl delete -f deployment.yaml<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s now create a third secret, that we won&#8217;t put into the SecretProviderClass:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>az keyvault secret set --vault-name aks-secret-nf \\\n    --name secret3notinclass --value verySecure<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And let&#8217;s then recreate the secretProviderClass and the deployment:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl create -f secretProviderClass.yaml \nkubectl create -f deployment.yaml<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">We can get the list of secrets by doing an <code>ls <\/code>in the pod:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl exec -it nginx-secret-59d7747b84-pv4xs ls \/mnt\/secrets<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And this shows &#8211; as expected &#8211; that the third secret was not loaded:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"957\" height=\"89\" src=\"\/wp-content\/uploads\/2020\/05\/image-27.png\" alt=\"\" class=\"wp-image-1001\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-27.png 957w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-27-300x28.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-27-768x71.png 768w\" sizes=\"auto, (max-width: 957px) 100vw, 957px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">If we now update the secretProviderClass, I expect the new secret to <em>not<\/em> show up in the existing pod, but to show up if we were to recreate the pod. Let&#8217;s verify:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s update the secretProviderClass.yaml to:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>apiVersion: secrets-store.csi.x-k8s.io\/v1alpha1\nkind: SecretProviderClass\nmetadata:\n  name: aks-secret-nf\nspec:\n  provider: azure\n  parameters:\n    keyvaultName: aks-secret-nf\n    objects:  |\n      array:\n        - |\n          objectName: secret1\n          objectType: secret\n        - |\n          objectName: secret2\n          objectType: secret\n        - |\n          objectName: secret3notinclass\n          objectType: secret\n    tenantId: \"72f988bf-86f1-41af-91ab-2d7cd011db47\"<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And apply the changes:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl apply -f secretProviderClass.yaml<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And if we check the running pod:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"953\" height=\"83\" src=\"\/wp-content\/uploads\/2020\/05\/image-28.png\" alt=\"\" class=\"wp-image-1002\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-28.png 953w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-28-300x26.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-28-768x67.png 768w\" sizes=\"auto, (max-width: 953px) 100vw, 953px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The new secret isn&#8217;t there yet. But let&#8217;s delete this pod and have the deployment create a new one:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"959\" height=\"90\" src=\"\/wp-content\/uploads\/2020\/05\/image-29.png\" alt=\"\" class=\"wp-image-1003\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-29.png 959w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-29-300x28.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/image-29-768x72.png 768w\" sizes=\"auto, (max-width: 959px) 100vw, 959px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Given the experience with the previous question, this behavior is more in the realm of expectations. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Summary<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The CSI-driver provides a way to connect secrets, keys and certificates in Key Vault to deployments in Kubernetes. It&#8217;s pretty easy to setup, and straightforward to use. There&#8217;s one gotcha in my mind, which is that secrets that are updated in Key Vault, aren&#8217;t automatically updated in the pods they mounted into. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Overall, this CSI-driver provides a common interface to connect to secrets in a vault. I&#8217;d say it&#8217;s early days for this project, since there are only two implementations (Azure and Hashicorp Vault). I would hope other providers come along. Having a quick look online, shows me that other providers have some other implementations, such as <a href=\"https:\/\/github.com\/godaddy\/kubernetes-external-secrets\">kubernetes-external-secrets for AWS<\/a> (developed by GoDaddy). Only time will tell if other providers will develop their own CSI driver.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Note: There&#8217;s a new post available combining CSI driver + AAD pod identity. When you store secrets in a Kubernetes cluster, by default those are stored in the etcd database within the master nodes. The same is true for secrets stored in an AKS cluster on Azure. The best practice for storing secrets is to [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1005,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[2,58,4,85],"tags":[115,117,116,50],"class_list":["post-990","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-kubernetes","category-management","category-security","tag-azure-kubernetes-service","tag-key-vault","tag-secrets","tag-security"],"jetpack_featured_media_url":"https:\/\/nillsfblog.blob.core.windows.net\/media\/2020\/05\/2020-05-11-21_25_34-PowerPoint-Slide-Show-Parcel.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/990","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/comments?post=990"}],"version-history":[{"count":4,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/990\/revisions"}],"predecessor-version":[{"id":1551,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/990\/revisions\/1551"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/media\/1005"}],"wp:attachment":[{"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/media?parent=990"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/categories?post=990"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/tags?post=990"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}