Synkroniser fra Azure Key Vault til Azure Kubernetes Service

I denne gennemgang opretter vi en ny Azure Key Vault, derefter en ny Azure Kubernetes Service, og til sidst synkroniserer vi certifikater og hemmeligheder fra Azure Key Vault til Azure Kubernetes Service.

Nyttige links:

• Azure Key Vault Provider for Secrets Store CSI Driver
• Use the Secrets Store CSI Driver for Kubernetes in an Azure Kubernetes Service (AKS) cluster (preview)

Vi bruger Powershell 7 og antager, at alle kommandoer køres i samme session. Lad os starte med at definere de nødvendige variabler:

$SUBSCRIPTION_ID = '...'
$LOCATION = '...'
$RG_NAME = '...'
$AKS_NAME = '...'
$AKV_NAME = '...' # must be globally unique

Sørg for, at vi kører kommandoerne under det rigtige Subscription:

az login
az account set --subscription $SUBSCRIPTION_ID

Aktivér Secrets Store CSI Driver-funktionen:

az feature register --namespace "Microsoft.ContainerService" --name "AKS-AzureKeyVaultSecretsProvider"

Det tager et stykke tid, før funktionen er aktiveret. Vi kan tjekke status med denne kommando:

az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-AzureKeyVaultSecretsProvider')].{Name:name,State:properties.state}"

# Eventually, it must return "Registered":

# Name                                                         State
# -----------------------------------------------------------  ----------
# Microsoft.ContainerService/AKS-AzureKeyVaultSecretsProvider  Registered

Genregistrér nu Container Service-udvidelsen og sørg for, at den er opdateret:

az provider register --namespace Microsoft.ContainerService
az extension add --name aks-preview
az extension update --name aks-preview

Opret en resource group:

az group create --name $RG_NAME --location $LOCATION

Opret en Azure Key Vault med én hemmelighed og ét certifikat:

az keyvault create --name $AKV_NAME --resource-group $RG_NAME --location $LOCATION

az keyvault certificate get-default-policy > policy.json # get the default policy

az keyvault certificate create --name cert-demo --vault-name $AKV_NAME -p "@policy.json"
az keyvault secret set --vault-name $AKV_NAME --name "foo" --value "bar"

Lad os nu oprette en Azure Kubernetes Service:

az aks create `
  --resource-group $RG_NAME `
  --name $AKS_NAME `
  --node-vm-size Standard_B8ms `
  --node-count 1 ` # AKS creates 3 nodes by default, but for the demo we need only one
  --generate-ssh-keys `
  --network-plugin azure `
  --enable-addons azure-keyvault-secrets-provider ` # enable the Secrets Store CSI Driver
  --enable-managed-identity ;

  # Expected output:

  # {
  #   "aadProfile": null,
  #   "addonProfiles": {
  #     "azureKeyvaultSecretsProvider": {
  #       "config": {
  #         "enableSecretRotation": "false",
  #         "rotationPollInterval": "2m"
  #       },
  #       "enabled": true,
  #       "identity": {
  #         "clientId": "...",
  #         "objectId": "...",
  #         "resourceId": "/subscriptions/.../resourcegroups/MC_resourse-group-name_region/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azurekeyvaultsecretsprovider-aks-name"
  #       }
  #     }
  #   },

Læg mærke til addonProfiles.identity — en managed identity, der automatisk oprettes i MC_-resource group'en. Vi bruger denne identitet til at forbinde til Azure Key Vault.

Lad os gemme addonProfiles.identity.cliendId i en variabel:

$SERVICE_PRINCIPAL_CLIENT_ID = 'a819baaa-4aeb-43fc-92ce-b367176d5b88'

Hvis du opdaterer en eksisterende AKS-klynge, skal du køre denne kommando:

az aks enable-addons --addons azure-keyvault-secrets-provider --name $AKS_NAME --resource-group $RG_NAME

Mens vi er i gang, lad os forbinde til AKS-klyngen og aktivere automatisk rotation af hemmeligheder:

az aks get-credentials --resource-group $RG_NAME --name $AKS_NAME
# check the CSI driver and the store provider statuses
kubectl get pods -n kube-system -l 'app in (secrets-store-csi-driver, secrets-store-provider-azure)'

# Expected output:
# NAME                                     READY   STATUS    RESTARTS   AGE
# aks-secrets-store-csi-driver-h52sr       3/3     Running   0          0h17m
# aks-secrets-store-provider-azure-7qlgd   1/1     Running   0          0h30m

az aks update -g $RG_NAME -n $AKS_NAME --enable-secret-rotation

Lad os nu give vores managed identity adgang til Azure Key Vault:

az keyvault set-policy -n $AKV_NAME --secret-permissions get --spn $SERVICE_PRINCIPAL_CLIENT_ID
az keyvault set-policy -n $AKV_NAME --certificate-permissions get --spn $SERVICE_PRINCIPAL_CLIENT_ID

Disse kommandoer giver den managed identity lov til at læse hemmeligheder og certifikater fra Azure Key Vault.

Næste skridt er at oprette en SecretProviderClass — en brugerdefineret Kubernetes-ressource, der bruges til at forbinde til Azure Key Vault:

# secretproviderclass.yml
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: azure-keyvault-name # use the name of your Azure Key Vault
spec:
  provider: azure
  secretObjects:
  # The following section describes how AKV secret is mapped to the Kubernetes secret:
  - secretName: foo
    type: Opaque
    data:
    - objectName: foo
      key: foo
  # If we store a certificate as a Kubernetes secret, the secret type must be kubernetes.io/tls
  - secretName: cert-demo
    type: "kubernetes.io/tls"
    data:
    - objectName: cert-demo
      key: tls.key
    - objectName: cert-demo
      key: tls.crt
  parameters:
    keyvaultName: "azure-keyvault-name" # The name of the Azure Key Vault
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "..." # The clientId of the addon-created managed identity
    # this section describes the objects pulled from Azure Key Vault
    objects:  |
      array:
        - |
          objectName: foo
          objectType: secret
        - |
          objectName: cert-demo
          objectType: secret
    # the tenant ID containing the Azure Key Vault instance, you can find it in Azure Portal
    tenantId: "..."

Anvend SecretProviderClass:

kubectl apply -f ./secretproviderclass.yml

Lad os til sidst teste det:

Opret en test-pod.yml med følgende indhold:

kind: Pod
apiVersion: v1
metadata:
  name: busybox-secrets-store-inline
spec:
  containers:
  - name: busybox
    image: k8s.gcr.io/e2e-test-images/busybox:1.29
    command:
      - "/bin/sleep"
      - "10000"
    volumeMounts:
    - name: secrets-store-inline
      mountPath: "/mnt/secrets-store"
      readOnly: true
  volumes:
    - name: secrets-store-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "azure-key-vault-name" # the name of your key vault

kubectl apply -f ./test-pod.yml

kubectl exec busybox-secrets-store-inline -- ls /mnt/secrets-store/
# Expected output:
# cert-demo
# foo

kubectl exec busybox-secrets-store-inline -- cat /mnt/secrets-store/foo
# Expected output:
# bar

kubectl exec busybox-secrets-store-inline -- cat /mnt/secrets-store/cert-demo
# Expected output:
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDVYhtyud6rbRJT
...
3fic6VM3cQR9FJxBxAq4vro=
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDQjCCAiqgAwIBAgIQSRZYP7ncTSGCw6IEOxTIhjANBgkqhkiG9w0BAQsFADAe
...
5STNJyO/kEBkBMjlzZKlDkhuf4Tr1g==
-----END CERTIFICATE-----

kubectl get secrets
# Expected output:
# NAME                                    TYPE                                  DATA   AGE
# cert-demo                               kubernetes.io/tls                     2      9h
# foo                                     Opaque                                1      9h