Roman Klimenko
EN

Synkronisering fra Azure Key Vault til Azure Kubernetes Service

azuredevops

I denne gennemgang opretter vi en Azure Key Vault og en Azure Kubernetes Service-klynge. Derefter synkroniserer vi certifikater og secrets fra Azure Key Vault til klyngen.

Nyttige links:

Vi bruger PowerShell 7 og kører alle kommandoer i den samme session. Start med at definere de nødvendige variabler:

$SUBSCRIPTION_ID = '...'
$LOCATION = '...'
$RG_NAME = '...'
$AKS_NAME = '...'
$AKV_NAME = '...' # must be globally unique

Før du opretter ressourcer, skal du sikre dig, at du bruger det rigtige Azure-abonnement:

az login
az account set --subscription $SUBSCRIPTION_ID

Aktivér Secrets Store CSI Driver-funktionen:

az feature register --namespace "Microsoft.ContainerService" --name "AKS-AzureKeyVaultSecretsProvider"

Det tager lidt tid at aktivere funktionen. Tjek status med denne kommando:

az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-AzureKeyVaultSecretsProvider')].{Name:name,State:properties.state}"

# Eventually, it must return "Registered":

# Name                                                         State
# -----------------------------------------------------------  ----------
# Microsoft.ContainerService/AKS-AzureKeyVaultSecretsProvider  Registered

Registrér Microsoft.ContainerService-provideren igen, og installer eller opdater derefter aks-preview-udvidelsen:

az provider register --namespace Microsoft.ContainerService
az extension add --name aks-preview
az extension update --name aks-preview

Opret en resource group:

az group create --name $RG_NAME --location $LOCATION

Opret en Azure Key Vault med Ă©n secret og Ă©t certifikat:


az keyvault create --name $AKV_NAME --resource-group $RG_NAME --location $LOCATION

az keyvault certificate get-default-policy > policy.json # get the default policy

az keyvault certificate create --name cert-demo --vault-name $AKV_NAME -p "@policy.json"
az keyvault secret set --vault-name $AKV_NAME --name "foo" --value "bar"

Opret derefter en Azure Kubernetes Service-klynge:

az aks create `
  --resource-group $RG_NAME `
  --name $AKS_NAME `
  --node-vm-size Standard_B8ms `
  --node-count 1 ` # AKS creates 3 nodes by default, but for the demo we need only one
  --generate-ssh-keys `
  --network-plugin azure `
  --enable-addons azure-keyvault-secrets-provider ` # enable the Secrets Store CSI Driver
  --enable-managed-identity ;

  # Expected output:

  # {
  #   "aadProfile": null,
  #   "addonProfiles": {
  #     "azureKeyvaultSecretsProvider": {
  #       "config": {
  #         "enableSecretRotation": "false",
  #         "rotationPollInterval": "2m"
  #       },
  #       "enabled": true,
  #       "identity": {
  #         "clientId": "...",
  #         "objectId": "...",
  #         "resourceId": "/subscriptions/.../resourcegroups/MC_resourse-group-name_region/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azurekeyvaultsecretsprovider-aks-name"
  #       }
  #     }
  #   },

Bemærk addonProfiles.identity, som er en managed identity, AKS automatisk opretter i resource groupen MC_. Vi bruger identiteten til at få adgang til Azure Key Vault.

Gem addonProfiles.identity.clientId i en variabel:

$SERVICE_PRINCIPAL_CLIENT_ID = 'a819baaa-4aeb-43fc-92ce-b367176d5b88'

Kør denne kommando for at aktivere tilføjelsen i en eksisterende AKS-klynge:

az aks enable-addons --addons azure-keyvault-secrets-provider --name $AKS_NAME --resource-group $RG_NAME

Forbind derefter til AKS-klyngen, og aktivér automatisk rotation af secrets:

az aks get-credentials --resource-group $RG_NAME --name $AKS_NAME
# check the CSI driver and the store provider statuses
kubectl get pods -n kube-system -l 'app in (secrets-store-csi-driver, secrets-store-provider-azure)'

# Expected output:
# NAME                                     READY   STATUS    RESTARTS   AGE
# aks-secrets-store-csi-driver-h52sr       3/3     Running   0          0h17m
# aks-secrets-store-provider-azure-7qlgd   1/1     Running   0          0h30m

az aks update -g $RG_NAME -n $AKS_NAME --enable-secret-rotation

Giv denne managed identity adgang til Azure Key Vault:

az keyvault set-policy -n $AKV_NAME --secret-permissions get --spn $SERVICE_PRINCIPAL_CLIENT_ID
az keyvault set-policy -n $AKV_NAME --certificate-permissions get --spn $SERVICE_PRINCIPAL_CLIENT_ID

Disse kommandoer giver identiteten adgang til at læse secrets og certifikater fra Azure Key Vault.

Opret nu en SecretProviderClass, den brugerdefinerede Kubernetes-ressource, som opretter forbindelse til Azure Key Vault:

# secretproviderclass.yml
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: azure-keyvault-name # use the name of your Azure Key Vault
spec:
  provider: azure
  secretObjects:
  # The following section describes how AKV secret is mapped to the Kubernetes secret:
  - secretName: foo
    type: Opaque
    data:
    - objectName: foo
      key: foo
  # If we store a certificate as a Kubernetes secret, the secret type must be kubernetes.io/tls
  - secretName: cert-demo
    type: "kubernetes.io/tls"
    data:
    - objectName: cert-demo
      key: tls.key
    - objectName: cert-demo
      key: tls.crt
  parameters:
    keyvaultName: "azure-keyvault-name" # The name of the Azure Key Vault
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "..." # The clientId of the addon-created managed identity
    # this section describes the objects pulled from Azure Key Vault
    objects:  |
      array:
        - |
          objectName: foo
          objectType: secret
        - |
          objectName: cert-demo
          objectType: secret
    # the tenant ID containing the Azure Key Vault instance, you can find it in Azure Portal
    tenantId: "..."

Anvend SecretProviderClass:

kubectl apply -f ./secretproviderclass.yml

Opret til sidst en test-pod.yml med følgende indhold for at teste opsætningen:

kind: Pod
apiVersion: v1
metadata:
  name: busybox-secrets-store-inline
spec:
  containers:
  - name: busybox
    image: k8s.gcr.io/e2e-test-images/busybox:1.29
    command:
      - "/bin/sleep"
      - "10000"
    volumeMounts:
    - name: secrets-store-inline
      mountPath: "/mnt/secrets-store"
      readOnly: true
  volumes:
    - name: secrets-store-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "azure-key-vault-name" # the name of your key vault

kubectl apply -f ./test-pod.yml

kubectl exec busybox-secrets-store-inline -- ls /mnt/secrets-store/
# Expected output:
# cert-demo
# foo

kubectl exec busybox-secrets-store-inline -- cat /mnt/secrets-store/foo
# Expected output:
# bar

kubectl exec busybox-secrets-store-inline -- cat /mnt/secrets-store/cert-demo
# Expected output:
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDVYhtyud6rbRJT
...
3fic6VM3cQR9FJxBxAq4vro=
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDQjCCAiqgAwIBAgIQSRZYP7ncTSGCw6IEOxTIhjANBgkqhkiG9w0BAQsFADAe
...
5STNJyO/kEBkBMjlzZKlDkhuf4Tr1g==
-----END CERTIFICATE-----

kubectl get secrets
# Expected output:
# NAME                                    TYPE                                  DATA   AGE
# cert-demo                               kubernetes.io/tls                     2      9h
# foo                                     Opaque                                1      9h