Cloud & Deployment

HashiCorp Vault for Secrets & AI Key Management

Use Vault to centrally manage secrets, rotate AI API keys automatically, and inject credentials into applications without ever touching a .env file.

What Is HashiCorp Vault?

HashiCorp Vault is a secrets management platform that solves a fundamental problem: how do you securely store, access, and rotate secrets across services, environments, and teams?

Vault provides:

Centralized secret storage — one source of truth for all credentials
Dynamic secrets — generate database credentials on-demand that expire automatically
Automatic rotation — rotate API keys before they expire without redeploying apps
Fine-grained access control — each service gets exactly the secrets it needs
Audit logging — every secret access is logged with who accessed it and when

For AI workloads, Vault is especially valuable: LLM API keys (OpenAI, Anthropic, Gemini) are high-value targets. If a model server is compromised, a stolen API key can be used to run expensive inference. Vault limits the blast radius and enables immediate revocation.

Core Vault Concepts

Secrets Engines — plugins that store or generate secrets. The most common are:

KV (Key-Value) — stores static secrets like API keys
Database — generates dynamic, temporary database credentials
AWS, GCP, Azure — generates cloud IAM credentials on-demand
PKI — issues TLS certificates

Auth Methods — how applications and humans authenticate to Vault:

AppRole — for machine/application authentication (most common for services)
Kubernetes — uses Kubernetes ServiceAccount tokens to authenticate pods
GitHub, LDAP, OIDC — for human authentication

Policies — define what paths a token can read/write.

Getting Started: KV Secrets Engine

bash

# Enable the KV v2 secrets engine
vault secrets enable -path=secret kv-v2

# Store AI API keys
vault kv put secret/ai/openai \
  api-key="sk-your-openai-key" \
  org-id="org-abc123"

vault kv put secret/ai/anthropic \
  api-key="sk-ant-your-key"

# Read a secret
vault kv get secret/ai/openai

# Read a specific field
vault kv get -field=api-key secret/ai/openai

Writing Vault Policies

Policies use HCL (HashiCorp Configuration Language) to define permissions. They follow the principle of least privilege.

hcl

# policy: ai-inference-service.hcl
# Allows reading OpenAI key only
path "secret/data/ai/openai" {
  capabilities = ["read"]
}

# Deny all other paths explicitly
path "secret/*" {
  capabilities = ["deny"]
}

bash

# Upload the policy
vault policy write ai-inference-service ./ai-inference-service.hcl

Kubernetes Auth Method

The Kubernetes auth method lets pods authenticate to Vault using their ServiceAccount token — no static credentials needed.

bash

# Enable Kubernetes auth
vault auth enable kubernetes

# Configure it with the cluster's API server URL and CA cert
vault write auth/kubernetes/config \
  kubernetes_host="https://kubernetes.default.svc" \
  kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt

# Create a role binding ServiceAccount to Vault policy
vault write auth/kubernetes/role/inference-role \
  bound_service_account_names=inference-sa \
  bound_service_account_namespaces=inference \
  policies=ai-inference-service \
  ttl=1h

Vault Agent: Automatic Secret Injection

Vault Agent runs as a sidecar container and automatically authenticates to Vault and injects secrets — without your app needing any Vault SDK code.

hcl

# vault-agent-config.hcl (as a ConfigMap)
auto_auth {
  method "kubernetes" {
    mount_path = "auth/kubernetes"
    config = {
      role = "inference-role"
    }
  }
}

template {
  destination = "/vault/secrets/config"
  contents = <<EOT
{{ with secret "secret/data/ai/openai" }}
OPENAI_API_KEY={{ .Data.data.api_key }}
{{ end }}
EOT
}

yaml

# Pod with Vault Agent sidecar
spec:
  serviceAccountName: inference-sa
  initContainers:
    - name: vault-agent-init
      image: hashicorp/vault:latest
      args: ["agent", "-config=/vault/config/agent.hcl", "-exit-after-auth"]
      volumeMounts:
        - name: vault-config
          mountPath: /vault/config
        - name: secrets-vol
          mountPath: /vault/secrets
  containers:
    - name: model-server
      image: myapp:latest
      envFrom:
        - secretRef:
            name: injected-secrets

Dynamic Secrets for AI Databases

Rather than storing static database passwords, Vault generates short-lived credentials on demand. When a pod starts, it gets a username and password that expire after its session ends.

bash

# Enable the database secrets engine
vault secrets enable database

# Configure a PostgreSQL connection
vault write database/config/ai-db \
  plugin_name=postgresql-database-plugin \
  allowed_roles="model-server" \
  connection_url="postgresql://vault-admin:admin-pass@db:5432/aidata"

# Define a role with a short TTL
vault write database/roles/model-server \
  db_name=ai-db \
  creation_statements="CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}";" \
  default_ttl="1h" \
  max_ttl="24h"

# When the pod starts, it requests credentials:
vault read database/creds/model-server
# Returns: username=v-model-abc123, password=A1B2C3... (expires in 1h)

AI API Key Rotation with Vault

Use Vault's KV v2 versioning to manage AI API key rotation without downtime:

bash

# Rotate the OpenAI key (generate new key in OpenAI dashboard first)
vault kv put secret/ai/openai api-key="sk-new-key-here"

# Apps using Vault Agent or the SDK pick up the new version
# on their next token renewal — zero downtime rotation

# View version history
vault kv metadata get secret/ai/openai

# Roll back to previous version if needed
vault kv rollback -version=2 secret/ai/openai

Audit Logging

Every secret access is logged. For AI workloads handling PII, audit logs are essential for compliance.

bash

# Enable file audit logging
vault audit enable file file_path=/var/log/vault/audit.log

# Each log entry records:
# - Timestamp
# - Which secret path was accessed
# - Which token (and thus which service) accessed it
# - The operation (read/write/delete)
# - HMAC-hashed secret values (never logged in plaintext)

Key Takeaways

Vault centralizes all secrets — no more .env files scattered across servers
Use the Kubernetes auth method so pods authenticate with their ServiceAccount — no static credentials
Dynamic secrets for databases give each pod short-lived credentials that expire automatically
AI API keys should be rotated regularly using KV v2 versioning — Vault makes this zero-downtime
Every secret access is audit-logged — essential for GDPR, HIPAA, and SOC 2 compliance

Example

bash

# Quick Vault setup for AI key management
# 1. Enable KV v2 and store keys
vault secrets enable -path=secret kv-v2
vault kv put secret/ai/openai api-key="sk-prod-key"

# 2. Write a least-privilege policy
cat > inference-policy.hcl << 'EOF'
path "secret/data/ai/openai" {
  capabilities = ["read"]
}
EOF
vault policy write inference inference-policy.hcl

# 3. Enable Kubernetes auth
vault auth enable kubernetes
vault write auth/kubernetes/role/inference \
  bound_service_account_names=inference-sa \
  bound_service_account_namespaces=inference \
  policies=inference \
  ttl=1h

# 4. Rotate key with zero downtime
vault kv put secret/ai/openai api-key="sk-new-rotated-key"

Try it yourself — BASH