HashiCorp Vault for Secrets & AI Key Management

What Is HashiCorp Vault?

HashiCorp Vault is a secrets management platform that solves a fundamental problem: how do you securely store, access, and rotate secrets across services, environments, and teams?

Vault provides:

• Centralized secret storage — one source of truth for all credentials
• Dynamic secrets — generate database credentials on-demand that expire automatically
• Automatic rotation — rotate API keys before they expire without redeploying apps
• Fine-grained access control — each service gets exactly the secrets it needs
• Audit logging — every secret access is logged with who accessed it and when

For AI workloads, Vault is especially valuable: LLM API keys (OpenAI, Anthropic, Gemini) are high-value targets. If a model server is compromised, a stolen API key can be used to run expensive inference. Vault limits the blast radius and enables immediate revocation.

Core Vault Concepts

Secrets Engines — plugins that store or generate secrets:

• KV (Key-Value) — stores static secrets like API keys
• Database — generates dynamic, temporary database credentials
• AWS, GCP, Azure — generates cloud IAM credentials on-demand
• PKI — issues TLS certificates

Auth Methods — how applications and humans authenticate to Vault:

• AppRole — for machine/application authentication
• Kubernetes — uses Kubernetes ServiceAccount tokens to authenticate pods
• GitHub, LDAP, OIDC — for human authentication

Policies — define what paths a token can read/write.

Getting Started: KV Secrets Engine

Enable the KV v2 secrets engine, store AI API keys, and retrieve them using vault kv commands.

Writing Vault Policies

Policies use HCL to define permissions. They follow the principle of least privilege — each service gets only the paths it needs with only the capabilities required.

Kubernetes Auth Method

The Kubernetes auth method lets pods authenticate to Vault using their ServiceAccount token — no static credentials needed. Create a role binding the ServiceAccount to a Vault policy with a short TTL.

Vault Agent: Automatic Secret Injection

Vault Agent runs as a sidecar container and automatically authenticates to Vault and injects secrets — without your app needing any Vault SDK code.

Dynamic Secrets for AI Databases

Rather than storing static database passwords, Vault generates short-lived credentials on demand. When a pod starts, it gets a username and password that expire after its session ends.

AI API Key Rotation with Vault

Use Vault's KV v2 versioning to manage AI API key rotation without downtime. Apps using Vault Agent pick up the new version on their next token renewal — zero downtime rotation.

Key Takeaways

• Vault centralizes all secrets — no more .env files scattered across servers
• Use the Kubernetes auth method so pods authenticate with their ServiceAccount — no static credentials
• Dynamic secrets for databases give each pod short-lived credentials that expire automatically
• AI API keys should be rotated regularly using KV v2 versioning — Vault makes this zero-downtime
• Every secret access is audit-logged — essential for GDPR, HIPAA, and SOC 2 compliance