Best Practices for Securely Managing Secrets

SAURABH SHARMA
3 min readMar 28, 2024

Never hardcoded secret, *.yaml or *.properties files

From creation through destruction, secrets must be handled securely: stored with the least privilege, encrypted in transit, appropriately rotated and revoked if compromised.

saurabhshcs: RDTR

We should use short-lived secrets wherever possible.

Secret rotation involves four steps that should always be considered, even if some actions don't apply to certain types of secrets.

  • Rotate: Where possible, generate a new secret alongside the previous secret
  • Deploy: Replace all instances of the previous secret in production with the new secret
  • Test: Where possible, check that the previous secret is no longer logging any uses
  • Retire: Delete the previous secret.

Caution: 1Password should never be used to share secrets across your team.

Securely Storing Secrets

Remember: Cloud environments are not inherently secure. You must configure and manage them securely, including managing your secrets securely.

IBM Cloud Secrets Managers

Cloud Native Secrets Managers

When choosing a third-party cloud provider, it's vital to evaluate their capability to handle sensitive information. #cloudsecurity #dataprivacy #cybersecurity #informationsecurity

Managing Secrets in Kubernetes

It is crucial to ensure the security of secrets in Kubernetes by going beyond their default settings.

  • Encrypt your secrets at rest with strong encryption. Key Protect or Hyper Protect Crypto Services can help. Kubernetes doesn't encrypt secrets by default. #cybersecurity #encryption #cloudsecurity
  • Enabling or configuring role-based access control (RBAC) rules with least privilege.
  • They are restricting Secret access to specific containers.
  • Consider using external secret store providers.
  • Storing confidential info in temporary systems is unsafe. Use encrypted databases with strict access controls instead. #datasecurity #confidentiality #encryption #accesscontrols #cybersecurity

Building Container Images Without Leaking Secrets

Secrets management doesn’t stop at source code, you also need to manage secrets appropriately when building container images and subsequently during orchestration.

To avoid leaking secrets from your build pipeline into the container images that you build, you should:

  • Avoid hardcoding secrets into your Dockerfile using Docker ENV or ARG commands.
  • Use Docker BuildKit Image Build Secrets with ‘buildx’ to mount secrets into your build. This prevents secrets from being leaked via the image metadata.
  • If your application requires injecting secrets at build time through environment variables, make sure their values are not revealed in system or application logs. To mitigate the fact that the environment variables are printed to logs, either process the logs to remove or obscure the secret values, or use a sidecar container to retrieve the secrets from a secure store.
  • If a secret is written to a file, ensure that it is deleted in the same build step. This prevents secrets from being leaked via intermediate image layers.
  • Use a multistage build where no secrets are used in the final stage. This additional layer of protection prevents image history or intermediate layers from the build stage(s) from being present in the shipped image.
  • For more information on best practices, refer to OWASP’s cheatsheet on Secrets Management: https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html

--

--

SAURABH SHARMA

Technology Enthusiast and Open Source Technology advocate