Credential Management with AWS Secrets Manager
Why Credential Management with Secrets Manager Is Important
In modern cloud architectures, applications frequently need to access databases, APIs, third-party services, and other resources that require authentication credentials. Hardcoding credentials in application code, configuration files, or environment variables is a significant security risk. If source code is exposed, leaked, or stored in version control, those credentials become compromised. AWS Secrets Manager addresses this critical security concern by providing a centralized, secure, and automated way to manage secrets throughout their lifecycle. For AWS Data Engineers, understanding Secrets Manager is essential because data pipelines, ETL jobs, and analytics workloads routinely connect to databases and services that require sensitive credentials.
What Is AWS Secrets Manager?
AWS Secrets Manager is a fully managed service that helps you protect access to your applications, services, and IT resources without the upfront cost and complexity of managing your own secret management infrastructure. It enables you to:
• Store secrets securely: Secrets are encrypted at rest using AWS KMS (Key Management Service) encryption keys.
• Rotate secrets automatically: Secrets Manager can automatically rotate credentials on a schedule without requiring application downtime or code changes.
• Control access with fine-grained policies: Integration with AWS IAM allows you to define precisely who and what can access specific secrets.
• Audit secret usage: Integration with AWS CloudTrail provides a complete audit trail of when secrets were accessed and by whom.
• Retrieve secrets programmatically: Applications retrieve secrets at runtime via API calls, eliminating the need to hardcode credentials.
Secrets Manager supports various types of secrets, including:
- Database credentials (Amazon RDS, Amazon Redshift, Amazon DocumentDB, etc.)
- API keys and tokens
- OAuth tokens
- SSH keys
- Arbitrary key-value pairs or plaintext secrets
How AWS Secrets Manager Works
1. Secret Creation and Storage
When you create a secret in Secrets Manager, you provide the secret value (e.g., a username and password combination). Secrets Manager encrypts this value using an AWS KMS key — either the default Secrets Manager KMS key or a customer-managed KMS key (CMK) that you specify. The encrypted secret is stored securely within the Secrets Manager service. Each secret is assigned an Amazon Resource Name (ARN) for unique identification.
2. Secret Retrieval
Applications, Lambda functions, AWS Glue jobs, or other services retrieve secrets at runtime by calling the GetSecretValue API. The calling identity must have the appropriate IAM permissions to access the secret. Secrets Manager decrypts the secret using the associated KMS key and returns it to the caller. This approach means credentials never need to be stored in code, configuration files, or environment variables.
3. Automatic Secret Rotation
One of the most powerful features of Secrets Manager is automatic rotation. Secrets Manager uses an AWS Lambda function to rotate secrets on a defined schedule. The rotation process typically involves:
• Step 1 - createSecret: A new version of the secret is generated.
• Step 2 - setSecret: The new credential is set in the target service (e.g., updating the password in the database).
• Step 3 - testSecret: The new credential is tested to ensure it works.
• Step 4 - finishSecret: The new secret version is promoted to the current version (AWSCURRENT staging label).
Secrets Manager provides built-in Lambda rotation functions for supported services like Amazon RDS (MySQL, PostgreSQL, Oracle, SQL Server, MariaDB), Amazon Redshift, and Amazon DocumentDB. For other types of secrets, you can write custom Lambda rotation functions.
4. Versioning and Staging Labels
Secrets Manager maintains multiple versions of a secret. Each version is identified by staging labels:
- AWSCURRENT: The current active version of the secret.
- AWSPREVIOUS: The immediately prior version (useful for rollback).
- AWSPENDING: The version currently being created during rotation.
This versioning mechanism ensures that during rotation, there is no downtime since the old credential remains valid until the new one is fully confirmed.
5. Access Control and Security
Access to secrets is controlled through:
- IAM Policies: Define which IAM users, roles, or services can perform actions like GetSecretValue, CreateSecret, RotateSecret, etc.
- Resource-based Policies: Attach policies directly to a secret to enable cross-account access or restrict access to specific conditions.
- VPC Endpoints: You can create a VPC endpoint for Secrets Manager to ensure that API calls to Secrets Manager do not traverse the public internet, enhancing security for data pipelines running within a VPC.
- KMS Key Policies: The KMS key used to encrypt the secret also has its own key policy, adding another layer of access control.
6. Integration with AWS Services
Secrets Manager integrates natively with many AWS services critical for data engineering:
• AWS Glue: Glue connections can reference Secrets Manager secrets to retrieve database credentials. Glue jobs can also use the boto3 SDK to fetch secrets at runtime.
• Amazon RDS and Aurora: Direct integration for automatic rotation of database master credentials.
• Amazon Redshift: Supports credential rotation through Secrets Manager.
• AWS Lambda: Lambda functions commonly retrieve secrets from Secrets Manager to connect to databases and external APIs.
• Amazon EMR: Can use Secrets Manager for managing credentials used in big data processing.
• AWS Step Functions: Orchestration workflows can include steps that retrieve secrets.
• Amazon ECS/EKS: Container services can inject secrets from Secrets Manager as environment variables at task launch.
7. Monitoring and Auditing
Every API call made to Secrets Manager is logged in AWS CloudTrail. You can also configure Amazon CloudWatch alarms and events to monitor secret access patterns, rotation failures, and other operational metrics. AWS Config rules can be used to ensure compliance — for example, verifying that all secrets have rotation enabled.
Secrets Manager vs. AWS Systems Manager Parameter Store
This is a common comparison tested on exams:
• Secrets Manager: Purpose-built for secrets. Supports automatic rotation natively. Charges per secret per month and per 10,000 API calls. Best for database credentials and secrets that need rotation.
• Parameter Store (SecureString): General-purpose parameter storage. Does not natively support automatic rotation (requires custom implementation). Free tier available for standard parameters. Better for configuration values, feature flags, and non-rotating secrets.
Key differentiator: If the question mentions automatic rotation of credentials, the answer is almost always Secrets Manager.
Secrets Manager vs. AWS KMS
KMS is a key management service for creating and managing encryption keys. Secrets Manager uses KMS to encrypt secrets. They are complementary, not competing services. KMS does not store secrets — it stores and manages encryption keys.
Cost Considerations
Secrets Manager pricing includes:
- A monthly charge per secret stored (approximately $0.40/secret/month as of current pricing).
- A charge per 10,000 API calls ($0.05 per 10,000 calls).
- Caching secret values in your application (using the AWS Secrets Manager client-side caching library) can reduce API call costs and improve performance.
Best Practices for Data Engineers
• Always retrieve credentials from Secrets Manager at runtime — never hardcode them.
• Enable automatic rotation for all database credentials.
• Use VPC endpoints when accessing Secrets Manager from within a VPC to avoid internet exposure.
• Apply the principle of least privilege with IAM policies for secret access.
• Use resource-based policies for cross-account secret sharing rather than duplicating secrets across accounts.
• Implement caching (using the Secrets Manager caching client) to reduce latency and API costs.
• Use tags to organize and manage secrets, and integrate with AWS Config for compliance monitoring.
• Monitor rotation failures using CloudWatch and set up notifications via Amazon SNS.
Exam Tips: Answering Questions on Credential Management with Secrets Manager
1. Automatic Rotation = Secrets Manager: If a question asks about automatically rotating database credentials, API keys, or any secrets, the answer is Secrets Manager. Parameter Store does not natively support automatic rotation.
2. Hardcoded Credentials Scenario: Any scenario where credentials are stored in code, configuration files, or environment variables and the question asks how to improve security — the answer involves migrating those credentials to Secrets Manager.
3. Cross-Account Access: If asked about sharing secrets across AWS accounts, look for answers involving resource-based policies on the secret. Remember that the KMS key used to encrypt the secret must also grant cross-account access via its key policy.
4. Glue and Database Connectivity: Questions about securely connecting AWS Glue jobs to RDS, Redshift, or other databases will likely involve Secrets Manager. Glue has native integration with Secrets Manager through Glue Connections.
5. Rotation Lambda Function: Understand that rotation is performed by a Lambda function. If rotation fails, the Lambda function's execution role, network connectivity (VPC configuration, NAT gateway), and Secrets Manager VPC endpoints may be involved in troubleshooting.
6. Encryption at Rest: Secrets are always encrypted at rest using KMS. If a question asks about encryption of stored credentials, mention the integration between Secrets Manager and KMS.
7. Cost Optimization: If a question mentions reducing costs for secret retrieval, consider the client-side caching library to reduce API call frequency. If the secret doesn't require rotation and you need a lower-cost option, Parameter Store SecureString might be appropriate.
8. Auditing and Compliance: For questions about auditing who accessed which secrets and when, the answer is CloudTrail. For ensuring rotation is enabled across all secrets, use AWS Config rules.
9. Staging Labels: Understand AWSCURRENT, AWSPREVIOUS, and AWSPENDING labels. Questions may test whether you understand how versions work during the rotation process and how applications can gracefully handle credential changes.
10. Eliminate Distractors: If you see options like storing credentials in S3, DynamoDB, or EC2 instance metadata, these are typically incorrect when Secrets Manager is an option. Secrets Manager is the purpose-built, secure solution for credential management.
11. Multi-Region Replication: Secrets Manager supports replicating secrets to multiple AWS Regions. If a question involves disaster recovery or multi-Region applications needing consistent access to secrets, look for this feature.
12. Read the Scenario Carefully: Pay attention to keywords like rotate, centralized management, secure storage of credentials, compliance, and least privilege. These keywords strongly point toward Secrets Manager as the correct answer.
