Encryption in Transit and Cross-Account Encryption – AWS Data Engineer Associate Guide
Why Is Encryption in Transit and Cross-Account Encryption Important?
In modern cloud architectures, data rarely stays in one place. It moves between services, across AWS accounts, and over networks — both public and private. Every time data is in motion, it is vulnerable to interception, eavesdropping, and man-in-the-middle attacks. Encryption in transit ensures that data remains confidential and tamper-proof while traveling between endpoints. Cross-account encryption adds another layer of complexity because it involves sharing encrypted resources across AWS account boundaries while maintaining strict access controls. For the AWS Data Engineer Associate exam, understanding these concepts is critical because data pipelines frequently span multiple services and accounts.
What Is Encryption in Transit?
Encryption in transit (also called encryption in motion) refers to protecting data as it moves from one location to another. This is achieved using cryptographic protocols such as:
• TLS (Transport Layer Security) – The most common protocol used to encrypt data in transit. Most AWS services enforce or support TLS 1.2 or higher.
• SSL (Secure Sockets Layer) – An older protocol largely replaced by TLS, though the term SSL is still commonly used interchangeably.
• IPsec – Used in VPN connections to encrypt data at the network layer.
• SSH (Secure Shell) – Used for secure administrative access and file transfers.
When data is encrypted in transit, it is converted into ciphertext using encryption keys. Only the intended recipient with the correct decryption key can read the data. This prevents unauthorized parties from reading data even if they manage to intercept it.
How Encryption in Transit Works Across AWS Services
Most AWS services support or enforce encryption in transit by default. Here is how it applies to key data engineering services:
1. Amazon S3
• S3 supports HTTPS endpoints (TLS) for all API calls. You can enforce encryption in transit by using a bucket policy that denies requests made over HTTP (by checking aws:SecureTransport condition key).
• Example bucket policy condition: "Condition": {"Bool": {"aws:SecureTransport": "false"}} — this denies any request that does not use HTTPS.
2. Amazon Redshift
• Redshift supports SSL connections for JDBC and ODBC clients. You can enforce SSL by setting the require_SSL parameter to true in the cluster parameter group.
• Redshift also supports encryption in transit for COPY and UNLOAD operations when communicating with S3 over HTTPS.
3. Amazon RDS and Aurora
• RDS instances support SSL/TLS connections. AWS provides SSL certificates for each region. You can enforce SSL at the database level (e.g., rds.force_ssl=1 for PostgreSQL).
4. Amazon Kinesis Data Streams and Firehose
• Data sent to Kinesis endpoints is encrypted in transit using TLS. Producers and consumers communicate with Kinesis over HTTPS by default.
• Kinesis also supports enhanced fan-out with HTTP/2, which uses TLS.
5. AWS Glue
• Glue jobs communicate with data sources over encrypted connections. Glue supports JDBC connections with SSL to databases and uses HTTPS for S3 access.
• Glue Data Catalog access is secured via HTTPS API calls.
6. Amazon EMR
• EMR supports in-transit encryption between nodes using TLS. You enable this through EMR Security Configurations, specifying a certificate provider (PEM certificates or a custom certificate provider).
• In-transit encryption in EMR covers Hadoop MapReduce, Spark, Tez, and other framework communications.
7. Amazon MSK (Managed Streaming for Apache Kafka)
• MSK supports TLS encryption for data in transit between brokers and clients, as well as between brokers themselves (inter-broker encryption).
• You can configure TLS, SASL/SCRAM, or IAM-based authentication.
8. AWS Database Migration Service (DMS)
• DMS supports SSL connections to source and target endpoints. You can upload SSL certificates and configure endpoints to use SSL mode.
9. VPN and Direct Connect
• AWS Site-to-Site VPN uses IPsec to encrypt data in transit between on-premises networks and AWS VPCs.
• AWS Direct Connect does not encrypt traffic by default. To encrypt Direct Connect traffic, you can layer a VPN over the Direct Connect connection or use MACsec (available on dedicated connections).
What Is Cross-Account Encryption?
Cross-account encryption refers to scenarios where encrypted data or encryption keys are shared between different AWS accounts. This is common in enterprise environments where separate accounts are used for development, staging, production, or for different teams and business units.
The primary service involved in cross-account encryption is AWS Key Management Service (KMS). KMS manages the encryption keys used to encrypt and decrypt data across AWS services.
How Cross-Account Encryption Works
AWS KMS Key Policies and Cross-Account Access
To share a KMS key across accounts, you must configure:
1. KMS Key Policy (in the key-owning account) – The key policy must explicitly grant the external account (or specific IAM principals in the external account) permission to use the key. Common actions to grant include kms:Encrypt, kms:Decrypt, kms:ReEncrypt*, kms:GenerateDataKey*, kms:DescribeKey, and kms:CreateGrant.
2. IAM Policy (in the consuming account) – The IAM principal in the consuming account must also have an IAM policy that permits the KMS actions on the key ARN in the owning account. Both the key policy AND the IAM policy must allow access — this is a two-step authorization model.
Cross-Account S3 Encryption Scenarios
When Account A writes objects to a bucket in Account B using a KMS key:
• If the KMS key belongs to Account B, Account A needs cross-account access to that key.
• If the KMS key belongs to Account A, Account B may not be able to decrypt the objects unless Account B is also granted access to Account A's key.
• Best practice: Use a KMS key owned by the bucket-owning account, or use a shared KMS key with appropriate cross-account policies.
• With S3 Bucket Keys, cross-account access patterns remain the same, but API costs to KMS are reduced.
Cross-Account S3 with SSE-S3 vs SSE-KMS
• SSE-S3 (S3-managed keys): Simpler for cross-account scenarios because S3 manages the key. No KMS key policy management is required. However, you have less control over who can decrypt.
• SSE-KMS (KMS-managed keys): Provides fine-grained control. You can specify exactly which accounts and principals can encrypt and decrypt. This is the preferred approach when governance and auditability are priorities.
Cross-Account Redshift and KMS
• When sharing encrypted Redshift snapshots cross-account, you must also share the KMS key used to encrypt the cluster. The receiving account must have access to the KMS key to restore the snapshot.
• Use kms:CreateGrant to allow Redshift in the consuming account to use the key for decryption during snapshot restore.
Cross-Account Kinesis and KMS
• When a producer in Account A writes to a Kinesis stream in Account B that is encrypted with a KMS key, the producer needs permissions on the KMS key as well as the Kinesis stream.
• The KMS key policy must allow the producer's IAM role to call kms:GenerateDataKey and kms:Decrypt.
Cross-Account AWS Glue Data Catalog
• AWS Glue Data Catalog can be shared cross-account using resource-based policies. If the catalog is encrypted with a KMS key, cross-account consumers also need access to that KMS key to read catalog metadata.
• Lake Formation can also manage cross-account sharing with encryption handled transparently.
Cross-Account with AWS Lake Formation
• Lake Formation supports cross-account data sharing via named resource grants or AWS RAM (Resource Access Manager).
• When underlying data in S3 is encrypted with KMS, the receiving account needs access to the KMS key, or Lake Formation must handle decryption on behalf of the consumer (depending on the sharing method).
Grants in KMS
• KMS Grants are used to delegate access temporarily or for specific AWS service integrations. For example, when you share an encrypted EBS snapshot cross-account, the receiving account creates a grant to allow EC2 to use the key for decryption.
• Grants are especially important in cross-account scenarios where AWS services need to call KMS on your behalf.
Key Concepts to Remember
• aws:SecureTransport condition key – Used in IAM or resource policies to enforce HTTPS (encryption in transit).
• TLS 1.2+ – The minimum recommended TLS version for most AWS services.
• KMS Key Policy + IAM Policy – Both must allow access for cross-account KMS usage. This is sometimes called the dual authorization model.
• Customer Managed Keys (CMKs) – Recommended for cross-account encryption because you have full control over the key policy.
• AWS Managed Keys – Cannot be shared cross-account. Only the owning account can use them. This is a common exam trap.
• S3 Bucket Keys – Reduce KMS API costs but do not change the cross-account access model.
• VPN over Direct Connect – Required if you need encryption in transit over Direct Connect.
Exam Tips: Answering Questions on Encryption in Transit and Cross-Account Encryption
Tip 1: Enforce HTTPS with Condition Keys
If a question asks how to ensure data is encrypted in transit to S3, the answer almost always involves a bucket policy with the aws:SecureTransport condition key set to deny HTTP access. Remember this pattern — it appears frequently.
Tip 2: AWS Managed Keys Cannot Be Shared Cross-Account
This is one of the most commonly tested concepts. If a scenario involves cross-account access to encrypted data and uses an AWS managed key (e.g., aws/s3, aws/redshift), the answer will typically involve switching to a customer managed KMS key (CMK) to enable cross-account access.
Tip 3: Two Policies for Cross-Account KMS
Always remember that cross-account KMS access requires both a KMS key policy in the owning account and an IAM policy in the consuming account. If only one is configured, access will be denied. Look for answer choices that mention configuring both.
Tip 4: Know Which Services Default to Encryption in Transit
Services like S3, DynamoDB, Kinesis, and most managed services use HTTPS by default for API calls. However, some services like RDS and Redshift require explicit SSL configuration. If a question asks about enforcing encryption in transit for a database, think SSL parameter configuration.
Tip 5: EMR In-Transit Encryption Requires Security Configuration
EMR encryption in transit is not enabled by default. You must create an EMR Security Configuration that specifies in-transit encryption and provides TLS certificates. This is a common exam scenario.
Tip 6: Direct Connect Is Not Encrypted by Default
If a question mentions data moving over Direct Connect and asks about encryption, the answer involves either a VPN overlay on top of Direct Connect or MACsec. Direct Connect alone does not provide encryption.
Tip 7: Cross-Account Snapshot Sharing
When sharing encrypted snapshots (RDS, Redshift, EBS) cross-account, you must share the KMS key as well. The receiving account must have decrypt permissions on the key. Look for this pattern in exam questions about disaster recovery or multi-account architectures.
Tip 8: kms:CreateGrant Is Important for Service Integrations
Many AWS services use KMS grants under the hood (e.g., EBS, Redshift, RDS). In cross-account scenarios, the consuming account often needs kms:CreateGrant permission to allow the AWS service to decrypt data. If an answer choice includes kms:CreateGrant, consider it carefully.
Tip 9: Lake Formation and Cross-Account Data Sharing
Lake Formation simplifies cross-account data sharing but does not eliminate the need for KMS key access when data is encrypted. If a question involves Lake Formation cross-account sharing with encrypted S3 data, ensure the consuming account has KMS key access.
Tip 10: Eliminate Answers That Rely on AWS Managed Keys for Cross-Account
In multiple-choice questions, if one option suggests using an AWS managed key for cross-account encryption, it is almost certainly wrong. Quickly eliminate that option and focus on answers involving customer managed KMS keys with proper key policies.
Tip 11: Read Scenarios Carefully for "Least Privilege"
Exam questions often require the solution that provides the minimum necessary permissions. For cross-account KMS, grant only the specific actions needed (e.g., kms:Decrypt for read-only access) rather than full administrative access to the key.
Tip 12: Know the Difference Between SSE-S3, SSE-KMS, and SSE-C
• SSE-S3: AWS manages keys, simpler but less control, not ideal for cross-account governance.
• SSE-KMS: Customer or AWS managed KMS keys, auditable via CloudTrail, supports cross-account with CMKs.
• SSE-C: Customer provides keys with each request, AWS does not store the key. Rarely used in cross-account scenarios on the exam.
By mastering these concepts and tips, you will be well-prepared to answer any exam question related to encryption in transit and cross-account encryption scenarios on the AWS Data Engineer Associate certification exam.
