Data Ingestion and Transformation

Ingesting data from streaming and batch sources, transforming and processing data across formats, orchestrating ETL pipelines, and applying programming concepts for data engineering on AWS.

5 minutes 5 Questions

Data Ingestion and Transformation are fundamental concepts in the AWS Certified Data Engineer - Associate exam, representing critical stages in any data pipeline. **Data Ingestion** refers to the process of collecting and importing data from various sources into a storage or processing system. AWS…

Concepts covered
Container-Based Data Processing with EKS and ECSData Migration with AWS DMSScheduling Data Pipelines with Airflow and EventBridgeFan-In and Fan-Out for Streaming DistributionData Replayability in Ingestion PipelinesMulti-Source Data Integration with JDBC and ODBCServerless Data Transformation with LambdaPipeline Orchestration with Step Functions and MWAAProgramming Best Practices for Data EngineeringInfrastructure as Code with CloudFormation, CDK, and SAMCI/CD for Data Pipeline DeploymentStreaming Data Ingestion with Kinesis and MSKBatch Data Ingestion with S3 and AWS GlueEvent-Driven Ingestion with EventBridge and S3 NotificationsAPI Data Consumption and Rate LimitingStateful and Stateless Data TransactionsData Format Transformation (CSV, Parquet, JSON)ETL Processing with AWS Glue and Amazon EMRCost Optimization in Data ProcessingBuilding Resilient and Fault-Tolerant PipelinesIntegrating LLMs for Data Processing
Test mode:
Start practice test

AWS DEA-C01 - Data Ingestion and Transformation Example Questions

Test your knowledge of Data Ingestion and Transformation

Question 1

A team wants to trigger a Lambda function whenever new files arrive in an S3 bucket. They are deciding between native S3 Event Notifications and EventBridge integration. Which statement correctly describes how S3 delivers events to EventBridge?

Correct Answer: S3 publishes events to EventBridge as structured JSON events with a 'detail' field containing bucket and object metadata

When Amazon S3 EventBridge integration is enabled on a bucket, S3 publishes events directly to Amazon EventBridge as structured CloudEvents-compatible JSON, including standard fields like source (aws.s3), detail-type, and a detail field containing bucket name, object key, size, and other metadata.

S3 does not route events through SNS before reaching EventBridge; they are sent natively to EventBridge. The source field identifies the service (aws.s3), not the object payload or bucket policy details. Events are not sent as CloudWatch Logs entries; EventBridge receives them directly from S3.

The key concept tested is understanding the native S3-to-EventBridge event delivery mechanism and its structured JSON event schema.

Question 2

A team wants to use an LLM through Amazon Bedrock to summarize text documents stored in Amazon S3. They are new to Bedrock and need to send their first request. Which AWS service or feature do they use to send text to the LLM and receive a summary response?

Correct Answer: The Amazon Bedrock InvokeModel API to send a prompt and receive the model's response

The Amazon Bedrock InvokeModel API is the correct way to send a prompt (such as a summarization request) to a foundation model hosted on Bedrock and receive the generated response in real time.

SageMaker batch transform is used for batch inference on SageMaker-hosted models, not for interacting with Bedrock foundation models. The Bedrock CreateModel API does not exist as described; Bedrock provides access to pre-trained foundation models without needing to build custom models from scratch. Amazon Comprehend is a natural language processing service for tasks like entity detection and sentiment analysis, not for LLM-based text summarization via prompts.

Key concept tested: understanding how to interact with foundation models using the Amazon Bedrock InvokeModel API.

Question 3

What is the primary purpose of a CI/CD pipeline in the context of data pipeline deployment on AWS?

Correct Answer: To automate the testing, building, and deployment of data pipeline code changes across environments

A CI/CD (Continuous Integration/Continuous Delivery) pipeline is specifically designed to automate the process of testing, building, and deploying code changes through development, staging, and production environments, ensuring reliable and repeatable releases of data pipeline code.

Scheduling and orchestration of data processing jobs is the role of workflow tools like AWS Step Functions or Apache Airflow, not CI/CD. Monitoring, alerting, and logging are handled by observability services like Amazon CloudWatch. Provisioning compute resources and scaling storage relates to infrastructure management and auto-scaling, which may be part of infrastructure-as-code but is not the primary purpose of CI/CD.

Key concept tested: understanding the role of CI/CD in software delivery lifecycle for data engineering.

More Data Ingestion and Transformation questions
879 questions (total)
Practice 100 questions
One session at a time, always new questions