Designing Data Processing Systems

Question 1

In Google Cloud KMS, what does a key ring primarily serve as?

A rotation policy template that governs the lifecycle of all keys within it A redundancy mechanism that replicates cryptographic keys across multiple regions An access boundary that enforces encryption algorithms for all contained keys A logical grouping and organizational container for cryptographic keys

Correct Answer: A logical grouping and organizational container for cryptographic keys

In Google Cloud KMS, a key ring serves as a logical grouping and organizational container for cryptographic keys, allowing you to organize keys by environment, team, or application and manage IAM permissions at the key ring level.

A key ring does not replicate keys across regions; it exists in a specific location and multi-region support is handled separately. Rotation policies are configured on individual keys, not inherited from the key ring. A key ring does not enforce encryption algorithms; algorithm selection is determined at the individual key or key version level.

The key concept tested is understanding the organizational hierarchy of Cloud KMS resources: project, location, key ring, key, and key version.

Question 2

In BigQuery, what is the hierarchy of resource organization from broadest to narrowest scope?

Project, Table, Dataset Project, Dataset, Table Dataset, Project, Table Organization, Dataset, Table and Project

Correct Answer: Project, Dataset, Table

In BigQuery, the resource hierarchy from broadest to narrowest is Project, Dataset, Table, where a project contains datasets and each dataset contains tables, views, and other resources.

One option incorrectly places Dataset above Project, but datasets exist within projects. Another option incorrectly places Table before Dataset, but tables are contained within datasets. Another option lists Organization as the broadest scope, and while Organization does exist in the broader Google Cloud resource hierarchy, within BigQuery's own resource model the standard hierarchy starts at Project, and the answer also lists the remaining items without proper ordering.

Key concept tested: BigQuery resource hierarchy and how data is organized within projects and datasets.

Question 3

In Apache Airflow, what does a DAG's 'max_active_runs' parameter control?

The maximum number of consecutive DAG runs that can be scheduled before the scheduler pauses The maximum number of DAG runs that can execute concurrently for that specific DAG The maximum number of task instances that can run in parallel across all DAGs in the environment The maximum number of DAG runs retained in the metadata database before older entries are archived

Correct Answer: The maximum number of DAG runs that can execute concurrently for that specific DAG

The max_active_runs parameter limits how many DAG runs of that specific DAG can be in an active (running) state at the same time, preventing resource contention when multiple scheduled or triggered runs overlap.

The option about task instances across all DAGs describes the Airflow parallelism configuration, not max_active_runs. The option about metadata retention describes a cleanup or archival concept unrelated to execution concurrency. The option about pausing the scheduler after consecutive runs is not a real Airflow behavior.

The key concept tested is understanding Airflow DAG-level concurrency controls and how max_active_runs governs parallel execution of runs for a single DAG.

Designing Data Processing Systems

GCP Data Engineer - Designing Data Processing Systems Example Questions

Question 1

Question 2

Question 3

Engineer Data Solutions on GCP