Pub/Sub

5 minutes 5 Questions

Google Cloud Pub/Sub is a fully managed, real-time messaging service that enables asynchronous communication between independent applications. It follows the publish-subscribe pattern, where message senders (publishers) and receivers (subscribers) are decoupled from each other. Key Components: 1. Topics: These are named resources where publishers send messages. Think of topics as channels or categories for organizing messages. 2. Subscriptions: These are named resources representing the stream of messages from a specific topic. Subscribers receive messages through subscriptions. 3. Messages: The data being transmitted, containing a payload and optional attributes. How It Works: Publishers create messages and send them to topics. Pub/Sub then delivers these messages to all subscriptions attached to that topic. Subscribers can pull messages on demand or configure push delivery to an endpoint. Key Features: - At-least-once delivery guarantee ensures messages are delivered reliably - Global availability with low latency - Auto-scaling handles millions of messages per second - Message retention for up to 7 days - Dead-letter topics for handling failed message processing Common Use Cases: - Event-driven architectures - Streaming analytics pipelines - Application integration - Load balancing workloads - Log aggregation and distribution For Cloud Engineers: When implementing Pub/Sub, you should consider: - Creating appropriate IAM roles for publishers and subscribers - Setting message retention policies based on requirements - Configuring acknowledgment deadlines appropriately - Monitoring subscription backlog to prevent message buildup - Using filtering to route specific messages to designated subscribers Pub/Sub integrates seamlessly with other Google Cloud services like Dataflow, Cloud Functions, and Cloud Run, making it essential for building scalable, event-driven solutions. Understanding Pub/Sub is crucial for designing loosely coupled, resilient cloud architectures.

Pub/Sub - Google Cloud Associate Cloud Engineer Guide

Why Pub/Sub is Important

Pub/Sub (Publish/Subscribe) is a foundational messaging service in Google Cloud that enables asynchronous communication between applications. It's critical for building scalable, decoupled architectures and is frequently tested on the Associate Cloud Engineer exam because it's used in real-world scenarios like event-driven systems, data streaming, and microservices communication.

What is Pub/Sub?

Google Cloud Pub/Sub is a fully managed, real-time messaging service that allows you to send and receive messages between independent applications. It follows the publish-subscribe pattern where:

• Publishers send messages to a topic
• Subscribers receive messages from a subscription attached to that topic
• Messages are persisted until acknowledged by subscribers

Key Components:

• Topic: A named resource to which publishers send messages
• Subscription: A named resource representing the stream of messages from a single topic
• Message: The data that moves through the service
• Publisher: Application that creates and sends messages to a topic
• Subscriber: Application that receives messages from a subscription

How Pub/Sub Works

1. Create a topic in your project
2. Create one or more subscriptions to that topic
3. Publishers send messages to the topic
4. Pub/Sub delivers messages to all subscriptions
5. Subscribers receive and acknowledge messages

Subscription Types:

• Pull Subscription: Subscribers explicitly request messages from the service
• Push Subscription: Pub/Sub sends messages to a specified endpoint (URL)

Message Delivery:

• At-least-once delivery: Messages are delivered at least once (duplicates possible)
• Acknowledgment deadline: Time allowed to process before redelivery
• Dead-letter topics: Handle messages that cannot be processed

Common Use Cases:

• Streaming analytics pipelines
• Event-driven architectures
• Load balancing work across workers
• Integrating systems and microservices
• Log aggregation and distribution

Key gcloud Commands:

Create a topic:
gcloud pubsub topics create my-topic

Create a subscription:
gcloud pubsub subscriptions create my-sub --topic=my-topic

Publish a message:
gcloud pubsub topics publish my-topic --message="Hello"
Pull messages:
gcloud pubsub subscriptions pull my-sub --auto-ack

Exam Tips: Answering Questions on Pub/Sub

1. Know when to choose Pub/Sub: Look for keywords like "asynchronous," "decoupled," "event-driven," "real-time messaging," or "multiple consumers"
2. Understand Pull vs Push: Pull is better when subscribers need control over message flow; Push is ideal when you have an HTTP endpoint ready to receive messages

3. Remember ordering: Standard Pub/Sub does not guarantee message ordering. If ordering is required, use ordering keys

4. Retention: Default message retention is 7 days, configurable up to 31 days

5. Fan-out pattern: One topic can have multiple subscriptions, allowing the same message to be processed by different systems

6. IAM Roles to know:
• roles/pubsub.publisher - publish messages
• roles/pubsub.subscriber - consume messages
• roles/pubsub.admin - full control

7. Integration scenarios: Pub/Sub commonly integrates with Dataflow, Cloud Functions, Cloud Run, and BigQuery

8. Exactly-once processing: Pub/Sub provides at-least-once delivery; implement idempotency in your application for exactly-once semantics

9. Global service: Pub/Sub is a global resource, but you can specify message storage locations for compliance

10. Cost factors: Pricing based on data volume, not number of messages
Test mode:
More Pub/Sub questions
30 questions (total)

Other Planning and Implementing a Cloud Solution concepts

AlloyDBAvailability policy for Compute EngineBigQueryBigtableChoosing and deploying data productsChoosing and deploying load balancersChoosing and deploying storage productsChoosing storage for Compute EngineCloud InterconnectCloud Next Generation Firewall (Cloud NGFW)Cloud RunCloud Run functionsCloud SQLCloud StorageCloud Storage classes (Standard, Nearline, Coldline, Archive)Cloud Storage object change notificationsCloud VPNCommand-line data uploadCompute EngineConfig ConnectorConfiguring OS LoginConfiguring VM ManagerCreating autoscaled managed instance groupsCreating a VPC with subnetsCustom machine typesCustom mode VPCDataflowDeploying containerized applications to GKEDeploying GKE clustersDeploying to serverless compute platformsEstablishing network connectivityEventarcFabric FASTFilestoreFirestoreFirewall ingress and egress rulesGKE AutopilotGoogle Cloud HyperdiskGoogle Cloud Managed Service for Apache KafkaGoogle Cloud NetApp VolumesGoogle Kubernetes Engine (GKE)Helm for KubernetesIaC versioning and state managementInfrastructure as code toolingInstalling kubectl CLIInstance templatesKnative servingLaunching a compute instanceLoading data into Google CloudMaintaining multi-region redundancyMemorystoreNetwork Service TiersPlanning and executing IaC deploymentsPrivate GKE clustersProcessing Pub/Sub eventsRegional GKE clustersRegional Persistent DiskSecure TagsSelecting compute choices for workloadsService accounts in firewall rulesShared VPCSpannerSpot VM instancesSSH keys for Compute EngineStorage Transfer ServiceTerraform on Google CloudUsing Tags in Cloud NGFW policy rulesVPC Network PeeringZonal Persistent Disk