Google Professional Data Engineer Exam Questions Instant Access

Question 1

You need to create a SQL pipeline. The pipeline runs an aggregate SOL transformation on a BigQuery table every two hours and appends the result to another existing BigQuery table. You need to configure the pipeline to retry if errors occur. You want the pipeline to send an email notification after three consecutive failures. What should you do?

ACreate a BigQuery scheduled query to run the SOL transformation with schedule options that repeats every two hours, and enable emailnotifications.

BUse the BigQueryUpsertTableOperator in Cloud Composer, set the retry parameter to three, and set the email_on_failure parameter totrue.

CUse the BigQuerylnsertJobOperator in Cloud Composer, set the retry parameter to three, and set the email_on_failure parameter totrue.

DCreate a BigQuery scheduled query to run the SQL transformation with schedule options that repeats every two hours, and enablenotification to Pub/Sub topic. Use Pub/Sub and Cloud Functions to send an email after three tailed executions.

Answer : D

To create a robust and resilient SQL pipeline in BigQuery that handles retries and failure notifications, consider the following:

BigQuery Scheduled Queries: This feature allows you to schedule recurring queries in BigQuery. It is a straightforward way to run SQL transformations on a regular basis without requiring extensive setup.

Error Handling and Retries: While BigQuery Scheduled Queries can run at specified intervals, they don't natively support complex retry logic or failure notifications directly. This is where additional Google Cloud services like Pub/Sub and Cloud Functions come into play.

Pub/Sub for Notifications: By configuring a BigQuery scheduled query to publish messages to a Pub/Sub topic upon failure, you can create a decoupled and scalable notification system.

Cloud Functions: Cloud Functions can subscribe to the Pub/Sub topic and implement logic to count consecutive failures. After detecting three consecutive failures, the Cloud Function can then send an email notification using a service like SendGrid or Gmail API.

Implementation Steps:

Set up a BigQuery Scheduled Query:

Create a scheduled query in BigQuery to run your SQL transformation every two hours.

Configure the scheduled query to publish a notification to a Pub/Sub topic in case of a failure.

Create a Pub/Sub Topic:

Create a Pub/Sub topic that will receive messages from the scheduled query.

Develop a Cloud Function:

Write a Cloud Function that subscribes to the Pub/Sub topic.

Implement logic in the Cloud Function to track failure messages. If three consecutive failure messages are detected, the function sends an email notification.

Reference Links:

BigQuery Scheduled Queries

Pub/Sub Documentation

Cloud Functions Documentation

SendGrid Email API

Gmail API

Question 2

Which software libraries are supported by Cloud Machine Learning Engine?

ATheano and TensorFlow

BTheano and Torch

CTensorFlow

DTensorFlow and Torch

Answer : C

Cloud ML Engine mainly does two things:

Enables you to train machine learning models at scale by running TensorFlow training applications in the cloud.

Hosts those trained models for you in the cloud so that you can use them to get predictions

about new data.

Question 3

Your company's on-premises Apache Hadoop servers are approaching end-of-life, and IT has decided to migrate the cluster to Google Cloud Dataproc. A like-for-like migration of the cluster would require 50 TB of Google Persistent Disk per node. The CIO is concerned about the cost of using that much block storage. You want to minimize the storage cost of the migration. What should you do?

APut the data into Google Cloud Storage.

BUse preemptible virtual machines (VMs) for the Cloud Dataproc cluster.

CTune the Cloud Dataproc cluster so that there is just enough disk for all data.

DMigrate some of the cold data into Google Cloud Storage, and keep only the hot data in Persistent Disk.

Answer : B

Question 4

Which Google Cloud Platform service is an alternative to Hadoop with Hive?

ACloud Dataflow

BCloud Bigtable

CBigQuery

DCloud Datastore

Answer : C

Apache Hive is a data warehouse software project built on top of Apache Hadoop for providing data summarization, query, and analysis.

Google BigQuery is an enterprise data warehouse.

Question 5

Which of the following is not true about Dataflow pipelines?

APipelines are a set of operations

BPipelines represent a data processing job

CPipelines represent a directed graph of steps

DPipelines can share data between instances

Answer : D

The data and transforms in a pipeline are unique to, and owned by, that pipeline. While your program can create multiple pipelines, pipelines cannot share data or transforms

Question 6

Which of the following is NOT a valid use case to select HDD (hard disk drives) as the storage for Google Cloud Bigtable?

AYou expect to store at least 10 TB of data.

BYou will mostly run batch workloads with scans and writes, rather than frequently executing random reads of a small number of rows.

CYou need to integrate with Google BigQuery.

DYou will not use the data to back a user-facing or latency-sensitive application.

Answer : C

For example, if you plan to store extensive historical data for a large number of remote-sensing devices and then use the data to generate daily reports, the cost savings for HDD storage may justify the performance tradeoff. On the other hand, if you plan to use the data to display a real-time dashboard, it probably would not make sense to use HDD storage---reads would be much more frequent in this case, and reads are much slower with HDD storage.

Question 7

You have important legal hold documents in a Cloud Storage bucket. You need to ensure that these documents are not deleted or modified. What should you do?

ASet a retention policy. Lock the retention policy.

BSet a retention policy. Set the default storage class to Archive for long-term digital preservation.

CEnable the Object Versioning feature. Add a lifecycle rule.

DEnable the Object Versioning feature. Create a copy in a bucket in a different region.

Answer : A

To ensure that important legal hold documents in a Cloud Storage bucket are not deleted or modified, the most effective method is to set and lock a retention policy. Here's why this is the best choice:

Retention Policy:

A retention policy defines a retention period during which objects in the bucket cannot be deleted or modified. This ensures data immutability.

Once a retention policy is set and locked, it cannot be removed or reduced, providing strong protection against accidental or malicious deletions.

Locking the Retention Policy:

Locking a retention policy ensures that the retention period cannot be changed. This action is permanent and guarantees that the specified retention period will be enforced.

Steps to Implement:

Set the Retention Policy:

Define a retention period for the bucket to ensure that all objects are protected for the required duration.

Lock the Retention Policy:

Lock the retention policy to prevent any modifications, ensuring the immutability of the documents.

Reference Links:

Cloud Storage Retention Policy Documentation

How to Set a Retention Policy

Google Cloud Certified Professional Data Engineer Exam Questions