Google Professional Data Engineer Exam Questions Instant Access

Question 1

You are designing a real-time system for a ride hailing app that identifies areas with high demand for rides to effectively reroute available drivers to meet the demand. The system ingests data from multiple sources to Pub/Sub. processes the data, and stores the results for visualization and analysis in real-time dashboards. The data sources include driver location updates every 5 seconds and app-based booking events from riders. The data processing involves real-time aggregation of supply and demand data for the last 30 seconds, every 2 seconds, and storing the results in a low-latency system for visualization. What should you do?

AGroup the data by using a tumbling window in a Dataflow pipeline, and write the aggregated data to Memorystore

BGroup the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to Memorystore

CGroup the data by using a session window in a Dataflow pipeline, and write the aggregated data to BigQuery.

DGroup the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to BigQuery.

Answer : B

A hopping window is a type of sliding window that advances by a fixed period of time, producing overlapping windows. This is suitable for the scenario where the system needs to aggregate data for the last 30 seconds, every 2 seconds, and provide real-time updates. A Dataflow pipeline can implement the hopping window logic using Apache Beam, and process both streaming and batch data sources. Memorystore is a low-latency, in-memory data store that can serve the aggregated data to the visualization layer. BigQuery is not a good choice for this scenario, as it is not optimized for low-latency queries and frequent updates.

Question 2

You are selecting services to write and transform JSON messages from Cloud Pub/Sub to BigQuery for a data pipeline on Google Cloud. You want to minimize service costs. You also want to monitor and accommodate input data volume that will vary in size with minimal manual intervention. What should you do?

AUse Cloud Dataproc to run your transformations. Monitor CPU utilization for the cluster. Resize the number of worker nodes in your cluster via the command line.

BUse Cloud Dataproc to run your transformations. Use the diagnose command to generate an operational output archive. Locate the bottleneck and adjust cluster resources.

CUse Cloud Dataflow to run your transformations. Monitor the job system lag with Stackdriver. Use thedefault autoscaling setting for worker instances.

DUse Cloud Dataflow to run your transformations. Monitor the total execution time for a sampling of jobs. Configure the job to use non-default Compute Engine machine types when needed.

Answer : B

Question 3

You are using BigQuery and Data Studio to design a customer-facing dashboard that displays large quantities of aggregated data. You expect a high volume of concurrent users. You need to optimize tie dashboard to provide quick visualizations with minimal latency. What should you do?

AUse BigQuery BI Engine with materialized views

BUse BigQuery BI Engine with streaming data.

CUse BigQuery Bl Engine with authorized views

DUse BigQuery Bl Engine with logical reviews

Answer : B

Question 4

The Dataflow SDKs have been recently transitioned into which Apache service?

AApache Spark

BApache Hadoop

CApache Kafka

DApache Beam

Answer : D

Dataflow SDKs are being transitioned to Apache Beam, as per the latest Google directive

Question 5

You have a data analyst team member who needs to analyze data by using BigQuery. The data analyst wants to create a data pipeline that would load 200 CSV files with an average size of 15MB from a Cloud Storage bucket into BigQuery daily. The data needs to be ingested and transformed before being accessed in BigQuery for analysis. You need to recommend a fully managed, no-code solution for the data analyst. What should you do?

ACreate a Cloud Run function and schedule it to run daily using Cloud Scheduler to load the data into BigQuery.

BUse the BigQuery Data Transfer Service to load files from Cloud Storage to BigQuery, create a BigQuery job which transforms the data using BigQuery SQL and schedule it to run daily.

CBuild a custom Apache Beam pipeline and run it on Dataflow to load the file from Cloud Storage to BigQuery and schedule it to run daily using Cloud Composer.

DCreate a pipeline by using BigQuery pipelines and schedule it to load the data into BigQuery daily.

Answer : B

The requirements are for a daily scheduled load, ingest, and transformation, and specifically a fully managed, no-code solution.

Ingest (Load): The BigQuery Data Transfer Service (DTS) is the fully managed, serverless, and no-code solution for batch loading files (including CSV from Cloud Storage) into BigQuery on a schedule. This is the 'ingest' part.

Transform: After loading the raw data into a staging table using DTS, the transformation can be done using BigQuery SQL. This transformation query can then be automated using a Scheduled Query in BigQuery, which is also a fully managed and no-code feature that runs on a schedule.

Fully Managed & No-Code: Both DTS for Cloud Storage and Scheduled Queries are native BigQuery features that are fully managed and configured through the console without requiring code, directly meeting the constraints.

Correcting other options:

A (Cloud Run + Script): Cloud Run requires writing a custom Python script, which violates the no-code requirement.

C (Dataflow + Apache Beam + Cloud Composer): This is a powerful, highly scalable ETL solution, but it requires writing custom code (Apache Beam) and requires setting up and managing a workflow orchestrator (Cloud Composer/Airflow), which violates both the fully managed (Dataflow is serverless, but the code/pipeline itself is custom and needs maintenance) and no-code requirements.

D (BigQuery pipelines): 'BigQuery pipelines' is not a distinct, official product name in the Google Cloud documentation that fulfills a no-code scheduled ETL. The closest product is the combination of DTS and Scheduled Queries, as described in option B.

'The BigQuery Data Transfer Service automates data movement into BigQuery on a scheduled, managed basis... The BigQuery Data Transfer Service supports loading data from Cloud Storage in one of the following formats: Comma-separated values (CSV)...' (Source: What is BigQuery Data Transfer Service? and Introduction to Cloud Storage transfers)

'A scheduled query is a query that BigQuery automatically runs at regular intervals. When you configure a scheduled query, you specify the GoogleSQL SELECT statement to run, the destination table for the query results, and the frequency of the query.' (Source: Scheduling queries)

This combination delivers a fully managed, no-code ELT (Extract-Load-Transform) pipeline.

Question 6

Which of these operations can you perform from the BigQuery Web UI?

AUpload a file in SQL format.

BLoad data with nested and repeated fields.

CUpload a 20 MB file.

DUpload multiple files using a wildcard.

Answer : B

You can load data with nested and repeated fields using the Web UI.

You cannot use the Web UI to:

- Upload a file greater than 10 MB in size

- Upload multiple files at the same time

- Upload a file in SQL format

All three of the above operations can be performed using the 'bq' command.

Question 7

You are working on a linear regression model on BigQuery ML to predict a customer's likelihood of purchasing your company's products. Your model uses a city name variable as a key predictive component in order to train and serve the model your data must be organized in columns. You want to prepare your data using the least amount of coding while maintaining the predictable variables. What should you do?

AUse SQL in BigQuery to transform the stale column using a one-hot encoding method, and make each city a column with binary values.

BCreate a new view with BigQuery that does not include a column which city information.

CCloud Data Fusion to assign each city to a region that is labeled as 1, 2 3, 4, or 5, and then use that number to represent the city in the model.

DUse TensorFlow to create a categorical variable with a vocabulary list. Create the vocabulary file and upload that as part of your model to BigQuery ML.

Answer : C

Google Cloud Certified Professional Data Engineer Exam Questions