Databricks Certified Data Engineer Associate Exam Questions Instant Access

Question 1

A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE.

The table is configured to run in Production mode using the Continuous Pipeline Mode.

Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?

AAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.

BAll datasets will be updated once and the pipeline will persist without any processing. The compute resources will persist but go unused.

CAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.

DAll datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.

EAll datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.

Answer : C

: In Production mode, the pipeline runs continuously and updates the output tables whenever new data is available in the input sources. The compute resources are allocated on demand and released when the pipeline is stopped. This mode is suitable for production workloads that require high availability and reliability.Reference:Configure pipeline settings for Delta Live Tables,Tutorial: Run your first Delta Live Tables pipeline,Building Reliable Data Pipelines Using DataBricks' Delta Live Tables

Question 2

The Delta transaction log for the 'students' tables is shown using the 'DESCRIBE HISTORY students' command. A Data Engineer needs to query the table as it existed before the UPDATE operation listed in the log.

Which command should the Data Engineer use to achieve this? (Choose two.)

ASELECT * FROM students@v4

BSELECT * FROM students TIMESTAMP AS OF '2024-04-22T 14:32:47.000+00:00'

CSELECT * FROM students FROM HISTORY VERSION AS OF 3

DSELECT * FROM students VERSION AS OF 5

ESELECT * FROM students TIMESTAMP AS OF '2024-04-22T 14:32:58.000+00:00'

Answer : A, B

Question 3

An engineering manager uses a Databricks SQL query to monitor ingestion latency for each data source. The manager checks the results of the query every day, but they are manually rerunning the query each day and waiting for the results.

Which of the following approaches can the manager use to ensure the results of the query are updated each day?

AThey can schedule the query to refresh every 1 day from the SQL endpoint's page in Databricks SQL.

BThey can schedule the query to refresh every 12 hours from the SQL endpoint's page in Databricks SQL.

CThey can schedule the query to refresh every 1 day from the query's page in Databricks SQL.

DThey can schedule the query to run every 1 day from the Jobs UI.

EThey can schedule the query to run every 12 hours from the Jobs UI.

Answer : C

Databricks SQL allows users to schedule queries to run automatically at a specified frequency and time zone. This can help users to keep their dashboards or alerts updated with the latest data. To schedule a query, users need to do the following steps:

In the Query Editor, click Schedule > Add schedule to open a menu with schedule settings.

Choose when to run the query. Use the dropdown pickers to specify the frequency, period, starting time, and time zone. Optionally, select the Show cron syntax checkbox to edit the schedule in Quartz Cron Syntax.

Choose More options to show optional settings. Users can also choose a name for the schedule, and a SQL warehouse to power the query.

Click Create. The query will run automatically according to the schedule.

The other options are incorrect because they do not refer to the correct location or frequency to schedule the query. The query's page in Databricks SQL is the place where users can edit, run, or schedule the query. The SQL endpoint's page in Databricks SQL is the place where users can manage the SQL warehouses and SQL endpoints. The Jobs UI is the place where users can create, run, or schedule jobs that execute notebooks, JARs, or Python scripts.Reference:Schedule a query,What are Databricks SQL alerts?,Jobs.

Question 4

Which method should a Data Engineer apply to ensure Workflows are being triggered on schedule?

AScheduled Workflows require an always-running cluster, which is more expensive but reduces processing latency.

BScheduled Workflows process data as it arrives at configured sources.

CScheduled Workflows can reduce resource consumption and expense since the cluster runs only long enough to execute the pipeline.

DScheduled Workflows run continuously until manually stopped.

Answer : C

Question 5

A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table.

The code block used by the data engineer is below:

If the data engineer only wants the query to process all of the available data in as many batches as required, which of the following lines of code should the data engineer use to fill in the blank?

AprocessingTime(1)

Btrigger(availableNow=True)

Ctrigger(parallelBatch=True)

Dtrigger(processingTime='once')

Etrigger(continuous='once')

Answer : B

https://spark.apache.org/docs/latest/api/python/reference/pyspark.ss/api/pyspark.sql.streaming.DataStreamWriter.trigger.html

Question 6

A data engineer is attempting to drop a Spark SQL table my_table. The data engineer wants to delete all table metadata and data.

They run the following command:

DROP TABLE IF EXISTS my_table

While the object no longer appears when they run SHOW TABLES, the data files still exist.

Which of the following describes why the data files still exist and the metadata files were deleted?

AThe table's data was larger than 10 GB

BThe table's data was smaller than 10 GB

CThe table was external

DThe table did not have a location

EThe table was managed

Answer : C

An external table is a table that is defined in the metastore and points to an existing location in the storage system. When you drop an external table, only the metadata is deleted from the metastore, but the data files are not deleted from the storage system. This is because external tables are meant to be shared by multiple applications and users, and dropping them should not affect the data availability. On the other hand, a managed table is a table that is defined in the metastore and also managed by the metastore. When you drop a managed table, both the metadata and the data files are deleted from the metastore and the storage system, respectively. This is because managed tables are meant to be exclusive to the application or user that created them, and dropping them should free up the storage space. Therefore, the correct answer is C, because the table was external and only the metadata was deleted when the table was dropped.Reference:Databricks Documentation - Managed and External Tables,Databricks Documentation - Drop Table

Question 7

A data engineer only wants to execute the final block of a Python program if the Python variable day_of_week is equal to 1 and the Python variable review_period is True.

Which of the following control flow statements should the data engineer use to begin this conditionally executed code block?

Aif day_of_week = 1 and review_period:

Bif day_of_week = 1 and review_period = 'True':

Cif day_of_week == 1 and review_period == 'True':

Dif day_of_week == 1 and review_period:

Eif day_of_week = 1 & review_period: = 'True':

Answer : D

In Python, the==operator is used to compare the values of two variables, while the=operator is used to assign a value to a variable. Therefore, option A and E are incorrect, as they use the=operator for comparison. Option B and C are also incorrect, as they compare thereview_periodvariable to a string value'True', which is different from the boolean valueTrue. Option D is the correct answer, as it uses the==operator to compare theday_of_weekvariable to the integer value1, and theandoperator to check if both conditions are true. If both conditions are true, then the final block of the Python program will be executed.Reference: [Python Operators], [Python If ... Else]