Databricks Certified Data Engineer Associate Exam Practice Test Instant Access

Question 1

Which of the following benefits is provided by the array functions from Spark SQL?

AAn ability to work with data in a variety of types at once

BAn ability to work with data within certain partitions and windows

CAn ability to work with time-related data in specified intervals

DAn ability to work with complex, nested data ingested from JSON files

EAn ability to work with an array of tables for procedural automation

Answer : D

The array functions from Spark SQL are a subset of the collection functions that operate on array columns1.They provide an ability to work with complex, nested data ingested from JSON files or other sources2.For example, theexplodefunction can be used to transform an array column into multiple rows, one for each element in the array3.Thearray_containsfunction can be used to check if a value is present in an array column4. Thearray_joinfunction can be used to concatenate all elements of an array column with a delimiter.These functions can be useful for processing JSON data that may have nested arrays or objects.Reference:1:Spark SQL, Built-in Functions - Apache Spark2:Spark SQL Array Functions Complete List - Spark By Examples3:Spark SQL Array Functions - Syntax and Examples - DWgeek.com4:Spark SQL, Built-in Functions - Apache Spark:Spark SQL, Built-in Functions - Apache Spark: [Working with Nested Data Using Higher Order Functions in SQL on Databricks - The Databricks Blog]

Question 2

Which two components function in the DB platform architecture's control plane? (Choose two.)

AVirtual Machines

BCompute Orchestration

CServerless Compute

DCompute

EUnity Catalog

Answer : B, E

Question 3

In which of the following scenarios should a data engineer use the MERGE INTO command instead of the INSERT INTO command?

AWhen the location of the data needs to be changed

BWhen the target table is an external table

CWhen the source table can be deleted

DWhen the target table cannot contain duplicate records

EWhen the source is not a Delta table

Answer : D

The MERGE INTO command is used to perform upserts, which are a combination of insertions and updates, based on a source table into a target Delta table1. The MERGE INTO command can handle scenarios where the target table cannot contain duplicate records, such as when there is a primary key or a unique constraint on the target table. The MERGE INTO command can match the source and target rows based on a merge condition and perform different actions depending on whether the rows are matched or not.For example, the MERGE INTO command can update the existing target rows with the new source values, insert the new source rows that do not exist in the target table, or delete the target rows that do not exist in the source table1.

The INSERT INTO command is used to append new rows to an existing table or create a new table from a query result2. The INSERT INTO command does not perform any updates or deletions on the existing target table rows.The INSERT INTO command can handle scenarios where the location of the data needs to be changed, such as when the data needs to be moved from one table to another, or when the data needs to be partitioned by a certain column2.The INSERT INTO command can also handle scenarios where the target table is an external table, such as when the data is stored in an external storage system like Amazon S3 or Azure Blob Storage3.The INSERT INTO command can also handle scenarios where the source table can be deleted, such as when the source table is a temporary table or a view4.The INSERT INTO command can also handle scenarios where the source is not a Delta table, such as when the source is a Parquet, CSV, JSON, or Avro file5.

1:MERGE INTO | Databricks on AWS

2: [INSERT INTO | Databricks on AWS]

3: [External tables | Databricks on AWS]

4: [Temporary views | Databricks on AWS]

5: [Data sources | Databricks on AWS]

Question 4

Identify how the count_if function and the count where x is null can be used

Consider a table random_values with below data.

What would be the output of below query?

select count_if(col > 1) as count_

a. count(*) as count_b.count(col1) as count_c from random_values col1

NULL -

A3 6 5

B4 6 5

C3 6 6

D4 6 6

Answer : A

Question 5

A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE.

The table is configured to run in Development mode using the Continuous Pipeline Mode.

Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?

AAll datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.

BAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist until the pipeline is shut down.

CAll datasets will be updated once and the pipeline will persist without any processing. The compute resources will persist but go unused.

DAll datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.

EAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.

Answer : B

The Continuous Pipeline Mode for Delta Live Tables allows the pipeline to run continuously and process data as it arrives. This mode is suitable for streaming ingest and CDC workloads that require low-latency updates. The Development mode for Delta Live Tables allows the pipeline to run on a dedicated cluster that is not shared with other pipelines. This mode is useful for testing and debugging the pipeline logic before deploying it to production. Therefore, the correct answer is B, because the pipeline will run continuously on a dedicated cluster until it is manually stopped, and the compute resources will be released only after the pipeline is shut down.Reference:Databricks Documentation - Configure pipeline settings for Delta Live Tables,Databricks Documentation - Continuous vs. triggered pipeline execution,Databricks Documentation - Development vs. production mode.

Question 6

Which of the following describes a scenario in which a data team will want to utilize cluster pools?

AAn automated report needs to be refreshed as quickly as possible.

BAn automated report needs to be made reproducible.

CAn automated report needs to be tested to identify errors.

DAn automated report needs to be version-controlled across multiple collaborators.

EAn automated report needs to be runnable by all stakeholders.

Answer : A

Databricks cluster pools are a set of idle, ready-to-use instances that can reduce cluster start and auto-scaling times. This is useful for scenarios where a data team needs to run an automated report as quickly as possible, without waiting for the cluster to launch or scale up. Cluster pools can also help save costs by reusing idle instances across different clusters and avoiding DBU charges for idle instances in the pool.Reference:Best practices: pools | Databricks on AWS,Best practices: pools - Azure Databricks | Microsoft Learn,Best practices: pools | Databricks on Google Cloud

Question 7

A data engineer is running code in a Databricks Repo that is cloned from a central Git repository. A colleague of the data engineer informs them that changes have been made and synced to the central Git repository. The data engineer now needs to sync their Databricks Repo to get the changes from the central Git repository.

Which of the following Git operations does the data engineer need to run to accomplish this task?

AMerge

BPush

CPull

DCommit

EClone

Answer : C

: To sync a Databricks Repo with the changes from a central Git repository, the data engineer needs to run the Git pull operation. This operation fetches the latest updates from the remote repository and merges them with the local repository. The data engineer can use the Pull button in the Databricks Repos UI, or use the git pull command in a terminal session. The other options are not relevant for this task, as they either push changes to the remote repository (Push), combine two branches (Merge), save changes to the local repository (Commit), or create a new local repository from a remote one (Clone).Reference:

Run Git operations on Databricks Repos

Git pull