Hortonworks HDPCD Exam Practice Test Instant Access

Question 1

You need to perform statistical analysis in your MapReduce job and would like to call methods in the Apache Commons Math library, which is distributed as a 1.3 megabyte Java archive (JAR) file. Which is the best way to make this library available to your MapReducer job at runtime?

AHave your system administrator copy the JAR to all nodes in the cluster and set its location in the HADOOP_CLASSPATH environment variable before you submit your job.

BHave your system administrator place the JAR file on a Web server accessible to all cluster nodes and then set the HTTP_JAR_URL environment variable to its location.

CWhen submitting the job on the command line, specify the --libjars option followed by the JAR file path.

DPackage your code and the Apache Commands Math library into a zip file named JobJar.zip

Answer : C

The usage of the jar command is like this,

Usage: hadoop jar <jar> [mainClass] args...

If you want the commons-math3.jar to be available for all the tasks you can do any one of

these

1. Copy the jar file in $HADOOP_HOME/lib dir

2. Use the generic option -libjars.

Question 2

Given the following Hive commands:

Which one of the following statements Is true?

AThe file mydata.txt is copied to a subfolder of /apps/hive/warehouse

BThe file mydata.txt is moved to a subfolder of /apps/hive/warehouse

CThe file mydata.txt is copied into Hive's underlying relational database 0.

DThe file mydata.txt does not move from Its current location in HDFS

Answer : A

Question 3

Indentify the utility that allows you to create and run MapReduce jobs with any executable or script as the mapper and/or the reducer?

AOozie

BSqoop

CFlume

DHadoop Streaming

Emapred

Answer : D

Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run Map/Reduce jobs with any executable or script as the mapper and/or the reducer.

Question 4

Which one of the following is NOT a valid Oozie action?

Amapreduce

Bpig

Chive

Dmrunit

Answer : D

Question 5

Examine the following Hive statements:

Assuming the statements above execute successfully, which one of the following statements is true?

AEach reducer generates a file sorted by age

BThe SORT BY command causes only one reducer to be used

CThe output of each reducer is only the age column

DThe output is guaranteed to be a single file with all the data sorted by age

Answer : A

Question 6

The Hadoop framework provides a mechanism for coping with machine issues such as faulty configuration or impending hardware failure. MapReduce detects that one or a number of machines are performing poorly and starts more copies of a map or reduce task. All the tasks run simultaneously and the task finish first are used. This is called:

ACombine

BIdentityMapper

CIdentityReducer

DDefault Partitioner

ESpeculative Execution

Answer : E

Speculative execution: One problem with the Hadoop system is that by dividing the tasks across many nodes, it is possible for a few slow nodes to rate-limit the rest of the program. For example if one node has a slow disk controller, then it may be reading its input at only 10% the speed of all the other nodes. So when 99 map tasks are already complete, the system is still waiting for the final map task to check in, which takes much longer than all the other nodes.

By forcing tasks to run in isolation from one another, individual tasks do not know where their inputs come from. Tasks trust the Hadoop platform to just deliver the appropriate input. Therefore, the same input can be processed multiple times in parallel, to exploit differences in machine capabilities. As most of the tasks in a job are coming to a close, the Hadoop platform will schedule redundant copies of the remaining tasks across several nodes which do not have other work to perform. This process is known as speculative execution. When tasks complete, they announce this fact to the JobTracker. Whichever copy of a task finishes first becomes the definitive copy. If other copies were executing speculatively, Hadoop tells the TaskTrackers to abandon the tasks and discard their outputs. The Reducers then receive their inputs from whichever Mapper completed successfully, first.

Note:

* Hadoop uses 'speculative execution.' The same task may be started on multiple boxes. The first one to finish wins, and the other copies are killed.

Failed tasks are tasks that error out.

* There are a few reasons Hadoop can kill tasks by his own decisions:

a) Task does not report progress during timeout (default is 10 minutes)

b) FairScheduler or CapacityScheduler needs the slot for some other pool (FairScheduler) or queue (CapacityScheduler).

c) Speculative execution causes results of task not to be needed since it has completed on other place.

Question 7

Which one of the following statements is true regarding a MapReduce job?

AThe job's Partitioner shuffles and sorts all (key.value) pairs and sends the output to all reducers

BThe default Hash Partitioner sends key value pairs with the same key to the same Reducer

CThe reduce method is invoked once for each unique value

DThe Mapper must sort its output of (key.value) pairs in descending order based on value

Answer : A

Hortonworks HDPCD Hortonworks Data Platform Certified Developer Exam Practice Test