Hortonworks HDPCD Exam Questions Instant Access

Question 1

Which one of the following files is required in every Oozie Workflow application?

Ajob.properties

BConfig-default.xml

CWorkflow.xml

DOozie.xml

Answer : C

Question 2

You have just executed a MapReduce job. Where is intermediate data written to after being emitted from the Mapper's map method?

AIntermediate data in streamed across the network from Mapper to the Reduce and is never written to disk.

BInto in-memory buffers on the TaskTracker node running the Mapper that spill over and are written into HDFS.

CInto in-memory buffers that spill over to the local file system of the TaskTracker node running the Mapper.

DInto in-memory buffers that spill over to the local file system (outside HDFS) of the TaskTracker node running the Reducer

EInto in-memory buffers on the TaskTracker node running the Reducer that spill over and are written into HDFS.

Answer : C

The mapper output (intermediate data) is stored on the Local file system (NOT HDFS) of each individual mapper nodes. This is typically a temporary directory location which can be setup in config by the hadoop administrator. The intermediate data is cleaned up after the Hadoop Job completes.

Question 3

Which Hadoop component is responsible for managing the distributed file system metadata?

ANameNode

BMetanode

CDataNode

DNameSpaceManager

Answer : A

Question 4

What types of algorithms are difficult to express in MapReduce v1 (MRv1)?

AAlgorithms that require applying the same mathematical function to large numbers of individual binary records.

BRelational operations on large amounts of structured and semi-structured data.

CAlgorithms that require global, sharing states.

DLarge-scale graph algorithms that require one-step link traversal.

EText analysis algorithms on large collections of unstructured text (e.g, Web crawls).

Answer : C

See 3) below.

Limitations of Mapreduce -- where not to use Mapreduce

While very powerful and applicable to a wide variety of problems, MapReduce is not the answer to every problem. Here are some problems I found where MapReudce is not suited and some papers that address the limitations of MapReuce.

1. Computation depends on previously computed values

If the computation of a value depends on previously computed values, then MapReduce cannot be used. One good example is the Fibonacci series where each value is summation of the previous two values. i.e., f(k+2) = f(k+1) + f(k). Also, if the data set is small enough to be computed on a single machine, then it is better to do it as a single reduce(map(data)) operation rather than going through the entire map reduce process.

2. Full-text indexing or ad hoc searching

The index generated in the Map step is one dimensional, and the Reduce step must not generate a large amount of data or there will be a serious performance degradation. For example, CouchDB's MapReduce may not be a good fit for full-text indexing or ad hoc searching. This is a problem better suited for a tool such as Lucene.

3. Algorithms depend on shared global state

Solutions to many interesting problems in text processing do not require global synchronization. As a result, they can be expressed naturally in MapReduce, since map and reduce tasks run independently and in isolation. However, there are many examples of algorithms that depend crucially on the existence of shared global state during processing, making them difficult to implement in MapReduce (since the single opportunity for global synchronization in MapReduce is the barrier between the map and reduce phases of processing)

Question 5

You want to understand more about how users browse your public website, such as which pages they visit prior to placing an order. You have a farm of 200 web servers hosting your website. How will you gather this data for your analysis?

AIngest the server web logs into HDFS using Flume.

BWrite a MapReduce job, with the web servers for mappers, and the Hadoop cluster nodes for reduces.

CImport all users' clicks from your OLTP databases into Hadoop, using Sqoop.

DChannel these clickstreams inot Hadoop using Hadoop Streaming.

ESample the weblogs from the web servers, copying them into Hadoop using curl.

Answer : A

Question 6

Indentify which best defines a SequenceFile?

AA SequenceFile contains a binary encoding of an arbitrary number of homogeneous Writable objects

BA SequenceFile contains a binary encoding of an arbitrary number of heterogeneous Writable objects

CA SequenceFile contains a binary encoding of an arbitrary number of WritableComparable objects, in sorted order.

DA SequenceFile contains a binary encoding of an arbitrary number key-value pairs. Each key must be the same type. Each value must be the same type.

Answer : D

SequenceFile is a flat file consisting of binary key/value pairs.

There are 3 different SequenceFile formats:

Uncompressed key/value records.

Record compressed key/value records - only 'values' are compressed here.

Block compressed key/value records - both keys and values are collected in 'blocks' separately and compressed. The size of the 'block' is configurable.

Question 7

You are developing a MapReduce job for sales reporting. The mapper will process input keys representing the year (IntWritable) and input values representing product indentifies (Text).

Indentify what determines the data types used by the Mapper for a given job.

AThe key and value types specified in the JobConf.setMapInputKeyClass and JobConf.setMapInputValuesClass methods

BThe data types specified in HADOOP_MAP_DATATYPES environment variable

CThe mapper-specification.xml file submitted with the job determine the mapper's input key and value types.

DThe InputFormat used by the job determines the mapper's input key and value types.

Answer : D

The input types fed to the mapper are controlled by the InputFormat used. The default input format, 'TextInputFormat,' will load data in as (LongWritable, Text) pairs. The long value is the byte offset of the line in the file. The Text object holds the string contents of the line of the file.

Note: The data types emitted by the reducer are identified by setOutputKeyClass() andsetOutputValueClass(). The data types emitted by the reducer are identified by setOutputKeyClass() and setOutputValueClass().

By default, it is assumed that these are the output types of the mapper as well. If this is not the case, the methods setMapOutputKeyClass() and setMapOutputValueClass() methods of the JobConf class will override these.

Hortonworks Data Platform Certified Developer HDPCD Exam Questions