READ Free Dumps For Cloudera- CCD-410
| Question ID 12523 | Youve written a MapReduce job that will process 500 million input records and generated
500 million key-value pairs. The data is not uniformly distributed. Your MapReduce job will
create a significant amount of intermediate data that it needs to transfer between mappers
and reduces which is a potential bottleneck. A custom implementation of which interface is
most likely to reduce the amount of intermediate data transferred across the network?
|
| Option A | Partitioner
|
| Option B | OutputFormat
|
| Option C | WritableComparable
|
| Option D | Writable
|
| Option E | InputFormat
|
| Option F | Combiner
|
| Correct Answer | F |
Explanation Explanation: Combiners are used to increase the efficiency of a MapReduce program. They are used to aggregate intermediate map output locally on individual mapper outputs. Combiners can help you reduce the amount of data that needs to be transferred across to the reducers. You can use your reducer code as a combiner if the operation performed is commutative and associative. Reference: 24 Interview Questions & Answers for Hadoop MapReduce developers, What are combiners? When should I use a combiner in my MapReduce Job?
| Question ID 12524 | A combiner reduces:
|
| Option A | The number of values across different keys in the iterator supplied to a single reduce method call.
|
| Option B | The amount of intermediate data that must be transferred between the mapper and reducer.
|
| Option C | The number of input files a mapper must process.
|
| Option D | The number of output files a reducer must produce.
|
| Correct Answer | B |
Explanation Explanation: Combiners are used to increase the efficiency of a MapReduce program. They are used to aggregate intermediate map output locally on individual mapper outputs. Combiners can help you reduce the amount of data that needs to be transferred across to the reducers. You can use your reducer code as a combiner if the operation performed is commutative and associative. The execution of combiner is not guaranteed, Hadoop may or may not execute a combiner. Also, if required it may execute it more then 1 times. Therefore your MapReduce jobs should not depend on the combiners execution. Reference: 24 Interview Questions & Answers for Hadoop MapReduce developers, What are combiners? When should I use a combiner in my MapReduce Job?