How would you process a 20 gb file with an application only having access to 4gb memory ?
Javasearch.buggybread.com

Search Interview Questions


 More than 3000 questions in repository.
 There are more than 900 unanswered questions.
Click here and help us by providing the answer.
 Have a video suggestion.
Click Correct / Improve and please let us know.
Label / Company      Label / Company / Text

   



Interview Questions and Answers

 Q1. How would you process a 20 gb file with an application only having access to 4gb memory ?BigData
Ans. Load the file in chunks and then process. If we need to do analytic, we can process analytic information for those chunks and then reprocess the processed information from each chunk.

For example - we need to average all marks in the file. We can divide the file and load into 5 chunks and calculate average for each chunk. Then we can collect averages for all 5 chunks and then calculate the final average.

 Help us improve. Please let us know the company, where you were asked this question :   

   Like         Discuss         Correct / Improve     bigdata  processing big data     Asked in 1 Companies


Related Questions

 Is there any schema in mongo DB ?
  What is Apache Kafka ?
  What is a broker in Apache Kafka ?
  What is a Topic in Apache Kafka ?
  Have you used Kafka in your project ? If Yes, for what ?
  Does Kafka uses ZooKeeper ?
  What is a Sequence File?
  What is the input to the Reduce function ?
  Which of the following is the implementation language for Map Reduce Framework ?
 Can we have multiple threads consuming message stream from a single partition ?



Help us and Others Improve. Please let us know the questions asked in any of your previous interview.

Any input from you will be highly appreciated and It will unlock the application for 10 more requests.

Company Name:
Questions Asked: