Title:	EFFICIENT LOG MANAGEMENT USING OOZIE, PARQUET AND HIVE
Author(s):	Gopi Krishnan Nambiar
ISBN:	978-989-8533-66-1
Editors:	Yingcai Xiao and Ajith P. Abraham
Year:	2017
Edition:	Single
Keywords:	Data Management, Oozie, MapReduce, Parquet, Scalability
Type:	Short Paper
First Page:	325
Last Page:	328
Language:	English
Cover:
Full Contents:	click to dowload
Paper Abstract:	The software industry produces a large volume of data in the form of application logs. This data is useful since it can help debug issues in production, provide many hidden insights and is a treasure trove for data scientists and researchers. The volume of the data, however causes multiple issues with querying, storage and maintenance over a period of time. In this paper, we share the architecture and the steps we implemented for efficiently storing the application logs based on the data requirements from our partners and also discuss various optimization techniques adopted.
	Go Back