|
Title:
|
MAPREDUCE-BASED ALGORITHMS FOR OUTLIER DETECTION IN ENERGY DATA |
|
Author(s):
|
Maxim Shcherbakov, Anton Tyukov |
|
ISBN:
|
978-989-8704-10-8 |
|
Editors:
|
Ajith P. Abraham, Antonio Palma dos Reis and Jörg Roth |
|
Year:
|
2014 |
|
Edition:
|
Single |
|
Keywords:
|
Energy data, outlier detection, map reduce, python, streaming. |
|
Type:
|
Poster/Demonstration |
|
First Page:
|
257 |
|
Last Page:
|
259 |
|
Language:
|
English |
|
Cover:
|
|
|
Full Contents:
|
click to dowload 
|
|
Paper Abstract:
|
Increasing of data volume in energy domain requires distributed warehouses and techniques of data processing. Hadoop ecosystem could be considered as ad-hoc technology of big data handling. However, it requires adoption of classical data algorithms to MapReduce functional paradigm. Authors developed, deployed and tested outlier detection algorithms for energy data using Python language and MapReduce paradigm. They are called MapReduce-based batch algorithms for outlier detection and include (i) rule-based outlier detection algorithm, (ii) outlier detection algorithm based on time stamps analysis over daily segment of data. Algorithms implementation in Python and Streaming environment is shown in demonstration scenario. |
|
|
|
|
|
|
