Log file Processing using Machine Learning Methods

Author: Magda Tsintsadze
Co-authors: Manana Khachidze, Maia Archuadze
Keywords: log analysis, Machine Learning, Text processing
Annotation:

The paper discusses an example of how machine learning is used to automatically detect system failures. To understand the basic concept of machine learning for log file analysis, an example is discussed where there are three sources of data: 1) system counters, CPUs, hard drives, and network; 2) large number of diluted log files accumulated from system programs; 3) Error logs (application errors etc). For all information from these input sources, the appropriate type is determined using the Bayesian algorithm to categorize the logs into the corresponding solution (learn). As a result, the newly logged logs automatically fall into the appropriate category, making it easy to spot a specific error.

Lecture files:

Log file Processing using Machine Learning Methods [en]
ლოგ ფაილების დამუშავება მანქანური სწავლების მეთოდებით [ka]