DILAF: A framework for distributed analysis of large-scale system logs for anomaly detection

Name: DILAF: A framework for distributed analysis of large-scale system logs for anomaly detection
Author: Astekin, M., Zengin, H., Sözer, Hasan

İsim	DILAF: A framework for distributed analysis of large-scale system logs for anomaly detection
Yazar	Astekin, M., Zengin, H., Sözer, Hasan
Basım Tarihi:	2019-02
Basım Yeri	- Wiley
Konu	Anomaly detection, Big data, Distributed systems, Log analysis, Machine learning, Parallel processing, Software architecture
Tür	Süreli Yayın
Dil	İngilizce
Dijital	Evet
Yazma	Hayır
Kütüphane:	Özyeğin Üniversitesi
Demirbaş Numarası	0038-0644
Kayıt Numarası	caf9f40e-1ede-4dff-9b54-3568a043d6b7
Lokasyon	Computer Science
Tarih	2019-02
Örnek Metin	System logs constitute a rich source of information for detection and prediction of anomalies. However, they can include a huge volume of data, which is usually unstructured or semistructured. We introduce DILAF, a framework for distributed analysis of large-scale system logs for anomaly detection. DILAF is comprised of several processes to facilitate log parsing, feature extraction, and machine learning activities. It has two distinguishing features with respect to the existing tools. First, it does not require the availability of source code of the analyzed system. Second, it is designed to perform all the processes in a distributed manner to support scalable analysis in the context of large-scale distributed systems. We discuss the software architecture of DILAF and we introduce an implementation of it. We conducted controlled experiments based on two datasets to evaluate the effectiveness of the framework. In particular, we evaluated the performance and scalability attributes under various degrees of parallelism. Results showed that DILAF can maintain the same accuracy levels while achieving more than 30% performance improvement on average as the system scales, compared to baseline approaches that do not employ fully distributed processing.
DOI	10.1002/spe.2653
Cilt	49

Kaynağa git Özyeğin Üniversitesi

Aramaya Dön

Özyeğin Üniversitesi

Kaynağa git

DILAF: A framework for distributed analysis of large-scale system logs for anomaly detection

Yazar Astekin, M., Zengin, H., Sözer, Hasan

Basım Tarihi 2019-02

Basım Yeri - Wiley

Konu Anomaly detection, Big data, Distributed systems, Log analysis, Machine learning, Parallel processing, Software architecture

Tür Süreli Yayın

Dil İngilizce

Dijital Evet

Yazma Hayır

Kütüphane Özyeğin Üniversitesi

Demirbaş Numarası 0038-0644

Kayıt Numarası caf9f40e-1ede-4dff-9b54-3568a043d6b7

Lokasyon Computer Science

Tarih 2019-02

Örnek Metin System logs constitute a rich source of information for detection and prediction of anomalies. However, they can include a huge volume of data, which is usually unstructured or semistructured. We introduce DILAF, a framework for distributed analysis of large-scale system logs for anomaly detection. DILAF is comprised of several processes to facilitate log parsing, feature extraction, and machine learning activities. It has two distinguishing features with respect to the existing tools. First, it does not require the availability of source code of the analyzed system. Second, it is designed to perform all the processes in a distributed manner to support scalable analysis in the context of large-scale distributed systems. We discuss the software architecture of DILAF and we introduce an implementation of it. We conducted controlled experiments based on two datasets to evaluate the effectiveness of the framework. In particular, we evaluated the performance and scalability attributes under various degrees of parallelism. Results showed that DILAF can maintain the same accuracy levels while achieving more than 30% performance improvement on average as the system scales, compared to baseline approaches that do not employ fully distributed processing.

DOI 10.1002/spe.2653

Cilt 49