Hybrid job scheduling for improved cluster utilization

Name: Hybrid job scheduling for improved cluster utilization
Author: Arı, İsmail, Kocak, Uğur

İsim	Hybrid job scheduling for improved cluster utilization
Yazar	Arı, İsmail, Kocak, Uğur
Basım Tarihi:	2014
Basım Yeri	- Springer Science+Business Media
Tür	Kitap
Dil	İngilizce
Dijital	Evet
Yazma	Hayır
Kütüphane:	Özyeğin Üniversitesi
Demirbaş Numarası	978-3-642-54420-0
Kayıt Numarası	2ae1bd3f-83a6-4cb2-8e8d-763e096e8df5
Lokasyon	Computer Science
Tarih	2014
Notlar	Due to copyright restrictions, the access to the full text of this article is only available via subscription.
Örnek Metin	In this paper, we investigate the models and issues as well as performance benefits of hybrid job scheduling over shared physical clusters. Clustering technologies that are currently supported include MPI, Hadoop-MapReduce and NoSQL systems. Our proposed scheduling model is above the cluster-specific middleware and OS-level schedulers and it is complementary to them. First, we demonstrate that we can effectively schedule MPI, Hadoop, NoSQL jobs together by profiling them and then co-scheduling. Second, we find that it is better to schedule cluster jobs with different job characteristics together (CPU vs. I/O intensive) rather than two CPU-intensive jobs. Third, we use the learning outcome of this principle to design of a greedy sort-merge scheduler. Up to 37% savings in total job completion times are demonstrated. These savings are directly proportional to the cluster utilization improvements.
DOI	10.1007/978-3-642-54420-0_39
Cilt	8374

Kaynağa git Özyeğin Üniversitesi

Aramaya Dön

Özyeğin Üniversitesi

Kaynağa git

Hybrid job scheduling for improved cluster utilization

Yazar Arı, İsmail, Kocak, Uğur

Basım Tarihi 2014

Basım Yeri - Springer Science+Business Media

Tür Kitap

Dil İngilizce

Dijital Evet

Yazma Hayır

Kütüphane Özyeğin Üniversitesi

Demirbaş Numarası 978-3-642-54420-0

Kayıt Numarası 2ae1bd3f-83a6-4cb2-8e8d-763e096e8df5

Lokasyon Computer Science

Tarih 2014

Notlar Due to copyright restrictions, the access to the full text of this article is only available via subscription.

Örnek Metin In this paper, we investigate the models and issues as well as performance benefits of hybrid job scheduling over shared physical clusters. Clustering technologies that are currently supported include MPI, Hadoop-MapReduce and NoSQL systems. Our proposed scheduling model is above the cluster-specific middleware and OS-level schedulers and it is complementary to them. First, we demonstrate that we can effectively schedule MPI, Hadoop, NoSQL jobs together by profiling them and then co-scheduling. Second, we find that it is better to schedule cluster jobs with different job characteristics together (CPU vs. I/O intensive) rather than two CPU-intensive jobs. Third, we use the learning outcome of this principle to design of a greedy sort-merge scheduler. Up to 37% savings in total job completion times are demonstrated. These savings are directly proportional to the cluster utilization improvements.

DOI 10.1007/978-3-642-54420-0_39

Cilt 8374