Augmenting reservoirs with higher order terms for resource efficient learning

Name: Augmenting reservoirs with higher order terms for resource efficient learning
Author: Oztop, Erhan, Asada, M., Celebi, Bedirhan

İsim	Augmenting reservoirs with higher order terms for resource efficient learning
Yazar	Oztop, Erhan, Asada, M., Celebi, Bedirhan
Basım Tarihi:	2024-01-01
Basım Yeri	- IEEE
Konu	Resource efficiency, Order neuron, High, Echo state networks, Reservoir computing
Tür	Belge
Dil	İngilizce
Dijital	Evet
Yazma	Hayır
Kütüphane:	Özyeğin Üniversitesi
Demirbaş Numarası	979-8-3503-5932-9
Kayıt Numarası	18772edd-36af-4a65-be11-db63ba0a5181
Lokasyon	Computer Science
Tarih	2024-01-01
Notlar	New Energy and Industrial Technology Development Organization ; Japan Science and Technology Agency ; Japan Society for the Promotion of Science ; Core Research for Evolutional Science and Technology
Örnek Metin	Reservoir computing provides an attractive alternative for time series data representation due to their gradientfree learning and energy efficient operation. In a 'reservoir computer' (RC), the reservoir is a random recurrent neural network that forms a high dimensional non-linear dynamical system which can be mapped to a desired sequence or dynamical system through linear regression. Although, the non-linearity of the reservoir gives its power, there is no guarantee that it is the most suitable for the task at hand. To this end, in this study, we propose to amplify the effectiveness of reservoirs by augmenting them with higher order terms computed based on reservoir unit activations. We name this model as Higher-orderaugmented Reservoir Computer (ha-RC). We test the efficacy of ha-RCs by using two types of tasks: time series representation and long-term prediction, i.e. learning a dynamical system. Our experiments show that with the proposed model, a given learning accuracy can be achieved with significantly less memory resource compared to the baseline of standard Reservoir Computer (sRC). This becomes possible as ha-RC can handle complex learning problems with much smaller reservoirs compared to sRC models. This memory efficiency directly translates to energy efficiency as less number of arithmetic operations are needed with ha-RCs to reach the same level of accuracy compared to sRCs. These points make our proposed model ideal for hardware implementation and edge computing.
DOI	10.1109/IJCNN60899.2024.10650952

Kaynağa git Özyeğin Üniversitesi

Aramaya Dön

Özyeğin Üniversitesi

Kaynağa git

Augmenting reservoirs with higher order terms for resource efficient learning

Yazar Oztop, Erhan, Asada, M., Celebi, Bedirhan

Basım Tarihi 2024-01-01

Basım Yeri - IEEE

Konu Resource efficiency, Order neuron, High, Echo state networks, Reservoir computing

Tür Belge

Dil İngilizce

Dijital Evet

Yazma Hayır

Kütüphane Özyeğin Üniversitesi

Demirbaş Numarası 979-8-3503-5932-9

Kayıt Numarası 18772edd-36af-4a65-be11-db63ba0a5181

Lokasyon Computer Science

Tarih 2024-01-01

Notlar New Energy and Industrial Technology Development Organization ; Japan Science and Technology Agency ; Japan Society for the Promotion of Science ; Core Research for Evolutional Science and Technology

Örnek Metin Reservoir computing provides an attractive alternative for time series data representation due to their gradientfree learning and energy efficient operation. In a 'reservoir computer' (RC), the reservoir is a random recurrent neural network that forms a high dimensional non-linear dynamical system which can be mapped to a desired sequence or dynamical system through linear regression. Although, the non-linearity of the reservoir gives its power, there is no guarantee that it is the most suitable for the task at hand. To this end, in this study, we propose to amplify the effectiveness of reservoirs by augmenting them with higher order terms computed based on reservoir unit activations. We name this model as Higher-orderaugmented Reservoir Computer (ha-RC). We test the efficacy of ha-RCs by using two types of tasks: time series representation and long-term prediction, i.e. learning a dynamical system. Our experiments show that with the proposed model, a given learning accuracy can be achieved with significantly less memory resource compared to the baseline of standard Reservoir Computer (sRC). This becomes possible as ha-RC can handle complex learning problems with much smaller reservoirs compared to sRC models. This memory efficiency directly translates to energy efficiency as less number of arithmetic operations are needed with ha-RCs to reach the same level of accuracy compared to sRCs. These points make our proposed model ideal for hardware implementation and edge computing.

DOI 10.1109/IJCNN60899.2024.10650952