Intelligent sensing in dynamic environments using Markov decision process
File(s)
Author(s)
Nanayakkara, Thrishantha
Halgamuge, Malka N
Sridhar, Prasanna
Madni, Asad M
Type
Journal Article
Abstract
In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically put the sensor nodes to sleep. The purpose of the learning algorithm is to couple the sensor’s sleeping behavior to the natural statistics of the environment hence that it can be in optimal harmony with changes in the environment, the sensors can sleep when steady environment and stay awake when turbulent environment. This paper presents theoretical and experimental validation of a reward based learning algorithm that can be implemented on an embedded sensor. The key contribution of the proposed approach is the design and implementation of a reward function that satisfies a trade-off between the above two mutually contradicting objectives, and a linear critic function to approximate the discounted sum of future rewards in order to perform policy learning.
Date Issued
2011-01-20
Date Acceptance
2011-01-18
Citation
Sensors, 2011, 11 (1), pp.1229-1242
ISSN
1424-8220
Publisher
MDPI AG
Start Page
1229
End Page
1242
Journal / Book Title
Sensors
Volume
11
Issue
1
Copyright Statement
© 2011 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article
distributed under the terms and conditions of the Creative Commons Attribution license
(http://creativecommons.org/licenses/by/3.0/.)
distributed under the terms and conditions of the Creative Commons Attribution license
(http://creativecommons.org/licenses/by/3.0/.)
License URL
Identifier
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000286592400065&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=1ba7043ffcc86c417c072aa74d649202
Subjects
Science & Technology
Physical Sciences
Technology
Chemistry, Analytical
Engineering, Electrical & Electronic
Instruments & Instrumentation
Chemistry
Engineering
sensor network
Markov decision process
sensing
reward shaping
REWARD
Publication Status
Published
Date Publish Online
2011-01-20