1. Neeraj Jaggi ASSISTANT PROFESSOR DEPT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE WICHITA STATE UNIVERSITY 1 Rechargeable Sensor Activation under Temporally Correlated Events

2. Outline Sensor Networks Rechargeable Sensor System  Design of energy-efficient algorithms  Activation question – Single sensor scenario Temporally correlated event occurrence  Perfect state information  Structure of optimal policy  Imperfect state information  Practical algorithm with performance guarantees 2 Neeraj Jaggi Dept of EECS Wichita State University

3. 3 Sensor Nodes  Tiny, low cost Devices  Prone to Failures  Redundant Deployment  Rechargeable Sensor Nodes Range of Applications Important Issues  Energy Management  Quality of Coverage Sensor Networks Neeraj Jaggi Dept of EECS Wichita State University

4. 4 Rechargeable Sensor System Quality of Coverage Discharge Recharge Event Phenomena Renewable Energy Randomness Spatio-temporal Correlations Activation Policy Control Rechargeable Sensors Neeraj Jaggi Dept of EECS Wichita State University

5. Research Question 5 How should a sensor be activated (“switched on”) dynamically so that the quality of coverage is maximized over time ? A sensor became ready. What should it do ?  Activate itself now :  Gain some utility in the short-term  Activate itself later :  No utility in the short term  Activate when the system “needs it more” Neeraj Jaggi Dept of EECS Wichita State University

6. Temporal Correlations 6 Event Process (e.g. Forest fire)  On period (HOT)  Off period (COLD)  Correlation probabilities 0.5 < (, ) < 1 ( = = 0.8) Performance Criteria – Single Sensor Node  Fraction of Events Detected over time Neeraj Jaggi Dept of EECS Wichita State University

7. Sensor Energy Consumption Model 7 Discrete Time Energy Model  Operational Cost (  1 )  Detection Cost (  2 )  Recharge Rate (qc)  Probability (q)  Amount (c) recharge sensor not activated (no discharge) qc activation policy K δ1δ1 discharge - Off period discharge - On period δ1+δ2δ1+δ2 sensor activated Neeraj Jaggi Dept of EECS Wichita State University

8. System Observability 8 Perfect State Information  Sensor can always observe state of event process (even while inactive) Imperfect State Information  Inactive sensor can not observe event process Neeraj Jaggi Dept of EECS Wichita State University

9. Approach/Methodology 9 Perfect State Information  Formulate Markov Decision Problem (MDP)  Structure of Optimal Policy Imperfect State Information  Formulate Partially Observable MDP (POMDP)  Transform POMDP to equivalent MDP (Known techniques)  Structure of Optimal Policy  Near-optimal practical Algorithms Neeraj Jaggi Dept of EECS Wichita State University

10. Perfect State Information 10 Markov Decision Process  State Space = {(L, E); 0 ≤ L ≤ K, E є [0, 1]}  L – Current Energy Level, E – On/Off period  Reward r– one if event detected; zero otherwise  Action u є [0, 1]; Transition probabilities p Optimality equation (average reward criteria)  h* – state variables  λ* – optimal reward Neeraj Jaggi Dept of EECS Wichita State University

11. Perfect State Information (contd.) 11 Approximate Solution  Closed form solution for h* does not seem to exist Value Iteration  Activation Algorithm  L << K  Sensitive to system parameters when L ~ K Optimality equation (average reward criteria)  H* – variables  Lambda* – optimal reward Neeraj Jaggi Dept of EECS Wichita State University

12. Perfect State Information (contd.) 12 Optimal Policy Structure  Randomized algorithm  P* is directly proportional to the recharge rate  Energy balance  Average recharge rate equals average discharge rate in steady state On Period ? Activate Yes No Sufficient Energy ? No Do Not Activate Yes Prob. ≤ P* ? Yes No Neeraj Jaggi Dept of EECS Wichita State University

13. Imperfect State Information 13 Partially Observable Markov Decision Process  State Space  Observation Space  Optimal actions depend on current and past observations (y) and on past actions (u) Transformation to equivalent MDP 1  State – Information vector Z t of length |X|  Z t+1 is recursively computable given Z t, u t and y t+1  Z t forms a completely observable MDP  Equivalent rewards and actions 1 Neeraj Jaggi Dept of EECS Wichita State University

14. Equivalent MDP Structure 14 Active Sensor – Observation = (L, 1) or (L, 0)  State is the same as observation  Z t has only one non-zero component Inactive Sensor – Observation = (L, Φ)  Let state last observed = E, number of time slots inactive = i  Z t has only two non-zero components  Let p i = prob. that event process changed state from E to 1- E in i time slots  State = (L, E) with prob. 1 - p i  State = (L, 1 – E) with prob. p i  Z t is a function of (L, E, i) Neeraj Jaggi Dept of EECS Wichita State University

15. Transformed MDP State Space – (L, E, t)  L – Current Energy Level  E – State of Event process last observed  t – Number of time slots spent in inactive state Optimal Policy Structure f 0 – (L, 0, t), f 1 – (L, 1, t)  [  1 =c=1,  2 = 2, = 0.6, = 0.9, q = 0.1] Imperfect State Information (contd.) 15 Off Period – Reluctant Wakeup On Period – Aggressive Wakeup Neeraj Jaggi Dept of EECS Wichita State University

16. Practical Algorithm 16 Correlation dependent Wakeup (CW)  Activate during On Periods; Deactivate during Off Sleep Interval (SI*)  Derived using energy balance during a renewal interval  -optimal (  ~ O(1/β)); β =  2 /  1 A – Active I – Inactive Y – On, N – Off SI – sleep duration t 1, t 2 – renewal instances Y YYYY N AAAAAAI Y YYN AAAAI SI I t t2t2 t1t1 Neeraj Jaggi Dept of EECS Wichita State University

17. Simulation Results 17 Energy balancing Sleep Interval SI* [ = 0.6, = 0.9, SI * = 7][ = 0.7, = 0.8, SI * = 18] [  1 = c = 1,  1 = 6, q = 0.5, K = 2400] Neeraj Jaggi Dept of EECS Wichita State University

18. Contributions 18 Structure of Optimal Policy EB Policy is Optimal for Perfect State Information EB Policy is near Optimal for Imperfect State Information  Coauthors Prof. Koushik Kar, Rensselaer Polytechnic Institute Prof. Ananth Krishnamurthy, Univ. of Wisconsin Madison  5 th International Symposium on Modeling and Optimization in Mobile Ad hoc and Wireless Networks (WIOPT) April 2007  ACM/KLUWER Wireless Networks 2008 (Accepted ) Neeraj Jaggi Dept of EECS Wichita State University

19. Q & A 19 THANK YOU !! Neeraj Jaggi Dept of EECS Wichita State University

20. Policies – AW, CW 20 AW (Aggressive Wakeup) Policy  Activate whenever L ≥  2 +  1  Ignores temporal correlations  Optimal if no temporal correlations CW (Correlation dependent Wakeup) Policies  Activate during On periods; deactivate during Off  Upper Bound ( U * CW )  State unobservable during inactive state  Performance depends upon sleep duration Neeraj Jaggi Dept of EECS Wichita State University How long should sensor sleep ?

21. MDP – State Transitions 21 State (L, 1): L ≥  2 +  1 Action u = 1 (activate)  Next state :  (L + qc – δ 1 – δ 2, 1) with probability q.p c on  (L + qc – δ 1, 0) with probability q.(1 – p c on )  (L – δ 1 – δ 2, 1) with probability (1 – q ).p c on  (L – δ 1, 0) with probability (1 – q ).(1 – p c on )  Reward r = 1 with probability p c on ; 0 otherwise. Action u = 0 (deactivate)  Next state :  (L + qc, 1) with probability q.p c on  (L + qc, 0) with probability q.(1 – p c on )  (L, 1) with probability (1 – q).p c on  (L, 0) with probability (1 – q).(1 – p c on )  Reward r = 0 Neeraj Jaggi Dept of EECS Wichita State University