Building a Reliable Remote Communication Device with Java ME JavaOne San Francisco 2013 - CON8004  Leonardo Lima – V2COM (llima@v2com.mobi) Tiago Padovani – V2COM (tpadovani@v2com.mobi)

Introduction

About the presenter (Leonardo) Software Development Manager at V2COM, leading the development of both embedded software for edge/field devices and server-side software that receives field data and enables fast communication with remote devices. Eight years of development and architectural experience on mobile, embedded and server platforms to enable management and operation of highly scalable and fast telemetry and network control systems.

About the presenter (Tiago) Lead Software Developer for Embedded at V2COM, leading the development of embedded software for edge/field devices. Six years of development and architectural experience on embedded Java platforms, building reliable communication devices that stays on the field for years.

About the presentation Explore how to build reliable remote communication devices that must stay online, connected, and transmitting field data without interruptions. Using Java ME, a solid foundation was built that can be used to deploy many kinds of applications in the field, with fast, reliable development. In this session, see what the difficulties, ideas, and insights were that led to this foundation, which is currently in use in more than 50,000 devices in Brazil and Latin America to connect more than 800,000 field devices.

V2COM is a leading Latin American provider of Smart Grid technologies and Advanced Metering Infrastructure (AMI). Our offer includes hardware, software and services that can reduce losses and increase water and energy efficiency, currently connecting more than 700.000 devices. Besides the economic gains for our customers, we develop technologies with significant impact on the environment (avoiding losses of important natural resources) and with social responsibility (our Smart Grid systems can be used to offer free internet access in challenging areas)

Awarded 2013 Java Business Innovation Award JCP member, nominated for EC election More interaction with community to foster Java ME development in this age of IoT!

V2COM clients Meter Reading Network Control

The scenario

Intelligenceware Suite Legacy Systems Intelligenceware Suite WAN Terminal LAN Devices IP Network Wireless Network Local Network

Use cases Automated Meter Reading (AMR) Network control Read “dumb meters” and send data over to Utility Both energy and water meters Can be any meter, really! Like our Zigbier refrigerator’s temperature sensor, with alarms for too cold or hot! Network control Remotely connect SCADA systems to field devices like reclosures “transparent pipe” between SCADA and Device DNP3/IEC104 protocols and proprietary as well

Use cases 1 3 6 4 2 5b 5a

The techonology used Gateway: Oracle Java Micro Edition 1 (Java profile IMP-NG version 2 & CLDC 1.1 Hl), embedded in a Gemalto TC65i module. Started with Siemens TC45, 6 years ago! Intelligenceware Suite: Oracle Java Enterprise Edition 5 Oracle Java Standard Edition 6 Oracle Utilities Meter Data Management 2.0

Living an embedded life

The 3 commandments “If the device has a power from source, it must be on” Our first commandment is to keep the device always on. If it can be on, it must be on, no matter what. “The firmware must be remotely upgradable” One of few certainties in software developments is that bugs always happen! So, we must be able to OTAP (over-the-air provision) the device “If it’s on, it must be online and connected (or trying to be...)” It doesn’t suffice to be on. It must be also be avaiable so that communication can happen

Hardware choices WatchDog Dual Sim Card Dual Serial Java ME One of the few always-must-have circuit for any device is a watchdog to turn it on in case it turns off or there’s a lock-up Dual Sim Card GPRS infrastructure is not very reliable, so it should be able to have option in case of carrier failure Dual Serial Many field devices have two serial ports, so being able to connect to both at the same time is very important Java ME Being able to run Java ME allows many advantages over C

Software choices - Separation of concerns “Hardcore” Abstracts board specifics details (like GPIO) “Zion” Handles common infrastructure (backend connection, message queues, locks and so on) Abstracts Modem commands, so there’s one for each avaiable module (Cinterion/Gemalto; Motorola; Telit) “App” Holds application logic - As tailored as clients need it to be Ranges from a simple Transparent (TN) to Automated Meter Reading (AMR)

Hardcore Handles board specifics Modem/AT Connection SMS/GPRS logic GPIO Watchdog

Zion Defines command shell Handle TCP communications Kitchen sink utils (math, string, locks, queue) Event Bus Extendable to handle “fatter” functionalities CSD SNMP monitoring

Application Handles only business logic and protocol Fast changes to acommodate business logic After the first few cicles, it stabilizes

Package Diagram mobi.v2com.zion... Application Control Hardcore Communication Event Protocol Tools Utilities ... and application specifics as well

Challenges Problems faced, solutions provided!

How to keep the device on? Problem: Module can throw Exceptions (EXIT CODEs) that shuts down the application Solution: Hardware Watchdog! Dedicated QG8 to handle only this (power on/off) 10 minutes timeout: either the firmware “swings” the GPIO, or it gets powered off then on Many other software watchdogs... Many other conditions can mean a “lock up”

How to keep the device online? Problem: GPRS connections can be unstable, specially when needed most: thunderstorms and harsh weather condition Solution: Dual-SIM cards, so if a network fails, there can be another Many soft watchdogs to retry attachments Turn on/off modem, like our phones Attach/Deattach

How to get behind public APN firewall? Problem: Can’t use a server socket while communicating on public networks! Solutions: Device polls server, asking for new commands This doesn’t work for always-on connection needs Expensive on resources Software Switch (server side)! Meet-in-the-middle approach

How to keep the TCP connection up? Problem: You don’t know the connection is broken until you have a blocking read() or write() TCP keep-alive is not enough No module support for this... Solution: Keep-Alive packets Application-level bytes sent to make sure connection is not off

How to keep the firmware updated? Problem: There are MANY devices deployed, each with a software version No one wants to mass-deploy... Not all updates are required for all devices Solution: Axon has control of each version Updates are dispatched as needed and/or cleared by client

How to debug in-device? Problem: Solution: No simple logging framework to write log to Log redirection works for serial port only, not remote connections Solution: Built a simple logging framework that handles debug level, message, threads

How to debug off-device? Problem: How can we run/test outside the framework? Many classes you depend on are hardware specific (Files, Connections) Solution: Built a simple “java abstraction layer” that implements javax.microedition classes (and proprietary ones too) that runs on Java SE on the desktop.

How to keep historical, debugging info? Problem: No long-term storage on devices to keep logs Flash memory can hold at most 30 minutes of logs The logs you need are the ones when device is unreachable, but alas, you can’t download it. Solution: Have a “black box” flight recorder Small, coded files that can hold critical information for weeks

How to synchronize access to resources? Problem: Access to system resources must be synchronized Serial device, system files synchronized can lead to unknown lock ups Some race conditions hard to track ando also some bugs... Solution: Custom implementation of Java SE’s Lock ReentrantLock inspired Added few methods for convenience tryLock, so that you lock if you can, but don’t fail if you can’t fastLock, for locks that should last than 2 secods

How to default parameters? Problem: Same firmware for different clients with different needs, lots of parameters Parameters for APN, Serial, Timings and such Flash memory can get corrupt and you lose your parameter files! Solution: Use JAD parameters JAD can be custom tailored to each client Can provide good defaults

Pending challenges... ... because you can’t win them all...

Many pain points are still there Make smaller binary fragments? And small updates over the air. Run multiple applications at the same time? More “permanent” and/or remote logging Security issues Better management of keys Anti-tampering measures

Incoming changes Java ME (r)evolution

Java ME 8 has much new features! Java ME 8 seems to have addressed some, if not most, this problems and abstractions Device I/O Access Point NIO ... And many others! Some still need works No concurrency support might mean no locks... We’ll work with JSRs to try and address the rest of them V2COM is joining the JCP EC to help foster Java ME evolution

Q?

Thanks! llima@v2com.mobi www.v2com.mobi