1. Alexis Maldonado & Georg Bartels
Specifications & Requirements from Universität Bremen
Alexis Maldonado & Georg Bartels
Universität Bremen

2. Table of Contents
Scenario Contributions
Hardware Platforms & Testbeds
Software Components Specifications
Software Components Requirements

3. Scenario Contributions

4. Bremen sees itself driving both scenario processes
Autonomous scanning unit as robotic platform
Continuous updates of semantic store maps
Know which SKU per facing
Know how full per facing
Know background information of SKU (meshes, weight, etc.)

5. 1b) Shelf monitoring: perceiving products on shelves
Scenario 1
1a) Layout identification: perceiving bar-codes on price labels per facing
1b) Shelf monitoring: perceiving products on shelves
No counting of individual products with fixed sensor
Distinguish full, partially full and empty facings
Detect misplaced products at the front of facings

6. Bremen sees itself only in a minor role for this scenario.
Not involved in the multi-robot control /coordination.
Not involved in depalletizing.
Bremen does provide the semantic store maps
Provide query interface for accessing the required data for this scenario.
Model the required SKU background knowledge for this scenario.

7. Scenario 3
Bremen sees itself driving this scenario together with other scientific partners
Keep specs and evaluation criteria as open as possible
Autonomous transport unit and mobile handling unit as robotic platforms
Focus on issues in mobile manipulation not complex multi- robot coordination

8. Hardware Platforms & Testbeds

9. Platforms at Bremen Mockup
Mockup environment almost ready and available to all partners as of right now
“Boxy” robot as stand- in for mobile handling unit and autonomous scanning unit
Wrist-mounted Intel sensor
Additional omni- directional robot as autonomous transport unit.

10. Platforms at Bremen Mockup
Strategy until 1st review
Push Scenario 1 and Scenario 3 in Bremen as far as possible
Have prototypes running with partner software fully integrated on Bremen platforms
Obtain a realistic impression of needs of (and holes in?) both Scenarios
To do this, Bremen has to go beyond its role as a scientific partner and also play the role of an integrator
Willing to do this but only on our platforms.
Committed to open source and open data research.

11. Platforms at DM pilot shop
To ensure successful transfer of software developed in 1st phase
Robotic units developed in REFILLS need to come with equivalent (ideally identical) ROS interfaces
Delivery dates in work plan WP2 mean hardware and software ready and tested
Location of DM pilot shop?
Technology transfer to industry partners not a goal

12. Software Component Specifications

13. Contain the ‘Realogram’ information
Semantic Store Maps
Contain the ‘Realogram’ information
3D representation of the shelves with product facings over time
3D representation of products on shelves over time
Background knowledge of SKUs
Runtime interface to query, add, edit, and delete of information
Export information into files for exchange and persistent storage
Import information from files
Graphical 3D user interface

14. Visual Perception Framework
Detection, recognition, and localization of products on shelves, free shelf space, and bar-codes
Robust, flexible, and fast performance
Render expected images using current belief state
Specify specific perception tasks through DSL
Optimized low-level routines
No counting of individual products; free shelf space only qualitatively

15. Plan Interpretation System
Provide plan-based and knowledge- enabled plan interpretation systems
Acts as high-level executive with failure handling
Logs performed actions and failures for subsequent analysis and learning
Programming on an abstract level using entity descriptions
Tight integration with perception system, reactive controllers, and manipulation knowledge base

16. Manipulation Knowledge Base
Calculates context- specific motion commands for reactive control layer
Collision free paths
Parametrize low-level controllers with object- specific information
Parametrize low-level event detection routines
Configure low-level event reactions
Aim for stereotypic motions

17. Software Component Requirements

18. SLAM maps and semantic store maps need to be aligned
Coordinate with partners that do SLAM on REFILLS robots
Clearly defined file format for data input and output
Unclear requirements
How many shelves?
How many facings?
How many products per facings?
How many scans stored in maps?
Multi-client support?
Support for certain batch queries?
Runtime requirements for specific queries

19. Visual Perception Framework
Various ROS interfaces needed
Consumes streams of RGB-D and RGB data via topics from sensors
Offers service interface to trigger perception tasks (with DSL to specify perception tasks and return perception results)
Uses service interface to semantic store maps to obtain environment maps and object information
Uses service interface to manipulation knowledge bases to obtain robot information (proprioception, sensors placements, telemetry over time)

20. Plan Interpretation System
Various ROS interfaces needed
Queries perception framework for perception tasks
Queries semantic store maps for environment knowledge
Queries manipulation knowledge bases for motion commands
Commands reactive control layer
No direct direct interface to robots
Reactive control layer for commands and non-visual perception
Perception framework for visual perception
Manipulation knowledge bases for proprioception

21. Manipulation Knowledge Base
Need URDF models of REFILLS robots
Autonomous scanning unit, autonomous transport unit, mobile handling unit
REFILLS robots need to provide various ROS interfaces
Telemetry through ROS topics
SLAM via ROS topics
Base, gripper, pan-tilt, and arm movements via ROS actions or topics (to be detailed in D2.1, more informed after first project phase)
Uses service interface to semantic store maps to obtain environment models
Uses service interface to motion planning suite (probably MoveIt!)

22. Suggested target operating systems and ROS versions
Basic requirements
Suggested target operating systems and ROS versions
First phase: Ubuntu and ROS Kinetic
Rest of REFILLS: Ubuntu and ROS M
Unclear requirements
Required computation performance
Required network characteristics

23. Thank you for your attention!
Q&A
Thank you for your attention!