At the terminus of every robotic manipulator is an end-effector. These are the hand-like devices or tools that directly interact with the environment. Collecting data from the vantage point of an end-effector is highly desirable, though difficult to accomplish in practice. Wired data transmission solutions which use internal cables suffer from signal degradation due to proximity to power conduits; while solutions which use external cables are subject to tangling and often impose limits on the robot’s kinematics. Finally, existing wireless data transmission solutions are rarely used due to the low transmission rate, high latency time and unstable connection.
This research focuses on developing a general approach to stream visual (RGBD) and force (haptic) data wirelessly to other devices. This consists of an architecture which interfaces with a suite of end-effector sensors and handles signal compression, transmission, and decompression efficiently. Additionally, this project involves a hardware design portion which aims to neatly package and mount this infrastructure at the robot end-effector while respecting power and kinematic constraints for the robot.
This research chose the Intel Joule, an embedded Linux board, as the infrastructure to handle wireless transmission. The embedded board uses a low-level Python networking package, which forms the connection to transmit data with other devices in the same network. This project implements packages to encode, transmit and decode the data stream with Python. The receiving device will decodes the compressed data and makes the data available to other devices in the same network.
This has been used by Food Manipulation Team for demo shown in NeurIPS, Montreal, Canada, 2018
~Still in Process~
And currently, we are developing the second generation, we will keep it updated here as well. We decided to use Jetson Nano to replace Intel Joule and developing couple food object detaction model especially for Embedded system like Nano.