Robotics technology such as live streaming is moving into mainstream business. Be ready to win.
There's a huge increase in people building 360 applications with live streaming and API control. Building your own robot is a great way to learn how to use these technologies in your business strategy.
Jesse Casman
President of Oppkey360° RTSP Streaming From Your Own Robot
Stream 4K 360 video with 400ms to 1000ms latency.
Software and parts are fully explained in the meetup. Build your own robot for under $200 in parts (excluding cost of camera).
Easily build this demonstration with the RTSP plug-in on a THETA V or Z1. Extend the architecture by connecting the camera to a Jetson and using gstreamer or Janus WebRTC server to get the stream to another computer.
Learn more at the meetup
5 Critical Components in Your 360° Robot
Design a strategy for transmission, viewing, processing, control and power
A 360° camera has slighty different requirements compared to a standard USB or CSI camera. Learn what it takes to add 360° vision to your project.
1. Transmission
The meetup demonstration will use software that runs inside the RICOH THETA V or RICOH THETA SC2 to stream video over Wi-Fi. You can use other technologies such as using a modified version of libuvc to live stream high-quality video over a USB cable to a Jetson Nano. The Nano can then relay the stream over an IP network to another computer running a program such as gscam on ROS.
2. Viewing
Unlike a standard camera, the stream will be in equirectangular format. You can use technologies such as Unity or A-Frame to navigate the scene.
Process
You may need to adjust your object detection software to handle the equirectangular frames from a 360 camera.
Control
360° cameras generally use a modified version of the OSC API from Google. You can also use a USB API based on MTP.
Power
The camera receives power over USB. You can control on/off behavior as well as sleep/wake using the USB
Advanced Camera Power Control - On, Off, Sleep, Wake
Remotely control camera on and off or build a touchscreen for your robot.
Use ssh or HTTP to connect to a Raspberry Pi or Nano and control the THETA power. You can also build an touchscreen interface for your camera.
Remote Power On Using ssh from Windows 10 control dashboard
Touchscreen controller connected to Raspberry Pi
Using official Raspberry Pi touchscreen and Flutter to build interface. The Raspberry Pi is connected to the THETA with a USB cable.
Learn more at the meetup
MotionJPEG Streaming
You can quickly test streaming from your robot with MotionJPEG.
In this demonstration, we use an SC2 for Business, which doesn't support streaming over a USB cable.
Learn more at the meetup
A Future Business Opportunity in 360 Live Streaming
Today, you can transmit live video in 360 degrees and video to provide a unique virtual experience for your customers and stakeholders.
Linux allows an easier way to use technologies like GStreamer to get the video stream to Video4Linux2-compliant applications.
Learn more at the meetup
Open Pose running on NVIDIA Jetson Nano
The output of the gst sample program can be used with the
v4l2loopback kernel module
to create V4L2 loopback devices
that appear on /dev/video1 or /dev/video2.
These devices can then be accessed by programs such as
OpenPose.
Learn more at the meetup
Building libuvc-theta and gstreamer sample program on Ubuntu 20.04 Focal Fossa.
- Grab the patched version of libuvc from GitHub.
- Clone libuvc-theta-sample from GitHub
- build and install libuvc. Make sure you build the patched branch.
- build the sample app
- Plug RICOH THETA into Linux USB port. Put into live mode with the mode button on the side of the camera or with the USB API for StillCaptureMode which you need to set to Live streaming with value 0x8005.
Learn more at the meetup