object_detection_openvino

Overview

An implementation of YOLO and Mobilenet-SSD object detection with a ROS2 interface and enhanced processor utilization using OpenVINO model optimization tools. It can be use with any Myriad X, i.e.: Intel Neural Compute Stick 2. This package is designed on async api of Intel OpenVINO and allows an easy setup for object detection.

If you provide a pointcloud image (from an Intel RealSense Camera, for example) you can obtain 3d boxes and markers in Rviz2.

Keywords: ROS, ROS2, OpenVino, RealSense, OpenCV, object_detection

License

The source code is released under a Apache license 2.0.

Author: Alberto Tudela

The object_detection_openvino package has been tested under Rviz2 Galactic on Ubuntu 20.04. This is research code, expect that it changes often and any fitness for a particular purpose is disclaimed.

Installation

Building from Source

Dependencies

• Robot Operating System (ROS) (middleware for robotics),
• Intel OpenVINO 2021.4.1 (toolkit for deep learning),
• OpenCV 3 (Computer Vision library),
• Boost (C++ source libraries)

Building

To build from source, clone the latest version from this repository into your catkin workspace and compile the package using

cd colcon_workspace/src
git clone https://github.com/ajtudela/object_detection_openvino.git
cd ../
rosdep install -i --from-path src --rosdistro galactic -y
colcon build --symlink-install

Usage

Run the main node with:

ros2 launch object_detection_openvino default.launch.py

Download weights file

The weights file is very large and needs to be downloaded separately.

Download the weights file to object_detection_openvino/models/ to install it.

cd object_detection_openvino/models
./downloadYoloModels.sh

Nodes

object_detection_openvino_node

Perform object detection using OpenVino.

Subscribed Topics

• /camera/color/image_raw (sensor_msgs/Image)

  Color image topic where detection will be performed.

• /camera/color/points (sensor_msgs/PointCloud2) (Optional)

  Registered pointcloud topic. Is this topic is not empty, it will extract a 3d bounding box of the objects detected. This pointcloud must be ordered.

Published Topics

• image_raw (sensor_msgs/Image)

  Image with the bounding boxes surrounding the detected objects.

• detection_info (vision_msgs/VisionInfo)

  Provides meta-information about the detection pipeline: method, database location,...

• detections2d (vision_msgs/Detection2DArray)

  List with the detected objects in the image.

• detections3d (vision_msgs/Detection3DArray)

  List with the detected objects in the image if pointcloud is enabled.

• markers (visualization_msgs/MarkerArray)

  3d markers of the objects detected.

Parameters

• camera_frame (string, default: "camera_link")

  Frame that all measurements are based on.

• class_labels (string array, default: "-")

  String array with the labels of the classes.

• model_thresh (float, default: 0.3)

  Detection threshold.

• model_iou_thresh (float, default: 0.4)

  Bounding box intersection overlaping threshold.

• model_xml (string, default: "-")

  Filepath to the network configuration file.

• model_bin (string, default: "-")

  Filepath to the network weigths fie.

• model_labels (string, default: "-")

  Filepath to the network labels file.

• model_type (string, default: "YOLO")

  Network type. Can be YOLO or SSD.

• device_target (string, default: "CPU")

  Device target. Can be CPU, GPU or a MYRIAD device.

• show_fps (bool, default: false)

  Option to show the fps in the image.

Future work

• Convert nodes to LifeCycleNodes.