RAIL Object Detector

OpenCV 3.0+ Warning:

Many files within Darknet will no longer compile on OpenCV 3+. We have a branch titled ‘opencv3_compilation’ which fixes these errors for CPU, but there are many more to fix before Darknet will compile for GPU use. As such, if you are on OpenCV 3+, we suggest cloning the opencv3_compilation branch and using the DRFCN detector instead of Darknet.

Two Minute Intro

This package now includes two object detectors which you may choose between, YOLOv2 and Deformable R-FCN (DRFCN). Detections from YOLOv2 are a bit faster, >10fps compared to ~4fps (on a Titan X), but less accurate than the detections from DRFCN.

The YOLOv2 detector uses darknet to perform object detection. It provides the ability to query for objects in an image through both services as well as from a topic.

The Deformable R-FCN detector is built on MXNet, and provides the ability to query for objects from a topic.

The response to all queries contains a list of objects, each of which has the following properties:

1. label
2. probability - confidence value in recognition
3. centroid_x - X pixel value of the centroid of the bounding box
4. centroid_y - Y pixel value of the centroid of the bounding box
5. left_bot_x - X pixel value of bottom-left corner of bounding box
6. left_bot_y - Y pixel value of bottom-left corner of bounding box
7. right_top_x - X pixel value of top-right corner of bounding box
8. right_top_x - X pixel value of top-right corner of bounding box

Querying through services (Darknet Only)

There are two modes of querying:

• Scene Queries
• Image Queries

Scene Queries are served by first subscribing to an existing camera sensor topic. Then at the moment of the query, we run object recognition on the latest frame from the camera and the resulting objects in that scene are returned after however long darknet takes.

Image Queries require an image to be sent along with the query. Object recognition is performed on this input image, and the detected objects as well as the original image are sent back.

Query through topic (Darknet and DRFCN)

If enabled (default with DRFCN), the detector subscribes to an existing camera sensor topic and grabs images from this camera as they are published. After performing object detection on the grabbed image, the detector publishes the list of objects that were found to the query topic along with the timestamp of the image which was grabbed for detection.

With the Darknet detector, the interval for grabbing images is specified in the form of a frequency. If the desired frequency exceeds the maximum frequency of operation of the detector (~1 Hz on CPU), we limit to the maximum frequency of operation.

Menu

• Installation
• Testing your Installation
• ROS Nodes
• Startup
• GPU Mode
• Scope for Improvement

Installation

Darknet Installation:

1. Put this package into your workspace
2. Assuming WS as the top level directory of this package (where this README is located), navigate to ${WS}/libs/darknet
3. Download the weights into this directory from this Google Drive link
4. Run catkin_make and enjoy the ability to use object detection! (If you need to update the file paths, use absolute paths!)

DRFCN Installation

Below is a list of python libraries that are required for the Deformable RFCN network in MXNet to work:

• Cython
• EasyDict
• mxnet-cu80==0.12.0b20171027
• Pillow
• pyyaml

Note that the version of MXNet is the same as in the Deformable ConvNets repo. You may change the version if your setup requires something different, but it may not work properly.

There is a requirements.txt file included in the repo, which lists the above libraries. You can install all of them at once by running:

pip install -r requirements.txt

Once you have all of the requirements, installation procedes as follows:

1. Put this package into your workspace
2. Navigate to the top level of this package (where this README is located)
3. run sh bin/build_drfcn.sh
4. Download the model parameters from this Dropbox link and move them into the libs/drfcn/model subdirectory of this package.
5. Run catkin_make and enjoy the ability to use object detection! (If you need to update the file paths, use absolute paths!)

Testing your Installation

Testing either detector is possible by starting a camera node or by adding some testing .jpg images to the libs/darknet/data directory. Then follow the procedures below according to the detector you wish to test.

Darknet Testing

Three optional test scripts are included in the scripts directory (test_image_query.py, test_scene_query.py, and test_detections_topic.py). To test your installation, do the following:

• Run a camera with your favorite ROS camera node.
• Launch the detector_node node with the image topic of your camera and image queries enabled:

roslaunch rail_object_detector darknet.launch use_image_service:=true image_sub_topic_name:=[camera image here]`

• Run the scene query test script; this should periodically detect and recognize objects in images from your camera:

``` rosrun rail_object_detector test_scene_query.py

File truncated at 100 lines see the full file

Changelog for package rail_object_detector

3.0.2 (2018-11-01)

• Update the default branch

3.0.1 (2018-09-07)

• Create a separate package for the messages
• Contributors: Siddhartha Banerjee, banerjs

2.0.1 (2017-12-11)

• Added Deformable RFCN detector.
• Change darknet node name.
• Change directory structure and file names for darknet.
• Update README with valid weights file, and fix broken links
• Contributors: Andrew Silva, Ryan Petschek, Siddhartha Banerjee, Weiyu Liu

1.0.4 (2017-02-09)

• Adding in an automatic build for 32 bit
• Contributors: Siddhartha Banerjee, banerjs

1.0.3 (2017-02-09)

• Adding in base travis build
• Fixed the 32-bit compile error, hopefully
• Contributors: Siddhartha Banerjee

1.0.2 (2017-02-03)

• Completed the build of GPU with flags
• Pushing fixes in master back to gpu_devel. Merge branch 'master' into gpu_devel
• Updated the README. Unfortunately, the mangling of data between publisher and service still exists and I cannot get rid of it
• Fixed timing bugs with the object detector and possibly even the bug between service and topic contention. Need to test
• Merging updates on master into the GPU branch
• Created and Tested install of the package
• Removed the need to update internal config file values
• Returning defaults
• Changed naming for a public release.
• Testing out GPU functionality
• Adding in a gitignore
• Updated the detector to also use a topic for publishing object detections
• removed unused parameters
• removed service name parameter
• Updated path in test script to use package-based absolute path
• package-location-based absolute paths for ros params, minor cleanup and debugging, documentation
• Adding in the cfg and data folders
• Completed final details on object detection. Delivering project for now
• Cursory stress test of the node is complete
• Finally done with a functioning prototype of the object detector
• Successful linkage of C++ with C
• Completed ROS Skeleton for the detector
• Initial commit of the object detector
• Contributors: David Kent, Siddhartha Banerjee