node-red-contrib-post-object-detection

This is a custom Node-RED node that handles the prediction results of an Object
Detection model.

Installation

$3

This module requires @tensorflow/tfjs-node as a peer dependency. You need
to install it within Node-RED manually. TensorFlow.js on Node.js
(@tensorflow/tfjs-node
or
@tensorflow/tfjs-node-gpu),
depend on the TensorFlow shared libraries. Putting TensorFlow.js as the
dependency of a custom Node-RED node may cause the situation where multiple
custom nodes each install their own tfjs-node module as a dependency. This causes
an attempt at loading multiple TensorFlow shared libraries in the same process, which
subsequently causes the process to abort with a protobuf assertion error.

Therefore, this module puts @tensorflow/tfjs-node as a peer dependency. You need
to install it with Node-RED manully.

Install @tensorflow/tfjs-node:
``
npm install @tensorflow/tfjs-node
`

This custom Node-RED node leverages the node-canvas npm package to draw the bounding
boxes on a image. Please make sure the platform running Node-RED fulfills the
prerequisites listed here. For
example, while using Ubuntu, you need to run the following command to install
the dependencies:
`
sudo apt-get install build-essential libcairo2-dev libpango1.0-dev libjpeg-dev libgif-dev librsvg2-dev
`

$3


Once you install the peer dependency and the prerequisites, you can install this module:
`
npm install node-red-contrib-post-object-detection
`

Usage

There are two custom Node-RED nodes in this package:
- post-object-detection: This is used to process the output of an Object Detection
model.
- bbox-image: This is used to annotate an input original image with bounding boxes.

$3

The input for this node should be an array of tf.Tensor objects with a length of 2.
The first tensor in this array corresponds to the detected objects with a
[1, number of box detectors, number of classes] shape where 1 is the batch size.
The second tensor is the bounding boxes with a
[1, number of box detectors, 1, 4] shape where 4 is the four coordinates of the
box. This node also requires class information through the use of the Class URL property.
The file specified here should be a JSON file containing the id and className
for each class. For example:
`
{
"0": "person",
"1": "cup",
...
...
}
`
The following node properties can also be altered from the defaults:

- IoU: The intersection over union threshold for determining whether boxes overlap too much with
respect to IOU during non-max suppression. Must be between [0, 1].
Defaults to 0.5 (50% box overlap).
- Min Score: Minimum score needed for a box to be accepted during non-max suppression.
Defaults to 0.5.

The node then calculates the object detection results and returns the detected
objects as an Object[]. Each object contains bbox, className and
score properties.
- bbox: The coordinates of the box, width and height: [x, y, w, h].
These values are float number between 0.0 and 1.0.
- className: The name of the class.
- score: The confidence value between 0.0 and 1.0.

$3


The msg.payload passed to this node should be an object containing these two
properties:
- image: The image data in Buffer data type
- objects: An object array containing a list of detected objects.
Each object has the following information:
`
{
bbox: [x, y, w, h],
className: string,
score: number
}
`

This node annotates the image by drawing the bounding boxes of detected objects onto it.
This annotated image is then output as a Buffer` for the next node.