Scailable.
Search
K

Post-processing

The Edge AI Manager is equipped with a number post-processing algorithms triggered contextually based on the ONNX's input/output names. In other words, the output JSON returned after an inference run might differ from the model's raw output, because that output has been processed by one of the algorithms to extract more insights from those raw values.
Below is a list of the currently integrated post-processing algorithms, along with the output JSON structure returned by the AI Manager.

Line-crossing algorithm

Use cases

This algorithm is mainly used to count the number of unique entities moving from one side of the image to the other, over a period of time. It can be used for instance to count:
  • number of people entering and exiting a building
  • number of vehicles passing by a road

Algorithm requirements

The line-crossing algorithm should be fed an output that represents bounding-boxes delimiting entities. Thus, the output should be something like a N by 4 matrix, where N is number of detected entities, and each row contains at least entity's coordinates.

Output JSON structure

Other than the usual meta-data included in the JSON, additional fields are included, namely:
{ ..., "output":[ a, b], ...}
Where:
  • a: number of unique entities moving from left to right side of line
  • b: number of unique entities moving from right to left side of line
For example, when using this model from the Scailable Free Models catalog, here's a complete JSON output:
[
{
"deviceId": "8e410fb88d924ab2900ffe79cc6bdc5e-145862338700776",
"modelId": "3a21caaa-2ea7-4e32-83cf-9994cb65545a",
"sourceId": "input-0",
"sourceName": "",
"timestamp": 1675421846075,
"output": [
2,
1
]
}
]
The used model is a people locator. The post-processor inferred that 2 people moved from left to right, and 1 person moved left.
For more information regarding the line-crossing parameters and where to specify their parameters, check the AI Manager's page.

Automatic Number Plate Recognition (ANPR)

Use cases

This post-processor is capable of decoding characters of a license plate.
  • Toll automation
  • Restricted area monitoring
  • Vehicle tracking
  • Unique cars counting

Algorithm requirements

Given the location of a license plate (LP) in an input image, the ANPR algorithm deciphers its characters and returns them as a readable string. In other words, an ML model is needed to locate the LP and return its location in this format: (x1, y1, x2, y2, confidence threshold, the 4 LP corners coordinates). So, the output should should contain 13 values.

Output JSON structure

Other than the usual meta-data included in the JSON, additional fields are included, namely:
[
{
...
"output": {
"licensePlate": "",
"lastEstimatePlate": "",
"coordinates": [
0,
0,
0,
0
]
},
"size": ,
"direction": ,
...
}
]
Where:
  • licensePlate: A string containing the rectified license plate characters based on previous detections.
  • lastEstimatePlate: A string representing the last detected license pate characters.
  • coordinates: The (x1, y1, x2, y2) coordinates of the detected license plate.
  • size: The estimated area of the license plate (Height x Width)
  • direction: An integer denoting whether the license plate (and hence the car) is moving away from camera (value = 2), moving closer to camera (value = 1), not moving (value = 0) or unknown when no license plate is detected for instance (value = 3).
For example, when using this model from the Scailable Free Models catalog, here's a complete JSON output:
[
{
"deviceId": "8e410fb88d924ab2900ffe79cc6bdc5e-145862338700776",
"modelId": "1eda1ec4-cb15-4919-9be4-bf32bb87c439",
"sourceId": "input-0",
"sourceName": "",
"timestamp": 1676024792051,
"output": {
"licensePlate": "",
"lastEstimatePlate": "G241GH",
"coordinates": [
0,
0,
0,
0
]
},
"size": 6764,
"direction": 3,
"sourceDate": "",
"outputType": "json"
}
]
This model detected a car with a license plate G241GH. Since this is the result of the first inference run, the coordinates are zero-ed and licensePlate is empty.

Alarm trigger

Use cases

If a model returns boolean values, they can be exploited to trigger an alarm somewhere based on those values. This trigger can serve to:
  • detect intruders
  • monitor restricted areas

Algorithm requirements

To support alarm trigger for a model, it needs to have an output following a specific pattern. The output name should start with alarm-. For more information, please check the named IO page.

Output example

Other than the usual meta-data included in the JSON, additional fields are included, namely:
[
{
...
"alarm": 0,
"baseRGB": "",
"h": 0,
"w": 0,
"c": 0,
...
}
]
Where:
  • alarm: is a boolean value. It's 1 when an alarm is raised, 0 otherwise.
  • baseRGB: A base64-encoded string containing the raw pixel values of the image triggering the alarm.
  • h, w, and c: stand for height, width and number of channels respectively. They are used to re-construct the original image with the right dimensions.
Please note that baseRGB, h, w, and c may not be present in the output JSON based on your configuration of the AI Manager. For information, check the output configuration section.
For example, when using the people detector from the Scailable Free Models catalog, here's a complete JSON output:
[
{
"deviceId": "8e410fb88d924ab2900ffe79cc6bdc5e-145862338700776",
"modelId": "6e7cb493-f310-4491-a387-3ef59f16b58e",
"sourceId": "input-0",
"sourceName": "",
"timestamp": 1676035054620,
"alarm": 1,
"outputType": "json",
"outputFormat": "namedObject",
"outputDims": [
[
1
]
],
"outputDataTypes": [
9
],
"output": {
"alarm-0:person": [
1
]
}
}
]
The model detect at least one person, and the alarm is raised but without sending the RGB image to the cloud.

Non Maximum Suppression algorithm

Use cases

If a model returns two outputs: one containing proposed bounding-boxes and an other for their respective scores, the NMS algorithm is used to only keep pertinent bounding-boxes, ie. remove those which the model is less confident about. NMS serves to:
  • Locate and count faces when coupled with the face locator model

Output example

Other than the usual meta-data included in the JSON, additional fields are included, namely:
[
{
...
"output": {
"count": 2,
"boxes": [
288,
16,
321,
65,
290,
182,
320,
232
]
}
...
}
]
Where:
  • count: is a non-negative integer representing the number of pertinent boxes.
  • boxes: a sequence of 4-tuple containing (x1, y1, x2, y2) coordinates.
For example, when using the face locator from the Scailable Free Models catalog, here's a complete JSON output:
[
{
"deviceId": "8e410fb88d924ab2900ffe79cc6bdc5e-145862338700776",
"modelId": "0d1a91bd-1474-11ec-af2f-9600004e79cc",
"sourceId": "input-0",
"sourceName": "",
"timestamp": 1678280224597,
"sourceDate": "",
"outputType": "json",
"outputFormat": "namedObject",
"outputDataTypes": [
1
],
"output": {
"count": 12,
"boxes": [
288,
16,
321,
65,
290,
182,
320,
232,
-1,
8,
30,
68,
64,
17,
102,
67,
139,
11,
178,
68,
215,
10,
258,
71,
26,
101,
70,
150,
110,
100,
149,
155,
217,
98,
253,
152,
56,
180,
97,
234,
143,
182,
182,
232,
220,
179,
257,
235
]
}
}
]
This post-processor is tested only on the face locator model in the Scailable free catalog.

Blurring algorithm

Use cases

This post-processor blurs the face recognized in a stream and makes them accessible locally for use. This blurring algorithm can serve to:
  • Preserve users privacy
  • Build a GDPR-compliant application

Output example

Other than the usual meta-data included in the JSON, additional fields are included, namely:
[
{
...
"output": {
"count": 2,
"boxes": [
288,
16,
321,
65,
290,
182,
320,
232
]
}
...
}
]
Where:
  • count: is a non-negative integer representing the number of pertinent boxes.
  • boxes: a sequence of 4-tuple containing (x1, y1, x2, y2) coordinates.
For example, when using the face locator from the Scailable Free Models catalog, here's a complete JSON output:
[
{
"deviceId": "8e410fb88d924ab2900ffe79cc6bdc5e-145862338700776",
"modelId": "0d1a91bd-1474-11ec-af2f-9600004e79cc",
"sourceId": "input-0",
"sourceName": "",
"timestamp": 1678280224597,
"sourceDate": "",
"outputType": "json",
"outputFormat": "namedObject",
"outputDataTypes": [
1
],
"output": {
"count": 12,
"boxes": [
288,
16,
321,
65,
290,
182,
320,
232,
-1,
8,
30,
68,
64,
17,
102,
67,
139,
11,
178,
68,
215,
10,
258,
71,
26,
101,
70,
150,
110,
100,
149,
155,
217,
98,
253,
152,
56,
180,
97,
234,
143,
182,
182,
232,
220,
179,
257,
235
]
}
}
]
This post-processor is tested only on the face locator with blurring model in the Scailable free catalog.

Color detection algorithm

Use cases

This post-processor determines the dominant color inside each recognized bounding-box. It could be used to:
  • detect color of cars

Output example

Other than the usual meta-data included in the JSON, additional fields are included, namely:
[
{
...
"colors": [
"Black"
],
...
}
]
Where:
  • colors: is a list recognized colors.

Bar/QR code reading algorithm

Use cases

This post-processor is used to decode bar and QR codes which are recognized in an image, for instance it can serve to:
  • Product counting
  • Identify products in store
  • Read links, information from images

Output example

Other than the usual meta-data included in the JSON, additional fields are included, namely:
[
{
...
"output": {
"barCodes": [
"welcome to scailable!",
"Scailable-2023"
]
},
...
}
]
Where:
  • barCodes: is a list of data encoded in each detect bar/qr code.

Upload images based on threshold value

Use cases

This post-processor is used to send input images with the output JSON based on the model's output on that image. More precisely, this works with object detection models where an image is uploaded if the model is less confident about a certain image. To configure this post-processor on the device, check out this page.
This post-processor can be used to:
  • Grab images from deployment environment to finetune the model.

Output example

Other than the usual meta-data included in the JSON, additional fields are included, namely:
[
{
...
"uncertain": 0,
"baseRGB": "",
"h": 512,
"w": 512,
"c": 3,
...
}
]
Where:
  • uncertain: is a boolean value. It's 1 when model is uncertain about the input, and 0 otherwise.
  • baseRGB: A base64-encoded string containing the raw pixel values of the image triggering the uncertainty condition. It's empty if the condition is not met.
  • h, w, and c: stand for height, width and number of channels respectively. They are used to re-construct the original image with the right dimensions.
This post-processor is only available for EdgeImpulse models.