Named model inputs & outputs
Both our library models and any user generated model can be used to trigger contextual pre- and post-processing and easy visualizations depending on named outputs of the initial ONNX model graph.
This mechanism enables automatic configuration to the rest of the AI Manager functionalities by just upload an ONNX model having certain properties, namely the input and output names.
In other words, by naming an output in a certain way, it'll be, automatically, subject to one or more post-processing algorithm(s).
For example, if a model has an output named
linecrossing_bboxes example, then the line-crossing post-processor will count unique entities, "stored" in that output, passing by a line in both direction.Below is a summary of the supported output names, and post-processing algorithm each one of them triggers.
- licenseplate_bboxes: will be post-processed to return the license plate characters.
- linecrossing_bboxes: will be post-processed to return number of unique which crossed the user-specified line in both directions.
- barcode_bboxes: will be post-processed to return all recognized bar-codes
- color_bboxes: will be post-processed to return dominant color inside each bounding-box
- alarm: will be used to raise an alarm and upload the image online (depending on how the user configures the AI Manager).
- bboxes: will be used in the visualization part in AI Manager (ie. draw the bounding-boxes detected by the model on input image).
- scores: will be used to return the most probable class id and its corresponding probability value.
Below we list the named inputs and outputs that provide contextual options and we provide some examples of their use.
The syntax used to name an input or output is:
[input/output name]-[key1:value1];[key2:value2];...;[keyN:valueN]
The key-value pairs are integrated to provide more information about the named IO.
- If a model contains an input representing an image, then it's named:
image- - If a model contains a binary mask, as input, to ignore certain areas of the image, then that input is named:
mask- - To enable a model to raise an alarm when number of occurrences of an object is above a specific threshold, then the model should contain an input named as follows:
thresholds-0:class_name1;1:class_name2; ...;K-1:class_nameK- Example: thresholds-0:car;1:person
- If a model is fed data from a sensor, then that input is named:
sensor-
- If a model generates bounding-boxes after execution, then the corresponding output is named as follows:
bboxes-format:{xyxyc, xyxysc, xyxy};class_id1:class_name1;...;class_idK:class_nameK- Example: bboxes-format:xyxysc;0:person;2:car
- xyxysc: means (x1, y1, x2, y2, score, class)
- xyxyc: means (x1, y1, x2, y2, class)
- xyxy: means (x1, y1, x2, y2)
- If a model generates bounding-boxes delimiting barcodes and QR codes, then the corresponding output is named:
barcode_bboxes-format:xyxy - If a model generates bounding-boxes delimiting license plates, then the corresponding ouptut is named:
licenseplate_bboxes-format:{xyxy,xyxyxsxyxyxyxy}- Example: licenseplate_bboxes-format:xyxy
- xyxy: means (x1, y1, x2, y2)
- xyxyxsxyxyxyxy: means (x1, y1, x2, y2, score, ... landmark coordinates)
- If a model generates probabilities per class after execution, then the corresponding output is named as follows:
scores-0:class_name1; ...;K-1:class_nameK- Example: scores-0:not face;1:face
- The scores output is expected to have a shape of [N, K] or [K] where each column i is the probability of that object belonging to the i-th category.
- If a model generates number of objects per class after execution, then the corresponding output is named as follows:
counts-0:class_name1;1:class_name2; ...;K-1:class_nameK - If a model generates an alarm based on number of objects per class, then the corresponding output is named as follows:
alarm-0:class_name1;1:class_name2; ...;K-1:class_nameK - To configure a model's output to be post-processed using the line-crossing counter, then that output is named as follows:
linecrossing_bboxes-format:xyxysc- xyxysc: means (x1, y1, x2, y2, score, class)
- To detect the dominant color within bounding-box, then that output is named as follows:
color_bboxes-format:xyxysc
- A model that locates and counts people and cars has the following IO names:
- Input:
image- - Output 1:
bboxes-format:xyxysc;0:person;2:car - Output 2:
counts-0:person;1:car
- A Model that locates, counts, and raises alarms for the people and cars categories, has the following IO names:
- Input 1:
image- - Input 2:
mask- - Input 3:
thresholds-0:person;1:car - Output 1:
bboxes-format:xyxysc;0:person;2:car - Output 2:
counts-0:person;1:car - Output 3:
alarm-0:person;1:car
To allow any user to add our post-processing algorithms to the uploaded model, we provide functions to rename the inputs and outputs of the model to match the predefined naming of the desired functionality.
- rename_input_image(graph, image_input_name): this function take an ".onnx" model and the current name of the image type input then rename it as
image-. - rename_input_mask(graph, mask_input_name): this function take an ".onnx" model and the current name of the binary mask input then rename it as
mask-. - rename_input_sensor(graph, sensor_input_name): this function take an ".onnx" model and the current name of a sensor input then rename it as
sensor-. - rename_input_threshold(graph, threshold_input_name, class_list): this function take an ".onnx" model, the current name of the threshold input and a list of the classes name then rename the input as
thresholds-0:class_name1;1:class_name2; ...;K-1:class_nameK.
- rename_bbox_output(graph, bboxes_output_name, format, class_list): this function take an ".onnx" model, the current name of the bounding boxes output, its format and a list of the classes name then rename the output as
bboxes-format:{xyxyc, xyxysc, xyxy};class_id1:class_name1;...;class_idK:class_nameK. - rename_barcode_output(graph, barcode_output_name): this function take an ".onnx" model and the current name of the barcode output then rename the output as
barcode_bboxes-format:xyxy. - rename_licenseplate_output(graph, licenseplate_output_name, format): this function take an ".onnx" model, the current name of the licenseplate output and its format then rename the output as
licenseplate_bboxes-format:{xyxy,xyxyxsxyxyxyxy}. - rename_class_probabilities_output(graph, output_name, class_list): this function take an ".onnx" model that generates probabilities per class after execution, the current name of its output and a list of the classes name then rename the output as
scores-0:class_name1; ...;K-1:class_nameK. - rename_object_count_output(graph, output_name, class_list): this function take an ".onnx" model that generates number of objects per class, the current name of its output and a list of the classes name then rename the output as
counts-0:class_name1;1:class_name2; ...;K-1:class_nameK. - rename_alarm_output(graph, output_name, class_list): this function take an ".onnx" model that generates an alarm based on number of objects per class, the current name of its output and a list of the classes name then rename the output as
alarm-0:class_name1;1:class_name2; ...;K-1:class_nameK. - rename_linecrossing_bboxes_output(graph, output_name): this function take an ".onnx" model that will be post-processed using the line-crossing counter and the current name of its output then rename it as
linecrossing_bboxes-format:xyxysc.
"model_test.onnx" is an onnx model whose input is an image and the output is bounding boxes and we want to rename the I/O, we use thus:
rename_input_image("model_test.onnx", "images:norm")rename_bbox_output("model_test.onnx", "nms_output", "xy", ['person', 'car', 'bycicle'])
