Network Optix Integration
This page is to detail how to set up Network Optix so that it integrates with Scailable Edge AI Manager
The Scailable Network Optix Plugin is currently only available for ARM64 and X86_64 devices with Ubuntu 20.04 or newer installed. Please contact us for support for different configurations.
Consult the Network Optix website to install the proper version for your platform. If you have an account, navigate to the download page.
https://www.networkoptix.com/my/download
www.networkoptix.com
Network Optix Download Page
Either the NX Witnesss or NX Meta platform will be compatible with the Scailable AI Plugin.
We provide a preconfigured detection package available for install. This will install the standard Scailable Network Optix Plugin and preconfigure it to work with Network Optix, and provide a single function:
People, bikes and cars detection. This function will detect and count any people, bikes and cars in the frame and provide labelled bounding boxes and counting events. See Visualizing Bounding Boxes and Adding Event Rules.
This package can be installed with:
sudo bash -ic "$(wget -q -O - https://get.sclbl.net/nx_plugin)" '' -p detection-package
This command will download and execute the standard AI plugin installation script, see AI Plugin Installation, with an added flag to configure the Edge AI Manager with the detection package. The script will attempt to detect as much of the environment as possible, and will prompt the user on what it should do.
- Would you like to install the Scailable Edge AI Manager on this device? (y/n)? If the install script detects that the Scailable Edge AI Manager is not installed in the default location, it will prompt if it should install it. If yes, the script will execute the default installation script for the Scailable Edge AI Manager before continuing.
- Would you like to attempt to restart server to load plugin? (y/n)? If yes, the Network Optix server will be restarted. This is a necessary step to load the plugin. Otherwise, the Network Optix can be restarted later through its web interface.
After installation, the Scailable Edge AI Manager will need to be registered. See Registration. Since the Edge AI Manager has been configured by the installation script, the Model, Input, and Output configuration steps can be skipped. You can directly continue with Running the model. After this, setup should be completed. You can now head over to the NX Client and enable the Scailable AI plugin.
We provide a preconfigured security plugin available for install. This plugin makes it significantly easier to get up and running with minimal configuration.
The plugin comes preconfigured with a few functions:
- Illegal Dumping Detection: A method to detect if an object gets left behind in the frame, such as a backpack or suitcase.
- Laying Down Detection: A method to detect if an person lays down in the frame for a period of time.
- Loitering Detection: A method to detect if the same person remains within the frame for an extended period of time.
This plugin can be installed with:
sudo bash -ic "$(wget -q -O - https://get.sclbl.net/nx_security)"
This will download and execute an interactive script. The script will attempt to detect as much of the environment as possible, and will prompt the user on what it should do.
- Would you like to install the Scailable Edge AI Manager on this device? (y/n)? If the install script detects that the Scailable Edge AI Manager is not installed in the default location, it will prompt if it should install it. If yes, the script will execute the default installation script for the Scailable Edge AI Manager before continuing.
- Would you like to configure Edge AI Manager to integrate with Network Optix plugin? (y/n)? It is highly recommended to reply yes to this option. This will ensure that the Scailable Edge AI Manager is properly configured to work with the NX Security plugin.
- Would you like to attempt to restart server to load plugin? (y/n)? If yes, the Network Optix server will be restarted. This is a necessary step to load the plugin. Otherwise, the Network Optix can be restarted later through its web interface.
If all goes well, the plugin should now be installed. Head over to Camera Settings in NX Client to see if the plugin is detected by the NX server:
After installation, the Scailable Edge AI Manager will need to be registered. See Registration. Since the Edge AI Manager has been configured by the installation script, the Model, Input, and Output configuration steps can be skipped. You can directly continue with Running the model. After this, setup should be completed. You can now head over to the NX Client and enable the Scailable AI Security plugin.
An install script is provided with the Scailable One-Line Install package. This script can be executed with:
sudo bash -ic "$(wget -q -O - https://get.sclbl.net/nx_plugin)"
This script will attempt to detect the installation directory of your NX Server. If non-default paths were used during Network Optix installation, this script will fail. In this case, follow the Manual Installation steps.
The script will download the plugin and install it in the appropriate plugins folder. Once installed, the script will attempt to detect an existing Scailable Edge AI Manager installation at the default path. If found, it will prompt if it should configure the Edge AI Manager to integrate with Network Optix. This involves changing some settings, see the Edge AI Manager Configuration section. Before the script configures these settings, it automatically makes a backup of your settings file in the same folder.
The plugin can be downloaded from the following link:
https://download.sclbl.net/network_optix/libscailable_ai_plugin.so
Scailable Network Optix Plugin Download
Once downloaded, copy the plugin file to the appropriate Network Optix plugins folder.
In the case of NX Witness the path will be:
/opt/networkoptix/mediaserver/bin/plugins
In the case of NX Meta the path will be:
/opt/networkoptix-metavms/mediaserver/bin/plugins
Once installed, the NX server will need to be restarted before the plugin is detected. This can be done through the NX Server web interface.
After installation, the Scailable Edge AI Manager will need to be registered. See Registration. Since the Edge AI Manager has been configured by the installation script, the Model, Input, and Output configuration steps can be skipped. You can directly continue with Running the model. After this, setup should be completed. You can now head over to the NX Client and enable the Scailable AI Security plugin.
In order for the Scailable plugin to operate with the Edge AI Manager, a number of settings need to be adjusted in the Edge AI Manager settings file.
These steps can be automatically performed by the NX Plugin install script. See NX Plugin Install Script. If this was done, this section can be skipped.
First, since the Scailable Network Optix plugin sends information to the Edge AI Manager over a Unix domain socket, the Edge AI Manager needs to be told to expect inputs on its socket. This can be done by setting the InputDriver to "SOCKET" .
Second, the Edge AI Manager sends its results back to the plugin via a separate Unix domain socket. This is configured by setting the OutputProtocol to "SOCKET" as well.
Finally, the Edge AI Manager needs to know where to send the output. Since a Unix domain socket communicates via a filepath, we can set the OutputLocation setting to "/opt/sclbl/sockets/output_socket.sock" , since this is where the Scailable Network Optix plugin is expecting input.
Once these settings are in place, and the browser has not overwritten them, the Edge AI Manager should be ready to operate alongside Network Optix! Select a model, and run. If the plugin is enabled in Network Optix, the Edge AI Manager should receive inputs and return results which can then be handled in Network Optix
If the plugin was detected, a 'Plugin' tab should be visible when opening a camera's settings:
Camera Settings Navigation
The plugin can be enabled/disabled per camera on the Plugins tab:
Scailable Plugin Tab
If a model which outputs compatible bounding boxes is selcted and running the Edge AI Manager, bounding boxes can be visualized by visiting the Objects tab on the right pane in the NX Witness Client:
Visible Bounding Boxes in NX Witness Client
The Scailable plugin can generate a number of events. Network Optix gives the user control over what these events should do, from sending an email to showing a notification. In this example, we will show a text overlay on the video.
Rules can be added by navigating to the Camera Rules menu:
On the Event Rules window, click the Add button to add a new rule.
From here, several options are available. First, on the left side, select Analytics Event from the When context menu. This is the type of event the Scailable plugin generates.
Select any, or multiple, camera(s) in the At context menu.
Under the Event Type context menu, you will find a number of event types the Scailable plugin can generate, from alarms to counting objects. In this example, we will select the Objects Counted type.
On the right side, it can be configured what should be done when one of these events is triggered. Here you could set up an email or notification trigger. In this example we will select the Show text overlay option in the Do context menu at the source camera
Example Settings for Objects Counted Rule
If set up correctly, and a model generating counts is selected and running, we should see an overlay on the camera feed showing us the object counts:
Example of Objects Counted text overlay
The Network Optix plugin contains a number of settings to manage the behaviour. These settings work in tandem with the Edge AI Manager settings to cater to a number of use cases.
Navigate to the camera settings and enable the Scailable AI plugin. A number of settings should appear.
It is possible to have multiple assigned models in the Edge AI Manager, see AssignedModels. In this case, it might be desirable to choose which models should be active on which camera. For example, it might be desirable for an internal surveillance camera to only detect people, while an external camera detects cars, and a third camera detects both. All of this is possible to manage from the Network Optix plugin.
Example of model routing
By default, only the first model is selected. This means that only the first assigned model will be used to run inference on each frame received from Network Optix. Any combination of models can be selected to be used for inference. The order of models in this list correspond to the assigned models in the AssignedModels array setting.
Currently up to four models is supported through the Network Optix plugin. Please contact us if more models are desired.
In this section, some general settings are listed which can be used in a wide array of cases.
Extract object counts from bounding boxes: When this is enabled, an Objects Counted event will be generated from the received object bounding boxes. This is to enable object counting in cases where the model itself has no such output.
In some use cases, it is desired to run additional inferences on certain objects. For example, let's suppose you trained a model which can detect if a person is smoking or not. This can then be used in tandem with a person detection model. The person detection model would be used to detect people from, say, a surveillance camera, and generate bounding boxes for them. If feature extraction is enabled, these bounding boxes will be used to extract parts of the larger image, and will then be sent back to the Edge AI Manager, so that the smoking detection model can be run on each of the detected people.
In this example, make sure that the person detection model is selected in the Model Routing section.
Feature Extraction Settings
Enable Feature Extraction: If feature extraction is enabled. When enabled, bounding boxes will be hidden, and instead be used to extract parts of the image to be sent back to the Edge AI Manager for inference.
Feature Extraction Model: Which model to use for the extracted features. This model will be used for inference on each extracted bounding box. In the example, this would be set to the index of the smoking detection model.
Feature Extraction Type: The type of object to extract from the full image. This is useful for when a detection model produces multiple types of bounding boxes. This setting will be matched to bounding box types, and types matching this setting will be sent to the Edge AI Manager for further inference.
The Network Optix plugin has the functionality to detect loitering. This is a tool to detect if a certain person or object is detected for longer than the desired amount of time.
To enable this, a number of settings need to be in place. Feature Extraction needs to be enabled, and routing to a ReID type model, or a model with similar output. This model allows for the plugin to keep track of the same person or object, even if they leave the frame and return.
For example, to detect if a person is loitering in a camera's view, assign and route to a person detection model in the Edge AI Manager and plugin settings. This will generate bounding boxes for persons detected in the frame. Enable feature extraction for the 'person' type, or whichever type the person detected model generates. Select the Feature Extraction Model as the index where a ReID type model, or a model with similar output, is assigned in the AssignedModels array. The plugin expects an output with either "Identity" or "embedding" as the name, and it expects this output to be a vector of any length, which uniquely identifies the subject.
When all the settings are in place, the plugin can uniquely track a subject in the frame, even if they exit and return into view. Once it is detected that this subject is loitering, a bounding box will be generated with the type Loiterer to clearly show where this is happening.
Loitering Detection Settings
Loiter Detection Time Threshold: A subject detected for at least this amount of time, in seconds, will be considered loitering.
Loiter Detection Forget Threshold: When a subject has not been detected for at least this amount of time, in seconds, it will be forgotten. If the same object returns after this amount of time, it will be considered as a new detection.
For the plugin to be detected after install, make sure to restart the NX Server. This can be done through the NX Server web interface.
Navigate to "<Server IP Address>:7001" in your web browser. Go to the Servers tab, and click the restart button.