Hailo-8 NPU

Please note that Hailo-8 support is currently included in our runtime (i.e., the AI manager), but on our platform only a limited number of Hailo 8 models is available (see below).
If you wish to deploy custom models (or one of our other library models) to a Hailo device this is currently possible but the model wil run only on CPU: do contact us for conversion support to Hailo 8 at [email protected].

Dependencies

The Scailable AI Manager currently provides Hailo-8 acceleration exclusively for Hailo-8 type ONNX models. However, it's important to note that this Hailo-8 acceleration feature is currently available only on Linux x86_64 systems (with GLIBC version: 2.35) and requires the latest Hailo drivers to be installed (version: Hailo 4.14.0).
Hailo drivers can be obtained by registering at: https://hailo.ai/developer-zone/sw-downloads/

Installing the Scailable middleware

You can easily install the Scailable AI Manager with Hailo-8 accelerator support through our standard convenient one-line installation process. Simply SSH into the device and run the following command:
sudo bash -ic "$(wget -q -O - https://get.sclbl.net)"
During the installation process, Hailo-8 will be automatically detected. You will be promted to enable Hailo 8 support during the installation. Upon answering y, the Scailable AI Manager will be downloaded and installed with support for Hailo-8:

Accessing the AI manager on the device

After the installation completes you are able to simply browse to the Administrative UI of the AI manager installed on the device by entering one of the IP addresses of the device directly into your browser followed by port :8081. The URLs are conveniently displayed:
After a simple registration process of your device (as simple step-by-step process, see here), you can now configure your AI pipelines and models.

Choosing a Hailo-8 supported model

For full AI manager documentation and configuration please see: https://docs.scailable.net/the-ai-manager/introduction.
To test a Hailo model, first select the "Hailo Models" catalogue from you available catalogues.
If you are unable to view the Hailo Models catalogue, please contact us at: [email protected].
Among the three currently available Hailo accelerated models, the Hailo object detection model stands out as it has been specially adapted to leverage our named input/output driver system. Hence, it will automatically generate visualizations in our pipeline (as is true for our other library models). However, consequently, it is the only model that provides advanced Scailable postprocessing options including live visualisation at this point in time.
The other two models (center pose & superresolution) can be chosen and run accelerated on the Hailo-8 but their output is not automatically visualized. However, the output can easily be inspected by clicking the Peek last outpu button after running the model.

Example run of the Hailo object detection model

After selecting the Hailo object detection model you can easily test it by selecting one our example inputs in the input tab:
Once you run the model you can view the live visualization: