SHM (shared memory)
Connect to camera sources through shared memory access
The GStreamer module shmsink is responsible for allocating a shared memory area, communicating the filename representing the shared memory area to Scailable through a communication pipe, feeding video frames into the shared memory and signalling the receiver that frames are ready to be read. Scailable will then use the frames and signal back to the shmsink when a frame has been used enabling the module to reuse the shared memory area for new frames.
To use the SHM interface for a camera set its input camera format to "Gstreamer SHM". By default, the shared memory interface expects an input of 640 x 360, at a frame rate of 25 frames per second. The default settings for the Gstreamer SHM interface can be changed through the settings file. For more information on these settings, see the advanced settings page.
Next, set the camera address to a (per camera and instance) unique SHM path. For instance,
/tmp/localcam Then, install and then launch a GStreamer instance including, for instance, a video test source:
# clean up, just in case
rm /tmp/localcam
gst-launch-1.0 -e videotestsrc ! videoconvert ! \
video/x-raw,format=RGB,width=640,height=360,framerate=25/1 ! \
queue ! shmsink socket-path=/tmp/localcam perms=1777 sync=true \
wait-for-connection=false shm-size=26000000
# shm-size here room for 4k video 3840 × 2160 x 3 + some additional space
# though, generally, since most models have lower input tensor dimensions
# it is always better to resize as early as possible in the chain,
# here, to 640x360
Now, on running a model, the Scailable AI Manager will make use of this custom GStreamer source for video based inferences.
Another example of integration with GStreamer, here using v4l2src on an IMX based platform:
# clean up, just in case
rm /tmp/localcam
# start
gst-launch-1.0 -e v4l2src device=/dev/video1 ! \
video/x-raw, height=1080, width=1920, framerate=30/1 ! \
imxvideoconvert_g2d ! video/x-raw,width=640,height=360 ! \
videoconvert ! video/x-raw,format=RGB ! \
shmsink perms=1777 \
socket-path=/tmp/localcam \
wait-for-connection=false \
shm-size=20000000
Do contact us if you are interested in further integration of your GStreamer plugin or video source with the Scailable AI Manager through shared memory (or otherwise).
Last modified 6d ago