Klepsydra and SMART CONNECT
Within SMART CONNECT, Klepsydra is providing software-based acceleration for different data processing algorithms. The acceleration is based on components of Klepsydra AI, a state-of-the-art software framework for edge devices.
In the first step, we focus on accelerating video encoding and decoding algorithms, provided by GeoVille, and thus improve the overall performance of the video delivery to and from the SMART CONNECT system to end-users.
In the second step, the acceleration framework will be provided as an SDK, allowing for an overall performance increase for edge devices using SMART CONNECT.
Video Compression with Klepsydra
The video compression in SMART CONNECT is currently based on algorithms available within the FFmpeg library. The “classical” approach usually requires multiple calls to the “ffmpeg” tool, resulting in high CPU workloads.
In SMART CONNECT, the Klepsydra Streaming Framework (part of Klepsydra AI) is used to accelerate the execution of the FFmpeg algorithms, utilising the ultra-low latency ring buffers developed by Klepsydra.
In order to utilise the benefits of the Klepsydra Streaming framework, the FFmpeg algorithm was split up into different steps. This creates a lock-free pipeline to run the ffmpeg compression task as a series of configurable steps much more efficiently. The framework also provides a way to further split the H.265 library compression into component configurable steps, which can provide further gains.
In a first implementation, we already achieved a reduction of average CPU usage by 45%. This means, e.g. longer run-time of battery-powered devices when streaming videos to/from the SMART CONNECT platform.
The application will now be tested on a drone streaming live video for re-distribution within SMART CONNECT.
Further Klepsydra AI has been applied to different AI models from partners like VITO to identify possible performance improvements. AI inference has been done on CPU and GPU-based systems using Klepsydra and shows promising first results. These are the first steps to utilise Klepsydra across different platforms and algorithms within SMART CONNECT to improve efficiency at the edge.
Klepsydra Explained
Klepsydra’s acceleration technology combines two-dimensional parallelization with lock-free programming to deliver a flexible and efficient inference engine, achieving up to 4x lower latency, 10x higher throughput, and 50% reduced power consumption. Its autotuning tool optimizes AI inference for various constraints, while a unified API eliminates processor lock-in, enabling greater flexibility and improved performance across hardware platforms.
The underlying technology, Klepsydra Streaming Framework, is also used to accelerate “classical” non-AI algorithms like image and video compression based on the ffmpeg library.