Meeting with Hailo: bidding to build an edge-AI training centre — and asking to take part in the development

We met the commercial leadership of Hailo in person in Trenčín to discuss building a training and application centre for edge computing. Our intent is not only to teach how these processors are used, but to take part in the development itself — in reference designs, model porting, a methodology for validating quantisation, and the security layer that industrial deployment in Europe requires.

Abstract

The Institute of Advanced Studies has met with Hailo, the Israeli maker of processors for edge computing — neural-network inference on the device itself (edge AI), to discuss building a training and application centre dedicated to designing and building systems on this platform. Alongside education we are declaring an interest in taking part in the development — in reference hardware designs for industrial environments, in porting and optimising models, in a methodology for verifying quantisation loss, and in the security and audit layer without which nothing falling under NIS2 or the AI Act can be deployed in Europe today. This article describes in detail the architecture and product range of Hailo, the consequences this architecture has for system design as we have verified them in our own projects, and the competences the bid rests on — from nanometre-scale thin films at the Institute of Physics of the Slovak Academy of Sciences, through microprocessor design and modelling and FPGA work, to AI systems in production with a library of more than 462 models.

Keywords: edge AI · inference accelerator · dataflow architecture · INT8 quantisation · FPGA · training centre · NIS2 · data sovereignty

1. What the meeting was about

The talks were held in person in Trenčín with representatives of Hailo’s commercial leadership. We thank them for travelling such a long way for this meeting — a visit in person says more about serious interest than any presentation can. The talks covered two connected intentions.

First — the training centre. In Slovakia and the wider Central European region there is a marked gap between what companies hear about artificial intelligence and what they are able to build and keep running. A graduate of a technical university can train a model in PyTorch on a laptop. Far fewer people can take that same model, compile it onto an inference accelerator, measure how much it degraded after quantisation, and design an enclosure that will survive five years in a hall with aerosols and thermal cycling. It is precisely this part of the chain — engineering, not research — that is missing.

Second — participation in the development. We do not want to be merely the operator of a training room. We are interested in contributing to development: to reference designs for industrial deployment, to porting and optimising models for specific verticals, to an independent methodology for measuring quantisation loss, and to a security layer for the European regulatory framework. Our projects on this platform did not arise as demonstrations but as answers to a customer’s brief — and that is exactly where the blank spots in the ecosystem became visible.

We take this as a challenge — technical as much as organisational. To build a centre that teaches people to build systems that work, and at the same time to take part in developing the platform we build them on, is exactly the kind of task the institute was founded for.

2. Hailo — who they are and what they do differently

2.1 The company

Hailo is an Israeli fabless semiconductor company founded in 2017 (Orr Danon, Avi Baum, Hadar Zeitlin). It does not make general-purpose graphics accelerators or server-class training chips — it concentrates on edge computing: on devices where the model runs where the data is created, on a budget of a few watts and without a connection to the cloud. According to publicly available information, following its Series C round (2024) the company has raised total funding in the order of hundreds of millions of dollars, and its architecture is deployed in hundreds of customer programmes across automotive, security, smart city, Industry 4.0 and retail.

2.2 Why dataflow rather than another GPU

The key is the architecture Hailo calls structure-defined dataflow. The difference from the classical computing model is fundamental and worth stating precisely, because everything else follows from it — performance, power and the limitations alike.

The classical von Neumann model (CPU, and to a large extent the GPU): the computing unit is general-purpose, program and data flow from memory across a bus, and at every layer of the neural network weights and intermediate results move between the compute core and external DRAM. For convolutional networks this movement is more expensive in energy than the multiplication itself. An access to external DRAM costs roughly two orders of magnitude more energy than an arithmetic operation — which is why a graphics card consumes hundreds of watts during inference.

The dataflow model (Hailo): the network is not executed as a sequence of instructions but is mapped into the structure of the chip. The compiler splits the model graph into layers and allocates to each of them its own compute resources and its own local memory directly on the silicon, according to what that layer actually needs. Data then flows between layers across the chip, not back and forth to DRAM. The Hailo-8 has no external memory at all: all memory is integrated on the die.

The consequences:

Anyone who has ever designed for an FPGA will recognise a familiar pattern here: this is spatial computing. Instead of one general-purpose unit switching between tasks in time, the task is spread across the area of the silicon and data flows through it. The same trade-offs — area versus throughput, pipeline depth versus latency, placement and routing of resources — appear here as well. We return to this in the section on our competences, because it is not merely an analogy; it is directly transferable experience.

2.3 The product range

According to the manufacturer’s data (as of August 2026):

Additional parameters that decide industrial deployment more than TOPS do:

• Temperature range: industrial grade −40 °C to +85 °C, automotive grade −40 °C to +105 °C. The modules need no active cooling — they fit into a cabinet beside the line.
• Qualification: A-SPICE L2 development flow, AEC-Q100 Grade 2 and ISO 26262 ASIL-B(D) compliance for automotive deployments.
• Host: x86 and ARM; Linux and Windows, plus Android for the Hailo-10H.

A note on the numbers: TOPS is a comparative figure, not the performance of a specific model. It states how many integer operations the chip can perform in the ideal case, not how many frames per second will pass through your model at your resolution. We use it for rough sizing; the decision is always made after measuring the specific model on the specific data.

2.4 The software chain

For this class of hardware, software is half the product — and in practice it is precisely the half that decides whether a project succeeds. The Hailo AI Software Suite has four parts:

1. Dataflow Compiler (DFC) — translates the model from standard formats (TensorFlow, TensorFlow Lite, Keras, PyTorch via ONNX) into the chip’s binary format. It performs graph optimisation, quantisation and — the heart of the matter — allocation and placement of layers onto the compute resources of the chip.
2. HailoRT — the runtime and driver. Lightweight, production-grade, available as open source; C/C++ and Python APIs, running on x86 and ARM.
3. Model Zoo — a library of pre-trained and pre-compiled models including an environment for retraining them. The Model Zoo GenAI branch contains pre-compiled generative models (LLM, VLM, ASR) for the Hailo-10H.
4. Examples and integrations — reference applications (detection, licence-plate recognition, multi-stream pipelines, re-identification, segmentation, pose estimation, tiling) and connections to common environments: GStreamer, ONNX Runtime, Ollama, and a REST interface compatible with OpenAI/Ollama.

From our own work with this chain we know two details that determine the architecture of the whole solution:

• The DFC runs on x86_64 Linux. Not on ARM. The compilation step therefore belongs on the server side — not on the embedded device, not on an ARM workstation. We had to account for this when designing the architecture for an industrial customer.
• The calibration set must be representative. Quantisation is tuned to the distribution of activations on real data; typically on the order of 1,000 frames from the target installation. Calibrating on a public dataset and deploying into a production hall is the most common and most expensive mistake.

And this is precisely the area we want to become active in: deploying Hailo processors in our cluster platform, the IOAS Security Cluster (IOASC) — from the choice of form factor and node sizing, through model compilation and validation, to operations, monitoring and field service. We have something to build on: IOASC is built on hardware-accelerated inference and a library of more than 462 models whose registry has from the outset assumed several compiled forms for different target hardware.

3. What the architecture implies for system design

A specialised accelerator is a deliberate engineering choice: the high efficiency and the constant inference time are paid for by moving part of the work out of operations and into the design phase. The following four points are therefore not objections — they are tasks for the designer. We formulated them while designing a real system for an industrial customer (automated evaluation of fluorescent penetrant inspection on a production line) and we consider them a mandatory part of any honest design:

1. It is an inference device — training belongs on a GPU. The accelerator executes the model; training it has to happen elsewhere. If the customer needs continuous retraining as the process drifts (a new batch of chemicals, an ageing light source, changed illumination), this is not a one-off operation but a recurring operational need and must have its own hardware and its own budget in the design.
2. Quantisation is verified by measurement. Going from FP32 to INT8 (INT4 on the Hailo-10H) is exactly what gives this architecture its efficiency — and at the same time a change that, when detecting faintly glowing indications against a dark background, must be measured, not assumed. We therefore include in our deliveries a comparison of FP32 versus the compiled model on the same evaluation set as an acceptance criterion.
3. A defined set of operations is supported. Anomaly-detection models of the PatchCore/PaDiM type are therefore split in practice: the backbone (feature extractor) runs on the accelerator, while the memory bank and distance computation run on the host CPU. This is a standard and effective arrangement, but it has to be reflected in CPU and RAM sizing — otherwise the host becomes the bottleneck.
4. Performance here is won by design, not by brute force. By input size, tiling, choice of architecture and the division of work between chip and host. It is engineering work — and precisely for that reason it is worth teaching, and precisely for that reason we want to do it.

4. Our competences: four layers that have to meet

A system built on an inference accelerator is a boundary discipline. It always fails at an interface: the physics of sensing does not meet circuit design, circuit design does not meet the model, the model does not meet operations. Our bid rests on being able to staff all four layers at once — and not from review literature, but from our own practice.

4.1 The physics layer — nanometre-scale films and sensors

The founder of the institute spent more than eight years as a research scientist at the Institute of Physics of the Slovak Academy of Sciences, working on multilayer magnetic thin-film structures — systems in which the functional property is determined by layer thicknesses at the level of single nanometres and by the quality of their interfaces. The result is a series of papers in Vacuum, Thin Solid Films and the Journal of Physics on pseudo spin-valves as sensing elements of mechanical strain and on the effect of magnetic flux distribution on giant magnetoresistance in Ag/Co multilayers, as well as two granted patents on magnetostrictive sensors (WIPO PCT WO2009088378, Slovak patent SK 50001/2008). The work is cited by groups active in flexible spintronics, including a citation in Nature Electronics.

Why is this relevant to a project about AI processors? For three reasons:

• We understand silicon as a physical object, not an abstraction. Anyone who has worked with sputtered films, interfaces and their degradation understands why power density, heat removal and the −40 to +85 °C range are a hard constraint and not a line in a table.
• We understand the beginning of the data path. The quality of a model is bounded from above by the quality of sensing. Most of the “AI failures” we see in companies are in fact failures of sensing, optics or illumination — the model merely reproduced faithfully what it was given.
• We can design a sensor, not merely buy one. GMR, spin-valve and TMR magnetic sensors, magnetostrictive sensors, strain gauges, ultrasound, distributed fibre-optic sensors (DFOS).

4.2 The digital design layer — FPGA and microprocessor modelling

The second layer is digital system design: FPGA work, the development and modelling of microprocessor architectures, mixed-signal and RF design, PCB design including EMC pre-compliance, embedded systems, RTOS firmware and real-time signal-processing chains.

This experience is transferable to the Hailo platform directly, not metaphorically:

The difference between someone who “somehow compiles” a model and someone who knows why this particular layer did not fit and how to restructure the network is precisely the difference between a completed tutorial and engineering competence. This is what we want to teach in the centre.

4.3 The model and MLOps layer

We operate a library of more than 462 machine-learning models within the IntelliTwin platform and the IOAS Security Cluster (IOASC) — with hardware-accelerated inference, multiple parallel camera streams on a single physical unit, and a complete MLOps cycle: data collection and curation, GDPR-compliant de-identification, training, structured pruning, INT8 quantisation with calibration, layer fusion, ONNX → target runtime conversion including Hailo-RT, evaluation, model registration with full lineage, canary rollout and active learning with operator feedback.

The model registry includes compile_target and quantization_config — that is, the fact that a model exists in several compiled forms for different target hardware is built into our architecture from the outset, not bolted on.

4.4 The deployment, security and compliance layer

The last layer is the one most often missing from training courses and the one that in practice decides whether a system may be deployed at all:

• Cyber security and NIS2 — architecture with a structured audit record, log integrity, access control, multi-factor authentication, encryption of data at rest, backups with verified restore, patch management. We run our own security stack and perform security audits for municipalities and companies.
• Remote management of deployed devices — a permanent encrypted administrative channel, signed updates, the ability to restart and recover without a site visit. A device that cannot be updated remotely is a design defect from a life-cycle perspective.
• Regulatory framework — GDPR, the AI Act, the Cyber Resilience Act (CRA), the product liability directive. We run courses on preparing for cyber-security audits and GDPR audits and issue certificates with a verifiable identifier.
• Methodology and standards — in industrial deployments we work with sector standards (for example UV-A irradiance measurement to EN ISO 3059 in penetrant inspection), because without metrology system drift is not detected, only deepened.

5. Where this is already running

The bid rests not on intent but on deployments:

• IOAS Security Cluster / IntelliTwin — a distributed platform for analysing imagery from existing municipal camera systems that integrates already installed infrastructure instead of replacing it. Hardware-accelerated inference, more than 462 models, aggregation and correlation across cluster nodes.
• Industrial quality control — the design of a system for automated evaluation of fluorescent penetrant inspection on a production line: a monochrome global-shutter sensor, filtering of the exciting UV-A radiation, tiling of the frame to detect indications below 0.5 mm, inference on a Hailo accelerator and splitting of the anomaly model between chip and host. Part of the design is the requirement that the customer’s production photo-documentation must not leave their premises.
• Non-destructive testing — the lecture Next-Gen Methods in NDT at the ATG conference (2025), distributed fibre-optic sensors, predictive maintenance.
• Local language models — running language models on our own infrastructure to process client documents and data without sending them to third-party providers. This experience transfers directly to the Hailo-10H, that is, to generative AI in edge computing.

6. The proposed training centre

6.1 Why edge AI and why now

Three reasons why we consider edge computing the right subject for a training centre in this region:

1. Data sovereignty. A model running on an edge device has no reason to send raw data anywhere. For footage from cameras in public space, for a customer’s production photo-documentation or for health data, this is not an extra feature — it is the condition under which the project may be carried out at all.
2. Energy and economics. Single-digit watts instead of hundreds. Across dozens of devices in the field this is the difference between an operable and an inoperable solution — and the difference between needing active cooling or not, which is the most frequent cause of failure in dusty environments.
3. The missing qualification. Many people can train a model today. Few can turn a model into a system that runs for five years without intervention — and that is the profile the market is asking for.

6.2 The curriculum — seven modules

We propose seven consecutive modules, each with laboratory work on real hardware and real data. The share of practical work is not below 60 %.

Module M7 is being prepared in intensive cooperation with ATG, with whom we have long worked on automated non-destructive testing — from the lecture Next-Gen Methods in NDT at their conference (2025) to the design of a system for the automated evaluation of fluorescent penetrant inspection directly on a production line. It is this cooperation that gives the module what cannot be invented in a training room: real inspection tasks, real images of indications and the customer’s real acceptance criteria.

6.3 The laboratory

We propose equipping the laboratory so that participants work with what the customer will subsequently buy: x86 workstations with the Dataflow Compiler, M.2 and mPCIe modules for embedded assemblies, a server card for multi-stream tasks, a module for generative AI, camera stations with controlled illumination and machine-vision optics, an industrial network with power over Ethernet, a climatic chamber for verifying the temperature range, a GPU training server (because the accelerator does not train) and a fully segregated laboratory network segment.

6.4 Certification and links to academia

We build on an existing system: an IOAS certificate with a verifiable identifier in an online register, discounts for corporate groups, and the option of closed bespoke courses. For the academic sector we offer a format of lectures and technical seminars, mentoring for university students on research and project topics, and joint semester and diploma assignments — the director of the institute spent eleven years as a university lecturer at Slovak and Czech technical universities and continues to mentor students.

6.5 What we build on

Education is not a new activity for us. We run a catalogue of seven structured professional courses (cyber security, GDPR audit, preparing for a cyber-security audit, generative design in electrical engineering, modern methods of complex signal processing, introducing AI in a company, botnets), we lecture at conferences, we consult for municipalities and we publish open technical texts. A centre for edge AI is an extension of a working structure, not a greenfield build.

7. Our interest in taking part in the development

This is the second half of our intent and we consider it as substantial as the education. We offer the following areas as a concrete contribution — in all of them we have either finished outputs or directly transferable practice.

7.1 Reference hardware designs for industrial environments

A module is only a component; between it and a deployment stand the carrier board, cooling, power, ingress protection and cabling. We offer the design and verification of reference assemblies for environments where standard solutions fail — a cabinet beside a production line (aerosols, vibration, thermal cycling), a pole in the field (off-grid power, condensation, lightning protection), mobile deployment. Our own competences here are mixed-signal and PCB design with EMC pre-compliance, thermal design including generatively optimised passive heat sinks, and embedded firmware.

7.2 Model porting and contributions to the ecosystem

The Model Zoo covers common computer-vision tasks. Industrial verticals are missing from it — detecting indications in penetrant inspection, evaluating surface defects, reading technical documentation and delivery notes under real operating conditions, waste-management analytics, agronomic and environmental tasks. We offer to port, optimise and publish such models together with the corresponding processing pipelines (GStreamer / TAPPAS), including documentation and evaluation sets.

7.3 An independent methodology for validating quantisation

The missing link across the whole field is a verifiable answer to the question of how much the model degraded after quantisation and whether that is acceptable. We have practical experience with evaluation pipelines, annotation databases and attributing errors down to individual rules from operating our own systems. We offer to develop and publish a methodology: composition of the evaluation set, statistical significance of the difference, acceptance thresholds by risk class, and an acceptance-test protocol usable as an annex to a contract — precisely what is missing today both in public procurement and in audits under the AI Act.

7.4 Sensor fusion and non-standard modalities

Most of the ecosystem addresses the camera. Our research history lies elsewhere: magnetic and magnetostrictive sensors, strain gauging, ultrasound, distributed fibre-optic sensors. We are interested in inference over these modalities in edge computing — evaluating a DFOS chain, processing ultrasonic A/B/C-scans, classifying vibration signatures for predictive maintenance. It is an intersection where sensing meets accelerated inference and where we can bring our own substantive contribution rather than a borrowed one.

7.5 Generative AI in edge computing

With the Hailo-10H, a local language or vision-language model ceases to be a theoretical possibility. We run local language models on client data and we are interested in moving them into edge computing: a voice and text interface to an industrial system without the cloud, automatic description of a finding, an assistant for the operator, document processing in environments where data must not leave the site. This also includes quantifying what aggressive quantisation (INT4) does to the quality of a language model’s output — the same question as in 7.3, only for a different class of models.

7.6 The security and audit layer

What the European market buys today is not performance but demonstrability. We offer to develop the layer that turns an inference node into an auditable device: a structured security record in a uniform schema (time, actor, action, resource, outcome, severity) portable into a SIEM, record integrity, secure boot and signed updates using the platform’s hardware facilities, remote device management and a documented life cycle under the CRA. We have our own security stack and audit practice for this.

8. Economics: what actually counts

When choosing between an accelerator in edge computing and a server-side solution, the wrong quantities are usually compared. These are the ones that decide:

9. Next steps

Discussions with Hailo continue. In parallel we are approaching the partners whose participation we consider a precondition for a meaningful centre:

• technical universities — a joint laboratory, semester courses, semester and diploma topics, student mentoring;
• industrial companies — pilot tasks from real production as laboratory assignments instead of artificial examples;
• municipalities and the public sector — secure and data-sovereign deployment in municipal infrastructure;
• integrators and suppliers — training for their own teams and joint reference designs.

If any of these areas is of interest to you — whether as a partner of the training centre or as a participant in joint development — get in touch.

director@ioas.pro · +421 903 667 654 · contact form

References and sources

1. Hailo. Hailo-8 AI Accelerator — product page and specifications. hailo.ai, 2026.
2. Hailo. Hailo-10H AI Accelerator — generative AI in edge computing. hailo.ai, 2026.
3. Hailo. Hailo-15 AI Vision Processors. hailo.ai, 2026.
4. Hailo. Hailo AI Software Suite — Dataflow Compiler, HailoRT, Model Zoo, TAPPAS. hailo.ai, 2026.
5. Hailo. Hailo Model Zoo — GETTING_STARTED. GitHub, hailo-ai/hailo_model_zoo.
6. Anwarzai, B.; Áč, V.; Lúby, Š.; Majková, E.; Senderák, R. Pseudo spin-valve on plastic substrate as sensing elements of mechanical strain. Vacuum, vol. 84, no. 1, 2009, pp. 108 – 110.
7. Lúby, Š.; Majková, E.; Debnárová, A.; Senderák, R.; Áč, V.; Anwarzai, B. Effect of magnetic flux distribution on GMR in Ag/Co multilayers. Thin Solid Films, vol. 433, 2003, pp. 243 – 246.
8. Anwarzai, B. et al. Magnetostrictive sensor — WIPO PCT WO2009088378; Slovak patent SK 50001/2008.
9. IOAS. IOAS Security Cluster (IOASC) — architecture, model-building pipeline, parallel and distributed computing. ioas.pro, 2025.
10. European Parliament and Council. Regulation (EU) 2024/1689 (Artificial Intelligence Act). Official Journal of the EU, 2024.
11. European Parliament and Council. Directive (EU) 2022/2555 (NIS2). Official Journal of the EU, 2022.
12. European Parliament and Council. Regulation (EU) 2024/2847 (Cyber Resilience Act). Official Journal of the EU, 2024.
13. ISO. EN ISO 3059 — Non-destructive testing: penetrant testing and magnetic particle testing, viewing conditions.

Illustrative photograph: Jakub Pabis / Pexels.

The technical parameters of Hailo products are taken from the manufacturer’s publicly available documentation (as of August 2026) and serve technical exposition; binding data are provided by the manufacturer. The experience with compilation, quantisation and splitting a model between accelerator and host comes from IOAS’s own projects.

© 2026 Branislav Anwarzai · Institute of Advanced Studies. The text may be quoted with attribution.