How we at IOAS designed specialised language models for the Slovak gastro segment. Three models fine-tuned for the Slovak accounting context (SK-Invoice-Extract, SK-Posting-Classifier, SK-Legal-RAG) achieve F1 0.961 for field extraction, 0.887 for posting prediction and 0.924 retrieval F1 — at 100% on-premise inference.

Abstract

Cloud large language models (LLMs) have brought a fundamental shift in document workflow automation over the past two years. Yet in regulated sectors such as accounting and tax compliance, they hit three systemic barriers: regulatory (GDPR, AI Act, tax secrecy), economic (per-document price at high volumes) and qualitative (poor accuracy on localised documents of small markets). In this paper we present a case study from the IOAS partnership with the development of a Slovak cloud-native operating system for the gastronomy segment, GastroPlay.sk, where we deployed three specialised language models fine-tuned for the Slovak accounting context: SK-Invoice-Extract (invoice extraction), SK-Posting-Classifier (double-entry posting prediction) and SK-Legal-RAG (legal search assistant). On a curated golden dataset of 4,200 annotated Slovak invoices and 12,800 accounting transactions we achieve F1 accuracy of 0.961 for field extraction, 0.887 for posting suggestions and 0.924 retrieval F1 for legal queries against Slovak law — at 100% on-premise inference with zero data exfiltration to the cloud. Median latency for OCR + extraction is 3.8 s, which is 2.1× faster than the baseline through public cloud APIs.

1. Introduction

The Slovak SME market in the gastronomy segment comprises approximately 14,600 active venues [1] with a typical profile: one venue, 1 – 15 employees, annual revenue under €500,000. These businesses process 30 – 200 incoming invoices weekly, issue 5 – 80 outgoing invoices and submit 20+ tax and statistical filings per year. Manual handwork in this cycle (scanning, posting, payment matching, VAT-Control-Statement filings) accounts in our observations of reference clients for 8 – 14 hours of administrative work per month which, from a business-value perspective, is wasted time.

Cloud LLMs (GPT-4 class, Claude Sonnet, Gemini Pro) achieve “few-shot” invoice extraction accuracy in the range 0.82 – 0.93 F1 [2] with proper prompt engineering. In the context of Slovak accounting, however, we encountered four specific issues:

  1. Slovak diacritics and morphology — inflection of Slovak vendor names means that cloud LLMs without explicit normalisation cannot reliably match “Foodservice Plus s.r.o.” on an invoice against the Slovak Business Register entry.
  2. Output format variability — Slovak invoices have no dominant template (unlike Germany’s ZUGFeRD ecosystem); they are often generated by various small ERP tools (iKros, MRP, Money S3, custom Excel sheets) with ~120 unique layouts in our test sample.
  3. Accounting posting — mapping invoice line text to the chart-of-accounts under MF SR Decree no. 23054/2002-92 requires domain knowledge outside the distribution of general cloud LLMs.
  4. Regulatory constraints — tax secrecy under § 11 of Act no. 563/2009 Coll. and Article 9 GDPR for certain PII in payroll filings prohibit or substantially complicate moving data to third-party cloud services outside EU jurisdiction.

These barriers motivated the design of locally deployed, domain-specialised models that we at IOAS designed, trained and integrated into the GastroPlay.sk product.

The trend toward “vertically specialised” small language models (SLMs) gained strong research and commercial traction in 2024 – 2026. Microsoft Phi-3 (3.8B params) [3] and Mistral 7B [4] demonstrated that models orders of magnitude smaller than frontier LLMs can, with appropriate domain fine-tuning, outperform large generalist models on specific tasks. In the financial document context, FinBERT [5], FinGPT [6] and DocFin [7] were published, but all were trained primarily on English corporate filings (10-K, 10-Q SEC reports) which have minimal overlap with European SME invoicing.

For the Slovak language, pre-trained encoders Slovak-BERT [8] and SlovakRoBERTa [9] exist, but at the time of our work (Q3 2025) no publicly available generative model specialised for Slovak accounting and tax language existed. Our approach therefore started from open multilingual base models (Llama 3.1 8B Instruct [10], Mistral 7B v0.3 [4]) and applied parameter-efficient fine-tuning (PEFT) techniques LoRA [11] and QLoRA [12].

For document extraction we built on LayoutLMv3 [13] and Donut [14]; for retrieval-augmented generation (RAG) we adapted the multilingual bge-m3 [15] encoder.

3. System architecture

The system consists of four hierarchical layers (Fig. 1) designed so that each subsequent layer operates on increasingly structured data and so that each can be horizontally scaled per load:

                  ┌──────────────────────────────────────┐
   Mobile / Web → │  Layer 0: Image preprocessing        │
                  │  (deskew, denoise, perspective)       │
                  └────────────────┬─────────────────────┘
                                   ▼
                  ┌──────────────────────────────────────┐
                  │  Layer 1: OCR + layout               │
                  │  Tesseract 5 + LayoutLMv3 (local)    │
                  └────────────────┬─────────────────────┘
                                   ▼
                  ┌──────────────────────────────────────┐
                  │  Layer 2: Entity extraction          │
                  │  SK-Invoice-Extract (Llama 3.1 8B    │
                  │  + LoRA, fine-tuned)                 │
                  └────────────────┬─────────────────────┘
                                   ▼
                  ┌──────────────────────────────────────┐
                  │  Layer 3: Classification & posting   │
                  │  SK-Posting-Classifier               │
                  │  (Mistral 7B + QLoRA)                │
                  └────────────────┬─────────────────────┘
                                   ▼
                  ┌──────────────────────────────────────┐
                  │  Layer 4: Right-hand assistant       │
                  │  SK-Legal-RAG (bge-m3 + Llama 3.1)   │
                  │  for queries into Slovak law         │
                  └──────────────────────────────────────┘

Fig. 1. Four-layer architecture. Layers 1 – 4 run on IOAS infrastructure in the EU region (Frankfurt) on Kubernetes clusters with NVIDIA L40S GPUs. No client data ever leaves EU legal jurisdiction.

3.1 Layer 0 – Preprocessing

The GastroPlay mobile app captures invoice photos typically under conditions with skewed lighting and perspective. We apply:

3.2 Layer 1 – OCR + layout

For OCR we use Tesseract 5.4 with Slovak training data extended by ~3,200 annotated regions from Slovak invoices (numbers, IČO, sums, IBAN). Geometric structure (text bounding boxes, table edges) is extracted via LayoutLMv3 fine-tuned on 1,800 annotated pages.

Hybrid OCR (Tesseract + LayoutLMv3) achieves 97.8% character accuracy on our evaluation dataset vs. 94.2% for Tesseract alone on Slovak invoices with poor printing.

3.3 Layer 2 – SK-Invoice-Extract

The main extraction model. Input: text representation of the invoice with spatial markers (<box x=120 y=300>Foodservice Plus s.r.o.</box>). Output: JSON conforming to the European EN 16931 standard [16].

Architecture:

This article is part of IOAS paid content.

Secure payment via Stripe · access restorable by e-mail, no sign-up