VIGILANT/CRS
COGNITIVE & RESILIENT SYSTEMS
Stuttgart · applied AI research & systems

AI SYSTEMS
THAT HAVE TO
WORK OUTSIDE
THE LAB.

Licensable runtimes, model-to-device optimization and applied R&D. We work where a standard model or generic AI stack stops being enough: real hardware, shared accelerators, uncertain sensors, autonomous decisions and persistent worlds.

01 / Core systems

Six systems built around real constraints.

Each system starts with a concrete failure mode: a model that does not fit the target device, stale inference, sensor data without context, scripted game populations, unreliable positioning or language that cannot represent the intended experience precisely enough.

MODEL → DEVICE / HARDWARE-AWARE ADAPTATION

TANS

INPUTBase model · domain data · target hardware · workload
COREFine-tune · distill · prune · quantize · export · measure
OUTPUTDeployment artifact that fits explicit quality and device budgets

ProblemA model can benchmark well and still be unusable on the target machine because latency tails, peak RAM, sustained thermals, energy draw or accelerator constraints were treated as an afterthought.

SolutionTANS adapts the model and evaluates the actual target together. It can fine-tune foundation models or detectors for the domain, distill from a larger teacher, prune unnecessary structure, quantize precision, export to the required runtime and measure every change on the device.

OutcomeNot simply “a smaller model”, but a reproducible deployment candidate selected against quality, p95/p99 latency, memory, thermal throttling and power/energy budgets — with changes accepted, rejected or rolled back from measured evidence.

TARGETS → Jetson / edge GPU · automotive SoC · phone / tablet · workstation GPU · CPU-only edge · EXPORT → ONNX · TensorRT · GGUF · device-specific runtimes · USE CASES → VLM/LLM adaptation · specialist detector tuning · edge vision · thermally constrained devices
PRE-PRODUCTION / VIGILANT INFERENCE QOS

Vigilant Inference Governor

INPUTDetector · pose · depth · VLM/LLM sharing one accelerator
COREFreshness · deadlines · admission · model-variant choice
OUTPUTUseful results before they become stale

ProblemA robot or vehicle can execute every requested model efficiently and still react late because an old camera frame has no value to the current control loop.

SolutionVigilant Inference QoS sits in front of NVIDIA Triton or TensorFlow Lite and understands age-of-information. It drops superseded work before dispatch, refuses work that would finish too late, protects periodic perception and can select a smaller model variant when only that variant still fits the time budget. vig autotune measures the actual models on the target hardware and can also conclude that no governor is needed.

OutcomeKeep the existing models and inference backend; change which work deserves scarce accelerator time. In the public public Inference Governor demos, including the public Inference Governor workload, the protected camera remains fresh in 100% of cycles instead of 0.1–0.2% under the demonstrated shared-GPU load.

Static local preview for the Vigilant Inference Governor public demo
VIDEO / INFERENCE GOVERNORYOUTUBE ↗

External video: this preview is stored locally. No YouTube player, thumbnail or tracking resource is loaded automatically. A connection to Google/YouTube begins only after you click the public demo video.

MEASURED → tuned Triton detector coverage 85% → Vigilant 99% on the measured GPU benchmark · ~20× fewer missed cycles there · four-camera demo: protected stream 0.1–0.2% → 100% fresh cycles under the demonstrated load · tested on NVIDIA GPU and Android/Adreno paths · 900 passing tests in the public repository

External links: GitHub and GitHub Pages are operated by GitHub, Inc. A connection is established only after you follow the respective link.

EDGE SECURITY / AUTONOMOUS SITUATIONAL AWARENESS / DISPATCH

Autonomes Reaktives Entscheidungs-System

INPUTCamera · thermal · radar · specialist detectors · local rules
COREDetect · track · correlate · evidence graph · scenario logic
OUTPUTLive local situation picture · priority · dispatch / workflow action

ProblemConventional security stacks generate detections and alarms, but a control room still has to correlate cameras, thermal feeds, radar and earlier events before deciding whether something is relevant and what should happen next.

SolutionThe system builds a continuously updated situation picture directly at the edge. Domain-specific detectors feed tracked entities and observations into a temporal knowledge graph; source, time, confidence and provenance remain attached. Graph retrieval, rules and reactive decision logic evaluate the evolving situation instead of treating each frame as an isolated event.

OutcomeOn-edge situational awareness can classify and prioritize security events, create a structured local situation report and trigger predefined workflows — for example cue another sensor, notify a control room, or dispatch the nearest available response resource — without requiring cloud inference.

USE CASES → critical-infrastructure perimeter monitoring · industrial / logistics sites · remote facilities · control-room retrofit · mobile or robotic security platforms · multi-sensor verification · ENGINEERING → specialist detector fine-tuning · temporal knowledge graph · provenance-aware graph retrieval / RAG · ONNX / TensorRT edge deployment · camera / thermal / radar fusion
UNSCRIPTED / COMMERCIAL EVALUATION / GAME AI / SOCIAL SIMULATION

UNSCRIPTED / Social Dynamics Runtime

INPUTPlayer action · witnessed event · authored world facts
COREKnowledge · memory · trust · rumours · groups · place mood
OUTPUTA world that changes socially without scripting every branch

ProblemMost open worlds have one invisible shared brain: a flag changes and every NPC effectively knows the same state. An LLM can make dialogue richer without fixing that underlying world model.

SolutionUNSCRIPTED is the runtime behind this approach. Each NPC holds local knowledge with source, confidence and history. Information can spread, mutate, be rejected or forgotten; trust and group membership affect belief; locations carry social state. The runtime decides what happened and what a character may know. A language model is optional and can be limited to wording.

OutcomeThe player changes the world dynamically. Witnesses react first, information travels through the social graph, a street or district can become more suspicious or closed, and characters encountered later can behave differently because of what actually reached them — not because a designer scripted that exact dialogue branch.

Local static preview of the Social Dynamics Runtime information propagation demonstration
VIDEO / RUMOUR · LIES · MEMORY · CONSEQUENCEYOUTUBE ↗

External video: this preview is hosted locally. No YouTube player, thumbnail or tracking resource is loaded automatically. Clicking opens YouTube and establishes a connection to Google/YouTube.

INTEGRATION → Godot 4.6 · Unity 6000 LTS / Unity 6 tech stream · Unreal 5.8 + MetaHuman bridge · HTTP/JSON for other engines · deterministic runtime · optional model wording layer · 204 passing tests · sealed evidence run: 2.99M turns / 59,745 invariant checks / 0 violations · 180-day free commercial evaluation for companies of any size

External links: the showcase is hosted on GitHub Pages and the source repository on GitHub. They are not embedded here; the connection begins only when you click.

R&D / INTEGRITY-AWARE FUSED GEOLOCATION

Fused Geolocation

INPUTRaw GNSS · Doppler · IMU · barometer · passive cell
COREES-EKF · clock state · exclusion · integrity / threat logic
OUTPUTPosition + confidence + degradation reason

ProblemIn tunnels, urban canyons or under jamming/spoofing, a normal location API may keep returning a coordinate without telling the application whether it should be trusted.

SolutionThe implemented stack fuses local sensor and satellite evidence with a receiver-clock-aware Error-State Kalman Filter, bounded RAIM/FDE-style exclusion and explicit integrity/threat diagnostics.

OutcomeThe consumer receives both the fused position and an explanation of degraded confidence. The architecture is built to continue conservatively when clean GNSS disappears rather than silently treating all fixes as equal.

IMPLEMENTED → ES-EKF · raw pseudorange / Doppler / carrier phase · clock bias + drift · barometer bias · passive LTE/NR + local tower DB · spoofing/jamming suspicion · replay / truth tracks · R&D → Hidden Markov mode model / Markov-chain transitions · snapping / map matching · multi-hypothesis tracking · independent device-to-device clock reference as future GNSS-time cross-check
R&D / EXPERIENCE SEMANTICS / POST-LINGUISTIC COMMUNICATION

Vigilant ESP / Experience Semantic Protocol

INPUTExperience · knowledge · intent · emotion · context
CORETyped semantic spaces · vector state · provenance · transformation
OUTPUTMachine-readable meaning that can be matched, transferred or rendered

TheoryHuman language compresses many different dimensions — factual knowledge, intention, emotion, sensory qualities, context and temporal continuity — into a linear sentence. ESP asks whether machines need that linguistic bottleneck at all. It represents those dimensions in typed semantic spaces so that an experience or knowledge state can be encoded as a structured vector state with metadata rather than first being rewritten as prose.

Protocol ideaA sender can transmit a machine-readable semantic state — for example what is known, how something felt, what outcome is desired and with what confidence or provenance. A receiving system can compare, transform or combine that state directly. Natural language, an image, sound or another interface becomes a rendering layer at the edge, not necessarily the transport format itself.

First implementationThe current media engine is the first practical implementation of this post-linguistic idea. A film is mapped into an experience genome; a user's desired experience is mapped into the same semantic space. “Like this film, but darker, faster and less violent” becomes a vectorized state transformation rather than only a keyword query.

DirectionThe longer-term goal is broader than media recommendation: agent-to-agent exchange of experience, knowledge and intention in a structured semantic representation that preserves dimensions which ordinary text can flatten or omit.

CONCEPT → typed semantic spaces for knowledge · intention · emotion · context · sensory state · temporal continuity · REPRESENTATION → structured/vector semantic state + metadata / provenance · IMPLEMENTATION → experience genome · tone-shift transformation · hybrid semantic / emotional / thematic retrieval · PAPER → “The Experience Semantic Protocol — A North Star for Post-Linguistic Communication” · DOI 10.5281/zenodo.20024213
02 / Applied systems

Narrower applications of the same engineering approach.

Useful proof that the principles above also survive ordinary operational workflows: local processing, explicit evidence and deterministic interfaces.

LOCAL DOCUMENT AI / IDENTITY

On-Device Identity Extraction

Passport, ID card and residence-permit images are processed locally. OCR, MRZ/check-digit validation, layout heuristics and a local model produce structured fields with confidence and can map them into an existing browser workflow without cloud OCR.

Pilot →
UNIVERSAL DOCUMENT TRANSFORMATION

Document Transformation Engine

Email, PDF, scan, image, spreadsheet, EDI or proprietary export in; the required machine-readable target format out. A canonical intermediate model and evidence per field connect heterogeneous input to CSV, XLSX, JSON, XML, EDIFACT or customer-specific schemas.

SOVEREIGN MESSAGING / DISTRIBUTED TRUST

Distributed Trust Messaging

Messages stay off-chain and use established end-to-end encryption. Transport, device identity/control and software updates are separate trust planes; a distributed ledger can anchor identity, key state and revocation without becoming a message database.

Discuss architecture →
DISCONNECTED SYSTEMS / OPTICAL TRANSPORT

QR-Based Data Transfer

Binary payloads can be segmented across a changing QR stream and reconstructed with packet ordering and integrity checks — useful where a screen and camera are available but network, pairing or radio transport cannot be assumed.

Discuss integration →
SPECIALIST VISION / DOMAIN DETECTORS

Detector & Vision Model Adaptation

Generic object taxonomies often stop where an operational domain begins. We build dataset taxonomies, hard negatives and evaluation sets, fine-tune detectors for domain-specific classes, then export and profile them for the actual edge target — including ONNX/TensorRT deployment where appropriate.

Discuss a detector →
03 / Research directions

Where we are pushing beyond the current product boundary.

Some work is valuable precisely because it is not yet a packaged product. These are recurring research directions that connect several of the systems above.

POST-LINGUISTIC COMMUNICATION

Meaning before wording.

Language is an extraordinarily useful human interface, but it is not necessarily the best transport format between machines. Our research asks whether experience, knowledge, intention, emotion and context can be represented directly as typed semantic/vector states, transferred between agents and only converted into text, speech, image or action when an interface requires it.

Vigilant ESP is the first implementation of this direction: an experience is encoded as a structured semantic state, another desired state is expressed in the same space, and the system can compare or transform the two without first reducing either one to a sentence.

ESP paper / Zenodo ↗
MACHINE ECONOMY / MULTI-AGENT / SWARM

Autonomous agents as market participants.

What is a Machine Economy? An economic system in which software agents, robots or connected machines are not only tools but autonomous market participants. They can discover demand, offer capacity, negotiate terms, buy data or compute, sell a service, reserve physical resources and transact with humans or with other machines.

Mechanism Design is the coordination layer. Instead of prescribing every bilateral interaction, we design the rules of the market: auctions, matching, pricing, bargaining, budgets, incentives, penalties and allocation mechanisms. The objective is to make individually rational local decisions produce useful system-level behaviour even when agents hold private information about costs, utility or availability.

Our decentralised architecture direction. Agents keep local decision logic and private utility functions, while a blockchain / distributed-ledger layer can provide machine identity, ownership and entitlement records, auditable settlement and programmable smart-contract rules. Humans and machines can therefore participate in the same market without requiring one central optimizer to know every participant's internal state.

Examples. A vehicle can bid for charging or loading capacity; a robot can procure compute or a spare resource; logistics agents can negotiate transport capacity; machines can sell unused energy, data or processing time; software agents can contract other agents to perform subtasks and settle the result automatically.

Mechanism design / joint R&D →
04 / Research infrastructure

When the deliverable starts as a question, not a product specification.

We use software-engineering discipline for research itself: explicit claims, executable tests, simulations, falsification and durable evidence.

SCIENCE AS SOFTWARE / AGENTIC RESEARCH

Research OS

Research OS treats a programme of research like a software project. Sources become claims, assumptions and dependencies; claims receive executable gates analogous to unit and integration tests; simulations and benchmarks produce evidence; holdouts keep calibration separate from validation.

Agents may generate theory, derivations, code and experiments, but they do not get to decide that their own result is true. The project state advances only when the defined evidence gates pass.

INPUTPaper · hypothesis · theory · dataset
COREClaim graph · derivation · test gate · simulation / benchmark
OUTPUTPromote · demote · falsify · durable research memory
EXAMPLE ENDPOINTS → Monte Carlo · Boltzmann / lattice-Boltzmann · numerical physics · cosmology / theoretical-model simulations · software benchmarks · ablation studies
05 / Foundations

The theory sits underneath the systems.

These are not decorative capability words. They recur because the products above repeatedly need to answer the same questions about information, uncertainty, incentives, time and collective behaviour.

GAME THEORYMECHANISM DESIGNINFORMATION THEORYKNOWLEDGE GRAPHSGRAPH RETRIEVAL / RAGBAYESIAN ESTIMATIONERROR-STATE KALMAN FILTERSHIDDEN MARKOV MODELSMARKOV CHAINSMULTI-AGENT SYSTEMSSWARMSPOST-LINGUISTIC COMMUNICATIONEDGE AI
The recurring engineering question: what does the system know, how certain and how old is that information, what can the hardware compute in time, and how does one local decision alter the larger system?
EXTERNAL LINKS ↗ → GitHub, GitHub Pages, Zenodo and YouTube are third-party services. No third-party player, page or remote thumbnail is embedded automatically; a connection to the respective provider begins only after you follow an external link.
06 / Contact

BRING THE
HARD PART.

Commercial evaluation · licensing · model-to-device adaptation · pilots · industrial R&D · research collaborations

info@vigilant-crs.de →