From Readiness To Accountability
On August 2, 2025, the EU AI Act’s obligations for providers of general-purpose AI models began to apply. Those obligations include technical documentation and information for downstream system providers; additional rules apply to models with systemic risk. The European Commission’s guidance also distinguishes models already on the market and explains the enforcement timetable.
Documentation at the model level doesn’t automatically make a particular operational decision auditable. Suppose a financial institution must review why a transaction was frozen six months ago. The review needs the evidence available at that time, the model and prompt version, the policy applied, the approving authority, and the action taken. A probability score or generated explanation can’t supply missing records.
The Black Box Problem
Large language models (LLMs) generate output from learned statistical relationships. Their behavior also depends on the model version, configuration, prompt, retrieved context, tools, harness and sampling settings.
When an LLM is asked why it produced an earlier output, the answer is another generated output, not an executable record of the earlier computationm which is a distinction that matters in an audit.
Training-data review, evaluation metrics, and explainability techniques answer useful model-governance questions. They don’t show which evidence and authority permitted a specific action at a specific time. Decision-level auditability must be designed into the workflow around the model.
The Glass Box Solution: Neurosymbolic Hybrid Engineering
One useful pattern separates probabilistic perception from explicit policy and action control. A neurosymbolic architecture can provide that separation, but it still needs evidence management, identity, authorization, observability, and recovery across the full workflow.
1. Neural Networks As The Perceptual Layer
Neural systems can extract information from documents, sensor data, images, or audio. So treat their classifications, confidence scores, and extracted facts as evidence with provenance and uncertainty, not as self-authorizing conclusions.
2. Symbolic Reasoning As The Explicit Logic Layer
Explicit rules can apply business constraints, thresholds, and exception conditions to structured evidence. Given the same recorded inputs and rule version, this layer can reproduce its result. Rules must still be governed, tested, and reviewed when the operating context changes.
3. Control Layers For Policy, Uncertainty, And Authority
- Policy evaluation governs which rules apply under which operational contexts.
- Uncertainty handling ensures that ambiguous or low-confidence inputs are surfaced, constrained, or escalated rather than silently acted upon.
- Authorization controls enforce who or what may recommend, approve, execute, delegate, or stop an action.
The execution layer should refuse or escalate when evidence, policy, or authority is missing. Deterministic logic can’t compensate for unreliable evidence or an incorrect rule, so representative evaluation must cover both.
4. Decision Records For Explanation And Execution
Capture:
- The specific evidence used
- The model, prompt, tool, rule, and policy versions
- The authorization state at execution
- The action, outcome, exception, and recovery steps
This creates an inspectable operational record. Formal verification may be appropriate for particular rules or components, but should not be claimed for the whole workflow unless its assumptions and scope are defined.
The tradeoff is real: explicit domain modeling, rule governance, evidence retention, and replay testing require ongoing ownership. That work is the cost of making consequential decisions inspectable.
Moving Towards Verification
The practical test is not whether every internal model operation can be reconstructed. Ask whether the organization can reconstruct the operational decision: the evidence presented, the model and policy versions, the authority in force, the permitted action, and the outcome.
Begin with one consequential workflow. Test routine cases alongside stale evidence, conflicting inputs, missing approval, model changes, and failed recovery. Use decision support or augmentation until the evidence justifies a narrower automation boundary.
Making one AI decision reconstructable? Book a Consultation.