The Problem with Jev: Can You Trust AI Decisions?
Jeff Crume examines Jev, a new AI approach that makes structured decisions rather than generating text, through a cybersecurity lens. He argues that confidence scores are not evidence, that AI decisions expand the attack surface, and that observability, decomposed reasoning, independent controls, and human oversight are needed before trusting them.
The trust problem with confident AI decisions
The presenter opens with a playful fortune-teller exchange to make a point: an answer delivered with confidence is not the same as an answer you can rely on. Viewing the question through a cybersecurity lens, where trust is never assumed, he examines Jev, a new AI approach that makes quick, structured decisions rather than generating words like an LLM. For him, Jev is a black box. You get an input, a decision, and a probability such as 97 percent, but you cannot see how the model reached its conclusion, so you cannot tell whether it is wrong, or how you would find out.
LLM outputs versus Jev decisions
LLMs, even when they hallucinate, give you something to inspect. You can ask why they produced an answer, challenge the reasoning, have a second model review the output, and examine the tools used, the training data, and the chain of evidence behind the response. Jev’s output is structured and easy to plug into software, which is a real advantage, since the software does not have to interpret free text. But the probability describes the model’s confidence, not the reason for the decision. As he puts it, confidence isn’t necessarily evidence, and it certainly isn’t proof.
Confidence is not evidence: a security alert
To make this concrete, he walks through an alert in which Jev recommends containing a host with 97 percent confidence. As the security engineer responsible, he needs to know why. Did a process match known malware? Was the IP address known to be malicious? Was there unusual behavior? Is the machine especially sensitive? Was there an authorization problem that the model recognized but did not understand? Or was the information feeding the model, or the model itself, manipulated? The answer matters because the decision may hinge on any of these factors, and the output alone does not reveal which one.
The AI attack surface
From a security perspective, both the input and the model are part of the attack surface. An attacker could alter a log, manipulate metadata, poison a training data source, or exploit a weakness in the workflow that feeds the model. The model could then make a perfectly consistent decision based on corrupted information. The result is what he calls ‘confident garbage out.’ The model still reports 97 percent confidence, which means only that it is confident given what it saw, not that the decision is trustworthy. It could be confidently wrong because of an attack, or simply wrong for an innocent reason, much as LLMs hallucinate.
Observability and the audit trail
When something goes wrong in a traditional system, you want to reconstruct what happened: what came in, what changed, which rules fired, what data was consulted, what decision was made, and who or what caused it. The goal is an audit trail that you can investigate and use to verify that the system behaved as intended. That is how you investigate incidents, detect attacks, and establish accountability. If an automated decision is going to shut something down, you need to be sure the advice is correct, not just confident. Jev’s structured outputs and probabilities do not, by themselves, provide that visibility.
Breaking decisions into verifiable steps
He highlights an idea attributed to the company behind Jev: rather than asking one big question, break it into smaller ones. In the alert scenario, you might first ask whether the activity was authorized, then what evidence supports that conclusion, then what the risk calculation looks like. Combining these intermediate results yields the final decision. This is better engineering because each step can be observed individually, rather than treating the entire process as an opaque box. It is stepwise refinement, where each decision builds visibly on the previous one.
Why verification is hard
He frames the security mindset as ‘verify, then trust,’ not the reverse. With Jev, you can verify that the output conforms to the expected type, record the probabilities, log the inputs, and audit the surrounding workflow. What you cannot easily do is independently verify the intelligence that produced the decision. You have a prediction, not a proof. He warns that attackers favor exactly the things defenders cannot see. In traditional systems, a compromise leaves logs, events, and indicators of compromise. A subtly compromised AI decision system might behave almost normally, becoming just slightly more likely to approve a bad action, or more inaccurate for a particular type of target. Each answer may look reasonable on its own, and accumulated trust, built on repeated good results, can make people less critical over time. Confidence becomes self-reinforcing, which makes the problem harder to detect.
Treating the AI as an untrusted component
He is not arguing against using Jev. He thinks the capability is extremely useful, but recommends treating the AI itself as an untrusted component and wrapping it in security controls. These include input validation, knowing exactly what goes in, relying on independent signals rather than a single source, auditing logs, watching for drift in model behavior, and looking for anomalies. For high-risk decisions, humans should be involved in escalation, with independent verification. The key principle is to keep a human in the loop and not let Jev be the only system that knows what happened.
Why the stakes rise as AI takes action
He distinguishes between an LLM writing a bad paragraph, which is one kind of problem, and an AI that blocks an account, pays an invoice, shuts down a server, or approves a transaction, which is an entirely different security problem. The question shifts from ‘can AI make the right decision?’ to ‘can I know when it makes the wrong decision, and can I tell whether someone caused it?’ He is clear that probabilistic or structured decisions are not inherently bad, and he finds the structured approach compelling. His concern is proportional: the more decisions we delegate to AI, the more essential it becomes to observe, audit, and verify them.
His closing message is that we should not just ask ‘what did AI decide?’ but ‘how do I know what it decided?’ Verify, then trust, so that AI decisions are not left as magic.