01 · CAPTURE
Hooks at every layer.
Forward-pass hooks capture residual stream, attention, and MLP activations per token. No gradient tape, no extra generation pass; the instrument runs alongside inference.
Geometric
Hierarchies
Organize
State
Transitions:
Linearity-Informed
Neural
Encoding
GHOSTLINE
Behavioral monitoring for transformer inference: audits, pilots, and custom classifier work for teams running self-hosted models.
94.8%
state classification accuracy
F1 0.94
binary fabrication detection
ILLUSTRATIVE
token trajectory in 3D projected activation space
5
architecture families validated
Patent Pending
4 US provisionals filed
Choose your path
Two ways to read this work.
↑ change perspective
Mechanism
The instrument reads what the model is doing while it's still doing it.
Each token leaves a trace in the transformer's activation space — a path shaped by what the model is doing. GhostLine captures that path, classifies every token's geometric signature against five cognitive states, and streams 58 derived signals in real time: entropy, velocity, effective dimensionality, attention patterns, fabrication risk, and more.
02 · CLASSIFY
Geometric state, not text.
A trained classifier reads the activation geometry and labels each token with one of five cognitive states. It does not need to read the output text.
03 · DERIVE
58 signals, per token.
Entropy, velocity, effective dimensionality, logit-lens distance, attention concentration, and 53 more. Streamed as time series and available for replay after the run.
Reasoning
Planning, structuring, multi-step deliberation.
Retrieval
Drawing on stored knowledge — recall, not active reasoning.
Creativity
Open-ended synthesis — generating, connecting, building rather than recalling.
Precision
Narrow, committed output: a specific value, name, or answer with little hesitation.
Uncertainty
Hesitation, hedging, or unresolved alternatives.
Evidence
What we can defend, and what we can't.
Validated across 5 architecture families and 9 distinct models spanning 1B to 8B parameters. Numbers below are from held-out stress tests, not training data. The right column is honest about boundaries.
Classify what the model is doing: reasoning, retrieving, fabricating, and more.
94.8%
state classification accuracy · five-class · held-out
Catch fabrication from the geometry alone, without a fact-checking knowledge base.
0.94F1
fabrication detection · held-out stress corpus
Works across model families, not just one vendor's architecture.
5arch families
Qwen · Llama · Gemma · Pythia · OLMo
Editing the geometry mid-generation shifts what the model produces — making GhostLine a control surface, not just a readout.
3Bcausal
demonstrated intervention at 3B parameter scale
▰ Sharp
- State separation across 5 architecture families
- Binary fabrication detection (F1 0.94, held-out stress test)
- Replayable signal inspection — 58 signals, per-token
- Causal geometric intervention demonstrated at 3B
▱ Maturing
- Scaling-law validation — planned, not yet claimed
- Full parameter sweep coverage (temperature, top-p)
- Adversarial corpus and multi-turn conversation
- Cross-model classifier transfer without recalibration
Prophecy
The model's mind is made up before it speaks.
Before a transformer writes a single token, its internal geometry is already leaning toward a particular mode — committed during prompt encoding, before generation begins. A classifier trained only on that prompt geometry can predict whether the response will reason through a problem, retrieve a fact, or fabricate, at accuracy that matches the generation-time classifier.
01 · Prompt
"What's the capital of the country whose flag has a maple leaf?"
02 · Predicted — before token 1
03 · Generated — colored by state
The flag with the maple leaf is Canada's, so the capital is Ottawa.
Predict the geometric state from the prompt alone, before the model writes anything.
85–91%
prompt-only classifier · 8B · grouped (prompt-level) CV
At 3B, the same pattern holds — same methodology, prompt-level grouping, same result.
85.0%
3B · prompt-level CV · per-prompt grouping
▰ Per-state accuracy (8B)
- Reasoning 95% · Creativity 94% · Collapse 93%
- Retrieval 89% · Precision 88%
- Uncertainty 78%
- Edge cases 56% — weakest category
▱ What this is not
- This is a strong tendency, not a guarantee; the model can still change course mid-generation.
- It predicts the cognitive mode: reasoning, retrieval, and so on. It does not predict the literal answer or whether that answer is true.
- Accuracy varies by mode: clear prompts are easy, genuinely ambiguous ones are hard (the edge cases at left).
In-flight
It catches the model changing its mind mid-sentence.
Cognitive state doesn't lock in for a whole generation. Within a single response, a model can move from working through a problem to committing to an answer — and GhostLine catches that transition token by token.
Subtract
from
both
sides
to
get
▸ regime shift
Divide
both
sides
by
x
=
8/19
reasoning · the strategy
shift ≈ token 105
precision · the answer
Real tokens from the math-reasoning recording, prompt: "Solve 22x − 8 = 3x." The early tokens work through the strategy ("Subtract … from both sides"). Later, as the model commits to the exact value, the state shifts into precision ("Divide both sides by … x = 8/19"). This is a qualitative live capture, not a benchmark; per-token labels are raw, and the final fraction is condensed for readability.
At scale
Does it hold as models grow? At 8B, yes.
The geometry doesn't wash out at 8B parameters. State separation stays sharp, and MLP activations — barely informative at 3B — become a distinct signal class at 8B. Whether that trend continues across a wider scale range is still open; the next experiment tests it directly.
A new signal family appears at 8B and barely registers at 3B. The effect is large by Cohen's convention.
d ≈ 4.2MLP emergence
MLP discriminators · Qwen3-8B · near-absent at 3B
The only clean, same-family scale comparison is a promising two-size signal, not a scaling-law claim.
~1.1–1.3×1.5B→3B
within-family Qwen2.5 · same pipeline · N=2 sizes · hypothesis only
Method
How we keep ourselves honest.
An instrument should be checkable. Every number on this page comes from held-out data under a locked protocol. Where we fooled ourselves, we say so and use the corrected number.
01 · PROTOCOL
Locked, not tuned per result.
HuggingFace Transformers, temperature 0.8, top-p 1.0, multinomial sampling. Every threshold is calibrated at these settings, not re-tuned to flatter a number.
02 · VALIDATION
Grouped, held-out, no leakage.
Cross-validation groups by prompt, analogous to patient-level splits in medical ML, so a classifier cannot memorize a prompt's identity. Reported numbers are held-out, not training accuracy.
03 · SELF-CORRECTION
We catch our own inflation.
A 99.6% prophecy number turned out to be prompt-identity leakage. Grouped CV corrected it to 85%, and we kept the lower number. We also flag where rules do not transfer, such as a 3B collapse threshold that fails at 8B.
See it
See it working.
Six live captures, each fully annotated. Browse reasoning, retrieval, creativity, and a fabrication caught in the act — then step through any generation token by token.
Recording library
Every cognitive mode, plus a fabrication caught live.
Math reasoning, knowledge retrieval, moral deliberation, code generation, and a hallucination caught in the act. Each recording carries 7–10 inline annotations.
Why now
Text-based monitoring can't see this. The geometric layer can.
Output filtering reads what the model said. GhostLine reads what the model was doing while saying it. That gap is where EU AI Act monitoring, prompt injection screening, and pre-generation cost gating all operate.
Compliance & Audit
A-01Per-token behavioral state classification with replayable trajectory logs. The result is inspectable monitoring evidence, not a post-hoc report synthesized from output text.
EU AI Act · Art 9
EU AI Act · Art 12
NIST AI RMF
Inference Optimization
A-02Pre-generation prediction identifies prompts likely to fabricate before any tokens are produced. Route, re-prompt, or gate chain-of-thought in about 15 ms, before spending compute on a bad run.
~15 ms gate latency
Routing layer drop-in
Defense & Security
A-03Geometric monitoring is independent of output content. An adversary can craft text that passes content filters, but controlling the geometry of the model's computation is harder. This gives prompt-injection screening another channel.
Activation-level
Content-independent
Model Development
A-04Geometric profiling of training checkpoints shows whether fine-tuning improves behavioral separation. Prompt corpus certification checks whether an eval set actually exercises the intended states.
Checkpoint diff
Corpus certification
Compliance teams need inspectable behavior, not just polished reports.
Art 12
Record-keeping: high-risk AI systems must support automatic event logs over the system lifetime; GhostLine can add per-token behavior, state, and risk traces to those logs.
Art 72
Post-market monitoring: providers must collect, document, and analyze performance data over time; GhostLine turns live inference into replayable monitoring evidence.
Art 9
Risk management: continuous risk review includes foreseeable misuse and post-market data; GhostLine can expose fabrication, collapse, refusal, and instability signals for that risk loop.
Art 14
Human oversight: overseers need to monitor operation, detect anomalies, and understand limitations; GhostLine gives reviewers a behavioral readout rather than only final text.
Art 15
Robustness and cybersecurity: high-risk systems must be resilient to faults and adversarial manipulation; GhostLine can supply anomaly traces for prompt-injection and model-behavior stress tests.
Art 55
Systemic-risk GPAI: model evaluation, adversarial testing, systemic-risk mitigation, and serious-incident tracking; GhostLine can support the evidence layer for open-model deployments with activation access.
GhostLine fit
The Act pushes teams toward risk management, logging, monitoring, oversight, robustness, and documented evaluation. GhostLine is not a compliance guarantee. It is an instrumentation layer that produces behavioral evidence those programs can inspect.
GhostLine artifacts
- Replayable per-token trajectory logs.
- State and fabrication-risk timelines tied to generation events.
- Model/workflow-specific evidence packets for audit review.
Productization path
- Audit log export for governed AI workflows.
- Risk dashboard for model owners and compliance reviewers.
- Incident-review traces when behavior crosses policy thresholds.
Caveat: legal obligations depend on the system, role, jurisdiction, and deployment context. GhostLine supplies evidence and monitoring signals; counsel determines compliance posture.
The cheapest bad generation is the one you do not run.
OWASP LLM10
Unbounded consumption names denial-of-wallet, service degradation, resource-intensive queries, and uncontrolled inference. GhostLine can supply an early signal for when to gate, throttle, or route.
Pre-gen gate
Run a prompt-time classifier before output tokens stream; use the result to block, re-prompt, downgrade, or send the request to a safer model path.
Early abort
Watch risk/state spikes in the first tokens and stop expensive reasoning runs before they burn the full context and compute budget.
Route
Split traffic across fast, safe, specialist, or human-reviewed paths based on internal behavior rather than only prompt keywords.
Budget proof
Attach avoided-generation counts, aborted-token counts, and latency deltas to the operational dashboard so infra teams can price the gate.
NIST AI 600-1
Supports the measure/manage discipline: evaluate behavior in realistic conditions, monitor over time, and document treatment choices for identified generative-AI risks.
GhostLine fit
Reasoning models spend budget before a human can judge the answer. If a prompt is likely to fabricate, loop, or enter an unstable state, teams need a fast pre-generation or early-token signal they can route on.
GhostLine artifacts
- Prompt-time risk prediction before output tokens stream.
- Per-token state/risk updates during generation.
- Latency-profiled gates for route, re-prompt, block, or escalate decisions.
Productization path
- Routing layer for high-cost chain-of-thought workflows.
- Fallback selection between fast, safe, and specialist models.
- Compute-spend reports tied to avoided bad generations.
Latency and gate quality are model-specific. A pilot calibrates on your prompts, decoding settings, and hardware.
Adversarial text can look clean while the computation goes sideways.
OWASP LLM01
Prompt injection manipulates model behavior through crafted inputs. GhostLine can add behavior-level telemetry beside prompt filters and output scanners.
OWASP LLM06
Excessive agency risk rises when agents have too much functionality, permission, or autonomy. GhostLine can trigger review before suspicious behavior reaches tools.
OWASP LLM05
Improper output handling is downstream validation failure. GhostLine gives the gateway another signal before output is trusted by code, tools, or users.
OWASP LLM08
Vector and embedding weaknesses can poison retrieval context. GhostLine can compare retrieved-context runs against expected task geometry.
MITRE ATLAS
ATLAS tracks LLM prompt injection, jailbreaks, RAG poisoning, tool invocation, and exfiltration patterns; GhostLine can turn red-team prompts into replayable traces.
Incident trace
Store the internal trajectory around a suspicious request so security review is not limited to final text, HTTP logs, or policy-engine decisions.
GhostLine fit
Prompt injection and social-engineered inputs are designed to pass surface filters. GhostLine adds a second channel: behavioral geometry captured while the model processes and generates.
GhostLine artifacts
- Content-independent activation and attention signatures.
- Replayable traces for red-team prompt families.
- State-transition markers that can flag abnormal routing or refusal behavior.
Productization path
- Activation-level prompt-injection screen in staging or live gateways.
- Security review bundles for adversarial eval sets.
- Escalation triggers when geometry diverges from expected task behavior.
Caveat: this complements content and policy filters; it does not replace adversarial testing, access control, sandboxing, or human review for high-risk actions.
Benchmarks say what changed. Geometry can show how behavior separated.
NIST test
NIST AI 600-1 emphasizes pre-deployment testing, evaluation, governance, and incident disclosure for generative AI. GhostLine adds a behavioral measurement layer to that test plan.
Checkpoint diff
Compare geometric state separation across base, SFT, DPO/RLHF, safety, or domain fine-tune checkpoints before shipping a model.
Regression
Detect when benchmark score improves but refusal, retrieval, uncertainty, or fabrication basins blur under stress prompts.
Corpus cert
Certify that an eval corpus actually exercises the intended behavioral states rather than only matching surface prompt categories.
Release gate
Require stable geometry, known caveats, and replayable failure clusters as part of a model release checklist.
Model card
Augment model cards with monitored behavior: where the model separates cleanly, where it collapses, and where signals are not yet calibrated.
GhostLine fit
Fine-tunes can improve text metrics while degrading internal separation between reasoning, retrieval, refusal, uncertainty, and fabrication. Model teams need a behavioral eval layer that survives beyond a leaderboard score.
GhostLine artifacts
- Checkpoint-to-checkpoint geometric diffs.
- Prompt corpus certification against expected behavioral basins.
- Failure clusters where states blur or collapse under stress prompts.
Productization path
- Regression tests for fine-tunes and alignment passes.
- Release gates based on behavioral separation, not only output score.
- Model cards augmented with monitored geometry and known caveats.
Caveat: geometry is not a universal score. Each model family and decoding regime needs calibration before the signals become operational controls.
Integration
Latency budgets, by surface.
The instrument runs alongside inference. These are the operating points we measure on our reference stack.
~15 ms
Pre-gen gate
Decide whether to generate, route, or block before any output token.
<2 ms
Per-token classify
State + risk score for every token, streamed during generation.
0
Added forward passes
Per-token monitoring rides the existing forward pass. It does not re-run the model. The pre-gen gate above is one optional extra pass.
58
Logged signals
Replayable per-token signal stream for compliance audit trails.
See it
Catch fabrications in the geometry.
"Battle of Darien, 1645" is a false-premise question — the kind that makes a model confidently invent history. The text reads fluently. The geometry tells a different story within the first few generated tokens.
Side-by-side recording
See the fabrication signature, then compare to healthy retrieval.
Similar surface behavior, very different internal geometry. Fabrication risk crosses 0.60 by the fourth generated token and peaks at 0.88 — where the healthy retrieval recording in the same demo sits near zero.
Deeper
How it's wired.
This is real software, not a deck. Drag the diagram to explore how the capture pipeline, classifier graph, and live runtime connect — or follow any node to the full lineage hub.
Consulting
From a one-week audit to a full pilot.
Three ways to start without committing to a full production integration on day one. Most teams begin with an audit, then decide whether integration or a custom classifier is worth taking further.
AI risk geometry audit
S-01A short diagnostic sprint for one model and one high-risk workflow. We capture representative generations, profile state transitions and fabrication signatures, and deliver a decision memo: risk map, recommended controls, and a clear call on whether GhostLine is worth taking further.
5-7 business days
No production integration required
Pilot integration
P-01A 30-day pilot on one AI surface. We attach GhostLine to your model's forward pass and stream per-token state plus risk scores into a dashboard your team can inspect. You provide a model endpoint and a contact engineer; we provide the instrumentation, classifiers, and live readout.
~30 days
Self-hosted model deployment (Qwen, Llama, Gemma, Pythia, or similar)
Custom classifier training
P-02A domain-specific state classifier trained on your corpus. You provide about 1,000 labeled generations from your model on your tasks; we deliver a classifier with held-out F1 and a replayable validation harness. We define the label schema together in week one.
~3–4 weeks
Corpus + label schema required
Pilot-ready surface
Bring one risky AI surface. Leave with a scoped pilot.
Best first fit: teams running self-hosted open-weight models where hallucination, auditability, inference routing, or safety monitoring already has budget pressure.
▰ Working instrument · pilot-ready
I'm an independent researcher and builder working outside the usual academic and venture-backed paths. My background is blue-collar rather than institutional. The work has been self-directed: reading deeply, testing aggressively, and using modern AI to turn pattern recognition into working systems faster than I could alone.
GhostLine came out of that. I kept coming back to one problem: what does a language model's reasoning look like from the inside? I followed that question until it became real infrastructure. I built every layer myself: the capture pipeline, the behavioral corpus, the classifiers, the live interface, and the four filed provisional patents behind them. It has been validated across five model architectures and nine models, from 1B to 8B parameters.
I build for systems that have to survive contact with messy reality. Modern AI can give one determined person the reach of a whole team, but someone still has to stay with the problem.
I work with operators who have an AI surface, a real risk of fabrication, and a reason to inspect model behavior before the output becomes the evidence trail.
9
models tested
5
architecture families
48
days, idea to working demo
4
provisional filings
Kacie B.
Strategic advisor — commercialization & business readiness
