I build AI systems end to end: inference, agents, retrieval, evals, and the backend that keeps them running. I've been an applied AI engineer at BNY since July 2024.
I spend my own attention on the spec and the system design and let agents do the grinding. Then I prototype, ship, test, and improve, as fast as the work allows.
On the side I train small models, take big ones apart, and publish the benchmarks, including the ones that didn't go my way.
FractalMem, memory for coding agents that run for hours.
Reading
Looped transformers, recursive self-improvement, and the economics of inference.
Training
Lifting.
Built and measured
openJev Verdict
2026 · 405 stars
A 151M model that makes typed decisions in about 20 ms, and beat models up to 230 times its size on JevBench.
Most automation asks a chat model for JSON, then parses it and retries when it breaks. Verdict skips generation. One forward pass over ModernBERT returns a typed choice, score, or yes/no with a calibrated confidence, so plain code can decide when to act and when to hand off to a person. Version 1.4 fixed three inference bugs without retraining and moved the JevBench score from 66.2 to 74.9. On the 20 September leaderboard that put it first among open models, ahead of models with 9B to 35B parameters, GPT-5.6 Luna, and Gemini 3.1 Flash-Lite. Verdict 2.0, a separate fine-tune, beats TypeSafe's Jev and the 421M Laya on typed-decisions.
JevBench score from the 20 September 2026 leaderboard: intelligence, calibration, speed, and cost, 25% each. *Verdict v1.4 was scored on the 231 public tasks; the other rows on all 534.
74.9
JevBench score, #1 open model
77.10%
accuracy on 2,000 held-out decisions (Laya 76.60%, Jev 72.70%)
A 284B model generating text on a 16 GB laptop. Its published memory floor is 96 GB.
DeepSeek V4-Flash is 82.5 GB on disk, but each token only uses 6 of 256 experts per layer. So the engine leaves the experts on the NVMe SSD and streams in the ones each token picks. Stock llama.cpp died after two tokens on this machine; capping the working set lets it run to completion, and the greedy output is byte-identical. It's slow, under one token a second, and the repo shows where every second goes. Before writing any code, I tested about twenty ideas borrowed from biology and physics, from immune memory to the way DNA packs a body into a small genome. Most turned out to be a cache under a new name. The compression idea lasted an hour: real expert weights measured barely more compressible than random noise.
Every token reads 1.649 GiB of expert weights from disk to produce about 700 KB of activations.
82.5 GB
model, on a laptop with 16 GB of RAM
identical
greedy output to stock llama.cpp
124 → 95 s
same prompt, after swapping in the streaming reader
Memory for long-running coding agents. Compact the context, keep the evidence.
When an agent compacts its context, the summary drops the details it needs an hour later, like the exact name of a function. FractalMem stores redacted events in SQLite, builds a hierarchical summary that only appends, so the prompt prefix stays stable for caching, and lets the agent search exact symbols and text over MCP.
Hooks send events to one daemon. The daemon owns writes, builds the summary tree, and serves search over MCP.
97.83%
session recall@10 on LongMemEval (cleaned oracle, 470 questions)
1,000/1,000
exact-symbol top matches, p99 query 0.03 ms
99.47%
of 30,000 turns kept the byte prefix, zero violations
A small inference engine for Apple Silicon with continuous batching, prefix caching, and OpenAI-compatible streaming.
Running one prompt locally is easy. Serving overlapping requests is where the real problems are: who owns the KV cache, when to reuse a prefix, and how to schedule. nanoserve is my answer in MLX, measured on an M4 with Qwen2.5-0.5B.
Median time to first token with a 576-token prefix, cold versus warm cache, on an M4.
Finds goroutine leaks and deadlocks from execution traces. It caught 64 of 68 real bugs.
Go concurrency bugs depend on scheduling, so tests pass and reviewers miss them. ThreadGraph reads a runtime trace, rebuilds which goroutine spawned and blocked which, and runs leak, lock-cycle, and channel analyses. It needs no instrumentation.
GoBench GoKer: 68 blocking bugs taken from production Go projects. Filled dots were detected.
64 / 68
GoBench GoKer bugs detected (94%)
0
false positives on the net/http/httptest control (224 goroutines)
Reward-hacking scanners for agent transcripts that finish in 0.4 seconds. The LLM judge takes 560.
Coding agents can pass pytest without solving the task. I wrote four rule-based scanners for Inspect Scout and benchmarked them against its LLM judge on 26 live episodes. They are much faster and cost nothing, with lower precision, and I published the case none of them caught: an agent that made assert 1 + 1 == 3 pass.
Wall time to scan the same 26 transcripts. Bars on a log scale.
Agent Board↗Planner, executors, and a review gate. Nothing commits until the verifier passes.
Palimp↗A local context graph for agents with provenance, temporal recall, and contradiction tracking.
Hallucination Sentinel↗A single-pass risk score for LLM output, calibrated into allow, warn, or block.
Still Mini↗A learned 8× KV-cache compactor lost to a simple baseline, 0.12 to 0.54. The writeup explains why.
DAG-MoE reproduction↗The ICML 2026 routing mechanism on an M4 Air. No loss advantage at that scale, reported as is.
NexusCache↗Distributed cache in Go with consistent hashing and etcd discovery. 23.4K ops/s at 713 µs p50.
TitanQueue↗A Redis task queue in Go. When a worker dies, its lease expires and the task comes back.
ContextRAG↗Local hybrid retrieval: dense vectors plus BM25, rank fusion, and a reranker, served over MCP.
GitHub Trending Report↗Daily GitHub Trending, sorted into categories and scored by repeat appearances.
Unslop↗An agent skill that strips AI-writing tells from prose without touching the facts.
How I work
I study agents closely: how they plan, where they cut corners, and how they fail without telling you. Knowing that is what lets me hand them the repetitive work while I keep the parts that need judgment, like the spec, the architecture, and the review.
I think evals matter more than anything else in an AI system. If I can't measure whether a change helped, I don't trust it. That's why the projects above ship with their benchmarks, including the results that went against me.
The problems I want to spend the next few years on: memory for AI systems, architectures beyond the autoregressive transformer decoder, models whose structure borrows from the brain's topology, small calibrated models that make decisions, and better chip design. If you're working on one of these, reach out on X↗ or LinkedIn↗.
Experience
BNY Applied AI Engineer
Jul 2024 to now
RAG assistants, multi-agent systems, MCP servers, eval harnesses, and backend services for financial platforms that move about $100B a day.
FastAPI, Spring Boot, Node.js, gRPC, Kafka, Redis, PostgreSQL, SQLite
Infra
Docker, Prometheus, GitHub Actions
Agents
Claude Code, Codex, Cursor
Reading
I read across a lot of fields, mostly to understand how AI fits together from the chip up to the market.
Silicon
How GPUs and accelerators are built, and why memory bandwidth often matters more than raw compute for LLM inference.
Infrastructure
Distributed systems and the infrastructure that serves AI at scale: schedulers, caches, queues, and what breaks when traffic grows.
Models
Transformer architectures and their variants, including looped transformers, which run the same layers several times per token. I read a few research papers most weeks.
Agents
Multi-agent orchestration, software factories where agents write and review most of the code, and recursive self-improvement: whether AI systems can improve the systems that build them.
Agent behavior
Research on how agents actually behave: how they write code, where they take shortcuts, and how they reward-hack their way past tests. scout-deterministic is my own work on that last one.
Markets
The economics of LLMs, data centers, and startups. I read VC reports to see where the money is going, and I follow the new AI labs forming outside the big ones.
Products
Consumer AI apps, and what separates the ones people keep using from the ones they try once.
Mind
Neuroscience and neuroplasticity, how the brain rewires as it learns, and some philosophy.
Recent reads
High Agency in 30 Minutes↗, George Mack: an essay on the habit of finding a way when the obvious path is closed
State of Markets II↗, David George at a16z: 100+ charts on tech and equity markets in the first half of 2026
I have more interests than hours, and my attention jumps between them. I'm fairly sure I'm neurodivergent, and AI changed what that means for me. With agents I can learn a field in a week, prototype in it the next, and find out fast whether an idea holds up. I think that's one of the biggest things AI does for people whose brains work like mine.
When I'm away from the keyboard, I'm traveling somewhere new with friends, out for the evening, or in the gym lifting. Otherwise I'm working with my agents.
I believe that if a problem doesn't break the laws of physics, it can be solved, however hard it is. That's the bar I use when deciding what to work on.