Root18D / Field notes
Thoughts in progress
Notes on Azure Commerce, agentic systems, operating at cloud scale, and the lessons that emerge between architecture and execution.
Agentic AI · Software Engineering · Research Perspective
A research perspective on agent-generated software, the shift from creation to verification, and an execution architecture for the next two years.
Agentic AI · Agent harnesses · Production systems
A systems field guide to the runtime, tool contracts, permissions, budgets, approvals, traces, and evaluation loops that turn a capable model into a governable agent.
Agentic AI · Model training · Production systems
A field guide to Transformer mechanics, LoRA and QLoRA adaptation, preference optimization, evaluation gates, and the operating system required to make a model useful in production.
Research Watch · Privacy · Governance
DP-SGD bounds how much any single training example can influence a model's parameters, replacing a policy promise about privacy with a formal, quantified guarantee — at a measurable cost in model accuracy.
Research Watch · Safety · Evaluation
DeepMind's automated red-teaming research uses a language model to generate adversarial test cases at a scale manual security review cannot match, surfacing a larger and more diverse set of failures in the same amount of time.
Research Watch · Governance · Provenance
Kirchenbauer and colleagues' watermarking scheme biases a model's token choices in a way that is statistically detectable without a model-owner API call, at a cost in robustness under paraphrasing that any deployment has to plan around.
Research Watch · Governance · Privacy
Microsoft Research's "Who's Harry Potter?" paper demonstrates approximate unlearning — suppressing a model's ability to reproduce specific content in about an hour of fine-tuning, versus the hundred-thousand-plus GPU-hours it took to pretrain.
Research Watch · Interpretability · Safety
Anthropic's sparse autoencoder research decomposes a model's tangled internal activations into individually interpretable, causally verified features — a path toward inspecting what a model actually represents.
Research Watch · Retrieval · Evaluation
Research on how language models use long contexts found a consistent U-shaped accuracy curve — strong at the start and end of a context, and measurably worse in the middle, even for models built for long context.
Research Watch · Reliability · Systems
Grammar-guided generation compiles a schema into a token-level index so an output is guaranteed structurally valid by construction — a stronger guarantee than asking a model nicely to follow a format.
Research Watch · Retrieval · Knowledge systems
Microsoft Research's GraphRAG pre-builds a knowledge graph and community summaries over a corpus, so a broad, corpus-wide question can be answered without depending on a single lucky retrieval hit.
Research Watch · Retrieval · Evaluation
RAGAS introduces reference-free metrics that separate retrieval quality from generation quality, so a team can tell whether a RAG failure came from search or from the model — without hand-labeled ground truth.
Research Watch · Reasoning · Agentic AI
Research on multiagent debate shows that having independent instances of the same model argue over an answer measurably improves factuality and reasoning — at a direct, linear cost in inference calls.
Research Watch · Reasoning · Agentic AI
A simple inference-time loop — generate, self-critique, revise — measurably improves output quality across seven tasks and multiple model families, without any additional training.
Research Watch · Alignment · Safety
Anthropic's Constitutional AI research replaces large volumes of human harm-labeling with a written set of principles the model uses to critique, revise, and eventually judge its own outputs.
Research Watch · Reasoning · Inference
Research on scaling test-time compute shows a fixed model can match or exceed a much larger one by spending more inference-time computation — but only if that compute is allocated by prompt difficulty, not spent uniformly.
Research Watch · Reasoning · Evaluation
OpenAI's process-supervision research shows that rewarding every correct reasoning step, not only the final answer, catches errors outcome supervision misses and produces a more reliable verifier.
Research Watch · Model architecture · Multimodal
CLIP and LLaVA describe the two layers most production vision-language systems are built from — a shared image-text embedding space, and a thin projection that lets a language model attend over visual features as if they were tokens.
Research Watch · Model architecture · Context
Position Interpolation, YaRN, and Ring Attention extend how much context a Transformer can use through three genuinely different mechanisms, each with a different binding constraint.
Research Watch · Data strategy · Alignment
Self-Instruct and Self-Rewarding Language Models turn a small human seed set into a large training signal — one by generating instruction data, the other by generating its own preference labels.
Research Watch · Model architecture · Efficiency
A decade-old idea — training a small student to match a large teacher's full output distribution — is how many production-grade small models are actually built today, and curated data can matter as much as scale.
Research Watch · Model architecture · Adaptation
Task Arithmetic and TIES-Merging treat fine-tuned weights as directions in weight space that can be added, negated, and combined directly — and show what breaks when teams merge naively.
Research Watch · Model architecture · Efficiency
How Switch Transformer and Mixtral decouple total model capacity from per-token compute through learned routing, and what that trade means for memory, quality, and production risk.
Azure · Microsoft Foundry · Work IQ · Agent architecture
An animated reference architecture that connects workplace context to Azure evidence, with delegated identity, typed tools, deterministic decisions, and verified outcomes.
AI infrastructure · Distributed inference · Production systems
From a lightboard explanation to a deployable serving design: memory arithmetic, parallelism, KV-cache handoffs, vLLM configuration, admission control, and measurable goodput.
AI infrastructure · Model serving · Production systems
A field guide to prefill and decode, continuous batching, KV-cache pressure, quantization, speculative decoding, admission control, observability, and cost per verified outcome.
Research Watch · AI infrastructure · Privacy
Trusted execution environments keep model computation isolated even from the cloud operator running the hardware — a foundational security research idea that modern GPU-level confidential computing has since extended to full production inference.
Research Watch · AI infrastructure · Cost
SCALM and related semantic-caching research match requests by meaning instead of exact text, cutting serving cost on repeated-intent traffic — with a similarity threshold doing all the safety work.
Commerce Billing · AWS · Agentic Systems
A companion to the 25-layer billing model: versioned entity releases, agent-assisted launch workflows, and deterministic controls from usage to settlement.
Commerce Billing · Azure · FinOps