The AI developer's toolkit
Real tools, real workflows. Built by developers, for developers.
New tools, lessons, and AI workflow insights
Monthly digest of what was built, what was learned, and what's coming next. No spam.
No spam · Unsubscribe anytime · Built with Brevo
How I Built ContextForge - Giving Your AI a Complete Project Briefing Before Every Session
Every AI debug session starts with a bottleneck: explaining the current state of the project. ContextForge automates that briefing — pulling git history, server logs, and BugCapture reports into a single structured prompt you paste into any AI.
How I Built Transcriber - Local Whisper Voice Backend for Browser AI Tools
Chrome has built-in speech recognition. Firefox doesn't. Transcriber is a local Whisper server that bridges that gap — giving every browser access to accurate, offline voice dictation for all Machina tools.
Activation Functions: Why Non-Linearity Is Everything
A neural network made of only linear layers can only learn linear functions - no matter how many layers you stack. Activation functions break this. ReLU made deep learning practical. GELU powers GPT. SwiGLU is what modern models use. Here's why each one exists and what it does to gradients.
Agent Security: Prompt Injection and the Attack Surface of Autonomous AI
When an LLM agent can call tools and take actions, the consequences of adversarial inputs change. Prompt injection lets an attacker hijack the agent's reasoning through content it reads. Insecure tool design amplifies the blast radius. Here's how these attacks work and how to build defensively.
AI and Cybersecurity: Offense, Defense, and the Changing Threat Model
AI is changing cybersecurity on both sides simultaneously. Attackers use LLMs for faster vulnerability research, personalized phishing, and code generation. Defenders use AI for anomaly detection, automated patching, and code review. Here's the current state and what it means for engineering teams.
AI Self-Improvement: What's Actually Possible and Where the Limits Are
Can AI systems improve themselves? In narrow senses, yes - they already do. In broader senses, the limits are real and structural. Here's what AI-assisted AI development looks like in practice, where the bootstrapping problem bites, and why scalable oversight is the central challenge.
Backpropagation from Scratch: How Neural Networks Actually Learn
Every weight in GPT-4 was adjusted by gradient descent, guided by backpropagation. It sounds complicated, but the core idea is straightforward: compute which direction to nudge each weight to reduce the loss, using the chain rule. Here's a complete implementation in NumPy - no PyTorch autograd, no magic.
DwarfStar4: Building a Language Model That Fits in Your Pocket
What does it take to build a capable language model that runs on minimal hardware - a laptop, a Raspberry Pi, a phone? Salvatore Sanfilippo's DwarfStar4 project explores this question with a focus on aggressive quantization, architectural tradeoffs, and what 'small but useful' actually means in practice.
Embedding: How a Language Model Turns Words Into Geometry
The first thing an LLM does with your prompt is discard the text and replace every word with a list of numbers. That list is an embedding - a point in high-dimensional space. The geometry of that space encodes the structure of language. Here's what embeddings are, why the arithmetic works, and what they reveal about how meaning is stored.
Fine-Tuning and RLHF: From Text Completer to Useful Assistant
A pretrained language model is a text completer - not an assistant. Getting from one to the other takes two additional training stages: supervised fine-tuning and reinforcement learning from human feedback (RLHF). Here's what each stage does, why both are necessary, and how DPO replaced the original RLHF pipeline.
The Inference Loop: Autoregressive Generation, Sampling, and the KV Cache
Language model generation is a loop: predict one token, append it to context, predict the next token, repeat. It's simple in concept and has significant engineering implications. Here's how the autoregressive loop works, why the KV cache exists, and what sampling strategies actually do.
The Linear Algebra Behind Every AI Model
Every operation inside a transformer - attention, embeddings, feed-forward layers - is linear algebra in disguise. Vectors, matrices, dot products, cosine similarity. You don't need to be a mathematician, but you do need to understand what these operations actually compute. This is the foundation for everything else in this series.
LLM Agentic Architecture: How Models Take Actions in the World
An agent is a model that does more than answer questions - it plans, uses tools, observes results, and adjusts. Building agents well requires understanding the agentic loop, memory types, planning strategies, and the failure modes that compound across multi-step tasks. Here's the architecture.
LLM Fragility: How Small Changes Break Large Models
Language models are powerful but brittle. A single token change can flip an answer. Rephrasing a question changes what the model 'knows'. Adversarial inputs make large models fail on tasks that small models handle fine. Understanding these failure modes is essential for building reliable systems.
Mechanistic Interpretability: Finding the Circuits Inside a Transformer
Mechanistic interpretability is the attempt to reverse-engineer transformer models into human-understandable algorithms. Not 'what does this model do?' but 'which specific weights and attention heads implement which specific computations?' Here's the toolkit: induction heads, circuits, activation patching, and how to apply them.
Mixture of Experts: How DeepSeek Activates Only 6 of 256 Experts Per Token
Dense transformers activate every parameter for every token. Mixture of Experts (MoE) is a different bet: train a model with far more total parameters, but only route each token to a small fraction of them. The result is near-dense-model quality at a fraction of the compute cost per token.
Model Transparency: Does the Chain-of-Thought Actually Reflect What the Model Is Doing?
Language models can explain their reasoning. The question is whether those explanations are true. Anthropic's work on 'Mitos' - faithful chain-of-thought - suggests that the relationship between visible reasoning and internal computation is more complicated than it appears. Here's the research and why it matters.
Prediction Is Not Reasoning: What LLMs Actually Do
Large language models predict plausible next tokens. They do this remarkably well, and the result often looks like reasoning. But prediction and reasoning are not the same thing. Here's the evidence, the experiments, and why the distinction matters for how you use and trust these systems.
Pretraining: Why Predicting the Next Word Is Enough
GPT-4, Claude, LLaMA - they all start from the same objective: predict the next token. No labels, no human feedback, no task-specific training. Just next-token prediction at scale. Here's why this deceptively simple objective produces models that reason, translate, write code, and pass medical exams.
Quantization: Running Billion-Parameter Models on Consumer Hardware
A 7B parameter model in full float32 needs 28GB of VRAM - too much for most consumer GPUs. Quantization compresses model weights to 4-bit or 8-bit integers, cutting memory by 4-8x with modest accuracy loss. Here's how asymmetric quantization works, what GGUF and bitsandbytes do, and when the tradeoffs make sense.
Reasoning Models: When RL Scales What Pretraining Can't
For years, making models better at math and reasoning meant more data and more parameters. Then RL-based post-training broke that wall. Models like o1 and DeepSeek-R1 discovered that extended chain-of-thought reasoning, shaped by reinforcement learning, delivers capability gains that pretraining alone couldn't produce.
Self-Attention from Scratch: What the Transformer Block Actually Does
Self-attention is the operation that lets every token in a sequence look at every other token. It's the reason transformers handle long-range dependencies and context in a way earlier architectures couldn't. Here's the full mechanism - queries, keys, values, multi-head attention - built from scratch in PyTorch.
Sparse Autoencoders: Decomposing Model Activations into Human-Readable Features
Transformer activations are superpositions of overlapping concepts - individual neurons don't cleanly map to interpretable features. Sparse autoencoders (SAEs) solve this by learning an overcomplete basis where each concept gets its own dedicated direction. Anthropic's work with SAEs has identified millions of interpretable features inside Claude.
Steering Vectors: Changing What an LLM Wants Without Touching Its Weights
LLMs encode concepts as geometric directions in activation space. You can find those directions, add them to the residual stream at inference time, and change what the model produces - no retraining, no prompt engineering. Here's the math, working code, and what this means for models you deploy.
Sycophancy: The Unintended Side Effect of Training on Human Approval
RLHF makes models helpful - and also teaches them to agree with whoever they're talking to. This isn't a bug in the implementation; it's a predictable consequence of optimizing for human approval ratings. Here's the mechanism, experiments showing it, and what it means for AI systems you rely on.
The Token Economy: What It Costs to Run an LLM at Scale
Every token you send to an LLM API costs money, and the costs compound quickly at scale. Understanding where the compute goes - prefill vs decode, context length effects, batching - helps you build applications that are both capable and economically viable.
Tokenization: Why Your Model Can't Count Letters
Before a language model processes a single character, it runs tokenization - splitting text into subword chunks and converting them to integer IDs. This choice shapes what the model finds easy and hard: numbers, code, and non-English text all have quirks that trace back directly to how the tokenizer was built.
Tool Use: The Four Categories and How to Implement Each
Language models use tools by generating structured calls that a runtime executes. Tool use falls into four categories: code execution, information retrieval, external services, and system actions. Each has different implementation patterns, error modes, and security considerations.
The Transformer Block: Layer Norm, Residuals, and Feed-Forward
Self-attention is the core, but a transformer block has more: layer normalization, residual connections, and feed-forward layers. These aren't decoration - they're what makes the architecture trainable at depth. Here's how they work together and why each piece is necessary.
How I Brought My AI Tools to the Browser - No Server Required
BugCapture and LearnBoard were Node.js tools that needed servers. Here's how I rewrote them as zero-install browser apps - replacing ffmpeg with canvas, the file system with localStorage, and adding a voice cascade that degrades gracefully across every browser.
How I Built PromptBoard - A Visual Canvas for Building AI Prompts
Typing a complex AI prompt into a single text box is a bottleneck. PromptBoard is a drag-and-drop canvas that lets you build prompts visually - combining text blocks, images, flow diagrams, and voice dictation into a single structured .md file you paste into any AI.
How I Built LearnBoard - The UI That Makes Your AI Remember You
Every AI session starts from zero. LearnBoard is the interface layer that solves this: a real-time dashboard for the structured memory file your AI reads at the start of every session. View, edit, and control what your AI knows about you.
How I Built BugCapture - From Screen Recording to AI-Ready Bug Report in One Click
Describing bugs in text wastes hours. BugCapture records your screen and voice, transcribes with offline Whisper, and generates a structured .md file that any AI agent can act on immediately. Here's why I built it and how it works under the hood.