Projects
Small AI projects, each with its code on GitHub.
- the-data-cant-tell ↗ Do language models invent causes when the data can't tell them why a number moved? Nine "why did it drop?" questions about an invented rental company, where sometimes the answer is in the data and sometimes the only honest answer is "the data doesn't say". Six models from Mistral and Anthropic, up to six ways of instructing them, about 320 answers graded on four questions, every prediction written before the run. Without a system message, Mistral Large invented a cause in 8 answers out of 10; one sentence took that to 1. With the figures computed by code and handed to the model, Mistral Large 4 without reasoning reached 9 fully right answers out of 10, level with the best set-up, at a third of the cost and 13 times faster.
- evals-kit ↗ A small evaluation kit for LLM applications, written by hand. Runs an agent against a golden set and prints one line per type of request. Graders check for expected words, a number, or an abstention. Crashed calls and wrong answers get separate retry budgets, so a timeout doesn't count as a failure, and the first-try rate is reported before the final one. First slice: run on a scripted assistant so far.
- toy-jepa ↗ A JEPA trained from scratch, and four ways to keep it from collapsing. A Joint Embedding Predictive Architecture that learns from frames of bouncing shapes by predicting later frames in embedding space. Four variants against collapse (stop-gradient alone, EMA, EMA with a variance term, EMA with normalization), compared with a linear probe. EMA with the variance term gave the best embeddings: 1.83 px of probe error, against 5.62 px for the mean baseline. The collapsed runs reached a loss of zero.