John Allsopp · Web Directions AI Engineer Melbourne · 3–4 June 2026 87 sessions · 19 interviews
Close to ninety speakers over two days — none of whom compared notes — kept agreeing and disagreeing about the same few things. Put together, it reads like this: the frontier is still moving, but the action has moved around it. Models are priced like commodities and chosen like components; the durable work is the harness — context, routing, memory, evals — and the organisational muscle to use it. Teams treating model selection as engineering (with a budget line) are shipping. Teams treating it as brand loyalty are paying 10× for the privilege. The unsolved problems are memory and organisational readiness, not raw capability. (In the full report, every claim links to its talk — check my reading.)
The six themes
01The harness beats the modelThe base model is a commodity; the moat is routing, context, cascades — and knowing when a smaller model (or none) does the job. Read more →
02Context isn't memoryBigger windows don't fix forgetting; agents need memory architecture. Still unsolved. Read more →
03Evals are the new testsNon-deterministic systems don't ship on vibes — deterministic checks first, judges on a leash. Read more →
04Spec is the sourceWhen code is regenerable, the durable artifact is intent. "We've been backing up the wrong files." Read more →
05The economics brokeToken spend is an engineering constraint now — Token Town, factory thinking, the AI tax. Read more →
06The org is the bottleneckCapability is ahead of organisational maturity; governance and culture gate adoption. Read more →
In their words — theme 01
"The model is a commodity; the harness around it is the moat."
"Everything through Opus: US$30,000 a year. Using BERT: $212."
"Switching everything to the updated model is a really lazy way to be selecting models."
Next steps (from theme 01)
Engineers
Default to a mid-tier model, retry upward on failure — ~70% of value comes from mid-range (Sennett). Watch the latency cost on the retry path.
Team leaders
Make "why this model for this task?" a review question. Reference pattern: SLM by default, escalate the strange 2% to the LLM (Bhatt, CommBank).
Org leaders
Ask for per-task unit economics, not the vendor lineup. If nobody can produce a $30k-vs-$212 comparison for your workloads, that's the gap (Hall).
Key takeaway
Stop auto-upgrading every workload to each new frontier model. It's more expensive, slower, and usually unexamined (Sachs).
Three questions to discuss with your team
1 · Do we know the cost per task and failure rate of our biggest AI workload?
2 · Where in our process does “why this model for this task?” get asked?
3 · Are we closer to overbuilt orchestration, or unexamined defaults?