This post isn’t really about the speed at which AI is showing recursive self improvement, but about how humans seem to be succumbing to “cognitive surrender”, a term coming out of research from Wharton. This paper is an excellent read for everyone who loved reading “Thinking Fast and Slow” by Daniel Kahneman. In my own life, I have found these 2-system mechanisms of explaining human cognition to be somewhat limiting, but very useful in a leadership context. Often when trying to explain a prediction that I might make intuitively (System 1), I reach for my System 2 analytical methods to reason through so I could explain it to myself and to others. Sometimes this is hard when there are decisions to be made quickly. Other times, you could take your time with these decisions, and take the System 2 path entirely.

Well - the paper and research introduces a new system, System 3 (Artificial), which is an AI system that sits “in silico” and helps you with decisions. When there’s a conflict between System 1 & System 2, usually people review their actions based on a detailed analysis from System 2 and come back and adjust their intuitions. The flow is illustrated by Shaw and Nave (authors of the paper).

The key new term is “Cognitive surrender”, which is entirely a new phenomenon studied by the authors and the findings are astonishing. The participants, who didn’t know ahead of time whether it was going to be accurate or not, relegated their decisions to AI 93% of the time when it was deemed accurate and 80% when it was deemed faulty. The more interesting find is that System 3 facilitated cognitive surrender.

And when compounded by time pressure, AI accuracy benefited human cognition and decision making owing to the accuracy vs. speed benefits of AI.

What is the take away from all this? In my own experience, I’ve followed the principles of “Co-intelligence” as suggested by Ethan Mollick of: inviting AI to the table, being the human in the loop, treating AI like a person and assuming it’s at the worst point in its life cycle, I have been skeptical of the results from AI and worked to ensure accuracy. However, Agentic AI is changing this entire approach from Co-intelligence to “Co-existence”, where increasing amounts of decision making is being delegated to the AI vs. being a human in the loop.

Knowing all of the above, Agentic Coding is a place where cognitive surrender I believe has already happened very quickly, as the agent can verify the actions that it takes through a feedback loop. And when the AI is accurate, we have less of a need to go back and look at it. As long as it “just works”, we worry less about the internals of a device. It’s easier to surrender when you don’t really have enough expertise in an area and you’re looking for answers quickly.

The agentic AI thesis is that whole companies can be put out of business quickly. Let us say that I believe there’s a market need for a search engine that doesn’t use AI, however small that market may be. To expand this market hypothesis, in a recent experiment, I asked AI to create a simple search engine using PageRank and Bloom Filters. The goal of this search engine would be to create a very simple interface without any AI assisted search. While this is just me kicking around the tires to see if “goal oriented” AI recursion can do the job for me entirely. I’ve used a lot of search engines over the years, and worked with enterprise search engines such as Elasticsearch and Solr, and while I’m not a domain expert on scaling search engines, I wondered how far I could go. It was done with Antigravity & Gemini 3.5 Flash with some clear guidelines for the AI around engineering practices. I’m sure Claude or ChatGPT will claim that they could do a better job, but models and code agents are supposed to be commodities in the long run, so you can’t blame the consumer yet.

I accepted the implementation entirely. It was written in Go, and knowing that building the search index could take time, concurrency was a key focus. The AI knew and suggested how to build it using BoltDB, a fast key-value store implementation, inverted search index, and a bloom filter with mutex locks to ensure safe database operations. I tried the search engine out, and I saw some interesting base results. Why there’s a UFO reference in a note from 2001 on XML signatures by W3.org (a standards body for the world wide web) begs some questions, but I digress.

Then I dove into the code a little bit, and even though I again knew not enough about the details of a search engine, I found something interesting. To understand this further, a Bloom filter ideally uses multiple hash functions, but with the FNV hash, you can avoid having to do k hash functions that increase time complexity. One of the parameters of the FNV hash is a prime number. And when I looked at the code for this implementation, I saw this constant called

Anyone with elementary school math knows that’s not a prime number, so I asked the AI, “why is this a composite number?” and out came this answer.

The point of the above anecdote, in conjunction with cognitive surrender, is hopefully clear. Relegated decisions (cognitive surrender) such as the above (granted I’m not ready to give Google a run for their money yet) could have risks buried deep in the codebase. I’m sure a more seasoned domain expert in search probably could have more substantial findings. Spotting these could be another AI reviewer’s job, insofar as domain expertise has been extracted and embedded into an AI. With larger and larger LLMs, the hope is that it will be, but what about all the proprietary knowledge inside of orgs around domains? How much surrender has already happened inside of frontier labs like Open AI and Anthropic? No one outside could know concretely, but based on their articles, the indication significant percentage, especially if any company’s output raises by 8X in terms of software. Cognitive surrender is here, and we all have to prepare by asking ourselves questions about where we should, and where we should not. The answers to those questions will help guide people toward their purpose, and avoid building a false sense of confidence based on a surrendered self.

Over the past few updates of Claude / Open AI and other frontier models, memory has been a key upgrade, including inside of code agents. Common hacks earlier included explicit instruction to your agent to capture anything about your style and need back into an AGENTS or CLAUDE file.

To all VLA builders and researchers, here’s something that I hope all of these models have - a sense of time. Even if I have memory, if I don’t know that my toast needs to be done before breakfast time, there’s a breakdown. If there aren’t any deadlines for my todos in long horizon tasks, I could keep doing them assuming people will wait for me, when they likely won’t. Every LLM, VLM, VLA type frontier model must have access to a clock on the side of the user it is serving. [As an aside, here’s a post on how analog clock reading hasn’t been great across frontier models]

A clock is within arm’s reach of models. Just ask for the time and you get a nice answer:

Just the way I can ask for time, any good agentic harness should provide access to a clock, along with a running time that the agent manages within tasks. LLMs do not have a sense of time, and in that, they are somewhat eternal. The power of an LLM is that eternal ability to be present and in the moment for you. However, while that magic is good for creative pursuits that define co-intelligence, it’s not great for autonomous agentic tasking. There’s a perfectly capable CPU clock available on the AI side of this interaction. Humans haven’t been able to tell time without clocks or bells for a long time now. We don’t make stuff up (at least not about time), but use tools, and that should be the standard for LLMs as well.

Yet, I don’t quite see evidence of this from frontier models that they use clocks well in longer horizon tasks with multi-step workflows. Take for example, asking an LLM to prepare me for a talk that’s happening 6 months from now on VLA. There should be a pause on this from the model because it doesn’t quite make sense. Who’s the audience? How much are they willing to learn about something from 6 months ago? How much more work is going to be done in the next 6 months in VLA? The questions pile up without good answers. But give this prompt to any LLM -

You’re an expert in Frontier models, VLMs and VLAs. Prepare me for a talk that’s happening 6 months from now on VLA and VLMs. Help me put together a story. I need a presentation that’s current and exciting for the audience with the latest reports. 

and it starts to faithfully execute without any questions.

I could put a stopper on it, such as “Ask me enough questions until we’ve reached a shared understanding”.

The list above keeps going (a total of 20). The fact that VLA is evolving rapidly over the past year and will continue to evolve a lot more in the next 6 months is not a factor that goes into the LLM agent decision initially, until prodded. I love the eagerness of Gen AI to solve problems, but until the user and their AI are aligned, zero-shot results are going to be somewhat sub-par, and will require further iterations.

Why might the above be? LLM pre-training without associated time history of where things are located in time, not just in the vector space of tokens and words makes you feel like it’s all happening at the same time. Likely a lot of this gets addressed in post-training and RLHF, but not entirely. For the next generation of LLMs (not just VLAs), the ability to use the concept of time more natively should be key. Loading more data and praying for emergent behavior is the same as using hope as a strategy.

There is some research in this area, and but seems pretty early stage. The most quoted paper seems to have examples such as the following:

There’s another paper here - which has a couple of other benchmarks, but there’s a lot more to think about and if there’s anyone out there doing a lot of work on this, please forgive my ignorance, but these benchmarks don’t seem to go below the surface, and LLMs still don’t seem to be doing great in this area. Here are some other references to papers in this space.

• Paper 1

• Paper 2

• Paper 3

A lot of them focus on math, reasoning and historical facts. Helpful, but how do I inject a sense of momentum into my model (not the momentum optimization inside a deep neural net, but an economic and market definition of how things build up over time).

If you think that’s not enough of an example, here’s another example of temporal reasoning failure. I’m in Boston, commuting to the suburbs. There are advisories around any delays available publicly on the local transit site (mbta.com). And here’s my convo with Google Gemini. The local time is 6:05 pm. I’ve cut off some preambles.

Gemini : the 320 pm train is delayed by 5-15 minutes and the next train is also likely delayed. Were you trying to take those?
Me: What timezone are you in?
Gemini: based on my location I’m in the Eastern Daylight Time zone.
Me: what time you got?
Gemini: 6:07 pm (mind you by this time a couple minutes passed)
Me: why are you telling me about trains from 320 pm when the current time is 6:07 pm?
Gemini: you are correct to point that out. That’s my mistake. Based on my current info - the 430 pm is delayed and the next train could be delayed as well. Are you planning to take those?
—- I repeat the time question again and get the same answer —
Me: Filter out any train timings prior to the current time. (Tool use hint)
Gemini: yes. Let me do that. Based on my current info - the 430 pm train is delayed and the next train is delayed as well. Are you planning to take those?
Me: please report to Google HQ that you didn’t do a great job on this one.

This is another one from Meta AI - where the time is just there on the message for the taking - but the AI hasn’t been fed that information by default.

My conclusions based on what I have experienced so far and read, would be the following

1. There is a significant opportunity to incorporate a user clock inside agentic LLM workflows, including a deadline, calendar and other artifacts. I understand nobody wants to be pressured, and time factor on LLMs is not really a thing because the inference speed is sufficient, but deadlines are always important for humans in the context of planning. We are still not quite there in terms of accepting zero-shotted code directly into production, and at least a question or two to understand deadlines is in order in the system prompt of these agents.

2. There is a need to evaluate and benchmark not only temporal reasoning, but temporal agentic planning and action. Does temporal data not need to be part of the LLM at all? I find that very odd, especially because if you are human, you need that for sure, and if the LLM / VLM training is not incorporating that as a path to AGI, I’d be worried.

3. Memory is a helpful factor here especially as you remember to go back and tick off things that you planned to complete during a long horizon task, or remember the user’s preferences for all of their tasks.

I have more ideas on this front, and will explore more in a future post. But I would like to end this with a quote from my other favorite song “Time” by Pink Floyd

The time is gone, the song is over
Thought I’d something more to say

Building software is a domain driven exercise, apart from the core algorithms that were invented and continue to get invented over time. Here’s a potential framework that can help with VibeSlop trickle based on generated code classification. We can broadly classify such AI generated code into

• The Muscle Memory Snippet: Code generated to help with testing, organizing and viewing existing artifacts as well as creating them. This is baffling at first but there’s so much code in tool use while an agent is running that it contributes a significant amount. Example below of a session.

  

  Proposal: There maybe opportunities to cache and reuse snippets, but prompt caching is likely a strong enough path. I call it muscle memory because this is something that engineers build into their workflow and some of them get really, really good at it.

• The Bespoke App Factory: Planning a trip? A new travel itinerary visual can help. Why communicate with docs when you can with Apps? And I don’t mean that in any sarcastic way, but when Apps are cheap, you can generate them on-demand for the right impact. Just be careful to not overdo this.

  Proposal: Given the token count spent and the human effort to get these artifacts generated (pretty low end), you may think about throwing them away after they’re useful life expires. The really good ones, like a flip book of pictures from a family trip or some well researched notes, maybe AI fodder for long term memory.

• The Consultant Prototype: An initial full prototype that’s been vibe coded using ad hoc prompts and some skills. An agent harness may or may not be used, and the user is iterating on an idea. There are a lot more of these now than 1 year ago, and growing exponentially.

  Proposal: Since the above is largely demo and illustration material, it could be used by code agents as a guide for building the next revision. If a prototype is really high quality (and I haven’t seen this happen personally), it is something that can become a reference implementation. VibeSlop creep is very high here, so be prepared to throw this out, no matter how much time might be invested in this. There could be value in extracting the conversation bits for developing new product requirements.

• The Agentic New Product: A strong initial product from scratch, which has been specified by using “grill-me” skills, specs, full fidelity UX wireframes in Figma, PRD, TRD, architecture guidelines and so on. Tasks specified by a user down to quite a bit of detail.

  Proposal: This is the future of software engineering as I see it. A lot of reusable skills, process guidelines, security requirements and the organizational knowledge gets extracted and placed in the hands of the machine and needs to be referenced all the time. While Vibeslop can be reduced by providing much clearer guidelines, beware the tech stack and the domain where this is applied. A lot more training data maybe needed to ensure success if there’s a completely new paradigm being figured out. As a long running process, context pollution, rot and confusion can take over here unless you break the work into atomic units. All agile software principles apply here along with expertise on how bigger chunks of work get broken down into smaller ones.

• The Agentic Maintainer: Existing production app where updates are made incrementally as features, bugs, hotfixes and so on.

  Production Updates for features should ideally follow the Agentic Product Dev experience above. For bug fixes and hotfixes, you need some serious human in the loop especially around the domain that your software operates in. But what happens when the original code was written by the machine? Could it be VibeSlop? Is VibeSlop unique to machines? I personally don’t believe so. The lessons from human software builds over decades apply here. Ongoing code audits, scans and levels of vigilance that you use to manage growth phase companies are significantly more valuable when evaluating machine coded software. There will be a lot of companies that emerge in this area, but the incumbents who do these scans and the principles behind them stay. The great news about agentic AI coded apps is if you had the foresight, you could have documented every decision to the last bit of detail. But even lacking that, LLMs and code agents are great for understanding any code base and explaining that to you in whatever form that suits you (diagrams, live apps, full on profiling and dependency tool builds). I strongly recommend that PR review comments add the right organizational context for such bug and hot fixes.

The great news is that newer variants of Claude Code, GitHub Copilot, Codex, Antigravity, Pi and others provide quite a bit of the above principles in the harness (planning, tasking, implementation against tasking, TDD and other such goal setting and achieving). But also, using agents for a first pass on the code reviews is a great way to help scale up some of the work. Also when you have access to multiple models, use one model to review another model’s work.

2025 was the first year of agentic AI in the mainstream. We have to assume that by that definition, these code agents today are the worst in their jobs, and they will improve exponentially. World models could be a new way to assess consequences prior to diving into implementation, but also needs significant domain knowledge transfer to machines. So while VibeSlop is a thing out there, applying the framework above for decisions and maintaining a strong set of organizational hierarchical rules for code agents (CLAUDE.md or AGENTS.md) will help cut it down in the longer term.