Is it ever justified to give an LLM your agency?
Last weekend, I did a hackathon at Y-Combinator in San Francisco. It was my first hackathon as a hacker since last Jan 2025. While I have generally moved past doing hackathons, I am appreciating the value from doing one annually. It keeps me warm to the way things can get built quickly, and gives a pulse-read on what kind of things are being hacked on.
While LLMs and AI-dev tools have really supercharged what’s possible at hackathons, it has and can really cripple human agency in many ways. This hackathon was only 11am - 5pm on a Saturday, and we started coding (or prompting) at around 1pm. Especially in time tight situations, the tendency to be lazy via taking short-cuts rises.
I had a mental boom moment – pondering:
- What LLMs have done to us?
- Now, what’s the purpose of having a team in hackathon?
- Whether my teammates trusted LLMs more than me.
- What parts of the old paradigm should be kept?
a now different paradigm in hackathons
who needs diagrams?
I have found diagrams extremely useful in hackathons. User flow, data flow, and high-level architecture diagrams all help align the team non-technically and technically. Perhaps this was a pre-vibe/LLM era learning.
As soon as we had decided on an idea, I started making a user flow diagram. In post-hack feedback, my teammate told me he had a mental boom watching me do that. Apparently, nowadays – all that’s needed to align a team is a shared PRD file.
Having a shared PRD file makes sense to me, but forgoing any diagram of any kind felt off. Are we prioritising the alignment of our different LLMs over our own mental model alignment?
who needs you?
Here’s the story. I reached out to someone to form a team, and a pair wanted to join us. I had proposed an idea and we were committed to it directionally. However, after getting some external feedback and realising the difficulty in a clear MVP for what we attempting – my intuition was shouting to me that we needed a narrowing of scope and to prioritise a couple of core features.
My teammates were pretty neutral in simply trusting my gut. However, the one thing I requested when the pair joined – was that I wouldn’t have to explain every opinion I had, that I was willing to elaborate and discuss alternatives, but only to a certain point, and if unconvinced – you would just have to trust to go with my decision. I requested this beforehand to manage the expectation, as I predicted a smaller time horizon would lead to having less time to team storm everything.
Another teammate was unconvinced and still wanted to demo two separate MVPs. He advised me to seek more feedback from a judge/sponsor. I relented, and did so – bringing one over, the judge gave vague feedback but hinted against demo’ing two MVPs. It became clearer to me we needed to reduce our feature set and focus on polishing one thing.
With the judges’ hints, I reasoned and presented my intuition again.
The next moment, my teammate who asked me to go seek more feedback – started describing our situation to Claude Code and asking what we should do.
That gave me a mental boom. I was stunned. I couldn’t believe that in a moment of differing opinions, he left it to an LLM to tiebreak. Why are we doing that? We each have done and won multiple hackathons – why are we trusting an LLM over our own experience and intuition?
I told him,
“I trust your intuition and opinion more than this LLM. And I expect you to trust mine too. Like that’s you wanted to team up with me right? Why are we trusting something with questionable intelligence? Something that encourages and responds in a positive leaning way, and even in an anyhow way depending on how you prompt it.
This bot doesn’t even have the full picture. It operates on imperfect context – it hasn’t heard everything we discussed, it hasn’t spoken to mentors/judges, and it doesn’t know our users. You do. I do.
Why are we even here if we’re going to ask the LLM something so core and fundamental? What value does having teammates even have then? Why not do this solo? Heck, why not just let the LLM do the entire hackathon for you?
I was so confounded by what I had witnessed. He had given his own, and the team’s agency over to a LLM.
my own recent hypocrisy
Yet, I realised my own recent hypocrisy. I have a colleague at work who is a better engineer than me and I have learnt a lot from.
He had reacted in a similar way when I had pasted some error logs into an LLM rather than try to understand the error first. At the time, on some level – I could understand, but I didn’t full accept it on another.
Primarily because I would believe an AI-dev tool LLM would know more about Chrome MediaPlayer protocols than I do in my brain at baseline. And when we faced a bug, he had pulled me into a 2 hour debugging session with him (thank you for taking the time to do that exercise with me) – before he finally relented and let me try using AI to debug, which it did in a couple minutes with a fix.
Here’s my current stance: there’s no ‘right’ answer. It’s all a trade-off. There is no ‘free’ win.
The tradeoff in spending human engineering time to develop a mental model vs handing off to an AI agent. I find it incredibly helpful to consider the longevity of the code and the blast radius of changes/interfaces in determining when to do which. Even if you review an AI’s work, you might consume less human time/energy, but instead develop a much weaker mental model that future work cannot reliably leverage from.
So on one-hand, at current levels of AI capabilities, I’m still disciplined in how I use AI in professional software engineering – but know when to just hand-off to it. On the other, I cannot accept giving agency to an LLM to let it decide the product or technical direction.
It’s literally your job to make decisions and own them.
There’s a fine line between using AI as a tool, but you the human must hold the wheel.
two analogies
Let me give you the two analogies I gave my team
-
Imagine you and your sibling are disagreeing on what to make for dinner, then your sibling turns around and ask Claude Code what it thinks they should eat.
-
Or imagine you went on a first date with someone you like, when you asked them if they would like to go on a second date – they asked Claude Code if they should.
I weep for humanity.
All Claude Code users are born good, but become evil.