As someone who uses AI coding assistants daily, I was struck by Anthropic’s recent research on how AI assistance impacts skill development. The findings challenge the assumption that AI makes us better programmers.
The Study at a Glance
Anthropic recruited 52 junior software engineers with Python experience and tasked them with learning Trio, an asynchronous programming library. Half used AI assistance; half coded by hand. Both groups knew a quiz would follow.
The results were stark: the AI group scored 17% lower on comprehension—averaging 50% versus 67% for hand-coders. That’s nearly a two-letter-grade difference.
Where AI Users Struggled Most
The largest performance gap appeared in debugging questions. Participants who used AI to write their code couldn’t identify what was wrong when things broke.
This makes intuitive sense. When you struggle through an error yourself, you build mental models of why things fail. When AI handles errors for you, that learning opportunity evaporates.
Not All AI Usage is Equal
The most interesting finding wasn’t the aggregate difference—it was the variation within the AI group. How people used AI dramatically affected their learning outcomes.
Low-Scoring Patterns (Average < 40%)
| Pattern | Behavior | Outcome |
|---|---|---|
| AI Delegation | Wholly relied on AI to write code | Finished fastest, learned least |
| Progressive Reliance | Started independently, then handed everything to AI | Failed to master second-half concepts |
| Iterative Debugging | Used AI to debug/verify code repeatedly | Slow and learned little |
High-Scoring Patterns (Average > 65%)
| Pattern | Behavior | Outcome |
|---|---|---|
| Generation-then-Comprehension | Generated code, then asked follow-up questions to understand it | Slower but strong understanding |
| Hybrid Code-Explanation | Asked for code + explanations together | Extra time paid off in comprehension |
| Conceptual Inquiry | Asked only conceptual questions, coded independently | Second-fastest overall, high scores |
The fastest high-scorers were those who used AI to clarify concepts but wrote their own code.
The Productivity Paradox
This study seems to contradict earlier findings that AI can reduce task completion time by 80%. But there’s an important distinction:
- Existing skills + AI = faster completion
- Learning new skills + AI = potentially stunted development
AI accelerates what you already know. It may impede what you’re trying to learn.
The Workplace Tension
Here’s the uncomfortable implication: organizations pushing AI tools to boost junior developer productivity might be undermining the very expertise those gains depend on.
“Productivity benefits may come at the cost of skills necessary to validate AI-written code if junior engineers’ skill development has been stunted by using AI in the first place.”
Junior developers under time pressure will naturally choose speed over learning. But if they never develop debugging skills, who catches the AI’s mistakes?
Practical Recommendations
Based on this research, here’s how I’m adjusting my own AI usage:
When Learning Something New
- Use AI for concepts, not code generation — Ask “how does async/await work?” not “write me an async function”
- Generate-then-question — If you do generate code, immediately ask for explanations and edge cases
- Embrace the struggle — Errors are learning opportunities; don’t immediately reach for AI help
- Use learning modes — Both Claude and ChatGPT offer modes designed to foster understanding
For Teams and Managers
- Be intentional about AI deployment — Don’t just roll out tools; consider the learning implications
- Protect struggle time — Some friction is productive, especially for juniors
- Separate contexts — Maybe AI assistance for routine tasks, but independent work for learning new domains
- Assess for understanding — Code reviews should probe why, not just whether it works
The Bigger Picture
This research landed at an interesting moment. As AI-written code becomes more prevalent, the ability to understand and validate that code becomes more critical, not less. We need humans who can provide meaningful oversight.
If AI use during skill formation undermines that capability, we have a problem. The engineers who grew up pair-programming with AI may struggle to catch AI’s mistakes—precisely because they never struggled themselves.
My Takeaway
I’m not abandoning AI tools. The productivity gains are real for tasks I already understand. But for learning new technologies, I’m going to be more intentional:
- More conceptual questions, fewer “write this for me” prompts
- More time sitting with errors before asking for help
- More explanations requested, more follow-up questions asked
The struggle is the point. It’s not inefficiency—it’s learning.
Further Reading: