I have recently spent some time on a call with a smart junior developer. While I was helping to clear up some confusion I kept thinking about the AI assistants. Wondering if they could've helped.
The problem was an API error complaining about missing introspection for one of the classes in a natively compiled Java application.
Junior had access to GitHub CoPilot and tried to solve the problem, but the solutions did not work. The troubled class was not part of our own code and was from an Apache library instead and neither AI, not the Junior could figure out what to do about an error in a library class.
I explained what the introspection is and why we need it, showed how to set it up for a library class. But I did not stop there. I pointed out how using such a class on the API surface is not a great idea. Then I mentioned that the class is only ever used for one specific purpose, and we can probably name it according to that purpose.
After the chat, our codebase and API were cleaner and better documented and the junior developer learned a few things.
AI agents fail miserably trying to replace any of the above.
AI fails to figure out what introspection is in Java. It only appears in Java applications compiled to a native image, which is not a common use case. Running as a native application (without JVM) there is no reflection, no bytecode to rewrite, it is just a plain executable. It could've been written in C++ for all that matters. AI kept suggesting various JVM tricks to work around the problem, none of which were even remotely applicable. It just could not shake off the assumption of JVM running Java code - because it is so prevalent in its training set.
AI also failed to spot higher level problems with the approach. Such as using library class in our own API surface or noticing that the class is used in a fairly narrow way and could be slimmed down by a lot.
Thinking back to my own junior times, I can remember one time where such conversation helped me become a much better developer. The senior developer I was asking for help with a specific problem fetched a book from his bookshelf and handed it to me: "Read this first, you'll figure the proper solution then". It was an awesome advice and a book I really needed to read at that time. AI never suggests for the developer to step back from the code and read a book instead. It is just not an option in its system prompt.
AI assistants are fun to use now. But they are far from being able to the software developer's job.