Hi guys,
as probably everyone is aware there are huge changes happening in software development, with very capable code generation being possible.
In my bioinfo work I had mostly used chatgpt for smaller modular functions with clear goals. So I was curious on how well agentic AI works (Defined as: you tell it in natural language, and the model is able to change files, run tests etc.). I got free access using Github Education to claude and chatgpt models, I think they were pretty advanced.
My toy project was an unrelated website idea I had had for years, and it worked ridiculously well. It walked me through lots of stuff I theoretically knew from studying CS, like setting up a frontend + backend + DB infrastructure and walking me through the entire deployment phase. It was really absurd how well and quickly it implemented any and all of my requests. One key thing for its working was that it quickly set up lots of testing infrastructure, which it could use to validate everything was ok.
So naturally I started being worried on the general future of work in CS / data analysis. So I tried using it for a different more work-related project. And I have to say it performed surprisingly poorly. Wrong scope of project, i.e. instead of doing a straightforward analysis it set up loads and loads of architecture. Another thing is that it works really badly with notebooks so far. So I have to say actually trying it made me a bit less worried about being replaced.
Now I am curious about your experiences. Have you tried using agentic AI for work? What were your experiences? I think one key issue is that testing frameworks are pretty much unusable, as the point of data analysis is to find currently unknown results, so we cannot write tests for that.