AI coding assistants do not boost productivity or prevent burnout, study finds

https://www.techspot.com/news/104945-ai-coding-assistants-do-not-boost-productivity-or.html

16 Comments

  1. It helps me type faster but thats about it. Most of my time Im blocked on people reviewing my stuff. It might make typing less tedious, but it doesnt matter all that much.

  2. deadisnowhere on

    AI assistants are shrouded in smoke, making their value unclear.

    For simple, repetitive, and common tasks, the tool should automatically handle these parts of the code, but for trickier, more complex requirements, most AI tools fail, making them unsuitable for the most stressful parts of software development.

  3. Does anyone know why people thought it would prevent burnout? Typically, tech that increases efficiency just means more work gets done in the same time frame — not that people could relax more in most environments. 

  4. absentmindedjwc on

    It absolutely does help quite a bit with throwing code together, but it 100% requires someone skilled on the other end to refactor and clean up jank-ass code.

    There is a more junior dev on my team that seems to use ChatGPT for *fucking everything*… he doesn’t realize I know, but recognizing GPT-output code is pretty simple if you’ve seen enough of it. Anyway… instead of taking the output and modifying bits and pieces of it to better fit within the needs of the application.. he just just copy/pastes it in, and if it works even a little bit: ship it. I reject practically 90% of his PRs, with the most recent one commenting essentially “see me after class” because it was so fucking bad.

    I even went and asked GPT to solve that problem, and with some back and forth, got a reasonably workable solution that just needed some slight modification on my end.

    IMO, AI is decent at getting a sort-of prototype up and running. It gives you a very low-fidelity version of code that might help you sort of get started…. but it is VERY MUCH not production ready… both because the implementation without a ton of back and forth is trash (or even better: personal touch-modifications that make it *truly* fit the application)… but – **and this is important** *-* ChatGPT-generated code is *fucking full* of security issues.

  5. As soon as you’re out of education, the bottleneck on programming is not in turning ideas into lines of code.

    It’s in understanding the technical and business requirements, and coming up with structures for getting things done that makes solving the immediate need easy, and the inevitable change less painful.

    It’s in building a consistent model of the world that lets your code do what it needs to do, and makes communicating about needed changes easy.

    It’s in constructing a platform that allows multiple devs with different skillsets and mental models to contribute according to their own strengths, and makes their task of doing the above easy for them.

    If you’re a programmer, and you don’t think these are major limitations, I suspect that’s because someone is managing them for you, and you haven’t hit a hard edge yet. May you be able to overcome it when you do. I trust you have built ways of working so you are not stuck the instant your prompt engineering fails.

    Edit: To wit, Burnout happens when management does not understand this, and treat programmers like factory workers. Productivity doesn’t increase because AI tooling does not address the bottleneck.

  6. I can’t use it at work but do use it for my personal projects. It helps a lot during the start of a project but once I’m a few days in I find it very unhelpful.

  7. ChatGPT is great for writing boilerplate, unit tests, util functions.

    None of those are the reason for burnout.

  8. I’m in no way surprised at this.

    Whilst on a personal level I’ve used GPT to give me ideas on how to attack a specific coding problem, there is no way in hell I would use code straight copied from GPT, but the allure of doing so would be strong for many.

  9. these must be really niche study cases; because ai coding definitely boosts productivity because i spend less time on good for my issues

  10. No way! But companies asking for money to use their services told me their services would make me a cool programmer without needing to work for it. Someone is lying here. /b

  11. Sounds like a user problem. I am not a Python programmer or web dev. Last week I got ChatGPT to help me deploy an idea I had for a web app over two days. A fully functional Flask app that does more or less exactly what I need it to. I didn’t even know what Flask was. This is stuff I would have spent weeks if not months figuring out in the past, done in two days. If AI assistance isn’t improving your productivity, you’re using it wrong.

  12. If I want to throw up a basic website skeleton it can handle that for me and then I will take it from there

    If I want it to show me some examples of a obscure feature, that’s not a terrible idea

    I can have it make me some unit tests or generate some getters or setters(granted there have been tools in my Ides to do that for ages)

    But I can also do all those by myself and I understand how to do them myself

    It can probably throw them together faster than I can, but I still need to you know Read it over. Make sure it works and I understand why it works and how it’s doing what

    And for anything beyond basic stuff I’m not going to trust it to not make me spend more time debugging than if I just coded it myself in the first place