Is GPT-4o actually better than Claude 3.5 for coding now? My results are all over the place.

I spent last weekend running the same 5 Python scripts through both models and the difference was kinda wild. GPT-4o nailed 4 out of 5 on the first try but then completely flopped on a basic data parsing task. Claude 3.5 got that one right but took 3 tries to fix a simple loop bug. This is after the big updates both companies pushed in late October. What changed in their training data or fine-tuning that made them trade strengths? Has anyone else noticed a clear winner for your specific workflow lately? I'm trying to decide which subscription to keep.

3 comments

3 Comments

black.amy26d ago

Lol me trying to follow the timeline of which update hit when is harder than actually coding itself. I swear I need a spreadsheet to track all these model versions now, my brain can barely remember what I ate for breakfast. But yeah you got a point about the timing thing, I was probably testing two different versions of each model without realizing it. I should just run all my scripts again this weekend and see if the November Claude update actually fixed that loop bug issue or if I'm just bad at prompting.

jesse77326d ago

@morgan316 actually the big GPT-4o update hit in late October, not September - check their changelog.

morgan31626d ago

You said "big updates both companies pushed in late October" but I think that's actually not right. The big GPT-4o update landed in September not October, and Claude 3.5 Sonnet got its major refresh in early November, not late October. So the timing might be throwing off your comparison a bit because those updates were rolled out a few weeks apart from each other. That could explain why your results feel so inconsistent, the models were in different versions when you tested them. I'd suggest running your test again with the current builds of both, especially Claude's November one, because that latest Claude update really focused on coding and logic stuff.