First impressions of GPT-4 🤖

---

To be honest, I wasn't immediately that hyped when ChatGPT came out using GPT 3. It was entertaining for awhile, but that's mostly what it was for me.

But after a fun handful of recent tweets around GPT-4's launch, I had to pull out my credit card out and give it a try.

And just like that, I was a ChatGPT Plus subscriber.

Link to this headingGetting started

One of the first things I decided to do to test things out was to build a Pokédex using GPT 4 without writing code myself... and it worked.

I was particularly blown away by how quickly it was able to build a working MVP.

Within seconds, I had this working prototype with the first 151 Pokemon. Pretty cool.

All it took was two queries. The first asking it to make a Pokedex web app, and the second asking it to make it look nice with CSS.

ChatGPT even gave me the following caveat:

Note that this implementation is quite simple and doesn't handle errors, pagination, or performance optimizations.

From here, I kept asking it to add features to the app, and it kept doing so.

For example, I asked to add filters for all the types of Pokemon, and then to add the remaining ~1000 pokemon to the app. And each query succeeded on the first shot.

I even got this response after the last feature request:

However, as I mentioned earlier, loading data for all Pokémon at once may not be optimal. To improve performance, consider implementing pagination, infinite scrolling, or using the react-query library to manage API calls and cache results. This way, you can load Pokémon data in smaller chunks and provide a better user experience.

But this is the point at which that things started to get hairy.

As soon as I asked it to implement pagination, it dropped a previous feature, filtering by type.

I was able to ask it to re-add filtering by type without losing anything else, but with the growing App.js size, ChatGPT would cutoff the code before finishing quite often.

Eventually, I had to ask ChatGPT to refactor the App.js so it wouldn't keep cutting off. Which fortunately worked quite well.

But nearly every other query at this point would result in needing to debug something with ChatGPT. Usually, additions or fixes to filtering would break pagination and vice versa.

Fortunately, pair programming with GPT-4 was still fun. Admittedly, this is around the point that it would've been faster to take over, but I wanted to see how far I could push it.

And there were some enlightening responses from ChatGPT along the way.

It's probably a good idea to thank robots.

The coolest part about the debugging process was learning that I could include code in my queries and get the same code back in the response with the needed fix.

After I got through the above kinks, the rest of the time was spent adding a bit more flair and features again. This time, without causing more things to break.

I was able to quickly get ChatGPT to automatically find and apply icons to the filter buttons for each type of Pokemon, list popular moves for each pokemon, and make the site look a bit nicer even.

With a little finagling, I pushed the React code to a NextJS app in this open-source repo and deployed it to Vercel.

You can also see the full conversation with ChatGPT here.

Link to this headingMy takeaway

Overall, I'm impressed. GPT-4 is more concise and better able to handle complex tasks and context.

I can finally see myself using this day-to-day.

At the very minimum, for kicking off new scripts or prototypes with GPT-4 (since it's going to be a lot faster than doing it myself), and refactoring code.

There's generally just a ton of one-off stuff that I won't have to google anymore, and boilerplate that I won't have to write anymore.

Even as-is, this feels like a bigger bump than copilot to my productivity, so I'm excited to see where this goes.

While GPT-4 is noticeably slower than GPT 3.5 currently, I'd use the waiting period to either queue up more queries or implement and check out how the code was working.

And in terms of learning, I think the potential is enormous. As I was building this Pokédex with GPT 4, I couldn’t help but to feel like this could be an interactive tutorial if I didn’t already know React.

It felt like a relatively natural way to learn pitfalls of web dev. I could query about anything regarding the code and the context was enough for ChatGPT to provide a reasonable answer. And often, ChatGPT would warn me and even suggest next steps.

As far as productivity and perceived competency, it felt like I had somewhere between a junior and mid-level engineer working directly with me while pair programming, except operating much more quickly.

With sufficient complexity that inevitably happens with debugging, it really feels like GPT needs to be integrated with my IDE and browser to access the full context without things being lost in translation to natural language.

Anyway, thanks for reading. Stay tuned for more updates on GPT-4 and my experiments with it.