New here? I'm George. I build software full time, mostly with AI right next to me, and I write down what I learn the hard way while I do it. This one humbled me more than anything has in a while.
Everyone says "AI slop" now, and everyone gets a little offended when someone says it about their product or their project. But when I tried to explain what actually makes 1 program slop and another one good, I couldnt really do it, and I dont think most people can either. We just kinda feel it.
I couldnt explain it until a few days ago, when I opened a Twitter thread about the biggest PHP bug of the year, fully ready to understand it, and couldnt get through the first line.
And I think the difference between AI slop and a good program is in the details. But I think that only makes sense if I tell the whole story of how I got there, so thats what this is. Its also me checking if I actually learned any of it, cause if I cant explain it to someone else I probably dont understand it.
I thought I learned this in March
Back in March I wrote a post called How AI Coding Agents Are Making Developers Dumber. I ended it with "AI is the best teacher I've ever had. But only when I stayed in the room." and I really believed it. I thought I was 1 of the people doing it right, using AI to learn and not just to get things done.
And I was learning, a lot, I always do. But I didnt notice what kind of learning it was until this month.
AI made all of us feel like we could do anything
My day job is full stack. I build automations, CRMs, internal tools, just things that makes a team move faster. Some of it uses AI for the fuzzy parts, the parsing, the semantic matching. And with AI next to me I was fast. You describe the thing, the agent writes it, you test it, it works, you ship it. And honestly it felt like a superpower.
The problem is when everything you describe turns into a working thing that fast, you stop asking what happens underneath it. You just trust that the compiler works, the framework works and the package works, cause nothing is broken so you never check.
And its not just me. The past couple of years the number of people calling themselves full stack or indie engineers went through the roof, and a lot of them say "AI" and mean ChatGPT or Claude, without knowing its machine learning underneath, with an inference side, a harness, and a lot of math. I knew that part, I've read papers on inference like AIBrix, but knowing what AI is didnt mean I knew what was under my own code. We all got really good at the top layer, and AI never really makes you go below it.
Everything I knew about security fit in 4 bullet points
Before any of this, I got handed a task that had nothing to do with my normal work -> figure out how to secure a passive system. A passive system is one that cant talk back. No internet, no connection out, it just sits there and listens and catches signals. And I had to work out how you protect something that can only receive.
This is everything I knew about security at that point:
- API tokens exist. How they actually worked -> no clue.
- You lock local systems off through the network. Also the most general, hand-wavy version of that idea.
- 2FA. I'd learned this one for fun -> you send a request, you get a code back, a little handshake happens, the code gets confirmed, and shabang, two factor authentication.
- You put your .env in .gitignore so your keys dont end up on GitHub. Yeah. That one made the list haha.
And that was it. I didnt know what red teaming was, and I had never heard of black box, white box or pen testing. But I had AI next to me, so I thought 4 things was enough.
Down instead of across
So I went digging, and fell straight down a hole on how SiriusXM controls a whole fleet of passive receivers, every radio in every car, and can lock any single one of them off whenever they want. The receiver can only listen and they still control it completely, and I really wanted to understand how.
And that meant learning the architecture underneath for real. AES, SHA-256, elliptic curves from the crypto world, a pile of encryption I still cant do the math behind (later problem). ECM and EMM, the control and management messages, and how they get built and split just right inside a transmitter signal. It was slow and hard, and instead of learning 5 more things around the topic like I usually do, I was going down into 1 thing, and I loved it.
Then the whole industry got hacked
Right after that I got my next task -> add a security gate to the CI pipeline. 3 checks. SAST, which reads your own code and flags the patterns known to be attacker bait, stuff like letting raw input pick which function runs. Dependency scanning, which checks your libraries for known holes. And secret detection, which makes sure no keys slip into the repo. If any of them trip, the PR cant merge.
And right as I'm learning what SAST even is, the whole industry starts getting hacked. OpenCode. PHP. Railway. A wave of npm packages. Company after company, and some of the tools I actually use got caught in it, which meant my own projects and everything I was contributing to were sitting in the exact same blast radius.
8 agents and a lot of confidence
So I decided to learn this properly and see if I could defend my own stuff, and I built a loop of agents for it. 8 agents. 4 hunting from the outside with no view of the code, black box. 4 reading the code from the inside, white box. All of them dumping what they found into 1 findings folder, and then Opus 5.5 picking those up and patching them one by one, with Fable 5.1 and Astra 6 as advisors arguing over every fix. I learned RBAC, more crypto, more of everything. And after all that I thought okay, Im finally ready to understand what happened with PHP.
I couldnt read the first line
So I opened the thread, and I didnt know what CVE meant.
I didnt know what any of the numbers meant either, or half the words around them. It was written by people who actually understand the machine and I couldnt follow it.
I got so good at pointing AI at problems that I thought I understood the problems, but I really just understood the prompt.
(For anyone sitting exactly where I was -> a CVE is just the public ID for one specific known vulnerability, like CVE-2026-109618, the PHP one. CVSS is the 0 to 10 score for how bad it is. I didnt know either of those.)
Back to the start
It wasnt really about PHP tbh. I always learn, thats just who I am, if I dont understand something it honestly pisses me off until I do. But that thread showed me what kind of learning I had been doing. It was horizontal -> I kept learning more and more things around a topic, the tools, the frameworks, the next thing the task needed, but I rarely went further down into any 1 of them. Vertical, going deeper into the same thing until you hit the bottom, I had mostly stopped doing.
And I used to do it all the time. In school I was always looking into why a physics formula looked the way it did, I wanted to figure out how someone got to that point and came up with it in the first place. Then I left university to work full time, and the tasks took over, every week needed something new, so I learned wide to keep up, and there wasnt much time left to go deep.
So I decided to get back into the vertical mode I used to be in. Every day I spend a couple of hours relearning the basics, even the things I already know, to get back to the level of understanding I used to have (or at least the level I always wanted to have).
I started with linear algebra, even though I finished it in university with an A. (Same thing happened a few weeks back with event loops, after a senior engineer I respect told me to go actually learn how they work. Turns out there was a lot more to it than I thought. Thats a whole other blog, coming soon.)
Then memory. I already knew about the heap, the stack, data and so on, I took an assembly class in university so I had a pretty good base in low level code. But that was also the first class where I had an agent write the code for me, so I didnt pay that much attention, and I remembered the basics without being sure my understanding was actually right. So I went through it again from the ground up, and I found a YouTuber called Low Level who breaks these hacks down with the real technicality but at a pace I could follow. This is how I understand it now.
What is actually underneath your code
Everything your program has, every variable, every function, your actual code, is just numbers sitting in a long line of boxes in memory. Each box has an address, which is just its spot in the line.
Those boxes get sorted by how long they live:
- Globals live the whole program. If you gave them a starting value they sit in an area called data. If you didnt, they sit in bss, which just reserves the space and fills it with zeros, so the file on disk stays small.
- Locals live for one function and then disappear. They sit on the stack.
- The heap is space you ask for by hand while the program runs, for when you dont know the size ahead of time. Like an input where someone might type one word or five thousand, so you dont reserve the max every time, you just ask for exactly what you need.
- And the text area holds your code itself, cause the instructions have to live in memory too for the CPU to read them.
Once I confirmed my understanding, the vulnerability finally started slowly to make more sense.
How you break in without breaking in
My whole idea of a hack was from movies, someone gets into your server, drops a file and runs it (whcih still could happend and there are a lot of different methods of hacks). But most of the time its the program attacking itself, and the attacker never gets inside the way I pictured.
Picture a program that reserves 8 boxes for a username and copies whatever you type into them without checking the length. Memory has no fence. Theres no wall that says the username ends here. So if you type more than 8, the extra just keeps writing into the boxes next door. And on the stack, the box next door is often the return address, the note that tells the program where to jump back to when its done. Overwrite that note, and you decide where the program goes next. So nobody broke in, someone just typed too much into a box nobody checked, and that 1 missing length check is the whole bug.
It gets more complicated than that though. You cant just drop your own code in, cause modern chips refuse to run code out of a data box (thats a protection called NX, no execute). So attackers dont bring code at all. They find tiny fragments of the program that already exist, each one ending in a return instruction, and they call those fragments gadgets. The overflow doesnt write code, it writes a list of addresses, one gadget after another. What I didnt know is that the return instruction has no fixed destination, it reads its next address straight off the stack. So the first return jumps to gadget one, gadget one hits its own return, that return pulls the next address off the list, jumps to gadget two, and on and on. A marker called the stack pointer walks down the list, one gadget per return. They assemble a whole new program out of the pieces of yours, all to reach one goal -> make the operating system hand them a command shell on your machine.
The heap and the globals work the same way, just the trigger is different. Theres no return address there, so instead you overflow onto a function pointer, or a vtable pointer (the pointer a program follows to find which code to run for an object). You flip that pointer to your target, then you wait for the program to call through it, and the call jumps to you.
The two bugs that made it real
The PHP one was a use after free. The program lets go of a piece of memory but keeps using the pointer to it on a later request, so you get it to read and write memory thats already been freed. That bug sat in PHP since version 5.4.0, which shipped in 2012. 14 years. Sitting under a huge part of the web, WordPress, Laravel, Drupal, anything running PHP through PHP-FPM or Apache. And it runs before your own code even gets a turn, which is why its a pre-auth bug, no login needed.
The Citrix one was a buffer overflow in a 35,840 byte scratch buffer, with pointers the program would later follow sitting in the memory right after it. Overflow the buffer, flip a pointer, wait for the call, take over. And the binary was shipped without basic protections, and the biggest one missing was a single compiler flag (its called PIE) that would have randomized its addresses. Without it, every target lives at the exact same address on every machine on earth, so the attacker already knows where everything is, and all of that came down to 1 flag.
Thats a very simple explanation of what happened.
Why did I tell you all of this
And these werent AI slop. PHP 5.4 shipped in 2012 and Citrix has been building this stuff for decades, way before anyone was writing code with agents, so the people who built them were real professionals who knew memory better than almost anyone. And they still missed 1 detail each, and that detail sat there for years until someone found it.
Now AI is on the other side too. It makes finding holes faster and easier than ever, so security gets weaker unless the people building the software actually understand whats underneath it. But that doesnt mean we stop, it means we have to go deeper.
So why did I tell you all of this? Because thats exactly where AI slop really made sense to me. If experts who understood every layer still left 1 detail open, then code built only at the top, by someone who never looked underneath at all, has every detail open.
Where my own code skipped the details
And I have the same problem in my own code. I shipped automations that trusted a third party package to always hand back the same shape of data, and I didnt think about what happens the day it doesnt. I built CRMs and internal tools that looked great in the demo and that I never load tested, never profiled, never checked what they did to memory, cause they worked, so why look. And my entire security practice was those 4 bullet points and a gitignored .env.
None of it broke in front of me, so I called it done, and thats exactly what slop is -> nothing is broken, nobody ever looked underneath. And I did that on a lot of projects.
So heres the definition
Before AI, good software got built by small teams of deeply niche knowledgeable people. The front end engineer whose whole world was making it feel right. The backend engineer who made it work. The low level engineer who built the fast machinery underneath. The security engineer who understood buffers and heaps and stacks better than anyone, so they found the holes before a stranger did. The reverse engineer who could read raw bits and turn them back into something human. That depth was the whole point, and the rare person who could hold more than one of those at once, the real full stack engineer, was a unicorn, cause theres just that much to know.
Then AI showed up, and we all stopped going down there.
So heres my definition, the one I couldnt give a few days ago:
AI slop is software built entirely at the top, just to work, by trusting that everything underneath it... is perfect. The compiler, the framework, the package, the memory, the security, all assumed flawless, none of it understood. It looks great in a demo, and then it breaks in the places nobody looked, cause thats where the details got skipped.
I was building slop. Now Im digging.
And by that definition I've been building AI slop. High level code that just worked, shipped fast, without enough thought about what was happening underneath it. I fell straight into the trap and I didnt even feel it, cause it felt exactly like productivity. But Im slowly going deeper now, into the low level stuff, the security, the math I skipped, 1 piece at a time, cause I actually want to earn the full stack title instead of just calling myself that.
The airplane and the jungle
Im not saying AI is bad. I think its 1 of the best things that ever happened to us. The people who already had that deep knowledge didnt get replaced, they got multiplied, cause they can steer the model and catch it the second its wrong, at a level someone newer just cant. And AI lets anyone who is actually curious become a kind of polymath, poly-niched, in anything, not just code, all you need is to actually be interested.
But the problem (and this is for me too) is that AI makes us feel like masters. And on the high level, we kind of are. But almost none of us are masters on the low level, and the low level is the foundation that actually makes software good. Its like flying over a jungle in a plane and calling yourself Tarzan. You saw the whole thing from above, and from up there you can see really wide, which is exactly how I was learning. But you never went down into it, never met the snakes, never got sick, never had to build a shelter and survive a night. You know what it looks like from above, but you dont actually know the jungle.
So I think the right way to use AI is to not let it convince you youre already a master, and to use it to actually go down into the jungle, cause now you have a guide that will go down there with you.
Im at the very tip of the iceberg, I know that now, but at least Im finally starting to survive inside the jungle with Tarzan next to me to teach me.
To summarize everything:
- AI slop is code built only at the top that works, while everything underneath it is trusted and never understood.
- The difference between slop and good software is the details underneath -> the length check, the compiler flag, the memory cleanup, the dependency you actually read.
- PHP and Citrix were built by real professionals before AI, and they still missed 1 detail each. Code built only at the top misses all of them.
- Start your digging with how a program actually runs -> memory, the stack, the heap, what your code turns into once its running.
- AI makes you feel like a master on the high level. On the low level, almost none of us are.
- Horizontal learning (more things around a topic) feels like progress and AI makes it really fast. Vertical learning (deeper into 1 thing) is where the details are.
- Use AI to go deeper, not just wider. Its the best guide you will ever have for climbing down into the hard parts.
- You dont have to master everything, you just have to know the layers you skipped are there.
Have questions? Want to chat?
I'd love to hear from you, especially if you think I got something wrong. Reach out on any of these:
Keep learning.
Keep growing.
Keep building.
George Babakhanov
Student for Life
Research & Blog Citations
