Visual effects have always been the most mysterious and addictive part of game dev for me. It's where being "just an artist" or "just a programmer" isn't enough. Great effects live in the overlap of both.
At Coplay, we decided to go after something sacred: teaching an AI to create at least simple, primitive VFX. And it's hard. AI models already struggle with spatial reasoning and VFX adds aesthetics, motion, and timing on top.
Where we started
The first problem was how to give the model extra instructions without overloading the context with a massive system prompt. The fix was simple: a recipe library. The model can call a tool and pull a structured template with instructions.
Where the model broke most often
The AI often picks the wrong shaders and builds effects with messy, inconsistent structure. It was fixed with detailed recipes that lock down the setup, recommend the right shaders per effect type, and enforce clear rules. For the first iteration, let's keep it simple: particle systems only. Shader Graph comes later.
Adding more power: texture generation
The next improvement that really boosts quality is generating textures for the effects. Coplay has a solid tool for this: you can specify resolution and a transparent background, and the model generates the masks/noise/sprites it needs for the job.
Visual feedback
The final necessary step is giving the model visual feedback. There's a catch: VFX matter in motion. This isn't a static image, it's movement and timing. But for the first approximation a practical approach was picked:
- simulate within 3 seconds
- find the frame with the maximum particle count
- render to texture and evaluate the result visually
Alright, let's generate some effects
Let's start with a campfire. The most obvious test. A campfire instantly shows whether the model can keep a clean structure.
Now let's try snow. Snow is a great stress test for subtlety.
And finally a mega cool colorful magical explosion. The final boss.
In the end, the AI didn't magically become a VFX artist, but with recipes, texture generation, and visual feedback, it started producing effects that actually look decent. Still rough, still simple, but real progress. Next step: Shader Graph and better iteration loops.