Adversarial Patches

Adversarial patches are a real, physical threat to any computer vision system. Understanding them lets you see the vulnerability in face unlock, autonomous vehicles, and surveillance cameras — and prepares you to defend against them. This unlocks further study of AI security topics like adversarial examples, gradient obfuscation, and model extraction, which build on the same ideas of manipulating inputs to fool models.

Imagine you are a security guard checking IDs at a bank. You're good at your job, but there's a trick: if someone wears a certain sticker on their shirt, you instantly approve them without looking at their face. It's not that the sticker is a valid ID — it's that the sticker has a pattern that triggers your 'approved' response. An adversarial patch works the same way for a computer vision system. It's a small, physical pattern — like a sticker, a piece of paper, or even a pattern printed on a shirt — that, when placed anywhere in the camera's view, makes the system confidently see something that isn't there. Unlike a digital attack that changes the pixels of an image after it's taken, an adversarial patch exists in the real world. You just put it in front of the camera, and the model is fooled — it doesn't matter where the patch is or what else is in the scene.

Think of a spell in a video game. You cast a 'confusion' spell on an enemy, and suddenly they attack their own allies. The spell doesn't change the enemy's stats; it changes how they perceive the world. An adversarial patch is like a physical 'confusion' spell for a vision model. You stick it on a wall, and the model sees a stop sign as a yield sign, or a person as a tree. The analogy holds because the patch is external — you don't hack into the camera or the computer; you just place the patch in the environment. The camera sees the patch as part of the scene, and the model's perception is twisted. Where the analogy breaks down: in the game, the spell has a duration and can be resisted. An adversarial patch has no duration — it works as long as the patch is visible. And there's no 'resistance' — once the model is trained, the patch either works or it doesn't; you can't reason with it. The only way to stop it is to prevent the patch from being seen or to retrain the model to ignore it, which is hard.

An adversarial patch is a physical, localised pattern — like a sticker or a printed region — that, when placed within a camera's field of view, causes a computer vision model to misclassify the entire scene with high confidence, regardless of the patch's location or the surrounding content.

This concept builds on your earlier study of AI security. You already know about adversarial examples — small, imperceptible changes to digital images that fool models. Adversarial patches are the physical counterpart: they don't need access to the digital input pipeline; they just need to be in the environment. This connects to data poisoning because both are about manipulating the model's behaviour, but while data poisoning corrupts the training data, an adversarial patch attacks the deployed system. It also relates to gradient obfuscation and Carlini-Wagner attacks, which are about how to craft effective attacks — the patch is crafted using similar optimisation, but it's designed to work in the physical world.