Intrusion Detection

Intrusion detection is the foundation of AI security. It's how you notice that something is attacking your system—whether that's a network, a host, or an AI model itself. Mastering this first gives you the mental model of detection: the trade-off between catching attacks and drowning in false alarms. This directly prepares you for the neighbouring topics in your library: liveness detection (is this a live face or a photo?), adversarial examples (is this input a subtle attack?), and model extraction (is someone probing my model?). Every AI security problem is, at its core, an intrusion detection problem. You'll also need this when you build agentic systems later—every tool your agent can call is an attack surface, and you'll need to detect when that surface is being probed. Start here, and the rest of AI security becomes variations on a theme.

Think of a security guard at a building entrance. The guard has two ways of spotting trouble. First, they have a list of known bad people—a ban list. When someone on the list shows up, the guard stops them immediately. That's signature detection: you know exactly what an attack looks like, and you match against it. It's fast, precise, and useless for anyone not on the list. Second, the guard has a sense of what normal looks like: people come in through the front door, at normal hours, with normal badges. When someone climbs in through a window at 3AM, the guard's alarm goes off, even if that person is not on any list. That's anomaly detection: you don't know what an attack looks like, but you know what normal looks like, and anything that strays too far from normal is suspicious. The problem is that normal is fuzzy. The cleaner who arrives early, the IT guy who logs in at 4AM to do maintenance, the intern who downloads the whole database—these are all deviations from normal, but they are not attacks. If the guard reacts to every deviation, they will be chasing innocent people all day and miss the real attacker who calmly walks in with a stolen badge. This is the core tension in intrusion detection: you cannot have a system that catches every attack without also flagging a huge number of legitimate activities. The art is in choosing where to draw the line. Real systems are hybrids: they combine a list of known attack patterns (signatures) with a model of what normal traffic looks like (anomalies). The signature catches the known, the anomaly catches the new, and a human analyst decides which alerts are worth investigating. The mathematics behind this is mostly probability and statistics—not calculus. It's about counting: how many attacks did I catch, how many normal events did I falsely flag, and how do I balance those two numbers?

The bank teller who knows her customers. She has a list of people who have been banned from the bank (that's signatures). She also has a feel for who usually comes in and when (that's a model of normal behaviour). When someone she doesn't recognise walks in and asks to withdraw a large amount of cash, her suspicion goes up. That's an anomaly. But the teller also knows that the quiet old man who comes every Tuesday afternoon is just withdrawing his pension—even though he also fits 'unusual' in some ways. The teller's skill is in balancing two kinds of mistakes: turning away a legitimate customer (a false positive) and letting a robber through (a false negative). A teller who is too cautious will annoy customers and slow the line. A teller who is too trusting will get robbed. The same trade-off applies to machine learning detectors. The analogy breaks down when you consider scale and patience. A human teller can use context, body language, and intuition—things a statistical model cannot. Also, a human can learn from a single incident: the teller who once got tricked by a fake ID will be more careful forever. A machine learning model typically needs thousands of examples to learn. And an attacker can patiently adapt to the model's idea of 'normal'—like a burglar who starts visiting the bank every week just to look normal, then strikes on the tenth visit. The model might not flag that because it has learned that this person is a regular. So the analogy teaches us the fundamental idea, but the reality is harder: machines are fast and consistent, but they are also naive and easily fooled by a clever adversary who knows how the model thinks.

Intrusion detection is the process of identifying malicious activity in a network or host by comparing observed behaviour against known attack signatures, or against a learned model of normal behaviour, or a combination of both.

This is your entry point into the AI Security topic. Your library lists neighbouring topics that you will study next: adversarial examples, which are inputs designed to fool a vision model; liveness detection, which is about telling a live face from a photo; and model extraction, where an attacker probes a model to steal its knowledge. All of these are intrusion detection problems—you are trying to identify when someone is attacking the system. The detection techniques you learn here—signatures and anomalies—apply directly to those. In prompt injection (which you'll meet in LLM security), the injected prompt is an anomaly in the input distribution. In model extraction, the query pattern is an anomaly in the API traffic. So mastering this concept gives you a unified lens for the rest of your library.