The other day I was in a car, hearing various ads on the radio, and one of them was an ad for a home alarm, a device that would call the police if it detected a break‍-​in. As part of the ad, it played a mock break‍-​in where a female voice said something to effect of “The police are on the way”.

This lead to thinking; if the burglar knows the police are coming, then ze will undoubtedly run like hell. The police could deduce this, so it seems they would have little incentive to actually follow‍-​up on this treat. Which leads to Complications.

(naturally this analysis ignores the obvious fact that police do have other incentives to visit the crime scene, for things like fingerprinting and such, but that’s less interesting)

This calls to mind the classic problem of deterrents; the Authority would have no incentive to actually carry out their threats, once a deed is done they gain no further utility through punishing. But this also falls apart once the would‍-​be criminal deduces this.

The dynamic is unstable, but for some reason this called to mind the prisoner’s dilemma. I was a bit happy at seeing this connection, since it would function as an introduction to the dilemma without the problems of the naive formulation (invoking an intuitive response designed precisely to that effect) and Yudkowsky’s formulation (requiring you to be so inferentially close enough to the LW memeplex that you already understand the dilemma).

Unfortunately, a closer look destroys this illusion.

In the beginning, the police responded to a call from these alarms by investigating it. The burglar would flee the scene. And since we’re ignoring reality as previously stated, the police have no incentive to continue arriving that the scene of the crime. The burglar learns this, and proceeds to steal even more. And so the police begin arriving once more. And on ad infintum.

This isn’t a prisoners dilemma, because it never stabilizes. For any configuration in ({burglar flees, burglar stays} and {police pursue, police ignore}), at least once party gains by altering their behavior (we’re neglecting randomized strategies).

There’s a quality to be found here; the police have it when they commit to ignoring any alarms, and the burglar has it when he commits to ignoring alarms. There isn’t a good word for it, as far as I know, so let’s call it an exploitable vulnerability.

An exploitable vulnerability opens up the agent to, you guessed it, being exploited by other agents. What’s interested is that they’re anti‍-​inductive: once you realize you’re presenting a vulnerable surface, you scramble to cover it.

This isn’t a new insight. It’s been noted that organism with no adaptive immune system aren’t flooded with illness; their infection rate is comparable to an ordinary mammal’s. The reason is simple: antibiotic resistance. Erecting a defense, merely prompts attackers to find away through or around it. Which leads to evolutionary arms race, etc. You know story.