DEFEAT: Decoupled feature attack across deep neural networks.

Adversarial attacks pose a security challenge for deep neural networks, motivating researchers to build various defense methods. Consequently, the performance of black-box attacks turns down under defense scenarios. A significant observation is that some feature-level attacks achieve an excellent success rate to fool undefended models, while their transferability is severely degraded when encountering defenses, which give a false sense of security. In this paper, we explain one possible reason caused this phenomenon is the domain-overfitting effect, which degrades the capabilities of feature perturbed images and makes them hardly fool adversarially trained defenses. To this end, we study a novel feature-level method, referred to as Decoupled Feature Attack (DEFEAT). Unlike the current attacks that use a round-robin procedure to estimate gradient estimation and update perturbation, DEFEAT decouples adversarial example generation from the optimization process. In the first stage, DEFEAT learns an distribution full of perturbations with high adversarial effects. And it then iteratively samples the noises from learned distribution to assemble adversarial examples. On top of that, we can apply transformations of existing methods into the DEFEAT framework to produce more robust perturbations. We also provide insights into the relationship between transferability and latent features that helps the community to understand the intrinsic mechanism of adversarial attacks. Extensive experiments evaluated on a variety of black-box models suggest the superiority of DEFEAT, i.e., our method fools defenses at an average success rate of 88.4%, remarkably outperforming state-of-the-art transferable attacks by a large margin of 11.5%. The code is publicly available at https://github.com/mesunhlf/DEFEAT.