Adversarial AutoMixup (ICLR 2024 spotlight)

Huafeng Qin^1,*,†, Xin Jin^1,*, Yun Jiang¹, Mounim A. El-Yacoubi², Xinbo Gao³

¹Chongqing Technology and Business University

²Telecom SudParis, Institut Polytechnique de Paris

³Chongqing University of Posts and Telecommunications

^* Equal Contribution ^† Corresponding Author

We propose AdAutoMix, an adversarial automatic mixup augmentation approach that generates challenging samples to train a robust classifier for image classification, by alternatively optimizing the classifier and the mixup sample generator. AdAutoMix comprises two modules, a mixed example generator, and a target classifier. The mixed sample generator aims to produce hard mixed examples to challenge the target classifier while the target classifier’s aim is to learn robust features from hard mixed examples to improve generalization. To prevent the collapse of the inherent meanings of images, we further introduce an exponential moving average (EMA) teacher and cosine similarity to train AdAutoMix in an end-to-end way.

Mixed Images of Various Mixup-based Approaches.

📬 You can contact me by email: 158398730@qq.com or WeChat: xinxinxinxin_j.

If you are interested in palm or finger vein research, please contact us!

---

🛠 Installation

💥News! ! !💥
2024-01-18: you can clone Openmixup training "AdAutoMix"[ICLR 2024].
2023-12-19: We update some analysis tools code such as: Calibration, FGSM calibration_fgsm.py and Occlusion Robustness occlusion_robustness.py experiments, also we support a mix augmentation method "SnapMix"[AAAI 2021].
Big thanks to Siyuan Li(@Lupin1998).

🔧How to install?🔧
In fact, you can add our python file in OpenMixup.
There, you can see how to use it and the environment required. What you need to do is add or replace our files by folder inside OpenMixup, and then add the function names of the files in the __init__.py file.
You also can download or find other Mixup methods in OpenMixup("https://github.com/Westlake-AI/openmixup")
Thanks contributors: Siyuan Li(@Lupin1998), Zichen Liu(@pon7) and Zedong Wang(@Jacky1128).

---

Here are the commands to install OpenMixup

conda create -n openmixup python=3.8 pytorch=1.12 cudatoolkit=11.3 torchvision -c pytorch -y
conda activate openmixup
pip install openmim
mim install mmcv-full
git clone https://github.com/Westlake-AI/openmixup.git
cd openmixup
python setup.py develop

Here are the commands to git clone AdAutoMixup

git clone https://github.com/JinXins/Adversarial-AutoMixup.git

---

📊 Experiments

CIFAR-100

Name	alpha	Conference	ResNet18	ResNeXt50	Swin-Tiny	ConvNeXt-T
Vanilla	-		78.04	81.09	78.41	78.70
MixUp	1.0	ICLR2018	79.12	82.10	76.78	81.13
CutMix	0.2	ICCV2019	78.17	78.32	80.64	82.46
SaliencyMix	0.2	ICLR2021	79.12	78.77	80.40	82.82
FMix	0.2	ArXiv	79.69	79.02	80.72	81.79
ResizeMix	1.0	CVMJ2023	80.01	80.35	80.16	82.53
PuzzleMix	1.0	ICML2020	81.13	82.85	80.33	82.29
AutoMix	2.0	ECCV2022	82.04	83.64	82.67	83.30
AdAutoMix	1.0	ICLR2024	82.32	84.22	84.33	83.54

---

Tiny-ImageNet & ImageNet-1K(denote *)

Name	alpha	Conference	ResNet18	ResNeXt50	ResNet18*	ResNet34*	ResNet50*
Vanilla	-		61.68	65.04	70.04	73.85	76.83
MixUp	1.0	ICLR2018	63.86	66.36	69.98	73.97	77.12
CutMix	0.2	ICCV2019	65.53	66.47	68.95	73.58	77.17
SaliencyMix	0.2	ICLR2021	64.40	66.55	69.16	73.56	77.14
FMix	0.2	ArXiv	63.47	65.08	69.96	74.08	77.19
ResizeMix	1.0	CVMJ2023	63.17	65.87	69.50	73.88	77.42
PuzzleMix	1.0	ICML2020	65.81	67.83	70.12	74.26	77.54
AutoMix	2.0	ECCV2022	67.33	70.72	70.50	74.52	77.91
AdAutoMix	1.0	ICLR2024	69.19	72.89	70.86	74.82	78.04

---

CUB-200, FGVC-Aircraft and Standford Cars

Name	alpha	Confrence	CUB R18	CUB R50	FGVC R18	FGVC RX50	Cars R18	Cars RX50
Vanilla	-		77.68	82.38	80.23	85.1	86.32	90.15
MixUp	1.0	ICLR2018	78.39	82.98	79.52	85.18	86.27	90.81
CutMix	0.2	ICCV2019	78.40	83.17	78.84	84.55	87.48	91.22
ManifoldMixup	2.0	ICML2019	79.76	83.76	80.68	86.6	85.88	90.20
SaliencyMix	0.2	ICLR2021	77.95	82.02	80.02	84.31	86.48	90.60
FMix	0.2	ArXiv	77.28	83.34	79.36	86.23	87.55	90.90
ResizeMix	1.0	CVMJ2023	78.5	83.41	78.1	84.08	88.17	91.36
PuzzleMix	1.0	ICML2020	78.63	83.83	80.76	86.23	87.78	91.29
AutoMix	2.0	ECCV2022	79.87	83.88	81.37	86.72	88.89	91.38
AdAutoMix	1.0	ICLR2024	80.88	84.57	81.73	87.16	89.19	91.59

If you want see more results, please check this Experiments.md.

---

😉 Citation

If you feel that our work has contributed to your research, please cite it, 🥰 and please don`t forget to cite OpenMixup if you use this project ! 🤗 Thanks.

@inproceedings{iclr2024adautomix,
      title={Adversarial AutoMixup},
      author={Huafeng Qin and Xin Jin and Yun Jiang and Mounim A. El-Yacoubi and Xinbo Gao},
      booktitle={International Conference on Learning Representations},
      year={2024},
}

@article{li2022openmixup,
  title = {OpenMixup: A Comprehensive Mixup Benchmark for Visual Classification},
  author = {Siyuan Li and Zedong Wang and Zicheng Liu and Di Wu and Cheng Tan and Stan Z. Li},
  journal = {ArXiv},
  year = {2022},
  volume = {abs/2209.04851}
}

---

💬 Other info

🏫 Chongqing Intelligence Perception and Block Chain Technology Key Lab, Biometric Identification Group

Head of Laboratory: Prof. Huafeng Qin. Email: qinhuafengfeng@163.com

---

📖 Some Publications

• Finger Vein & Palm Vein Recognition
  • Finger-Vein Quality Assessment by Representation Learning from Binary Images, 2015, ICONIP PDF
  • Deep Representation for Finger-Vein Image-Quality Assessment, 2017, TCSVT PDF
  • Deep Representation-based Feature Extraction and Recovering for Finger-Vein Verification, 2017, TIFS PDF
  • Finger-Vein Quality Assessment by Joint Representation Learning from Grayscale and Binary Images, 2018, ICPRAI PDF
  • Multi-Scale and Multi-Direction GAN for CNN-based Single Palm-Vein Identification, 2021, TIFS PDF
  • Local Attention Transformer-based Full-View Finger-Vein Identification, 2022, TCSVT PDF
  • Transformer based Defense GAN Against Palm-Vein Adversarial Attacks, 2023, TIFS PDF
  • Label Enhancement-Based Multiscale Transformer for Palm-Vein Recognition, 2023, TIM PDF
  • Adversarial Learning-based Data Augmentation for Palm-vein Identification, 2023, TCSVT PDF
  • AG-NAS: An Attention GRU-based Neural Architecture Search for Finger-Vein Recognition, 2023, TIFS PDF
  • Adversarial Masking Contrastive Learning for Vein Recognition, 2024, arXiv PDF
• Computer Vision
  • Adaptive Deep Feature Fusion for Continuous Authentication with Data Augmentation, 2022, TMC PDF
  • Memory-Augmented Autoencoder based Continuous Authentication on Smartphones with Conditional Transformer GANs, 2023, TMC PDF
  • Adversarial AutoMixup, 2024, ICLR (spotlight 5%) PDF
• Eye Movement Recognition
  • EmMixformer: Mix Transformer for Eye Movement Recognition, 2024, arXiv PDF

---

📝 Interest

  If you are interested in vein research (palm veins, finger veins, etc.), why not contact us and we will be happy to discuss the research or questions with you.
  Of course, we're also trying to research how we can use AI to predict and detect diseases, potentially Alzheimer's, Parkinson's, etc., but of course, this is new research for us, so we're still figuring it out.
  Email: 158398730@qq.com