Skip to content

A list of papers that studies out-of-distribution (OOD) detection and misclassification detection (MisD)

License

Notifications You must be signed in to change notification settings

Impression2805/Awesome-Failure-Detection

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 

Repository files navigation

Awesome Failure Detection / Reliable Prediction

Failure Detection is a machine learning problem, which aims to detect out-of-distribution (OOD) and misclassified samples based on reliable confidence estimation. This topic is important for risk-sensitive applications (e.g., autonomous driving, clinical decision making), and is gathering much attention in the research community.

Here, we provide a list of papers that studies OOD detection and misclassification detection (MisD).

Misclassification Detection / Selective Classification / Failure Prediction

  • Unified Out-Of-Distribution Detection: A Model-Specific Perspective (ICCV 2023) [paper]
  • OpenMix: Exploring Outlier Samples for Misclassification Detection (CVPR 2023) [paper] [code]
  • Towards More Reliable Confidence Estimation (TPAMI 2023) [paper]
  • Failure Detection for Motion Prediction of Autonomous Driving: An Uncertainty Perspective (ICRA 2023) [paper]
  • A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification (ICLR 2023) [paper] [code]
  • What can we learn from the selective prediction and uncertainty estimation performance of 523 imagenet classifiers (ICLR 2023) [paper] [code]
  • Towards Better Selective Classification (ICLR 2023) [paper] [code]
  • AUC-based Selective Classification (AISTATS 2023) [paper]
  • Failure Detection in Medical Image Classification: A Reality Check and Benchmarking Testbed (TMLR 2022) [paper] [code]
  • Rethinking Confidence Calibration for Failure Prediction (ECCV 2022) [paper] [code]
  • Improving the Reliability for Confidence Estimation (ECCV 2022) [paper]
  • Trust, but Verify: Using Self-Supervised Probing to Improve Trustworthiness (ECCV 2022) [paper]
  • Confidence estimation via auxiliary models (TPAMI 2021) [paper]
  • Learning to predict trustworthiness with steep slope loss (NeurIPS 2021) [paper] [code]
  • Confidence-Aware Learning for Deep Neural Networks (ICML 2020) [paper] [code]
  • Self-Adaptive Training: beyond Empirical Risk Minimization (NeurIPS 2020) [paper] [code]
  • Selectivenet: A deep neural network with an integrated reject option (ICML 2019) [paper]
  • Addressing Failure Prediction by Learning Model Confidence (NeurIPS 2019) [paper] [code]
  • Deep Gamblers: Learning to Abstain with Portfolio Theory (NeurIPS 2019) [paper] [code]
  • To Trust Or Not To Trust A Classifier (NeurIPS 2018) [paper] [code]
  • Selective classification for deep neural networks (NeurIPS 2017) [paper]
  • A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (ICLR 2017) [paper]
  • An Optimum Character Recognition System Using Decision Functions (IRE Transactions on Electronic Computers 1957) [paper]

OOD Detection

2023

  • Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection (Arxiv 2023) [paper]
  • WDiscOOD: Out-of-Distribution Detection via Whitened Linear Discriminant Analysis (ICCV 2023) [paper]
  • DIFFGUARD: Semantic Mismatch-Guided Out-of-Distribution Detection Using Pre-Trained Diffusion Models (ICCV 2023) [paper] [code]
  • Understanding the Feature Norm for Out-of-Distribution Detection (ICCV 2023) [paper]
  • Nearest Neighbor Guidance for Out-of-Distribution Detection (ICCV 2023) [paper]
  • Residual Pattern Learning for Pixel-Wise Out-of-Distribution Detection in Semantic Segmentation (ICCV 2023) [paper] [code]
  • Unified Out-Of-Distribution Detection: A Model-Specific Perspective (ICCV 2023) [paper]
  • Revisit PCA-based technique for Out-of-Distribution Detection (ICCV 2023) [paper] [code]
  • Hierarchical Visual Categories Modeling: A Joint Representation Learning and Density Estimation Framework for Out-of-Distribution Detection (ICCV 2023) [paper]
  • Out-of-Distribution Detection for Monocular Depth Estimation (ICCV 2023) [paper]
  • Simple and Effective Out-of-Distribution Detection via Cosine-based Softmax Loss (ICCV 2023) [paper]
  • Unsupervised Out-of-Distribution Detection with Diffusion Inpainting (ICML 2023) [paper]
  • Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection (ICML 2023) [paper] [code]
  • In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation (ICML 2023) [paper] [code]
  • Hybrid Energy Based Model in the Feature Space for Out-of-Distribution Detection (ICML 2023) [paper] [code]
  • Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability (ICML 2023) [paper] [code]
  • Detecting Out-of-distribution Data through In-distribution Class Prior (ICML 2023) [paper] [code]
  • Concept-based Explanations for Out-of-Distribution Detectors (ICML 2023) [paper] [code]
  • Decoupling MaxLogit for Out-of-Distribution Detection (CVPR 2023) [paper] [code]
  • Block Selection Method for Using Feature Norm in Out-of-Distribution Detection (CVPR 2023) [paper] [code]
  • LINe: Out-of-Distribution Detection by Leveraging Important Neurons (CVPR 2023) [paper] [code]
  • Uncertainty-Aware Optimal Transport for Semantically Coherent Out-of-Distribution Detection (CVPR 2023) [paper] [code]
  • Rethinking Out-of-Distribution (OOD) Detection: Masked Image Modeling Is All You Need (CVPR 2023) [paper] [code]
  • Balanced Energy Regularization Loss for Out-of-Distribution Detection (CVPR 2023) [paper]
  • Detection of Out-of-Distribution Samples Using Binary Neuron Activation Patterns (CVPR 2023) [paper] [code]
  • GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection (CVPR 2023) [paper] [code]
  • OpenMix: Exploring Outlier Samples for Misclassification Detection (CVPR 2023) [paper] [code]
  • Packed-Ensembles for Efficient Uncertainty Estimation (ICLR 2023) [paper] [code]
  • A framework for benchmarking Class-out-of-distribution detection and its application to ImageNet (ICLR 2023) [paper] [code]
  • Out-of-Distribution Detection based on In-Distribution Data Patterns Memorization with Modern Hopfield Energy (ICLR 2023) [paper] [code]
  • Extremely Simple Activation Shaping for Out-of-Distribution Detection (ICLR 2023) [paper] [code]
  • The Tilted Variational Autoencoder: Improving Out-of-Distribution Detection (ICLR 2023) [paper]
  • Out-of-distribution Detection with Implicit Outlier Transformation (ICLR 2023) [paper] [code]
  • Energy-based Out-of-Distribution Detection for Graph Neural Networks (ICLR 2023) [paper] [code]
  • How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection? (ICLR 2023) [paper] [code]
  • Harnessing Out-Of-Distribution Examples via Augmenting Content and Style (ICLR 2023) [paper]
  • Fake It Until You Make It : Towards Accurate Near-Distribution Novelty Detection (ICLR 2023) [paper] [code]
  • Non-parametric Outlier Synthesis (ICLR 2023) [paper] [code]
  • Out-of-Distribution Detection and Selective Generation for Conditional Language Models (ICLR 2023) [paper]
  • Turning the Curse of Heterogeneity in Federated Learning into a Blessing for Out-of-Distribution Detection (ICLR 2023) [paper] [code]

2022

  • Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD Training Data Estimate a Combination of the Same Core Quantities (ICML 2022) [paper] [code]
  • POEM: Out-of-Distribution Detection with Posterior Sampling (ICML 2022) [paper] [code]
  • Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognition (ICML 2022) [paper] [code]
  • Scaling Out-of-Distribution Detection for Real-World Settings (ICML 2022) [paper] [code]
  • Mitigating Neural Network Overconfidence with Logit Normalization (ICML 2022) [paper] [code]
  • Out-of-Distribution Detection with Deep Nearest Neighbors (ICML 2022) [paper] [code]
  • Training OOD Detectors in their Natural Habitats (ICML 2022) [paper] [code]
  • Out-of-Distribution Detection via Conditional Kernel Independence Model (NeurIPS 2022) [paper] [code]
  • Your Out-of-Distribution Detection Method is Not Robust! (NeurIPS 2022) [paper] [code]
  • Boosting Out-of-distribution Detection with Typical Features (NeurIPS 2022) [paper] [code]
  • RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection (NeurIPS 2022) [paper]
  • Provably Adversarially Robust Detection of Out-of-Distribution Data (Almost) for Free (NeurIPS 2022) [paper] [code]
  • Watermarking for Out-of-distribution Detection (NeurIPS 2022) [paper] [code]
  • Is Out-of-Distribution Detection Learnable? (NeurIPS 2022) [paper]
  • RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy & Out-of-Distribution Robustness (NeurIPS 2022) [paper] [code]
  • Out-of-Distribution Detection with An Adaptive Likelihood Ratio on Informative Hierarchical VAE (NeurIPS 2022) [paper]
  • GraphDE: A Generative Framework for Debiased Learning and Out-of-Distribution Detection on Graphs (NeurIPS 2022) [paper] [code]
  • Density-driven Regularization for Out-of-distribution Detection (NeurIPS 2022) [paper]
  • Delving into Out-of-Distribution Detection with Vision-Language Representations (NeurIPS 2022) [paper] [code]
  • Beyond Mahalanobis Distance for Textual OOD Detection (NeurIPS 2022) [paper]
  • SIREN: Shaping Representations for Detecting Out-of-Distribution Objects (NeurIPS 2022) [paper] [code]
  • Out-of-distribution Detection with Boundary Aware Learning (ECCV 2022) [paper]
  • Out-of-Distribution Detection with Semantic Mismatch under Masking (ECCV 2022) [paper] [code]
  • Data Invariants to Understand Unsupervised Out-of-Distribution Detection (ECCV 2022) [paper]
  • Out-of-Distribution Identification: Let Detector Tell Which I Am Not Sure (ECCV 2022) [paper]
  • Deep Hybrid Models for Out-of-Distribution Detection (CVPR 2022) [paper]
  • Rethinking Reconstruction Autoencoder-Based Out-of-Distribution Detection (CVPR 2022) [paper]
  • ViM: Out-of-Distribution With Virtual-Logit Matching (CVPR 2022) [paper] [code]
  • Neural Mean Discrepancy for Efficient Out-of-Distribution Detection (CVPR 2022) [paper]
  • Unknown-Aware Object Detection: Learning What You Don’t Know from Videos in the Wild (CVPR 2022) [paper] [code]
  • Revisiting flow generative models for Out-of-distribution detection (ICLR 2022) [paper]
  • Igeood: An Information Geometry Approach to Out-of-Distribution Detection (ICLR 2022) [paper]
  • A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Networks (ICLR 2022) [paper]
  • VOS: Learning What You Don't Know by Virtual Outlier Synthesis (ICLR 2022) [paper] [code]

2021

  • Understanding Failures in Out-of-Distribution Detection with Deep Generative Models (ICML 2021) [paper]
  • Exploring the Limits of Out-of-Distribution Detection (NeurIPS 2021) [paper]
  • STEP: Out-of-Distribution Detection in the Presence of Limited In-Distribution Labeled Data (NeurIPS 2021) [paper]
  • Locally Most Powerful Bayesian Test for Out-of-Distribution Detection Using Deep Generative Models (NeurIPS 2021) [paper]
  • Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection (NeurIPS 2021) [paper]
  • ReAct: Out-of-distribution Detection With Rectified Activations (NeurIPS 2021) [paper] [code]
  • Entropy Maximization and Meta Classification for Out-of-Distribution Detection in Semantic Segmentation (ICCV 2021) [paper]
  • Semantically Coherent Out-of-Distribution Detection (ICCV 2021) [paper] [code]
  • Triggering Failures: Out-of-Distribution Detection by Learning From Local Adversarial Attacks in Semantic Segmentation (ICCV 2021) [paper] [code]
  • MOOD: Multi-Level Out-of-Distribution Detection (CVPR 2021) [paper]
  • MOS: Towards Scaling Out-of-Distribution Detection for Large Semantic Space (CVPR 2021) [paper]
  • Out-of-Distribution Detection Using Union of 1-Dimensional Subspaces (CVPR 2021) [paper] [code]
  • Multiscale Score Matching for Out-of-Distribution Detection (ICLR 2021) [paper] [code]
  • SSD: A Unified Framework for Self-Supervised Outlier Detection (ICLR 2021) [paper] [code]

Early Works

  • Energy-based Out-of-distribution Detection (NeurIPS 2020) [paper] [code]
  • CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances (NeurIPS 2020) [paper] [code]
  • Why Normalizing Flows Fail to Detect Out-of-Distribution Data (NeurIPS 2020) [paper] [code]
  • Likelihood Ratios for Out-of-Distribution Detection (NeurIPS 2019) [paper]
  • Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem (CVPR 2019) [paper]
  • Deep Anomaly Detection with Outlier Exposure (ICLR 2019) [paper] [code]
  • A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (NeurIPS 2018) [paper] [code]
  • Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks (ICLR 2018) [paper]
  • Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples (ICLR 2018) [paper] [code]
  • Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (NeurIPS 2018) [paper]
  • Predictive Uncertainty Estimation via Prior Networks (NeurIPS 2018) [paper] [code]
  • Out-of-Distribution Detection using Multiple Semantic Label Representations (NeurIPS 2018) [paper]
  • A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (ICLR 2017) [paper]