Skip to content

ePillID Dataset: A Low-Shot Fine-Grained Benchmark for Pill Identification (CVPR 2020 VL3)

License

Notifications You must be signed in to change notification settings

usuyama/ePillID-benchmark

Repository files navigation

ePillID Benchmark Open In Colab

PWC

This repository contains data and code for ePillID - a benchmark for developing and evaluating computer vision models for pill identification. The ePillID benchmark is designed as a low-shot fine-grained benchmark, reflecting real-world challenges for developing image-based pill identification systems.

The characteristics of the ePillID benchmark include:

  • Reference and consumer images: The reference images are taken with controlled lighting and backgrounds, and with professional equipment. The consumer images are taken with real-world settings including different lighting, backgrounds, and equipment. For most of the pills, one image per side (two images per pill type) is available from the NIH Pillbox dataset.

Examples of reference and consumer images in ePillID

  • Low-shot and fine-grained setting: 13k images representing 9804 appearance classes (two sides for 4902 pill types). For most of the appearance classes, there exists only one reference image, making it a challenging low-shot recognition setting.

ePillID data distribution

  • The best performing model at the time of publication is a multi-head metric learning approach. Following a CNN encoder, a bilinear transformation layer is used to represent fine-grained embeddings.

Model overview

Please refer to our paper for more details:

@inproceedings{usuyama2020epillid,
  title={ePillID Dataset: A Low-Shot Fine-Grained Benchmark for Pill Identification},
  author={Usuyama, Naoto and Delgado, Natalia Larios and Hall, Amanda K and Lundin, Jessica},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
  year={2020}
}

Data

Downloadable from the releases page.

Docker

Docker image https://hub.docker.com/r/naotous/pytorch-image

sudo nvidia-docker pull naotous/pytorch-image:py36torch041-legacy

Train and evaluate models

sudo nvidia-docker run --ipc=host -it -v $HOME:$HOME naotous/pytorch-image:py36torch041-legacy /bin/bash

use -v mount option to access folders from Docker containers

source activate py36

python train_cv.py --appearance_network resnet18 --max_epochs 10 --data_root_dir /home/naotous/ePillID_data

NOTE: max_epochs 10 for quick testing

Train and evaluate models on Azure ML

python azureml_submit_runs.py ePillID_0430

Notes and acknowledgments

The ePillID dataset includes customized data from two public NIH NLM databases. Please refer to their websites for original data:

For bilinear pooling implementations, we forked fast-MPN-Cov GitHub repo.

Please consider citing their work as well when you find our work useful in your research.

Disclaimer

This dataset and software are released for research purposes only.