Skip to content

Python Implementation of Zero Shot Learning Algorithms (ALE, DeViSE, ESZSL, SAE, SJE) under ZSLGBU protocol

License

Notifications You must be signed in to change notification settings

mvp18/Popular-ZSL-Algorithms

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

32 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

This repository contains implementations of 5 classical zero-shot algorithms (SAE, ALE, SJE, ESZSL, and DeViSE) in the usual as well as the Generalized zero-shot learning (GZSL) settings using the Proposed Split and evaluation protocols (eg. Class-Averaged Top-1 Accuracy) outlined in Zero-Shot Learning - A Comprehensive Evaluation of the Good, the Bad and the Ugly (ZSLGBU) by Yongqin Xian, Christoph H. Lampert, Bernt Schiele, Zeynep Akata (TPAMI 2018).

This is the first public implementation of SAE, ALE, SJE and DeViSE under the ZSLGBU protocol. An existing implementation of ESZSL can be found here (thanks to @sbharadwajj). To this, I have added the GZSL functionality.

Reference Papers

The original papers corresponding to the 5 algorithms are:

[1] SAE (Semantic Autoencoder) - Semantic Autoencoder for Zero-Shot Learning. Elyor Kodirov, Tao Xiang, Shaogang Gong. CVPR, 2017.

[2] ALE (Attribute Label Embedding) - Label-Embedding for Image Classification. Zeynep Akata, Florent Perronnin, Zaid Harchaoui, Cordelia Schmid. TPAMI, 2016.

[3] SJE (Structured Joint Embedding) - Evaluation of Output Embeddings for Fine-Grained Image Classification. Zeynep Akata, Scott Reed, Daniel Walter, Honglak Lee, Bernt Schiele. CVPR, 2015.

[4] ESZSL - An embarrassingly simple approach to zero-shot learning. Bernardino Romera-Paredes, Philip H. S. Torr. ICML, 2015.

[5] DeViSE - DeViSE: A Deep Visual-Semantic Embedding Model. Andrea Frome*, Greg S. Corrado*, Jonathon Shlens*, Samy Bengio, Jeffrey Dean, Marc’Aurelio Ranzato, Tomas Mikolov. NIPS, 2013.

Data Splits

Dataset Total Images Attributes Class Split (Tr+Val+Ts) ZSL GZSL
tr val ts tr val tr+val ts seen ts unseen
SUN 14340 102 580+65+72 11600 1300 1440 9280 1040 10320 2580 1440
CUB 11788 312 100+50+50 5875 2946 2967 4702 2355 7057 1764 2967
AWA1 30475 85 27+13+10 16864 7926 5685 13460 6372 19832 4958 5685
AWA2 37322 85 27+13+10 20218 9191 7913 16187 7340 23527 5882 7913
aPY 15339 64 15+5+12 6086 1329 7924 4906 1026 5932 1483 7924

Code

Each folder above has its own README with running instructions, results and their comparisons with those reported in ZSLGBU. I have also put existing code references wherever relevant.

Setup

git clone https://github.com/mvp18/Popular-ZSL-Algorithms.git
cd Popular-ZSL-Algorithms
bash setup.sh

This downloads data (splits, Res101 features and class embeddings) corresponding to the Proposed Split for AWA1, AWA2, CUB, SUN and aPY. To know more about the individual files, refer to the README.txt file available inside xlsa17 folder.

TODOs

  • GZSL expts for ALE
  • GZSL expts for DeViSE
  • GZSL expts for SJE

Contributing

If you find any errors, kindly raise an issue and I will get back to you ASAP.

About

Python Implementation of Zero Shot Learning Algorithms (ALE, DeViSE, ESZSL, SAE, SJE) under ZSLGBU protocol

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published