Machine Learning for iOS

Tools and resources to create really smart iOS applications.

Last Update: Dec 28, 2015.

Curated list of resources for iOS developers in following topics:

• Machine Learning,
• Artificial Intelligence,
• Natural Language Processing (NLP),
• Computer Vision,
• General-Purpose GPU Computing (GPGPU),
• Data Visualization,
• Bioinformatics

Most of the de-facto standard tools from domains listed above are written in iOS-unfiendly languages (Python/Java/R/Matlab) so find something appropriate for your iOS application may be a difficult task. This list consists mainly of native libraries written in Objective-C, Swift and ports of C, C++, JavaScript libraries or libs which can be easily ported to iOS. Also links to some relevant web APIs, blog posts, videos and learning materials included.

Pull-requests are welcome here!

Where to learn about machine learning and related topics in general

• Courserra course on machine learning from Andrew Ng.
• Machine learning playlist on Youtube.
• Free online interactive book "Neural Networks and Deep Learning".
• "Natural Language Processing with Python" - free online book.
• Heterogeneous Parallel Programming course.
• Immersive Linear Algebra interactive book by J. Ström, K. Åström, and T. Akenine-Möller.
• Videos from Deep Learning Summer School, Montreal 2015.
• Deep Learning for Perception by Virginia Tech, Electrical and Computer Engineering, Fall 2015: ECE 6504
• Probabilistic Programming & Bayesian Methods for Hackers - An intro to Bayesian methods and probabilistic programming from a computation/understanding-first, mathematics-second point of view.
• CAP 5415 - Computer Vision by UCF
• CS224d: Deep Learning for Natural Language Processing by Stanford
• Machine Learning: 2014-2015 Course materials by Oxford
• Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition.
• Deep Learning for Natural Language Processing (without Magic)

Web APIs

• IBM Watson - Enable Cognitive Computing Features In Your App Using IBM Watson's Language, Vision, Speech and Data APIs.
  • Introducing the (beta) IBM Watson iOS SDK
• AlchemyAPI - Semantic Text Analysis APIs Using Natural Language Processing. Now part of IBM Watson.
• Microsoft Project Oxford
• Google Prediction engine
  • Objective-C API
• Google Translate API
• Google Cloud Vision API
• Amazon Machine Learning - Amazon ML is a cloud-based service for developers. It provides visualization tools to create machine learning models. Obtain predictions for application using APIs.
  • iOS developer guide.
  • iOS SDK
• PredictionIO - opensource machine learning server for developers and ML engineers. Built on Apache Spark, HBase and Spray.
  • Swift SDK
  • Tapster iOS Demo - This demo demonstrates how to use the PredictionIO Swift SDK to integrate an iOS app with a PredictionIO engine to make your mobile app more interesting.
  • Tutorial on using Swift with PredictionIO.
• Wit.AI - NLP API
• Yandex SpeechKit Text-to-speech and speech-to-text for Russian language. iOS SDK available.
• Abbyy OCR SDK
• Clarifai - deep learning web api for image captioning. iOS starter project
• MetaMind - deep learning web api for image captioning.
• Api.AI - Build intelligent speech interfaces for apps, devices, and web

General-Purpose Machine Learning Libraries

• FANN - Fast Artifical Neural Network library; an implementation of neural networks.

• lbimproved - DTW + kNN in C

• Shark - provides libraries for the design of adaptive systems, including methods for linear and nonlinear optimization (e.g., evolutionary and gradient-based algorithms), kernel-based algorithms and neural networks, and other machine learning techniques. CocoaPods. Official site

• YCML - A Machine Learning framework for Objective-C and Swift (OS X / iOS). The following algorithms are currently available: Gradient Descent Backpropagation, Resilient Backpropagation (RProp), Extreme Learning Machines (ELM), Forward Selection using Orthogonal Least Squares (for RBF Net), also with the PRESS statistic, Binary Restricted Boltzmann Machines (CD & PCD, Untested!). YCML also contains some optimization algorithms as support for deriving predictive models, although they can be used for any kind of problem: Gradient Descent (Single-Objective, Unconstrained), RProp Gradient Descent (Single-Objective, Unconstrained), NSGA-II (Multi-Objective, Constrained).

  • Github.

• Swix - Swift implementation of NumPy.

• Brain - Neural networks in JavaScript. Unmaintained.

• April-ANN - APRIL-ANN toolkit (A Pattern Recognizer In Lua with ANNs). This toolkit incorporates ANN algorithms (as dropout, stacked denoising auto-encoders, convolutional NNs), with other pattern recognition methods as HMMs among others. Additionally, in experimental stage, it is possible to perform automatic differentiation, for advanced ML research. Potentially can be ported for iOS.

  • Official Site.

• Recommender - A C library for product recommendations/suggestions using collaborative filtering (CF).

• SNNeuralNet - A neural network library for Objective-C based on brain.js, for iOS and Mac OS X.

• MLPNeuralNet - Fast multilayer perceptron neural network library for iOS and Mac OS X. MLPNeuralNet predicts new examples by trained neural network. It is built on top of the Apple's Accelerate Framework, using vectorized operations and hardware acceleration if available.

• MAChineLearning - An Objective-C multilayer perceptron library, with full support for training through backpropagation. Implemented using vDSP and vecLib, it's 20 times faster than its Java equivalent. Includes sample code for use from Swift.

• BrainCore - simple but fast neural network framework written in Swift. It uses Upsurge and the underlying Accelerate framework to be as fast as possible.

• Swift-AI - 3-layer NN.

• EERegression - General purpose multivaritate and quadratic Regression library for Swift 2.1

• Kalvar Lin's libraries

  • ios-BPN-NeuralNetwork - 3 layers NN + Back Propagation.
  • ios-Multi-Perceptron-NeuralNetwork / KRANN - Multi-perceptrons neural NN based on Back Propagation NN.
  • ios-KRHebbian-Algorithm
  • ios-KRKmeans-Algorithm - K-Means clustering and classification.
  • ios-KRFuzzyCMeans-Algorithm - Fuzzy C-Means fuzzy clustering / classification algorithm.
  • ios-KRGreyTheory - Grey Theory implementation (this one?).
  • ios-KRKNN - kNN implementation.
  • ios-ML-Recommendation-System

Deep Learning

• Torch-iOS - Torch port for iOS. Torch is a scientific computing framework with wide support for machine learning algorithms. One of the most popular deep learning frameworks.
• Caffe - A deep learning framework developed with cleanliness, readability, and speed in mind. GitHub. [BSD]
  • iOS port
  • C++ examples: Classifying ImageNet, Extracting Features
  • Caffe iOS sample
• Deep Belief SDK - The SDK for Jetpac's iOS Deep Belief image recognition framework
• Convnet.js - ConvNetJS is a Javascript library for training Deep Learning models by Andrej Karpathy. GitHub
  • ConvNetSwift - Swift port [work in progress].
• mxnet - MXNet is a deep learning framework designed for both efficiency and flexibility.
  • Deploying mxnet to smartphone
• TensorFlow - an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow Will have iOS support in the future.
• DeepLearningKit - Open Source Deep Learning Framework from Memkite for Apple's tvOS, iOS and OS X.

AI

• Mendel - Genetic algorithms in Swift.

Game AI

• Introduction to AI Programming for Games
• dlib is a library which has many useful tools including machine learning.
• MicroPather is a path finder and A* solver (astar or a-star) written in platform independent C++ that can be easily integrated into existing code.
• Here is a list of some AI libraries suggested on OGRE3D website. Seems they are mostly written in C++.
• GameplayKit Programming Guide

Natural Language Processing

• Parsimmon
• NSLinguisticTagger

Computational Semantics

• Word2Vec - Original C implementation of Word2Vec Deep Learning algorithm. Works on iPhone like a charm.

Text Mining

• Twitter text - An Objective-C implementation of Twitter's text processing library. The library includes methods for extracting user names, mentions headers, hashtags, and more – all the tweet specific language syntax you could ever want.

Speech Recognition (TTS) and Generation (STT)

• TLSphinx, Tutorial
• MVSpeechSynthesizer
• OpenEars™: free speech recognition and speech synthesis for the iPhone - OpenEars™ makes it simple for you to add offline speech recognition and synthesized speech/TTS to your iPhone app quickly and easily. It lets everyone get the great results of using advanced speech UI concepts like statistical language models and finite state grammars in their app, but with no more effort than creating an NSArray or NSDictionary.
  • Tutorial (Russian)

Computer Vision

• Vuforia Framework

  • Object detection with Vuforia tutorial. (Russian)

• OpenCV - Open Source Computer Vision Library. [BSD]

  • OpenCV crash course
  • OpenCVSwiftStitch
  • Tutorial: using and building openCV on iOS devices
  • ML algorithms:
  • Normal Bayes Classifier
  • K-Nearest Neighbour Classifier
  • Support Vector Machines
  • Expectation - Maximisation
  • Decision Tree
  • Random Trees Classifier
  • Extremely randomised trees Classifier
  • Boosted tree classifier
  • Gradient Boosted Trees
  • Multi-Layer Perceptrons
  • CV:
  • Haar-like Object Detection
  • Latent SVM Object Detection
  • HOG (Histogram-of-Oriented-Gradients) Descriptor and Object Detector
  • Data Matrix detection
  • LINE template matching algorithm

• trackingjs - Object tracking in JS

Text Recognition (OCR)

• Tesseract
  • Install and Use Tesseract on iOS
  • tesseract-ios-lib
  • tesseract-ios
  • Tesseract-OCR-iOS
  • OCR-iOS-Example
• ocrad.js - JS OCR

General Math

• Surge from Mattt
• YCMatrix - A flexible Matrix library for Objective-C and Swift (OS X / iOS)
• Eigen - A high-level C++ library of template headers for linear algebra, matrix and vector operations, numerical solvers and related algorithms. [MPL2]

GPGPU

Articles

• OpenCL for iOS - just a test.

• Exploring GPGPU on iOS.

  • Article
  • Code

• GPU-accelerated video processing for Mac and iOS. Article.

• Concurrency and OpenGL ES - Apple programming guide.

• OpenCV on iOS GPU usage - SO discussion.

Metal

• Simon's Gladman (aka flexmonkey) blog
  • Talk on iOS GPU programming with Swift and Metal at Realm Altconf.
  • The Supercomputer In Your Pocket: Metal & Swift - a video from the Swift Summit Conference 2015
  • https://github.com/FlexMonkey/MetalReactionDiffusion
  • https://github.com/FlexMonkey/ParticleLab
• Memkite blog - startup intended to create deep learning library for iOS.
  • Swift and Metal example for General Purpose GPU Processing on Apple TVOS 9.0
  • Data Parallel Processing with Swift and Metal on GPU for iOS8
  • Example of Sharing Memory between GPU and CPU with Swift and Metal for iOS8
• Metal by Example blog
• objc-io article on Metal

GPU-accelerated libraries

• GPUImage is a GPU-accelerated image processing library.

Data Visualization

• iOS-Charts
• Core Plot
• Awesome iOS charts
• JTChartView
• VTK
  • VTK in action

Bioinformatics

• BioJS - a set of tools for bioinformatics in the browser. BioJS builds a infrastructure, guidelines and tools to avoid the reinvention of the wheel in life sciences. Community builds modules than can be reused by anyone.
• BioCocoa - BioCocoa is an open source OpenStep (GNUstep/Cocoa) framework for bioinformatics written in Objective-C. [Dead project].
• iBio - A Bioinformatics App for iPhone.

Big Data

• HDF5Kit - This is a Swift wrapper for the HDF5 file format. HDF5 is used in the scientific comunity for managing large volumes of data. The objective is to make it easy to read and write HDF5 files from Swift, including playgrounds.

Opensource Applications

• DeepDreamer - Deep Dream application
• DeepDreamApp - Deep Dream Cordova app.

Other Lists

• Awesome Machine Learning
• Machine Learning Courses
• Awesome Data Science
• Awesome Computer Vision
• Speech and language processing
• The Rise of Chat Bots: Useful Links, Articles, Libraries and Platforms by Pavlo Bashmakov.

People to follow in iOS machine learning and related topics

(In alphabet order).

• Alejandro - BrainCore and Caffe for iOS author. Blog. @aleph7.
• Simon Gladman - Swift and Metal enthusiast and blogger. Blog. @flexmonkey.
• Andrej Karpathy - PhD student at Stanford. ConvNetJS author. Blog. @karpathy
• Kalvar Lin - wrote a set of small machine learning libraries for iOS. @ilovekalvar.
• Memkite team - Memkite is a Deep Learning framework for iOS that can be used to support Artificial Intelligence (AI) in apps. @memkite.
  • Torbjørn Morland @torbmorland.
  • Amund Tveit @atveit