Training Your Smartphone to Perform Visual Recognition via Deep Learning

By August 01, 2014
Jetpac Spotter

A startup pioneering image recognition has released an iOS software development kit and app based on Deep Learning architecture in order to popularise the use of Deep Learning.

San Francisco-based JetPac started out in 2012 by creating travel guides using an algorithm-generated selection of photos published on Facebook and Instagram. This ‘Deep Learning’ approach, which consists of applying a trained algorithm to a set of accumulated data, enabled the startup to put together travel guides by tracking the origin of photos which had not already been geolocated by the people who snapped them. The algorithm developed by the JetPac team enables recognition of the constant determining features of objects shown in the photos.  Since then, Technical Director Pete Warden has gone a step further, applying his Deep Learning knowhow to enable low-cost visual recognition on smartphones.

Visual, not facial, recognition

In April this year, JetPac released in open source an iOS software development kit called Deep Belief, plus a simple computer vision iPhone app called Spotter.  Using your iPhone camera to build a representative set of relevant images, you can now train the app to recognise chosen items and objects.  JetPac’s aim in putting the SDK for the iOS Deep Belief image recognition framework on code hosting platform Github is basically to explore the potential uses that independent developers can make of it. Jing Luo, a biologist from the University of California at Berkeley, USA, has already used the software for his Catalyst Frame Microscope project, which turns an iPhone into a portable microscope. Using the SDK, Catalyst Frame’s software programme can be trained to automatically identify different kinds of cells. At the moment the relative imprecision of the hardware used restricts visual recognition to general categories of objects or animals, such as cats, and the app cannot perform the sort of facial recognition which a soon-to-be-released app for Google Glass and smartphones is set to provide.  However, although JetPac’s Spotter and Google’s forthcoming app use different approaches, they both break with traditional data processing methods.  With these new algorithms, recognition/identification  is no longer carried out by detecting the fixed contours – edges, shape and colours –of an object on a pre-determined basis, but through a ‘convolutional neural network’ approach which scans for resemblances across a broad mass of data and images.

Matching items with accumulated data

An image which has been analysed is added to the machine memory. The algorithm then functions along the lines of a neural system, searching until it finds a relevant match between an object being studied and the data in the memory. As a technique spun off from Big Data analysis, Deep Learning is not just about object recognition but can also be used to enhance instantaneous machine translation and search engine crawlers. However, the usefulness of the Deep Learning approach depends entirely on having large volumes of accumulated data on which the algorithms can ‘train’.  For instance, Pete Warden admits that his app currently tends to associate any new object with a meal on a plate, because so many images of food were originally integrated into the algorithm during the early training phase!

Jetpac Spotter Deep Belief Object Recognition App from JetpacApp on Vimeo.

Legal mentions © L’Atelier BNP Paribas