What Is Labellio?

Labellio is a Web service that enables you to create your own original image recognition model in minutes without knowledge of programming nor image recognition. Labellio is intended to be used for PoC (Proof of Concept), i.e., you can use Labellio to see if your idea can be realized by image recognition.

Labellio lets you easily do trial-and-error by trying different combinations of input images and deep learning parameters. You can also chat with The Labellio Team to get help in improving your model or if you have any question on how to use Labellio.

Tutorial

Here we walk you through the complete flow of building your own classification model.

Login

Labellio supports OAuth sign-in with GitHub or Google. Go to Login page and click one of the buttons to be authenticated via OAuth mechanism.

Login

My Datasets

To create a image recognition model, you need a set of labeled images. In Labellio, we call these set of labeled image datasets. On this page, you can view the list of datasets you have created. Click the Create Dataset button to create a new dataset.

My Datasets

Create Dataset

On this page, you can create a new dataset. Enter a name in the Name field. Add labeled images into the dataset by clicking the +Add Data button. Then click on the Create button to create the dataset.

Create Dataset

How to add labeled images to datasets

In Labellio, you can add labeled images in the following four ways.

Add Data

Upload ZIP

You can upload a zip file that contains your training images. The zip file must follow these rules.

The folder structure should look as follows.

dog/dog_01.jpg
   /dog_02.jpg
   /dog_03.jpg
   /mydog.png
cat/cat_01.jpg
   /cat_02.jpg
   /cat_03.jpg
   /yourcat.png
bird/bird_01.jpg
    /bird_02.jpg
    /bird_03.jpg

Upload TSV

You can upload a TSV file containing a list of image urls with labels. Please note that:

The TSV file should look the following. Images without labels are ignored.

dog<tab>http://url.to/image
dog<tab>http://url.to/image
dog<tab>http://url.to/image
cat<tab>http://url.to/image
cat<tab>http://url.to/image
cat<tab>http://url.to/image

You can add labeled images by entering keywords to a search engine. First, choose either Bing or Flickr for the search engine.

Next, type keywords into the Labels input box. Keywords can be separated by tab, comma or by pressing the enter key. The number of images per keyword is 50 by default. This can be changed up to 100 in the Advanced Options.

The keyword is used as the label for those images, however, you can specify a different label by entering a colon : and the label name after the keyword like this keyword:label.

Finally click on the Add button to add the images to the dataset.

My Datasets

You can add labeled images from a dataset you have previously created. Just select the dataset and clicking the Add button.

Image Format

The following rules are applied regarding the format of images.

Download Datasets

You can download your dataset as a tar file by clicking on the Download Dataset button in the Browse Data tab in the dataset info page. If you are using non-ASCII characters for you labels, make sure to use a UTF-8 compatible program such as 7-Zip to untar the file.

My Model Parameters

To create a good model, it is import to do trial-and-error by tuning the deep learning parameters. However, since there are many parameters you can change, it is bothersome to set all the parameters each time you create a model.

In such as case, you can save a template of the parameters as My Model Parameters. Since this is just a convenience tool to make setting learning parameters easier, it is not mandatory to create a model parameter (default values are prepared, and all parameters are directly set for each model You can also enter by hand).

My Model Parameter

Create Model Parameter

On this page, you can customize learning parameters for creating models.

To create the model parameter, enter a value or choose one of the options for each field, and click the Create button at the bottom.

Create Model Parameter

Network

Pretrained Model

Iteration

Batch Size

Solver Type

Learning Rate

Learning Rate Policy

Fixed
Polynomial Decay
Exponential Decay

My Models

On this page, you can see the list of models you have created. Click the Create Model button to create a new model.

My Models

Create Model

On this page, you can create a model by following these steps: 1. Enter a name for the model in the Name field. 1. Click the Select Dataset button and select the dataset. Note that only datasets with two or more labels will appear. 1. Select the model parameters by clicking the Select Model Parameter button. * Note that his step is not mandatory because default values are provided. 1. Click on the Create button to start the training.

A snapshot of the model is created for each 1000 Iteration.

Create Model

Download Model

You can download your model by clicking the Download Model button in the Detail tab. Remember to select the iteration you want to download. Please note that this can only be done if you have subscribed the standard plan or the premium plan.

Download Model

How to use the model

You can use labellio_classify to use the model trained in Labellio locally.

Setting up labellio_classify

You can download labellio_classify from GitHub.

Using labellio_classify

First, place all of the images to classify in a directory.

$ ls images
label01.jpg  label02.jpg  label03.jpg  label04.jpg label05.jpg

Expand the downloaded model.

$ ls model-(iteration number)
model.ckpt-(iteration number).data-00000-of-00001 model.ckpt-(iteration number).index model.ckpt-(iteration number).meta label.txt checkpoint

Execute labellio_classify command providing the directory of the model, the directory of the images, and the name of the network name (mobilenet_v1, inception_v4 or resnet_v2_152).

$ python classify.py model_directory image_directory network_name
...
images/label01.jpg  {('label01', 0.81), ('label99', 0.1), ('label02', 0.05), ('label98', 0.03), ('label03', 0.01)}
images/label02.jpg  {('label02', 0.93), ('label01', 0.05), ('label03', 0.01), ('label99', 0.0), ('label04', 0.0)}
images/label03.jpg  {('label03', 0.74), ('label02', 0.2), ('label04', 0.02), ('label01', 0.02), ('label05": 0.01)}
images/label04.jpg  {('label04', 0.61), ('label03', 0.22), ('label05', 0.1), ('label02', 0.01), ('label06": 0.01)}
images/label05.jpg  {('label05', 0.56), ('label06', 0.2), ('label04', 0.05), ('label07', 0.03), ('label03": 0.03)}

Test

On the Test tab, you can see how the model classifies on a single image or a set of labeled images, and see its accuracy.

Test Model

Select the number of Iteration with theIteration button, and click the Add Test Data button to specify what images to classify. Upload Image: Upload a single image from your local computer of specify a url for a image. Upload ZIP: Upload a zip file containing images. Upload TSV: Upload a tsv file containing a list of urls for the images. My Datasets: Select one of your datasets.

Note that the same rules apply for the zip file, tsv files and image files as in creating a dataset.

Click the Download Result button to download the classification result as a tsv file.

Summary shows the Top-1 accuracy and the Top-5 accuracy. The Top-1 accuracy is the percentage of images where the true label matched the model classification. The Top-5 accuracy is the percentage of images where the true label was in one of the top 5 scores of the classification for that image.

Confusion Matrix is a table for checking the performance of the model. The row represents the true label, and the column represents the label that the model predicted. The integer in row A, column B is the number of images in the test set that is label A, but was predicted B by the model.

Deep Visualization

By clicking the Deep Visualization button in the classification result, you can see a visualization of the weight of each layer of the model when the model classifies the image. The jet color-map is used for the visualization.

Deep Visualization

Plans

Labellio offers these four plans. Check the TOS for more information about billing.

Free Light Standard Premium
Model Training 10 /month 10 /month 30 /month unlimited
Models Stored 5 20 50 100
Image Storage 1GB 10GB 50GB 100GB
Max Iteration 1,000 1,000 10,000 500,000
Network mobilenet_v1 mobilenet_v1, inception_v4, resnet_v2_152 mobilenet_v1, inception_v4, resnet_v2_152 mobilenet_v1, inception_v4, resnet_v2_152
Model Downloads - - 100 /month 100 /month
API Request 1,000 /month 5,000 /month 10,000 /month 100,000 /month

Disclaimer about Model Training

About Chat Support

How to use the API

Get the API key

On the models page, click on the API KEY button and a pop-up for creating the API key will appear.

get-api-key

Get Result

Specify the following parameters at https://www.labell.io/api/v1/predict/, and when requested, the json style response will be returned.

Parameter Required Description
Content-Type Yes must be multipart/form-data
Content-Length Yes The length of the request body in 8-bit bytes

Request Parameters

Parameter Type Required Description
key string Yes The API Key of the model
iteration integer Yes The iteration number to use for classification
file binary * The image to classify
url string * The URL of the image to classify

※ One of file or url is required. If both are provided, the file will be used.

Response Parameters

Parameter Type Description
name string The name of the file
result string Returns the status of the request:"Success", "Timeout Error", "Argument Error", "Server Error" or "Over Limit"
scores string The result of the classification is arranged as an array in the format "Label name": "Score". The length of the array is eqaul to the number of labels

Response example

``` { "name":"001.jpeg", "result":"Success", "scores":[ { "label01":0.81, "label02":0.10, "label03":0.05, "label04":0.03, "label05":0.01 } ] }