Build a Custom Food Image Recognition Classifier in 5 Steps

For this article, we hired Becca Miller, a freelance software developer and technical writer, to build a custom food image classifier with Nyckel image classification. Becca details her experience below and shares how you can build the image classifier yourself.

Food image recognition is an application of AI that uses computer vision and deep learning to quickly analyze, identify, and label food items in images. The technology is especially useful in the health and nutrition industries, including for companies that develop food-tracking applications or organizations that conduct nutritional research.

For example, a company with a food-tracking app can integrate a food image recognition API into their product to simplify food logging for their users, a notoriously tedious and time-consuming task. Users simply photograph their meals, and then the classifier can quickly identify the nutritional values of the food items, such as calories or nutrient composition. Nutrition tracking apps like MyFitnessPal can use food image recognition to offer their users a quicker, more effective way to monitor dietary habits.

How does food recognition work?

Food recognition uses machine learning models trained on large databases of food images to detect what food is displayed in a photo. Trained models often use image classification to analyze the distinct visual features of the foods, such as their shapes, colors, and textures and then categorize them into predefined classes or labels, like differentiating a hot dog from a hamburger.

Some food recognition APIs use object detection in addition to image classification. Object detection is especially useful when you want to count the number of similar food items in a single photo. For example, airports often have checkout kiosks where the customer places all the items they’re purchasing under a camera. The camera then quickly detects the items being purchased (e.g., two water bottles and a Snickers bar) and charges the customer accordingly.

How to easily build a custom food image recognition classifier in 5 steps

Creating and deploying a custom food image recognition function to label images of food is simple and easy with Nyckel. In this article, I’ll demonstrate how to build a function using Kaggle’s Food-101-Tiny dataset (a subset of the larger Food 101 dataset). The Food-101-Tiny dataset includes 2,000 images across 10 types of foods, with each image labeled by food type. In this specific example, I’ll create an image classification function to divide the images into two food categories: tiramisu or cannoli. However, you could create a Nyckel classifier for all 10 food types from the dataset, or you could use your own dataset and include up to 1,000 different labels.

I’ll assign a single label to each image, a technique that works well for images that predominantly feature a single food item. If your images will include multiple food items per image, you should instead use Nyckel’s image tagging function. Image tagging supports multi-label classification, which allows you to assign multiple labels to a single image. For example, an image of a breakfast plate featuring eggs, toast, and bacon would receive three distinct labels, each corresponding to a different food item.

Let’s jump into the 5 steps:

1. Collect training data

When building your own custom classifier, it’s important to start by determining what foods your users will need to classify, so you can gather annotated data samples for each label/food. We recommend at least 100 annotated samples per label. For this example, we use the 150 tiramisu images and 150 cannoli images included in the Food-101-Tiny dataset.

To ensure good model performance, try to collect a diverse array of images featuring food in various environments, including scenarios where the food isn’t perfectly displayed. This image variety will enable your image classifier to learn to recognize your food under a wide range of conditions.

2. Upload your data to Nyckel

To upload your data to Nyckel, you first need to create an image classification function. This is as simple as setting the input to “image” and the output to “classify.” Next, you can import your training data into Nyckel’s platform.

Nyckel lets you bulk-upload and bulk-label your images, simplifying the data annotation process. You can efficiently upload your data by selecting images in batch sizes (limited to 1,000 images per batch) and assigning labels accordingly. In this case, I simply saved the tiramisu and cannoli images into separate folders (e.g., “tiramisu” and “cannoli”) and then added the relevant label when I uploaded the folder of images into Nyckel.

After the data is uploaded and annotated, Nyckel immediately starts training your model. As the model trains, you’ll see an indicator on the side of your screen telling you how confident Nyckel is in its model predictions. This immediate feedback allows you to monitor and assess the accuracy and performance of your food image classifier in real-time.

3. Try out the model

After the model finished training, I checked its performance and saw that it correctly classified 147 (98%) examples as tiramisu and 146 examples (97%) as cannoli. The model got the vast majority of images correct, only misclassifying 7 out of 300 examples.

To improve the model’s performance, I can always add more training data to our model, focusing on data samples that are trickier to decipher. You can also test out the model by uploading some test images to the “Test” tab in Nyckel’s UI.

4. Deploy the model into your systems using Nyckel’s API

Once the food image classifier is trained and ready to go, the next step is to deploy the model into your systems using Nyckel’s API. Nyckel designed this deployment process to be simple to ensure you can easily integrate your model into your existing infrastructure.

The model is run on optimized hardware, so you don’t have to worry about finding the right hardware/scaling hardware to meet your usage. Because Nyckel handles scaling demands and performance optimization, you can focus on getting the most out of your food image classifier while enjoying the benefits of a high-performance food detection API.

5. Monitor + improve model performance

Nyckel provides tools that make it easy to monitor and enhance your food image classifier’s performance. Its invoke capture feature makes the process of refining and fine-tuning your model quick and intuitive. This feature automatically captures random data and data with low confidence predictions from the model’s invokes, so that customers can annotate this data to retrain the model — ultimately helping improve the model’s performance.

Demo: My experience building a food image recognition function with Nyckel

Once I gathered the necessary data, I was able to build a custom food image recognition classifier with Nyckel in fewer than five minutes. The platform offered real-time feedback during model training, so I was able to quickly see how the model was doing. Since the platform also handles deployment, I could roll out the model immediately without fussing with hardware.

See the step-by-step process of how I trained the food image recognition function with Nyckel:

Easily classify food images with Nyckel

Nyckel provides the flexibility, accuracy, and adaptability that businesses need to quickly and easily create a custom food image classifier. If you’d like to take the next step, sign up for a free Nyckel account to create your classifier. Run into any issues along the way? Reach out to us at any time.