Customized Auto-Tagging for Digital Asset Management

TL; DR

• AI-driven auto-tagging of digital assets (like images) aids in search, discovery, and workflow routing.
• Generic auto-tagging is often frustrating because it adds noise and doesn’t provide the tags you are looking for.
• Custom models that are trained on a handful of manual tags can tag exactly what you are looking for.
• It is a common misconception that training custom models require tons of data, a team of AI experts, and a lot of time.
• Custom auto-tagging can be implemented in hours, not weeks, and without any AI expertise. I show you how you can do it in three minutes in this short video.

Auto-tagging in digital asset management

Digital asset management (DAM) platforms help curate and manage large amounts of assets like photos. Businesses use DAM platforms to manage images for e-commerce, marketing departments, product inventory, art, photography, and much more.

Tagging images with metadata is important in DAM for two reasons:

• It allows search and discovery. For example, if you want to find images of “yellow Jeep Wranglers on a beach,” one way to do that is through semantic search, which allows you to search a database of images using text. However, tagging the images with metadata about the contents of the image is another common way to enable this.
• It enables workflows based on the contents of the image. For example, if you want to kick off a process to remove the background in all images tagged with “sneakers.”

Even though tagging is important, it is infeasible to tag large numbers of images manually. As a result, DAM platforms and users have been turning to AI-based auto-tagging to tag their assets without much effort.

The problem with generic auto-tagging

Several APIs provide generic auto-tagging of images. For example, this one from Google, this one from AWS, and this one from Microsoft Azure. These APIs are easy to use and integrate into your DAM solution, but they have two big flaws:

• They don’t always give you the tag(s) you care about.
• They add noise by adding tags that you don’t care about.

Let’s look at an illustrative example. Say you manage images of construction equipment, and you want to tag photos of telehandlers. Let’s look at the tags that Google’s API provides for an image of a telehandler:

There are two things worth noting:

• Google doesn’t tag the image as a telehandler (or anything close).
• It returns tags like “machine,” which you don’t care about because all your photos are of machines.

The solution: custom-trained auto-tagging

Unlike generic auto-tagging, you can train custom AI models to tag the exact things you care about and nothing else. Let’s look at some use cases we’ve seen from our customers:

• Tagging stock photography that matches a particular aesthetic
• Tagging content with IAB taxonomy
• Make, model, and viewport tagging of car photos (this public model does just that)
• Detecting rooms and features in real estate photos. For example, detecting that a photo is of a kitchen and that it has an island and stainless steel appliances.
• Custom logo detection for smaller brands that are not well-served by generic logo detectors
• Logo identification for identifying established brands

Myths about custom-trained models

You might wonder why everyone doesn’t use custom-trained auto-tagging if it’s so effective. Unfortunately, there are a few common misconceptions about what it takes to implement it that hold people back:

• Misconception 1: You need a lot of data to train a custom model. Modern models can be fine-tuned to your use case, using just a tiny amount of data. In this image classification benchmark, we saw ~75% accuracy from just five examples per tag.
• Misconception 2: You need a team of AI experts to implement it. AutoML platforms like Nyckel do the work of an AI expert and hide the gory details behind a simple API. Here is an example in python where we train a model to distinguish cats from dogs, and then invoke it:

from nyckel import User, ImageClassificationFunction

user = User(client_id="...", client_secret="...")

func = ImageClassificationFunction.new("IsCatOrDog", user)

# provide a few examples of cats and dogs to train a model
func.create_samples([
    ("cat1.jpg", "cat"),
    ("cat2.jpg", "cat"),
    ("dog1.jpg", "dog"),
    ("dog2.jpg", "dog")])

# check if an image has a cat or dog
prediction = func.invoke("cat_or_dog.jpg")

• Misconception 3: It takes a lot of time to implement custom auto-tagging: Nyckel’s models train in roughly 60 seconds. Several of our customers have gone from finding us to using their model in production within 24 hours. Some of our customers have trained 3,000+ models because it’s so easy, fast, and inexpensive.

These myths were all true a few years ago, so they are not outright lies. But recent developments in AI research and user-friendly AutoML platforms mean they are not true anymore and should not hold you back.

It doesn’t have to be that hard

Here is a short video showing me training, deploying, and using a custom trained model to detect telehandlers, all in under three minutes!

Get in touch

Try us out for free or get in touch with our team to chat about your DAM use case.