A Quick Guide to Semantic Image Search (with Examples)
In the early days of the internet, if you searched “the dark knight,” search engines like Google didn’t necessarily know that you were referring to Batman.
In the past decade, though, search engines have dramatically improved their ability to understand context and meaning and make associations between words to serve up content that better fits what the searcher is looking for. The ability of a search engine to understand the intent of a query is called semantic search.
Today, some search engines can run semantic searches not just on text databases but also on image databases. This is called semantic image search.
What is semantic image search?
Semantic image search uses either a text query or an input image to search a database of images to find images that are semantically similar to the search query. Thanks to advances in machine learning and computer vision, search engines can better understand what a user is looking for when they search for an image.
Semantic image search has become invaluable for businesses that need to help their customers quickly find what they need in a database of images. Let’s take a look at 11 different use cases to give you a better idea of how semantic image search works and how you could use it to support your business.
11 semantic image search use cases
1. Search e-commerce products
Online retailers need to make it as easy as possible for their customers to find what they’re looking for. Let’s say a customer saw a photo on Instagram of a blue sweater with polka dots she liked. Since she already has an idea of what she wants, she can use that image to search for a similar sweater rather than sorting through all of the results under the “sweaters” page. If the retailer sells an item similar to what the customer is looking for, she can quickly find it using a semantic image search.
2. Recommend a similar product
Similar to the example above, online retailers can use semantic image search to identify products to recommend to customers that are similar to the product they’re currently looking at. This makes it easier to help the customer find what they need and encourages them to keep looking for other items they might be interested in.
3. Stock photo search
Stock photography websites that offer semantic image search can help their users more efficiently find a photo to license. For example, let’s say a user found a photo of the Golden Gate Bridge against a sunset background that they want to use on their website, but the image is copyrighted. Instead of browsing manually through thousands of images to find one similar, they can do a quick semantic image search on a stock photo site using the image as the input to find the closest match. Or they could search for “golden gate bridge at sunset” with the same effect.
4. Duplicate image detection
Many companies store large amounts of digital images. For example, scientific organizations might have an inventory of ground-based, aerial, or satellite imagery for research purposes.
A semantic image search function allows the organization to detect duplicate images in its database, freeing up storage and improving search results. Highly accurate semantic image searches can find slightly altered versions of the duplicate image, including an image that has been recolored, reformatted differently, or slightly resized.
5. Manage, protect, and sell digital art
Online art marketplaces make it easier for people to discover unique art and support emerging artists. However, these marketplaces can be difficult to search, and they also make it easy for bad actors to copy, modify, and distribute artwork without the artist’s permission. Semantic image search can help with both of these problems.
Using a semantic image search, a customer can use an image to search a large marketplace to find art with similar elements. With something as subjective as art, the customer may be better able to express their art preferences using photos of other artwork they like versus trying to find the right words to describe the styles or color schemes they like best.
Online art marketplaces, museums, and other art organizations that need to maintain the integrity of their collections can also use semantic image search to find scammers who are selling or using plagiarized versions of their artwork elsewhere. This uses the same techniques as duplicate image detection mentioned above.
6. Marketing and product photo search
Many businesses maintain large databases of images for marketing purposes. Semantic image searches can improve the digital asset management experience for these businesses, helping them find what they’re looking for much faster than if they were searching using a more traditional approach (e.g., sorting through folders of saved images).
For example, a marketing agency needs to find the right photo from its catalog to use in an upcoming ad campaign for its client. Instead of manually sifting through image files, it could upload a similar picture of its client’s product to locate what it needs.
7. Real estate search
Semantic image search is useful for real estate agents, home buyers, and sellers who want to find a home similar to something they already saw and liked. Using a semantic image search, a buyer could upload a photo of a home (or even a specific room) they love to find the closest match in their buying area. This saves the home buyer time wading through pages of available houses and takes some of the stress out of the house-hunting process.
8. Image-based travel search
A travel booking site, like Expedia, Priceline, or Booking.com, could use a semantic image search to help its users find a travel destination based on an image they saw and liked. For example, a user may have seen a picture of the Múlafossur Waterfall in the Faroe Islands on social media. Unlike the most famous iconic landmarks like the Eiffel Tower, the user may not immediately recognize the waterfall or know where it’s located. If they want to travel somewhere similar, they could visit their favorite travel booking site to search using the image to locate the destination and find hotels and excursions to book in the area.
9. Pet and animal search
Many people turn to online pet marketplaces to find their furry friends. Many would-be pet owners have an idea in mind of the type of pet (e.g., breed, color, size) they want, but doing a basic search on an online pet adoption website can quickly become overwhelming because of the number of pet listings. With semantic image search, a visitor can use a specific image of an adorable puppy (or another animal) to quickly sort through the database of available pets to find similar matches.
10. Augment a museum experience
Museums have an opportunity to augment their visitors’ experiences by allowing them to snap photos of things they see and then provide them with more information about the object they’re looking at. After a visitor snaps a photo, a semantic image search uses it to sort through the museum’s database to locate more information about the relevant object. It then sends this information about the object back to the visitor.
11. Buying and selling on a resale marketplace
Resale marketplaces, like Craigslist or Facebook Marketplace, are popular ways to buy second hand products. But they can also be time-consuming and unwieldy. When you’re selling something on a marketplace, it can be tedious and time intensive to fill in all of the relevant details about a product. When you’re buying something, it can be a pain to sort through low-quality or unrelated items to find what you want to purchase.
Using a semantic image search, a marketplace could enable its users to upload a photo of what they’re selling and then auto-populate the description fields based on the database of items previously sold on the marketplace. Similarly, a user shopping on a marketplace could upload a photo of what they’re looking for to find a listing that matches what they want to buy.
How to create a semantic image search
As more companies implement semantic image search functionality into their products, more people will expect it to be a part of their experiences. It’s time to think about how you could better support your users or customers with this functionality.
You can easily implement semantic image search into your own platform by using a simple API that you can train on your own data. Nyckel’s semantic image search solution is both quick and easy to deploy, with no machine learning expertise required. In fact, we designed the product specifically for machine learning non-experts.
Ready to get started? Check out our image search quickstart guide and create a free account. You can try Nyckel with your own dataset and get up to 1,000 invokes per month for no cost.