Image Search Engines: Query by Image

These days, major search engines offer advanced image search options, enabling users to filter searches by size, colorization, and file type. Yet limitations still exist.

Until recently, most search engines had focused on alt tags and context surrounding images to categorize and index those images. A user would type in a text query, and the search engine would rank the images based almost exclusively on their relevancy to the keywords contained in the image’s alt text, surrounding body copy, and page metadata. This has made image search the target of spammers undertaking keyword stuffing or similar black-hat SEO (define) techniques.

New technologies are emerging that will take image search a step further, however. Instead of looking at the text associated with an image, these technologies can effectively scan and “see” what the image is to provide the user with information about it.

The applications of such technology are far reaching, including copyright protection, content moderation, censorship, and forensics investigations. Today, I’ll focus on its application to search specifically. Terms used to describe these advances include “visual search technologies,” “visual cataloguing,” “image recognition,” “image identification,” and “visual content analysis.” No matter what you call them, though, these technologies essentially attempt to do the same thing: decipher the content in an image so that queries can be performed.

Why should you care about all this? I’m sure you’ve experienced one of the following scenarios:

• When you go through your photos from the previous night, you find a picture with a random person in it. You want to find out who that person is.
• You take a picture of something and later notice something cool in the background (a painting, car, gadget, etc.) and want to know where to buy it.
• You vacation abroad and snap a photo of a historic landmark. Later, you can’t remember what the landmark is called.
• You find an image that you want to publish online or in print, but the resolution is too grainy.

The new visual search technology is the solution to all these conundrums. Instead of using text to querying for an image, you can upload a particular image and run a query based on the image’s contents. This is referred to as query by image, rather than query by keyword.

For example, according LTU Technologies, a leader in visual search technologies, you can upload an image from your computer or the Web and ask its visual cataloging product to show you:

• Images identical to your image
• Variations of the image
• Images similar to the image
• A high-resolution version of the image

The search results will include the images the engine deems to be the most closely matched to your query. If you asked for images identical to your image, the top results would be those that most closely resemble your image’s visual content, which would be followed by images that are very similar but perhaps not exact replicas of the queried image.

In addition, depending on the visual search technology, it is possible to weight queries to emphasize a specific color, shape, or both. Existing engines leveraging such technology include the progressive visual search engine TinEye and the mobile visual search engine SnapTell.

All these technologies will no doubt help searchers more easily retrieve information about existing images or find new images that meet their needs. Image searches will return more relevant results, improving the user experience and satisfaction with this technology.

Let’s consider the implications for marketers for a moment.

Moving forward, we may no longer need to emphasize keyword tagging or contextual placement of images since image analysis can interpret and understand the image without textual content. Any advertiser whose products rely on images to do the selling (e.g., fashion retailers and car manufacturers) stand to benefit from increased visibility, literally, in search engines. You may see more traffic coming to your Web site as a result of the images your site has indexed in these new image search engines.

It is unclear just how much impact these new technologies will have on individuals and businesses, but it is an important trend to be aware of for all those who use and leverage search engines.

Join us for Search Engine Strategies San Jose, August 10-14, 2009, at the McEnery Convention Center. Spend Day 1 learning about social media and video strategies with ClickZ.