“A lot of the future of search is going to be about pictures instead of keywords,” Ben Silbermann, Pinterest CEO
On a sunny day, you are sitting in a cafe watching people who are walking by. You notice someone with an interesting sport jacket with some distinct patterns. And you decide on the spot that you want to have it.
But there is a problem. You cannot discern the brand. And you have no idea where you would get the jacket even if you did.
Enter visual search.
You quickly take the snapshot of the jacket. Upload the picture to image search engine. After browsing the results, if lucky, voila, you not only get the name of brand, but also find the online shop which sells the product.
This is an example of technology use at its best. If perfected, it can save a lot of time and make shopping experience simple, quick and fun, allowing companies to offer instant gratification to its users.
Artificial technology is delivering
Although visual search as part of online shopping experience still has some way to go in terms of fulfilling users wishes, it is nevertheless an important innovation for e-commerce. 20 years ago such a shopping experience would be considered science fiction. Today, thanks to rapid advance in computer vision, a discipline of artificial intelligence, it is becoming reality.
Although visual search still represents only a small fraction of all searches, it is experiencing rapid growth. Pinterest Lens, one of the leaders in this field, has seen monthly number of visual searches grow by 140%, from 250 million in February 2017 to 600 million in 2018.
Major products in visual search – Google Lens, Bing Visual Search, Pinterest Lens, Amazon Camera Search
Google Lens allows users to take a photo or upload an existing photo. Some of the tasks you can perform with Google Lens:
- look up a meal from the menu,
- add events to calendar,
- get directions,
- make a phone call
- take a picture of clothes, furniture, home decor and other products
- look up information about places or locations by taking its picture
- identify plants and animals
Bing visual search offers similar features as Google. Main advantage of Pinterest Lens is its Pinterest ecosystem of pictures, although it offers less features when compared to Bing and Google. Amazon Search purpose is primarily to improve the shopping experience for the main Amazon platform.
In addition to visual search an interesting way to search for products is a hybrid approach of text and visual search. For example, let us say you are interested in brown leather couch. You enter the text query in Google search engine. You explore the image results and when finding the couch you really like, you use that image in Google image search to find potential online shops where you can check the prices and complete the purchase.
Why is visual search improving?
Visual search is based on image recognition programs that are using deep learning methods. Amount of images available for training deep learning models is continuously increasing, making the image recognition models trained on these datasets more accurate. Visual search engines are also learning from its interaction with users. Both trends will contribute to rise of visual search accuracy and thus to its increased adoption.
Which industries will be most affected by visual search
Pinterest has published industries and products that are currently most popular for visual searches in its Pinterest Lens.
Fashion and Home Decor are two of the industries leading the number of visual searches.
How can visual search benefit your business
One of the main benefits of using visual search is reduced friction in the process from search to conversion leading to quicker checkouts. If you are a larger online shopping company it makes sense to explore visual search technology in your offering to enable your customers an easier browsing of products on your site.
An illustrative case study of use in retail space is Target which partnered with Pinterest to integrate Pinterest Lens search technology into Target’s apps and desktop website.
Even if you are not active in retail space on larger scale you can still optimize your site for visual search. One area are meta data of images which you can rewrite to better describe your images.
The main takeaways:
- visual search popularity is rising due to increasing accuracy of image recognition models and platforms learning from interactions with its users
- it allows users to quickly perform many tasks from finding seen products to obtaining information about places or locations
- it is emerging as an important growth factor for retailers and online shopping platforms, by reducing friction in the process from search to conversion
