How Pinterest uses AI to learn (and sell) your style
Pinterest’s computer vision tools can be used to identify things you see in the world, but the real goal is to be able to recognize your style, even if it doesn’t have a name.
“We don’t have to know the name of your style but we [can match it], and I think that’s the magic part. That’s why we believe Lens is the first baby step to encourage users to use your phone and camera as an input of what you’re thinking; then we will figure out what we can do for you to finish the journey,” Li Fan, Pinterest’s head of engineering, said at VB Summit.
At Pinterest, Fan has led efforts to launch products like Lens, a computer vision-powered tool for image searches and ecommerce, and Shop the Look, which also made its debut earlier this year. Shop the Look places a blue dot on images that, when clicked, displays recommendations of similar items available for purchase.
“From the baseline, we noticed that merchants that are part of a partnership for Shop the Look, they’ve seen double clickthrough-rate to ecommerce sites and 2 to 6 times engagement on Pinterest of their product,” she said.
To create AI models, neural nets need clean data. Some of Pinterest’s data comes from contractors, but some also comes from its 200 million monthly active users. To date, more than 100 billion images have been pinned by Pinterest users.
“200 MAUs save those pins on their personal page; they will label saying ‘this is a dream dress for spring’ or ‘this is my sunglasses to buy’ or ‘this is my next baby room that I want to have’,” Fan said. “They already give you the signal of what the object is in the image they’re interested in.”
Both with Lens and Shop the Look, Pinterest is trying to identify your style, but it’s also attempting to use images as queries far beyond simple object identification. Pinterest users have, for example, used Lens for recipe recommendations based on items for sale at the supermarket.
“Originally you think a user may want to know if this is a Chanel dress or not, but turns out that’s not the case,” Fan said. “I think a lot of users told us they want to understand: How can I style it? How can I match other things in my wardrobe, things like that. Those are the use cases where you have to go deep, talk to users, and then use your technology to match what you have instead of just saying ‘OK, I gave you the object you asked for,’ which is not what they want.”