Pinterest has unveiled Canvas AI

Pinterest has unveiled Canvas AI, an artificial intelligence (AI) project designed to improve product visualization strategies. Specifically, as detailed at the VB Transform conference, it is a new tool that allows brands to change the background of product photos without altering the representation of the product, offering “a unique opportunity to revitalize their marketing strategies.”

Canvas uses AI to integrate products into various digital backgrounds without compromising product integrity, creating a more engaging display. This advancement broadens brands' creative horizons and enhances consumers' overall shopping experience by helping them visualize how the product would fit into different scenarios and backgrounds.

Anthony Suarez, senior director of developer enablement at Pinterest, was on hand to introduce Canvas: “Early on, we assumed that the pace of innovation would require us to adapt and evolve what we build to keep up with a changing landscape,” explained Suarez.

Canvas distinguishes itself from other text-to-image models by enhancing existing images rather than generating new ones from scratch. This approach aligns perfectly with Pinterest's core mission to inspire users through visual discovery. 

The new tool is easy to use, even with basic user commands. In fact, it transforms existing concepts, products, or goods into new insights using Pinterest's extensive database of product images. It aims to engage users by promoting creativity and offering diverse visual experiences.

Canvas uses a segmentation model to generate product masks and identify the foreground from the background.

Pinterest aims at provide a better visual experience 

In addition, it uses visually dominant large language models for full captions and trains all UNet layers using a select set of product images with high user interaction. This improves pattern recognition in user preferences, leading to more dynamic results.

In addition, Canvas uses product-to-plot ratios to guide the distribution of spatial attention. By focusing on meaningful details, this feature increases the system's responsiveness and leads to a more personalized user experience. Continuous learning fed back into the model during iterative system training strengthens the accuracy of product feature identification and improves overall performance and accuracy.

Canvas's primary goal is to repurpose existing Pin images into dynamic backgrounds, giving brands the freedom to choose any aesthetic that aligns with their brand image. Brands can enter descriptors, and the system generates potential backgrounds for their product photos that resonate with those style requirements.

Pinterest expects the successful implementation of this model to diversify Pins images and enhance the appeal of products in various patterns and design styles.