John Puterbaugh, Ph.D.: Director of Mobile/Digital Strategy, BlueSoHo
From QR to AR Part 1 reviews how far we’ve come in interactive print since 2005 and sets the stage for Part 2, which looks at how marketers can put the technology to work with real-world applications.
Considering what’s here to stay—it’s important to look ahead to what might happen next.
Three primary applications
Three everyday applications stand out as effective and practical uses of interactive print:
- Direct Response Marketing
- Visual Search
1. Direct Response Marketing
Two key tactics within mobile marketing stand out as best practices for any multichannel marketing program that leverages print (e.g., direct mail, signage, inserts, etc): social sharing and mobile response vehicles. Each acts as an amplifier, enabling your brand touchpoints to work harder. For digital brand touchpoints, social sharing is built into most viewable digital content (e.g., post on Facebook, send to friends), and mobile response vehicles take the form of links or actions (e.g., swipe). For physical / print touchpoints, social sharing must be paired with mobile response vehicles.
For best results, I’ve seen higher response rates when a call-to-action includes two or more response vehicles with the most effective being text short codes and QR codes, followed by short or vanity URLs.
2. Visual Search
For the past 10+ years we’ve found images by using descriptive terms in text queries (e.g., Google Search) and selecting the output as an image format. With advances in machine learning, mobile tech (cameras, faster CPUs, better bandwidth) and pattern recognition, you can use images themselves as the input. As such, you can take a picture—of a shoe, a handbag, a car—and get actionable results: find it and buy it, then see other relevant products or images. Gartner, in their 2018 predictions suggests that by next year more than half of all mobile searches will be voice-based or visual search.
For digital brand touchpoints, social sharing is built into most viewable digital content.
In 2014, Amazon released their Fire phone, with six sensors and the ability to recognize images. You could take a picture of a razor in a Gillette ad, then immediately purchase it on Amazon. The day the phone was released, Target launched an app called Snap concurrently with media buys in magazines (e.g., Real Simple). Readers could download the app and use its image recognition to point the camera at the magazine’s print ad, thus converting it into a mobile-optimized, shoppable experience.
During the same time, Google was progressively adding capabilities to its search using images. These early developments were really about image matching and not recognizing components within the images (e.g., products, people, etc.).
Companies such as Home Depot, Nike, JC Penney, and Nieman Marcus were early adopters of visual search, integrating the technology into their retail mobile applications. In the case of Nieman Marcus, you could snap a photo of shoes or handbags, then immediately buy them. Other examples included JC Penney activating their printed inserts, enabling users to shop directly from them by translating the 2D printed image into the component products within them.
In the past year both Google and Pinterest have released visual search capabilities. Google Lens was announced in 2017 at Google I/O. Pinterest has been marketing visual search as the ability to shop the look. Both Home Depot and Target have been early partners incorporating Pinterest visual search.
AR is getting the most adoption in retail within the home improvement space, primarily for the purpose of visualization.
Visual search will likely see the most significant growth area due to its utility, technical feasibility and ease of use for consumers and shoppers. As we’ve seen with voice assistants, the simplicity of not having to type textual content to describe something you want to do comes quite naturally. And, with mobile camera-based search, the ability to simply point a phone at an image and save it, shop it, share it and learn more about it is quite powerful and plays into an increasingly common behavior—snapping photos.
AR is getting the most adoption in retail within the home improvement space, primarily for the purpose of visualization (BI Intelligence). An early example was how Ikea enabled their catalog recipients to scan an image on one of the pages (e.g., a desk), then use AR (transparent overlays) to see what the product would look like in a house, inviting a user to point her or his camera at a wall, or toward the middle of a room. In essence, it virtually lifted an image off the page and placed it within a room. Since Apple and Google have released their AR-enabling tech, other retailers have entered the space:
- Wayfair merged its AR app, WayfairView into its main shopping app
- Build.com and Houz have launched AR apps
- Lowe’s has deployed 11 different AR / VR programs, and are quickly figuring out how to deploy AR at scale
Similar to visual search, AR is based on image recognition. Azuma succinctly defined the characteristics of AR systems as 1) combining real and virtual elements, 2) interactive in real-time, and 3) performing registration (the alignment of virtual and real) in three dimensions. There are many ways to implement augmented reality including marker-based systems (used by Ikea early on), markerless (used by Mateo, Layar and others), transparent overlays (popularized by Blippar), and geo-based (overlays based on location rather than image-tracking superposition).
For the past decade or so, handheld devices have been the dominant vehicle by which to experience AR. In the coming years, we’ll see more adoption of heads-up displays (e.g., car windshields) and head-mounted displays (return of glasses, etc.).
Journalists, authors, tech industry leaders and marketers have begun a dialog about whether or not AR is a new medium. In their article on the New York Times Olympic AR experience, Adweek quotes Graham Roberts, head of immersive platform storytelling at the Times: “There’s a whole language that needs to be learned on both sides, the producers and the consumers. It’s almost like introducing the mouse for the first time; it’s a new way of interacting with something.”
Visual Search will likely see the most significant growth area due to its utility, technical feasibility and ease of use for consumers and shoppers.
Helen Pagaiannis in her book Augmented Human notes, “AR is a new communication medium that has great possibilities for extending the human condition, reimagining the way stories are told and experienced.” She goes on to outline tropes that marketers have borrowed from past media and are using to tell stories with AR (e.g., virtual try-on ghosts, x-ray vision, living pictures, hole in the wall, etc.).
Going forward, brands and retailers deploying AR content will have two paths from which to choose—either build the capability into an app using enabling tech (likely from Google and Apple), or leverage a single third-party app.
For the past 10 years, only a handful of apps have boasted sufficient scale to appeal to marketers and serve as the single third-party app for AR. Blippar served this role until recently when both Snap and Facebook have entered the market.
Snap has had some recent high-profile wins in the AR space, with the ability to overlay digital images on a photo’s background using Snapchat. The first advertisers to use this capability include Adidas, Coty, King and STX entertainment (Business Insider).
For the past decade or so, handheld devices have been the dominant vehicle by which to experience AR.
At the May 1, 2018, Facebook F8, Mark Zuckerberg presented AR as the ability to essentially conjure physical objects out of thin air by overlaying “digital text and images on the physical world around you” (recode). They’ll embed their Camera Effects Platform (i.e., AR engine) within their apps (Messenger, Instagram), enabling developers to build AR experiences. Facebook announced that Sephora, Nike, ASUS and Kia will be launching AR effects for Messenger.
A crisis of measurement
There’s still a crisis of measurement within the area of interactive print. People are comparing “scans” to page-views, when in fact they are much more like “click-throughs.” Considering click-through rates on paid social or programmatic display programs are on average around a quarter of a percent, interactive print programs can deliver commensurate results.
Another problem related to measurement is related to the “cart before the horse” problem. Many marketers start with the technology then look for an application, rather than letting their strategy drive content. Only after you figure out your campaign goals (e.g., acquisition, activation/engagement, retention) and the required content (e.g., get a coupon, drive app download, view a video, sign-up) should you select which technology you’ll use to amplify the program and when (or even if). This is where you can determine how to match response vehicles to your audience, content, brand touch-points and channels.
Even when the right tech is deployed for the right audience, we often forget we need to use basic tried-and-true direct marketing tactics when introducing new tech, (e.g., use action terms in the CTA, make sure a clear benefit is defined, include instructions if it is really new, etc.).
When URLs started appearing in magazines in the early ‘90s, many readers had no idea what to do with them. There’s been a chorus of complaints in recent years about how long it takes to download apps and how slow mobile browsers are. (This was, of course, nothing compared to dialing up online via a modem on a personal computer that had much less horsepower than even the first iPhone!) We can easily avoid our audiences’ ire with a minimal explanation and guidance. Managing expectations is an integral part of marketing practice.
With all this set, it will be difficult in coming years to parse out which enabling technologies for interactive print will survive and remain a critical piece in your marketing toolbox. As with QR code eulogies, we will continue to see a range of other technologies in the interactive print space declared dead: NFC by TechCrunch, Beacons by Forbes, Shortcodes are dead… Lest we forget, even AR was declared a thing of the past a few months before the launch of Pokémon Go.
Managing expectations is an integral part of marketing practice.
Surely camera-based and voice-based interactions will continue to evolve. They’ll be the basis for a wide range of interactivity from direct response applications, visual and voice search, as well as a wide range of extended reality applications beyond visualization and into new forms of storytelling based on AR, VR, mixed reality, 360 video, etc.
Given the wide range of sensors and modalities that phones and IoT devices can support, multiple mobile response vehicles or triggers will thrive as technologies enable us to act with the physical world. They’ll continue to transform what were static printed images (signs, ads, articles) into dynamic, interactive and connected content. Smart signs and smart graphics will deliver relevant contextual content to the consumer, becoming easier to manage and maintain for the marketer. As adoption of new technologies becomes ubiquitous, you and your team will have ample options to enhance storytelling efforts, and to connect at a deeper level with your audience.
John founded Nellymoser in March of 2000, one of the first mobile agencies to build apps and mobile services. He is Managing Director, Digital for BlueSoHo. Educated at Princeton University, Dartmouth College and Oberlin College, John has taught courses and lectured on interdisciplinary topics related to cognition, video, interactive print and mobile.