Spotting Misleading Tech Statistics - Tech Insiders News, May 2023

Welcome to the Digital Explorers' Diary #37!

A curated collection of thought-provoking topics about interactive technologies, AI, web3, sense-making, entrepreneurship, and psychology.

Stay on top: It’s easy to be fooled

This is the second part of a series of articles on staying on top of the innovation news. Read part 1 here.

So, you have selected a small number of news sources from a diversity of opinions and views. You’re all set to stay on top, and here comes your next challenge: not falling for the «typical-misleading-statistical-pseudo-research-paper».

It’s very easy to be enthusiastic when seeing an article claiming that a high percentage of users want Augmented Reality experiences or that the XR market will be worth billions of dollars in 2025.

Alas, the reality is often different. And consequences are much more than just a distorted world vision: businesses might be created, decisions made, and investments funded based on these false metrics, only to face a much smaller market than they thought, leading to wasted money and human resources. Let’s dive into some tools for critical thinking. I’m using screenshots from this paper as an example… Sorry Snapchat!

One of the first clues to raise your awareness is if it seems too good to be true. It’s always helpful to Notice Your Confusion. When we pay attention, our minds are much more acute than we think; if you have doubts, dig deeper.

Then, follow the money: who profits from these numbers and these conclusions? It might sound conspiratory, but in some cases, it’s unfortunately true: even if the results are not false per se, the numbers are twisted towards a positive conclusion.

See that icon in the sponsor’s list? That’s Snapchat. Would they benefit from positive research on social commerce? I’ll let you reply!

Next: are they drawing results from opinions or actual data? Did they actually measure what they claim to? For example, look at the question in the screenshot below: «If social media platforms offered the following shopping features in the future, how likely would you be to take advantage of them?».
It will bring up completely different results than from a study where the panel is observed while shopping online and randomly assigned a page where AR is available.
Users might think they will use AR, but they might not do it when in front of a page.

Also, we have very different perceptions of probability words. Loom at the graph below: between people, “likely”’s meaning can vary from 50% to 90%!

And then comes the classic abuse of statistics: percentages can be very misleading. For example, an article claims that using Augmented Reality «Increases by 50% the effectiveness of ad campaigns».
Reading this quickly, you might think it’s an impressive number! But it’s missing the main point: what is the base rate of effectiveness? If it’s only 5% percent, the total effectiveness is 7.5%… It’s great, but maybe not as impressive, right?

Ok, enough for today; there’s obviously more to the topic. As usual, contact me to dig deeper, and share your feedback in the comments!

This Week's News:

1. Meta will add an AR feature to the Reels Ads and Facebook stories. Maybe they read Snapchat’s article ;-) Anyway, as social networks compete for the brand ads money, this is an intelligent move from Meta. Would you use it for your brand?
   Sephora was one of the beta testers of the feature.
2. Did you know that data poisoning could be a thing in AI? By intentionally inserting some biased elements into training data, ill-intentioned actors could alter the outcome of the AI model and lead to disastrous consequences. Imagine the potential geopolitical issues that this could cause. More than ever, as generative AI becomes ubiquitous, we need to spend much more time on security, regulation, and rationalization than on exponential scale.
3. The band Gorillaz, well known for their advanced use of virtual technologies, debuted an «AR Youtube Experience» at the Coachella festival.
   Let me translate that for you: they added 3D animations on top of the camera feeds, and the resulting video was sent to Youtube live.
   This technology is nothing new; we were already doing that 14 years ago. But it’s great to see it adopted by the entertainment industry on a large scale.
   You can enjoy it below!
4. Apple headset is supposed to be announced early next month, and suppositions are flourishing everywhere. One trend I’ve seen is a focus on health and fitness experiences, corroborated by the rumor that Apple is developing an AI-powered coach.
   My take: I’m not sure what type of experience they are thinking of; one thing I can tell is that doing fitness with a heavy headset and a battery on the waist is maybe not the best thing. We’ll see, Apple; please prove me wrong!

From the Podcast

Every week, I’m teaming up with Guillaume Brincin and Sébastien Spas on the Lost In Immersion podcast. This week:

1. We expand on the paper on AR in Social Commerce I mentioned last week. We dig deeper into the methodology and question the results. This was the inspiration for today’s article above!
2. Meta has lowered the allowed age in their virtual world Horizons to 13 years old. We both find it concerning, as moderation is complex on such platforms. How can we ensure the Metaverse is a safe space for kids, and for everyone for that matter? Should there be a kids-only version, such as YoutubeKids? How can we verify identities and age while still keeping privacy and freedom?

Tune in at 10 am UTC on Wednesdays on Twitch, and listen to the replays on Youtube!