Podcast math: we don’t know what we don’t know
Steve Safran November 2nd, 2006
By Steve Safran
Managing Editor
Lost Remote
Ze Frank calls it the Nerd Fight. He is disputing Rocketboom’s assertions that they have 350,000 viewers a day. At the center of this dispute is what we decide a “view” really is. Actually, we can all agree on what a “view” is – someone watched something. It’s how you measure it that’s up in the air. One thing’s for sure: just because you download a podcast doesn’t mean you’ve watched it. And just because you’ve started watching it doesn’t mean you’re going to finish it. There’s simply no way for us to measure viewership of podcasts. But we keep reporting numbers from the networks, big sites and podcasters without questioning them (guilty as charged) and we need to step back for a moment and ask: “How do we figure out who is really watching or listening to our podcasts?” Then we have to admit “We don’t know.”
What does a download number mean?
Last week, ABC News announced it had reached five million podcast downloads in a month. An impressive number. And there’s no reason to doubt it. But what does that translate to in terms of viewership? And by extension, how do you sell that to an advertiser?
I don’t know.
If you’re a podcast subscriber, you know the process: you subscribe to a bunch of shows, they download to your iPod (or whatever) and then… what? Maybe you watch. Maybe you don’t. Maybe you start it but that’s it. Come on – my iPod is jammed with the things. I get to a fraction of the ones actually on my iPod. It’s a bit like the TiNo syndrome: you gather all those episodes of “NOVA” and “Frontline,” you just don’t watch them.
Look at it another way: having the podcasts on your device is a victory for the producers, no doubt. You can’t watch the show if you didn’t download it. But all that means is that the show is available to you. It’s really no different than having a cable channel “make it” to your cable box. You have access to it – but you don’t always watch PBS. (Admit it.)
All the “downloads” metric does is outline your potential universe. It does not tell the advertiser the most important metric: who frickin’ watched us?
I don’t know.
What I do know is that paid downloads are another matter. It doesn’t mean a thing to ABC whether you watched that download of “Lost,” – as long as you paid for the download. So if there were 35,000 downloads of a show at $1.99 apiece, that’s real money with a real number. In that case viewership doesn’t matter any more than the fact that you bought the extended DVD of “Lawrence of Arabia” but you and I both know you’re never going to watch it. At least take it out of the plastic wrapper, will you?
Ze What?
Over at Ze’s Nerd Fight, (complete, as any war coverage should be, with a theme song) he fires a salvo at Andrew Barron, Rocketboom’s producer: “Andrew’s caught in the web of these fuzzy numbers. To him, “complete download” not only means a “completely watched show” but also a “unique audience member.” He also touts his large subscriber base and brags how a number of software products download his show by default after they’re installed — without even asking the user if he’s interested or wants that to happen.
“Those aren’t viewers, they’re nickels. Grab a thousand; swipe $50 from an advertiser.”
Still, Ze allows that everyone engages in the practice of reporting the download stat and not an actual view stat: “MSN, YouTube and other sites are also Rocketbooming. They’re all desperate to come across as popular, so they all present aggressive background statistics. On some gallery sites just browsing a page full of thumbnails increases the view count for all the items on that page. Other times you click a blurry video, immediately leave, but it still counts as a view.”
Barron fires back with his own post, Ze Errors, in which he points out that TechCrunch audited his numbers successfully. He also implies a certain jealously on Ze’s part - Ze gets 30,000 downloads to Rocketboom’s 300,000:
“The core of the matter is not hits or page views or uniques or subscribers or Alexa #’s it’s how many completed videos were served. The video carries the ad. So that is what the advertiser wants to know, that is the ultimate value number and that is what Mr. Frank stated he was most concerned about: ad revenue potential. For any website that uses Apache and keeps a log file (i.e. the majority of the websites on the internet), this is a no brainer to see. Again, its simply a video file and files are easily counted.”
But notice that Barron’s not really disputing Ze’s point: we just don’t know if people are watching.
A new metric: “Engagement”
What metric could we possibly figure out that would let us know if the podcast we sent out was actually effective? Web legend and current VP of Media Development, at the PodTech Network, Robert Scoble, has an interesting suggestion: a metric he calls “engagement.” It’s less definable than the actual number of downloads, but far more interesting.
At Streaming Media West 2006 this past week, Scoble gave an interesting anecdote about a form of engagement. (I’m going to quote my own transcription from the panel session now, which was more of a “quotes-meets-my-summary” liveblog.)
Scoble: My friend Buzz at ActiveWords (released new software), had a five star review in USA Today and had all of 32 downloads… he then went on a 50 minute podcast and late in it he mentioned he’d give a free download to people who were still listening. He had 450 downloads as a result of that mention, so that gave him a metric.
It’s an interesting idea and proof that traditional PR efforts like getting your client in a major newspaper are less valuable than targeting niche PR. It’s really not all that different from the notion of “direct response” advertising. You can measure effectiveness, at least in this example. And you can then go to an advertiser after you have a proven track record of that effectiveness.
Always question that PR number
So maybe we can know, after all, how effective our podcasts are. The networks should look to this example and try to steer clear of the Nerd Fights. Most important, as journalists, we should always question numbers handed to us on a silver email. There’s a reason PR offices exist. They are simply a pipeline of the good news that companies want you to know. Period. I can tell you I’ve never been able to get what I suspect to be a bad number out of them. (And you know who you are and to what service I refer.) We need to keep trying to be creative with new metrics, and not simply parrot the old ones.
14 Comments Add your own
1. thedetroitchannel | November 2nd, 2006 at 6:27 pm
good for ze.
barron has proven himself quite the bullshitter in the past.
if ze is the war planner i think he is, he may already “know” the real number and is just drawing out his opponent.
2. thedetroitchannel | November 2nd, 2006 at 6:42 pm
for your consideration… just clicked over to rb and there are 38 comments for today’s show. 1 in 9210 people took the time to comment?
considering 10 comments were from “drew” and ten more from some regulars who were around when BoomBoom was there, that takes that ratio to about 1 in 14, 500.
compare that to LR
3. John Furrier | November 3rd, 2006 at 7:54 am
Great Post. I’m glad you highlighed Robert Scoble’s engagement principle which PodTech is developing. We are experiencing the same stat issue… we get great feedback but no one really knows yet…the metrics are behind the consumption adoption curve.
PodTech is developing a ROI return on influence principle as well..
Great post.
4. jennifer jones | November 3rd, 2006 at 8:00 am
I am the host and creator of a podcast called Marketing Voices for Podtech.net (www.podtech.net) which delves into the impact of social media on marketing. I can tell you that metrics and how to make them real is THE most important and most discussed topic among all of PodTech’s corporate clients who include tech’s best…INTEL, NVidia, Juniper etc. No one has yet suggested or created a great means of measurement for podcasts. I have also interviewed many people for the show including Peter Blackshaw of Buzzmetrics and this topic and although many are working on it, no one has the solution. A company who breaks the code and comes up with the solution will truly have helped move the needle in greater acceptance of podcasts as being used in marketing.
5. Chris Way | November 3rd, 2006 at 8:49 am
Thinking for a moment regarding what we can track … while not all, but at least most people listen/watch to podcasts on an iPod, and iTunes tracks the play count, perhaps it would make sense to for Apple to track this data and provide back to content providers in a non-personally identifiable way. Not perfect, but it could be a start. Either way, if Podcasts will every be held accountable, something is going to have to change …
6. Alyssa | November 3rd, 2006 at 9:12 am
Good point about the number of downloads vs. the number of podcasts actually listened to/watched. I think that people are more inclined to watch something that they paid to download
7. Mitch | November 3rd, 2006 at 10:47 am
I’ve started watching a lot of Ze’s videos. Don’t remember ever finishing one.
8. Alex | November 3rd, 2006 at 12:58 pm
Also, keep in mind that a lot of people don’t download ‘podcasts’ to their ‘ipod’ - they just watch/listen online.
9. Gerald Buckley | November 3rd, 2006 at 1:04 pm
Messrs. Frank, Barron and Scoble are looking at this primarily from someone who has a patch of real estate to sell. Turn the thing around and look at it from the sponsor’s point of view. That entity typically wants results (unless they’re just fond of the site, in which case it’s purely sponsorship).
So, “engagement” and “influence” are all fun metrics to define, hoist and chase… at the end of the day an advertiser simply wants to know their money was well spent. ROI reigns supreme.
10. Pete Alcorn | November 4th, 2006 at 12:09 pm
To keep downloads roughly in line with actual plays, the iTunes client will automatically unsubscribe from a feed if the user stops playing episodes from the feed. Here’s the logic:
* If the user has not played an episode in more than 5 “download sessions” (there may be > 1 episode downloaded per session)
AND
* There have been more than 5 days between the first session and now,
THEN
* iTunes unsubscribes from the feed.
11. Alyssa | November 6th, 2006 at 8:12 am
iTunes hasn’t yet unsubscribed me from anything, even when I don’t get around to playing things for a while
12. Albert Maruggi | November 15th, 2006 at 9:12 am
We Americans are a fickle and impatient lot.
Bottom line – advertisers want a sure thing
Realty – in life there are very few sure things (sun rising in the east, stuff like that I wouldn’t bet against, but most other stuff is a gamble)
Reconciliation between the bottom line and reality is to test, assess, adjust, “and so it goes,” (attribution Ellerbe)
As long as the majority of people and dollars flock to “more” media and by more I mean whatever show has larger numbers than another show, so that means Rocketboom over something else for the purpose of this thread, then the system values quantity above quality.
In podcasting the quest to monetize has allowed some parts of the podsphere to fall into the mass (or more) media trap. So, as I’ve feared most of my life, size does matter. This of course explains the majority of the emails I receive.
Scoble’s point of audience engagement is an excellent measure and should be one of several measures that will give advertisers confidence to test and take calculated risks. But, having spent time in politics and as a journalist, the engagement assessment is a
”Snapshot” of the audience at the particular point in time at which it is taken.
In Scoble’s case he cited a situation where listeners of a podcast were asked to call in for a free download of software. OK that measures interest in free software. That metric assesses interest, but is not quite a true measure of value which is what dollar amount would they willing to pay to satisfy that interest. The engagement index does however, generate interested leads and that’s a milestone for direct engagement.
The Rocketboom Baron’s got a good thing going, generating excitement, advancing the medium and providing some metrics in the process, while others are wringing their hands and arguing how many angels can dance on the head of a pin. At the end of the day at this point this specific discussion it’s about generating excitement. On that measure, Rocketboom has done it and we are all benefiting.
Marketers don’t push too much for absolute metrics. The closer marketing gets to a formula, the less valuable you become.
13. Christopher Kenton | December 5th, 2006 at 11:13 am
Why is this such a big mystery? If your goal is a hard measure of ROI for acquisition, insert a premium with a source code into your podcasts and measure response. That’s all engagement is. We’ve done this in print. We’ve done it in radio. We’ve done it on the Web. We’ve done it on tv. Call now. 888.podcast.
If your goal is to measure impressions, then accept downloads as an indication, and not a guarantee of impressions. You can get a reasonable risk adjustment by looking at download growth. If people aren’t watching or listening to what they download, fewer are going to come back to download again. Advertisers make ad buys in print based on circulation, when there’s no guarantee that every reader is going to open their newspaper that day, much less turn to page A16 to view their ad. Advertisers make ad buys on television, knowing that users hold the remote and use TIVOs. There are negotiations and adjustments that shape how metrics relate to the pricing of space, and those shake out over many seasons.
If your goal is branding, than accept what advertisers have done for years, and go out and do some brand awareness measures before and after a round of podcasts. Is this realistic? Well, small companies have never been able to afford such measurements, but that hasn’t stopped them from accepting the effectiveness of advertising based on the studies of larger companies.
I suspect this is more about getting across the chasm–getting the mainstream to accept the change to a new medium, and providing them with the ass cover they need to justify their ad spending. The truth is, the metrics for podcasting are not likely to be more effective than metrics have been for television or print, at least until we can embed a chip in people’s brains to measure cognitive acceptance of a delivered message. And the tactics for getting close enough to a metric for the relative effectiveness of different podcasting channels will look a lot like they have for decades. Offer some bait, and see who takes it.
14. Sam | January 18th, 2008 at 7:29 am
Wow, thanks for the excellent information!
Leave a Comment
(Please keep URLs out of the comment body or the spam filter will block you.)Subscribe to the comments via RSS Feed