Jan 10

The Dark Matter Of Pop

FT/31 comments • 859 views

I have a new (fortnightly) column in the Guardian – the first instalment is here. I posted that link on Tumblr last night and The Lex left an interesting comment, which I’m going to excerpt:

“Illegal downloading is v much an elephant in the room here! Obviously there are no facts and figures but it’s so easy to find any given song for free – and almost all the time, way before it’s available to buy legally – that I really assume illegal downloads of singles vastly outstrip legal downloads. (And obviously there are plenty of demographic factors at play here too.) And that’s pretty much why I can’t take the charts seriously as a gauge of popularity.”

I know exactly what Lex means, but is he right to say “obviously there are no facts and figures”?

The question of how you measure the popularity of music is a knotty one. The charts have always been an imperfect solution – they don’t take how much buyers like a record into account, the separation of album and single makes it often hard to work out the popularity of a particular song, the sample of shops used to calculate them used to be grossly unrepresentative, and so on.

Now, as Lex says, with illegal downloading the charts simply can’t claim to be an accurate measurement of anything other than “tracks sold”. But measurements don’t need to be exact to be interesting – there’s no problem with an unrepresentative measurement if we can account for the ways it’s unrepresentative, and there’s no problem with an indicative measurement if we want to know the answer to a broad question. The question “Which is the most popular song in the country?” is fairly broad.

The problem of illegal downloading is that the taste profile of illegal downloaders and legal downloaders may be very different. Back in the days when only a certain number of sales outlets counted towards the charts, this was a real issue. The kinds of music being bought in, say, specialist dance music shops and mainstream shops were very different, and the charts only monitored the latter, on the assumption that they could generalise outward. A lot of the time, they could – the sales of most specialist records were miniscule. But sometimes sales weren’t miniscule, and it wasn’t uncommon for artists to report enormous hits through independent shops that were completely ignored in the actual charts.

Is something similar happening with downloading? Luckily, we have a couple of ways of checking.

We can, for example, look at what’s being downloaded on BitTorrent sites, which handily let you sort by number of seeders. Here the most popular acts are bands like Lady Gaga, Black Eyed Peas, Eminem – exactly the kind of global stars also doing well in the ‘real’ charts. Advance albums also do well here, of course – on the Pirate Bay Dr Dre is doing great “business” while Vampire Weekend’s new record isn’t getting a lot of traction.

BitTorrent stats suggest that at the top end the profile of illegal downloaders isn’t very different from the profile of legal downloaders. What about listening to songs, though? Here we can look at Last.FM for some data. Now, last.fm is problematic because it’s very clearly biased: its users are well known for deleting plays of particular tracks (mostly pop ones) from their libraries, and its safe to say that the service overweights rock and indie music. So for Last.FM what we’d need to do is look solely at a genre that is well-represented on the service, like rock, and see if its profiles differ much from how you see that genre behaving in sales charts. And here again we see acts like Muse, Coldplay, U2 and Depeche Mode – the big album and ticket sellers – coming out as most popular on the last.fm service.

So we can’t prove illegal downloaders have similar tastes to legal downloaders. But we can find information which points to that as a hypothesis at least. Big is big, whether paid for or not.

(Of course, our hypothesis only works at the top end, which can cope with wide margins of error. Further down the scale – asking, for instance, whether Joy Orbison’s “Hyph Mngo” is as ‘big’ as Little Boots’s “Remedy” – you do run into fog a bit.)


  1. 1
    Tom on 8 Jan 2010 #

    Dave Rawkblog mentioned Hype Machine popularity on Twitter as something else we could measure, and of course I’d completely forgotten YOUTUBE which is a really important (and COMPLETELY measurable) method of listening to music which has a broader based demographic than either Last FM or a torrent site.

  2. 2
    Scott Curtis on 8 Jan 2010 #

    Excuse me for not reading the entire article but one of the best ways to measure would seem to be google searches using common keywords like mp3, download, torrent, rapidshare, megaupload etc.

  3. 3
    Lex on 8 Jan 2010 #

    BitTorrent is something one would use mostly for albums though, isn’t it? (I still haven’t figured out how torrents work.) And in terms of both how they’re bought and how they’re downloaded I think albums are a different beast – obviously a less casual purchase but also less casual to d/l for free due to hard drive space, time and so on. I was mostly referring to randomly googling for a song and getting it off a zshare or usershare link, which I do most days and which is super-easy – in terms of inidvidual tracks, especially new ones, I think this is where the bulk of the non-legal downloading comes from, and I don’t think this is trackable.

    Youtube, yes – even easier than looking for an mp3 if you don’t mind not “owning” it, and EVERY new leak gets on youtube it seems.

  4. 4
    Tom on 8 Jan 2010 #

    Well, that method is trackable but only by Google – I guess this is also what Scott C is getting at at #2.

    But I still would be very surprised if the specific music looked for and found by that method is SO different from the music picked up by other, more trackable methods: I can’t think of any reasons it would be.

    Individual song torrents are common, and some are very popular, though it seems to me like using a sledgehammer to crack a nut! More common actually are torrents of individual song videos – “She Wolf”, “Bad Romance” etc.

  5. 5
    ian on 8 Jan 2010 #

    I think there might be an age bias – crudely put, kids are more likely to download, and perhaps more likely to download illegally. So their tapes are perhaps under represented against those of the richer, older types who are more likely to buy physical items.

    Or maybe I’m just a bit behind the times.

  6. 6
    swanstep on 8 Jan 2010 #

    @tom. Do you think the Bad Romance single itself is that great? Or is that not supposed to be a useful question to ask? My (I assume not especially idiosynchratic experience): it’s video blew me away- I couldn’t stop watching it on youtube and on tv for about a week. And then suddenly I was over it, and now I’m interested by neither the song nor the vid. (as if to underline this trajectory, the non-video mixes of the song seem very inferior – don’t have the howl! I swear it’s just like ‘into the groove’ vid mix being much preferable to most of album versions and remixes, which all lacked the really wound up ‘now i know you’re mine’. Gaga has a degree in advanced madonna studies, i’m sure of it – she’s just fascinating!).

    This was the trajectory for Beyonce’s single ladies too. It was a monster vid. hit first and foremost. At any rate, bad rom and Sing ladies in particular have raised for me the question for me of the emergence of a new single/ace vid combination entity that shocks and awes us very intensely for a short period – creating a kind of single-serving monoculture. I haven’t really thought the idea through. Maybe nothing’s really changed. We’ll see I guess.

  7. 7
    Tom on 8 Jan 2010 #

    #6 – I’ve actually heard the single a lot more than I’ve seen the vid, and I’ve had the hook in my head a lot more than I’ve heard the single, so it works on the basic pop-punch level, but yeah when I’m talking about Bad Romance as “event” it’s the song and the video and the performances all in one.

  8. 8
    Tom on 8 Jan 2010 #

    And yes Single Ladies is another example, definitely: it was USABLE in a lot of different ways –

    – big hook
    – excellent production which revealed a bit more of itself as you listened in
    – amazing video
    – pasticheable video
    – at the very basic level a nice little meme/joke, “if you liked it then you should have put a ring on it” has cropped up all over the place

  9. 9
    Mark M on 9 Jan 2010 #

    Re 8: All of which could be said for Umbrella, too. What Single Ladies has additionally (I report from the women’s magazine trenches) is an I Will Survive* anthem element. I must say when I first heard – or rather saw – Single Ladies, I thought it was pretty slight and would have never imagined that it would become such a big deal.

    *Being essentially rather more misanthropic than Tom, I’ve always hated songs that have lyrics that people sing along to collectively with a “that’s me, that is” expression. Still I have traumatic flashbacks to 1991 and students singing along to that Wonder Stuff one about being drunk a thousand time before. Aaargh…

  10. 10
    Pete on 9 Jan 2010 #

    At the time I hadn’t been drunk a thousand times, so felt unable to contribute.

    These days, that’s me that is!

  11. 11
    Lex on 9 Jan 2010 #

    “Single Ladies” works much much better as a pop culture meme than as an actual song! As a song it’s a poor retread of the amazing “Get Me Bodied” off B’Day – and I don’t even think the dancing is that great, it’s simple and copyable but it doesn’t have the wow factor of Ciara or Janet’s dancing.

    I am aware that all of that is the reason it became a pop culture meme where “Get Me Bodied” or Ciara’s “Promise” didn’t. All I can say is that it was very astute of Beyoncé to realise that taking her success to the next level would entail redoing what she’d already done, but less well.

  12. 12
    Lex on 9 Jan 2010 #

    But back on topic, “Single Ladies” is another single which proves my point about the charts – no one can deny how absolutely huge and ubiquitous it was for all the reasons listed, it felt like a song which stayed at No 1 for months, but if you just look at its chart performance (never got higher than No 7!) you’d never realise that.

  13. 13
    koganbot on 9 Jan 2010 #

    I think the crucial audience that gets overlooked and undercounted is the one for “adult contemporary” and “smooth jazz,” not ’cause they torrent but because they often don’t acquire the song at all; but this is the station they have on at the office and while driving, and presumably some of them purchase some of the goods that are advertised on the stations. And when they do buy, it’s probably albums, most likely back catalog, though this may be the audience that’s rearing its head on behalf of Susan Boyle*. These are surmises on my part, since I don’t know. These people do get counted in the Billboard system, which factors in airplay and videoplay as well as sales.

    Also, sales are suppressed for fourth and fifth singles from an album as opposed to first and second; “You Belong With Me” and “Already Gone” are still huge on radio but sales are way behind.

    *Except Boyle doesn’t have a track in either the AC or Hot AC top forties at the moment (or any other radio chart, as far as I can tell, though her “Silent Night” did get play last month). According to Mediabase, the top five on AC right now are Taylor Swift “You Belong With Me,” Colbie Caillat “Fallin’ For You,” Michael Buble “Haven’t Met You Yet,” Rob Thomas “Her Diamonds,” and Miley Cyrus “The Climb”; while on Hot AC (which actually has more listeners) they are Kelly Clarkson “Already Gone,” Owl City “Fireflies,” Rob Thomas “Someday,” Uncle Cracker “Smile,” and Train “Hey, Soul Sister.” But these stations are also playing a lot of oldies.

  14. 14
    koganbot on 9 Jan 2010 #

    If you scroll down to the bottom on the page of Billboard charts, you’ll see they’ve got charts for iLike Libraries: Most Added, iLike Profiles: Most Added, Lala.com, AOL Video, AOL Radio, Yahoo Video, and Yahoo Audio. Top five on AOL Video: “Here Without You” 3 Doors Down; “Thug Style” Ciara; “Rockstar” Nickelback; “Whatever You Like” T.I., “I Kissed A Girl” Katy Perry.

  15. 15
    koganbot on 9 Jan 2010 #

    (Actually, “Already Gone” and “You Belong With Me” are third singles; but they’re still hugely popular long after their albums came out. Interestingly, “Already Gone” never charted that high in its prime; it began its push as a single in August, didn’t hit the top twenty of the Hot 100 until October, didn’t peak until December at a relatively low 13, but as I say it’s still chugging along on airplay. What also doesn’t get measured is how much people play something after they’ve bought it; but a track that never seems to leave the radio may be a clue here; as is what people stream (you might stream something you already own when you’re not on your home computer, say at work; or when you want to watch the vid; or you’re too lazy to search for it on you mp3 player’s library).)

  16. 16
    koganbot on 9 Jan 2010 #

    Adding to what Lex said about “Single Ladies,” often the way people report the charts, or what tends to get listed as history, is misleading because what’s most available as information is a song’s peak; Kelly Clarkson’s “Never Again” jumped to number 3 shortly after it was released, then plummeted; whereas “Already Gone” built over months and months and months; it’s the far more popular song, but its peak is only 13, so it looks as if it wasn’t nearly as big a hit. Fortunately Billboard is now making chart histories readily available online. But for years good information was hard to get if you didn’t hoard back issues. People with information often want to sell it rather than simply give it away; selling, I suppose, is an incentive to collect the info in the first place, but makes it hard for people who want access.

  17. 17
    koganbot on 9 Jan 2010 #

    Country act Lady Antebellum’s drunk dialing masterpiece, “Need You Now,” is number 2 country, as you might expect, but isn’t so high nationally: is 18 in digital sales, 20 on the Hot 100, only 29 on the radio. But it’s Number 5 on the ringtones!

  18. 18
    Tom on 9 Jan 2010 #

    Yes to #16 – one of the misunderstood things about the “long tail” of digital distribution is that not only does the market as a whole have an l.t.* but that EVERY ITEM IN THE MARKET also has one. The charts have always reported 2 axes – position and duration – and we now have a market where the duration axis stops being a bit of nice to know info and becomes a really important way of gauging a song’s ubiquity. So Lex, the information is there in the charts, it’s just you’re looking at the wrong bit of it.

    *(which it does, even if the ways Chris Anderson claimed it would behave have been debunked)

  19. 19
    thefatgit on 9 Jan 2010 #

    At this time of year, the new acts get heaps of airplay. These new acts are held up before us by the media and we choose who to champion and who to reject. For every Fleet Foxes and Animal Collective, there’s an army of Twang or Bravery, that don’t cut it.
    This year we have Ellie Goulding, who has a distinctive voice and an unusual sound. I wonder how many people are willing to invest in this new artist with physical sales or legit downloads. Some might be tempted to go down the illegal route until they find out whether Miss Goulding has “legs”.

  20. 20
    koganbot on 10 Jan 2010 #

    we now have a market where the duration axis stops being a bit of nice to know info and becomes a really important way of gauging a song’s ubiquity.

    Wasn’t this always the case? I don’t think the Internet changes this fact any, just gives us more information on consumer-generated play (rather than just airplay), which can be tallied when it happens online, and the ‘Net gives word-of-mouth greater scope but also gives us file-sharing and more piracy, hence underground consumption.

    To give a fairly extreme example from music’s pre-‘Net days, L’Trimm’s “Cars With The Boom” was on the Hot 100 for 17 weeks in late 1988 but only peaked at 54, was on the hip-hop/r&b chart for 12 weeks and only peaked at 39, but this information doesn’t even tell you what was going on with the song, which started on an indie label, hit in clubs and got airplay in Miami and San Francisco and I’m guessing a few other of the major dance cities such as New York, so it was already having its run and peaking in a few locales before getting picked up by Atlantic Records and pushed nationally, though it never hit across the board, scoring only in some places and not all at the same time. I only know this because I had actually heard it at a club and then tried to find out what it was, and living in San Francisco I was near to some record stores that carried the album, and from there I paid special attention to what was happening with the song.

  21. 21
    Tom on 10 Jan 2010 #

    It’s always been the case in the US charts, which include airplay, but in the UK the “long tail” of a single’s duration could be greatly curtailed by either the label deleting it (having new product to promote) or shops ceasing to stock it. So between the behemoths of the 50s and recent mega-duration hits most singles had artificially shorter runs.

  22. 22
    lonepilgrim on 11 Jan 2010 #

    In what, to me seems a recent development, there has been some acknowledgement of UK airplay as a measure of a songs success. So, for example, Gabriella Cilmi’s ‘Sweet about me’ was the most played song of 2009 in UK – despite only reaching 6 in the chart.

  23. 23
    wichita lineman on 18 Jan 2010 #

    Re 22: Yes – same goes for Snow Patrol’s Chasing Cars which was most played song of the decade AND iirc broke Frank Sinatra’s ‘most weeks on chart’ record (for My Way) which had stood for the best part of 40 years. And to show how not toooo much has changed re long tails/highest positions/popularity, Chasing Card peaked at 6, My Way at 5.

    I think the chart is still the best gauge of a record’s popularity. The quirks of long tails, weirdly low peaks for acknowledged classics (Canned Heat’s Going Up The Country, current ad fave, no.19 in 1969, Journey’s Don’t Stop Believin’ didn’t even make the Top 40 first time out) or major stars having almost forgotten no.1s (Sinatra’s Three Coins In The Fountain) all tell us something that revisionism would warp. I heart the Guinness book.

  24. 24
    AndyPandy on 18 Jan 2010 #

    Tom makes a very pertinant point about the under-representation of certain types of music in the charts. And as he mentions this reached its apotheosis with dance music – even pre-1988 there was a very considerable network of dance (soul/funk/jazz)shops in existence – most major towns and cities had at least one and London and the south-east was rife with them – you just have to look in the back pages of an old edition of ‘Blues and Soul’ to see the evidence. Acts like Second Image, pre-pop Level 42 and Light of the World were massive in the early 80s and in the case of Second Image were just about ubiquitous on a large swathe of car stereos around 1983 but in the Top 75 pop chart they were fortunate to peak in the 60s.But then everyone was buying them in shops like Bluebird, Jifs or Slough Imports. I suppose this happened to a lesser extent with rock music and that genre’s specialist shops but the difference was that with the mainstream airplay they received (albeit often in the evening)a larger number of mainstream records shops at least stocked their records and therefore a proportionately higher number of their sales registered in the pop charts.

    Of course the situation snowballed post-1988 as along with a still greater proliferation of dance music specialists (now a large proportion of even medium-sized towns had one), many dance music fans stopped buying singles altogether and either bought compilations (from the pre-house “Streetsounds” onwards to the massive glut that followed from the Jackin albums in 86/87 onwards) always a far more significant presence amongst dance fans than albums, or the infamous mixtapes either market-stall, rave or specialist shop bought or taped directly from the pirates.

    Despite the deleterious effect this had on the music’s singles sales, by the early 90s (as I’ve mentioned on another thread before) some hardcore tracks were still shifting legendary amounts The Scientist “Exorcist” and Acen “Trip II The Moon” reputedly both well over 40,000 each and barely scraping the Top 50, Dance Conspiracy’s “Dub War” doing not much less and spending a solitary week in the Top 75 as did stuff like Nick O.D.’s (aka Nicky Blackmarket from just one such specialist dance store)”Spam” EP without even making the Top 75.

    Then you’d get some indie rock record selling 10,000 and getting into the Top 10. Not to get sidetracked or to over-generalise but isn’t that kind of thing also linked to possible theory that to many fans of more rock based music it isn’t about just music but about a slightly trainspottery desire to collect and obsess about the physical object of the disc whilst the generally the dance music fan just wants the tunes not particularly bothering how he gets them.

    Now hypothetically imagine if by some freak set of circumstances (and especially in the periods 1988-92 (house/hardcore) and 1997-02 (trance/hardhouse)) literally every purchase of a single had been logged to create the charts and there had been no such thing as compilations,mixtapes etc. In other words if a slightly more level playing field had existed (I say slightly as even in this slightly-surreal world dance music would have suffered due to its ignoring by virtually all the legal stations) and the charts had been compiled on raw figures of the tracks’ genuine popularity amongst record-buyers.

    Well it doesn’t need much imagination to realise that the old farts who control the music business and those at Radio 1 etc really would have had something to moan about.

  25. 25
    lonepilgrim on 18 Jan 2010 #

    re 25 but wasn’t part of the appeal of buying these records that one was outside of the mainstream – there was a connoiseurship of the obscure – whether past or present – which would have been threatened by chart success.

  26. 26
    Pete Baran on 19 Jan 2010 #

    I’ve got to say that the trainspottery desire to have a track was as, if not more, prevalent in the dance music scene of the early nineties especially, as fity percent of the London club audience seemed to fancy themselves as DJ’s. SO you would go clubbing, hear five massive new tunes and then you had have them. But without getting mainstream airplay, a lot of this would be a slowish burn sales crawl – with the tracks being hard to get hold of. And lets be fair, once they got a big commercial release, it was no long cool to like them.

  27. 27
    thefatgit on 19 Jan 2010 #

    @24 The Slinky/Frantic/Storm crowds were a generally knowledgable bunch and on that cusp between physical sales and downloads around 02/03 many were switching from buying vinyl/cd’s to using Kazaa/Limewire/Napster to find the tunes they were looking for rather than visiting their preferred stockist. It wasn’t until after 05 that iTunes began listing up to date compilations outside the sphere of the usual Ministry/Gatecrasher compilations.

  28. 28
    logged out Tracer Hand on 20 Jan 2010 #

    There’s a frustrating imprecision here about what is desired (not to mention why): is it a gauge of “what sells” or a gauge of “what people listen to”? The US charts attempt to steer toward the latter, the UK charts toward the former.

    But neither even pretend to reflect what people actually listen to. My dad listens to a hell of a lot of Sonny Stitt, but I think the last time he actually paid money for a Stitt album was some time in the mid 1980s, and it’s not like SS is exactly tearing up the airwaves.

    The answer presumably is that once people start paying a blanket fee for being able to play anything from “the cloud” – a la Spotify – these two categories of info – buying vs listening – will gradually become identical. Hurrah?

  29. 29
    AndyPandy on 20 Jan 2010 #

    Pete at 26: but those trendy trainspotters of clubbing lore made up a very small proportion of the club/electronic scene although they always received a disproportionate amount of coverage in shit magazines like Mixmag. The kind of tracks I was on about were sort of “mainstream underground” and usually viruently scorned by the “trendies” ie rave/hardcore, piano house, trance, hard house etc.

    And yes the djing fraternity did buy vinyl -and I never was saying people didnt buy the singles as my comment about the sales shows- but to many fans of the above genres it was about mixtapes (bought at raves, shops and market stalls or recorded off the pirates) and compilations that sufficed.

    But it’s probably in Happy Hardcore where the differential between chart success and popularity with British youth is most glaringly obvious.
    And on that scene there also existed the unique phenomenon of the multi-cassette pack which was almost became synonymous with the scene.

    And when Happy Hardcore was arguably at its most dominant ie 1994-96 I doubt there was one Top 75 emanating from the genre but you’d have been hard-pushed to spend much time at all on any council estate in England without hearing this dominance blaring out somewhere.

    I remember being at some of the last big old style multi-room multi- genre raves in London around 1995 (Ravelation etc)and having to pass through the happy hardcore rooms to get to the rooms playing my sounds and by then there were more people in these “arenas” (as they were rather grandiosly called) jumping around to Slipmatt, Dougal, that MC who used to wear the Arsenal shirt* or whoever than all the other rooms put together.Bloody hell did passing through those rooms full of hundreds of hornblowing, jumping up and down teenagers make me feel old!But at least it was better than the jungle rooms (this was before most of the London jungle clubs were forced to shut for a few years and well before students knew anything about it!)where everything smelled of crack and everyone seemed to be getting stabbed, assaulted and mugged!

    *just put that little memory about the Gooner MC in so I can say “We are Top of the League say We Are Top of the League”!

  30. 30
    logged out Tracer Hand on 21 Jan 2010 #

    I hope you mean crack cocaine. A sentence I never thought I’d say.

  31. 31
    AndyPandy on 21 Jan 2010 #

    yes that’d be even worse wouldn’t it? although some of the venues did smell like toilets…

Add your comment

(Register to guarantee your comments don't get marked as spam.)


Required (Your email address will not be published)

Top of page