I originally came from the entreprise software world (for 10 years) and before that I was in mobile & telecoms (8 years) so the last three years of immersing myself in consumer Internet, digital media & advertisings has been very eye opening. I arrived on this scene wet behind the ears assuming that the web was, as it seemed to me as a user, powered by the masses for the masses. Ah, the joys of youthful naivete.
I first learned the ropes around SEO (search engine optimization) and how for years this has been a cat-and-mouse game where people game the system (Google) through link exchanges, offshore SEO “agencies,” widgets, algorithmically optimized content and the like that degrades the quality of search results. I then learned about the large world of Internet arbitrage, lead gen, and “crap-taculous” revenue as one friend calls it. I learned about domain parking and how much money that drives for Google.
[update: I was not implying that all SEO programs are bad. To the contrary, I actively work with entrepreneurs on how to better drive SEO. I was simply pointing out that there is a "black hat" element to SEO that I (and many non SEO people) didn't know about. There are also "white hat" SEO people. The black hat element is what keeps Google trying to innovate to stay ahead of. That's all.]
And then I had another bubble burst. I had previously believed that the world of user-generated news sites were run on a open, crowd-sourced model. You submitted news to a website like Digg and if it was interesting content with a great headline and newsworthy text it would get driven up to the masses. I know the more experienced consumer web people will be laughing about now.
What I learned 18 months ago is that sites like Digg traditionally have not been “crowd sourced” so much as “mafia sourced.”
Here’s how I learned:
When I first started blogging 18 months ago or so I started asking people the best way to get distribution. I didn’t want to obsess with it (I have a day job!) but I wanted to be sure I wasn’t writing just for my mom. I had a lot of initial success publishing my content on Twitter and the truth is that it was fairly democratic. If I wrote an interesting story with a compelling enough title it tended to get good click throughs (2-4% each time I Tweeted & for reasons I’ve explained before I Tweeted 2-3 times into different day parts).
I had a few friends help with the initial distribution and explained how to do that in this post on how to blog effectively.
Then I started experiencing the magic of being “profiled” on a few sites. The first to hit was when I was on the WordPress home page. BOOM! Traffic rolled in. And then I got covered a couple of times on HackerNews and another major spike.
Next on my agenda – Digg. If HackerNews was big, Digg must be even bigger – right? I put up a button on my blog and noticed that I got a few clicks on the button. But it didn’t drive ANY traffic. Hmm. Not the expected reaction. I tried another experiment – I asked a friend of mine to submit a story on Digg. He had told me he used Digg all the time so I figured if people saw him there submitting a story it would bring at least some juice. Nada. Bagel. Zippo.
I called a friend of mine who is REALLY in the know. I’ll need to protect his identity I asked him why I sucked on Digg. He asked me who submitted my story. I told him and he said, “well, that’s your problem.” Huh?
“On Digg it really matters who submits your story. There is a small group of people that all work collectively to promote stories. We all know each other by online handles. We are all linked in IM (instant messenger). When we want a story promoted we ping each other and all of the power users will promote the story. When it starts breaking then the power of the crowds takes over. If you want a story to break on Digg just let me know. I’ll help promote it.
I never took my friend up on this offer. Blogging is a hobby for me and I love being able to learn about all of the technologies from a practitioners perspective rather than the Ivory Tower. But I don’t make a penny from blogging and I certainly wasn’t wanting to mafia source a bunch of users who probably would drive unfocused numbers to my blog. I care about quality entrepreneurs & investors engaging with me intellectually, not mob scenes.
I’m not talking about systems where a few friends conspire to get more coverage for their news – that obviously happens everywhere and in many ways is just part of hustling as an entrepreneurs. I’m talking about places that are systematically rigged by powerful trading networks of people who are paid to help propagate (and kill) stories. That’s a far cry from friends who are your personal user-generated marketing machines.
I called a bunch of other friends in the space and they all corroborated my initial friend’s story. Some offered to help promoted me on StumbleUpon, Fark and others. Uh, no thanks. Et tu, Brute? Yup. Seems these rings operate where they can. And I learned that some of these people I knew are paid to help promote stories. They have consultancies that guarantee you traffic and get paid to operate in these mafia rings.
One place that really has surprised me is HackerNews. At least from what I can tell (maybe I’m still PollyAna-ish about it because I love HackerNews) this user generated news is much less gamed. Not zero, but less so. They seems to make it hard to directly link to stories and I haven’t heard of HN mafia circles. It seems that they have a desired goal to control it as outlined by Paul Graham even this week.
So when I read the articles about the Digg V4 controversy (see: wikipedia & TechCrunch) it doesn’t surprise me that Digg would want to change its feature set to make it a more honest broker of user-generated news (the way Twitter seems to be to me). Yes, they are getting lambasted by their users for the recent changes but probably precisely because when you’re the mafia you don’t appreciate a little light shining. I must admit that I haven’t followed the Digg v4 controversy closely enough. I stopped watching when the Sopranos ended. So there might be more to users discontent than making the system harder to game.
I for one applaud any efforts to make user-generated news (and the web) more democratic & open.