Rick Ross Tries To Think

This morning I queried our database for DZone’s overall 2009 link statistics, grouped by link status. I thought you might like to see the results:

I won’t go into great length in my analysis (which would certainly bore you.) I will note, however, that DZone’s moderator team deserves our thanks for an AMAZING job of killing off spam before you ever have a chance to see it. 4.3 spams blocked for every link that makes it to the front page, and that doesn’t include the countless thousands that WOULD be submitted if we didn’t ban so many spammers!

For the past 9 months, since April of 2009, we have offered significant Twitter support for every link submitted to DZone. We make it easy for you to tweet any link, easy to sign up to follow DZone (and several thousand have), and easy to spot the biggest and most popular links.

As of this moment we have provided instant tweet support and shortened URLs for 399,864 links. In total, these links have garnered just 273,470 clicks. Now, some of you may say “a quarter-million clicks, that’s not too bad” but the truth is that this represents less than a week’s worth of normal clicks – a pathetic week at that!

In other words, after 9 months of steady support for Twitter, the sum total of the click-through to your blogs and websites really comprises only a few extra days worth of our normal traffic. Twitter adds less than 1 click to the attention the average DZone link receives. We’ve given lots of valuable screen real-estate and valuable time to supporting twitter, but I’m not sure the developer audience really cares.

What do you think? Has it been worthwhile to support twitter at your site? Have you captured any quantitative or qualitative data that supports how you feel? I know Twitter is all the rage, but it is honestly hard to believe that DZone members benefit much from all the space and attention we give to Twitter. In fact, it feels like we’re just helping to fuel the hype wave.

As DZone becomes more popular with users, it also becomes more popular with spammers! This past week Matt and I worked on several new tools to help manage the ever-increasing flood of spam that DZone attracts. I think we achieved a breakthrough on a key issue.

It used to be the case that all new links were visible to all users, and we relied on spam detection and vigilant moderators to remove spam quickly. This put a ton of pressure on our excellent moderator team, however, since it required constant scrutiny of to keep the new links queue clean. As the rate of spam submissions increased, so did the probability that some of that spam would be sitting in the top of the new links queue for a while (and also in the new links RSS feed, even worse!) Far too often, this spam would pollute the new links queue.

Now, we have come up with a way to segregate nearly 100% of that spam into an active moderation queue BEFORE it ever reaches the eyes of the general DZone audience. The difference is remarkable, and you may have noticed that DZone’s new links queue is a lot cleaner and more interesting now. (Well, I guess the spam links to “Asian Sex Vacations” and “Top 10 Recipes for Healthy Desserts” were colorful in some respect, but not interesting to DZone readers!)

Since we deployed the changes a week ago, over 1000 links have been blocked outright, and none of them ever got in front of our readers. Additionally, we have new tools for the moderators to manage the active moderation very conveniently. In short, it has never been so easy to spot and kill spam. It’s almost unfair, like shooting fish in a barrel, but somehow I doubt very many of you will argue that we should be more lenient or merciful to spammers!

If you’d like to be part of the DZone moderator team, just get in touch with me at rick@dzone.com, and I’ll be happy to consider you for the role. It’s easy and fun. You’d be surprised how satisfying it feels to block these junk links and ban the vermin who submit them!

Many thanks to the folks over at Clickfire. They were kind enough to do an interview with me which was published this morning. Much appreciated!

DZone Founder Rick Ross Interview

I noticed a tweet from @bigokro the other day saying “Spam has invaded DZone… I wonder how they’ll deal with it.” and it made me realize that most people thankfully have no idea how much spam gets submitted to DZone every day and how much effort we put into protecting you from it.

A quick check of the database showed that we have blocked over 70,000 spam links. We typically block 100 or more each day. In addition, there are thousands of users banned for spamming and tons of domains that are blacklisted so nobody can post links pointing to them. If not for those, the number would be a LOT higher than 70,000!

Here’s a short sample of the headlines we’re talking about. These were posted yesterday (along with many dozens of other fine examples.) I think you’ll get the idea pretty quickly:

• Free And Cute Babies Pictures
• Escort vip Madrid
• Madrid Escorts
• Carl Lewis: Olympic Medals through the Vegan Diet | Vegan123
• 10 Steps to Open Your Own Consulting Business and Work from Home.
• Ladat: Cures and Remedies
• looks like a unlimited bandwidth web hosting..cool site
• Free Laptops
• Leather bar stools

Just for fun, I exported a list of the 70,452 links (titles,dates and users – no urls) that have been blocked since we launched DZone. It’s just a zipped text file, but rather large (about 1.6 mb zipped.) Feel free to download it and have a look at the outrageous junk that “SEO Experts” and outright spammers think you need to know about. It’s good for a laugh, really it is!

I’m a product of my upbringing, and (using the term loosely) I grew up in the 60’s and 70’s. It was the time of Star Trek and a real-world space program, both of which had tremendous influence in shaping my belief that the pursuit of scientific knowledge leads to good things.

The voyagers of the starship Enterprise had an excellent situation. They simply had to do whatever it was they were good at doing, and somehow resources were available to do keep doing it continuously and with little regard for cost. It’s a model that appeals to the closet utopian in me, but it’s pretty far from the day-to-day economic reality most of us live in.

Today, I listened to Tim Berners-Lee’s TED talk where he urges us all to open up our data, all data, and make it available for linked use. I love the idea of “raw data now”, but it scares me. It happens that DZone is floating on an ocean of data. In our three years online we have tracked how millions and millions of developers have used hundreds of thousands of links from tens of thousands of domains. I imagine that intriguing insights about developer trends could be drawn from this data. It might be even more intriguing if it could be correlated to open source project activity and commit rates or some similar data pool that someone else possesses. Sir Tim’s idea of exposing “raw data now” challenges us to engage in a broad experiment and find out what happens.

I’m close to taking up Sir Tim’s challenge, really I am. My desire to see what we might learn confronts my business training, which suggests that possessing information exclusively is my competitive advantage. My instincts, however, tell me not to sweat it and that things will be alright.

Your input matters a lot, and I’d like to hear your ideas about how you would want to leverage DZone’s data if we opened it up (of course, no personal data would be shared!) What new and interesting possibilities would this create? Are there steps we could take in this direction without throwing the doors wide open and inviting the world into our databases? What would you do?

I’m going to give this serious thought, and I would genuinely like to hear from you. Thanks!