The guys behind WordPress had the foresight and energy to create Akismet to help control blog spam, and the blogging world wouldn’t be the same without it. A popular web application like WordPress is a big target for spammers, since they can write bots to pump their spew into it. Akismet helps cut that problem down to size.

I was really pleased to see a cool site called StopForumSpam.com, created by a couple of developers who believed they could help solve the problem. The idea of StopForumSpam.com is simple – make it easy for people to report spammers on their sites, so others can block them. It’s fast, free, easy to use, and I feel it can help control the damage caused by some of the worst offenders.

Over a half-million known spammers are already listed at StopForumSpam.com, and more are added every day. If you run a forum site you might want to take a look at the mods that are available for phpBB, vBulletin and simpleMachinesForum. There’s also an API available that is very easy to incorporate into your own application.

We liked StopForumSpam.com so much at DZone that we decided to adopt the site and provide hosting and support for it in our data center. The founders have been carrying the load all by themselves, and it’s a pleasure to be able to help them out in their worthy mission. StopForumSpam.com can’t solve the whole problem, but it can certainly be a piece of the solution. I hope it will be helpful to you!

This morning I queried our database for DZone’s overall 2009 link statistics, grouped by link status. I thought you might like to see the results:

I won’t go into great length in my analysis (which would certainly bore you.) I will note, however, that DZone’s moderator team deserves our thanks for an AMAZING job of killing off spam before you ever have a chance to see it. 4.3 spams blocked for every link that makes it to the front page, and that doesn’t include the countless thousands that WOULD be submitted if we didn’t ban so many spammers!

For the past 9 months, since April of 2009, we have offered significant Twitter support for every link submitted to DZone. We make it easy for you to tweet any link, easy to sign up to follow DZone (and several thousand have), and easy to spot the biggest and most popular links.

As of this moment we have provided instant tweet support and shortened URLs for 399,864 links. In total, these links have garnered just 273,470 clicks. Now, some of you may say “a quarter-million clicks, that’s not too bad” but the truth is that this represents less than a week’s worth of normal clicks – a pathetic week at that!

In other words, after 9 months of steady support for Twitter, the sum total of the click-through to your blogs and websites really comprises only a few extra days worth of our normal traffic. Twitter adds less than 1 click to the attention the average DZone link receives. We’ve given lots of valuable screen real-estate and valuable time to supporting twitter, but I’m not sure the developer audience really cares.

What do you think? Has it been worthwhile to support twitter at your site? Have you captured any quantitative or qualitative data that supports how you feel? I know Twitter is all the rage, but it is honestly hard to believe that DZone members benefit much from all the space and attention we give to Twitter. In fact, it feels like we’re just helping to fuel the hype wave.

We’ve been using subversion at DZone for several years, and looking backwards I can comfortably say it was a solid improvement over the CVS system we used before. Lately, the newer “git” and the popular GitHub project hosting site appear to be coming on strong, and I’ve started to use them in OSQA – an open source project I’m becoming involved in.

I’m willing to trust that there’s something really useful about git, something that qualifies it as a leap forward and not just a “newer must be better” alternative. If I’m completely honest, however, then I have to admit I don’t yet understand what it is? GitHub, in particular, confuses me.

The project I’m interested in has been forked dozens of times (since forking a project repository is apparently the first thing you do to get started.) As a result, there are now dozens of independent commit streams and lines of development branched off the primary original repository. It feels chaotic and disorganized.

To be sure, the Network Graph Visualizer feature is one of the best tools I have yet seen for viewing such a set of mostly diverging (and occasionally re-converging) streams. You can see each commit in a timeline view that also shows how and when the streams are being merged back together.

What escapes me, however, is any understanding of how this supports and encourages teamwork? While it appears to make it easy to working independently and resynch from time to time, my guess is that you’ll need more than GitHub to work effectively as a team. A dashboard that shows the various commit messages is simply not enough for most teams to function together in a well-organized, coordinated effort. Merely having a wiki available to the project as part of the GitHub service is not really enough for most teams – it can too easily remain unused or under-used. I still want something like Trac or Jira to help discuss and prioritize development tasks.

I’m sure I don’t yet fully understand how git and GitHub make it easy to merge these independent streams. It is very cool that git and GitHub are aware of and respect the ancestry of the project, which may actually be what represents the major leap forward. I just need to get my hands dirty with this tool and find out how it works in practice.

I’d be delighted to hear the pros and cons of your experiences with git and GitHub (or Gitorious and other git-related tools and services.)

This past weekend I was searching for open source implementations of a web application that has recently become popular. I came across several, but one in particular seemed to have really good community energy and direction, so I explored it further.

The app in question is built with Ruby-on-Rails, with which I have enough experience with to be reasonably confident. Rails seemed like a good choice, so I continued to dig deeper. I learned that the application is relatively new and has no deployments running at serious scale. Since I anticipate my little site could possibly blossom into a million pages per month or so, I was somewhat concerned that no existing sites approached anything like that level of traffic.

Still, a million pages is not that much compared to the big sites, and I didn’t really expect scalability to be a big problem ( at least not unless the site grew much larger.)

Then I hit the issue that stopped me dead in my tracks, and I realized I had truly become more conservative in the choices I am willing to make. This Rails app uses MongoDB for persistence, which was a step too far off the beaten path for me. Don’t get me wrong, as I think MongoDB is genuinely cool and probably works great. It may even be the best possible tool for the job.

But I will probably never know. I simply don’t have the stomach for the double-risk of choosing a fairly new open source web application AND ALSO the non-traditional data persistence system it mandates. If the web app supported multiple implementations of a persistence interface, then I would probably try the MongoDB with confident knowledge I could switch to a more familiar SQL environment if necessary. Lacking that, I just couldn’t go out on a limb with the persistence component. I just couldn’t.

I feel a little bit cowardly, but I have paid the price for being at the bleeding edge before and don’t tread into such territory casually. I’ve got a lot of SQL experience and have the tools and the resources to manage, backup, and maintain high-performance SQL databases. If I took the bet on MongoDB, I’d have to start all over, and I just couldn’t balance the risk-reward equation.

Maybe if I was still in my 20′s, or even still in my 30′s…

I’ve been wanting to create a new home for developers who want to blog, so that’s what I hope DeveloperBlogs.com will become. I’ve installed the latest version of WordPress-MU and a cool plugin called BuddyPress. Together these should provide the foundation for a reliable multi-user blogging community.

Would you like to have a blog here to share your thoughts? If you feel you’re going to be at least somewhat focused on developer-related topics, then I’d be happy to have you on the site. I have long admired ScienceBlogs.com, and there’s no reason we should enjoy a similar site for developers.

Shoot me an email if you want to request a blog. If you’d like to move your existing blog to a new home here, then we can probably arrange that, too.

As DZone becomes more popular with users, it also becomes more popular with spammers! This past week Matt and I worked on several new tools to help manage the ever-increasing flood of spam that DZone attracts. I think we achieved a breakthrough on a key issue.

It used to be the case that all new links were visible to all users, and we relied on spam detection and vigilant moderators to remove spam quickly. This put a ton of pressure on our excellent moderator team, however, since it required constant scrutiny of to keep the new links queue clean. As the rate of spam submissions increased, so did the probability that some of that spam would be sitting in the top of the new links queue for a while (and also in the new links RSS feed, even worse!) Far too often, this spam would pollute the new links queue.

Now, we have come up with a way to segregate nearly 100% of that spam into an active moderation queue BEFORE it ever reaches the eyes of the general DZone audience. The difference is remarkable, and you may have noticed that DZone’s new links queue is a lot cleaner and more interesting now. (Well, I guess the spam links to “Asian Sex Vacations” and “Top 10 Recipes for Healthy Desserts” were colorful in some respect, but not interesting to DZone readers!)

Since we deployed the changes a week ago, over 1000 links have been blocked outright, and none of them ever got in front of our readers. Additionally, we have new tools for the moderators to manage the active moderation very conveniently. In short, it has never been so easy to spot and kill spam. It’s almost unfair, like shooting fish in a barrel, but somehow I doubt very many of you will argue that we should be more lenient or merciful to spammers!

If you’d like to be part of the DZone moderator team, just get in touch with me at rick@dzone.com, and I’ll be happy to consider you for the role. It’s easy and fun. You’d be surprised how satisfying it feels to block these junk links and ban the vermin who submit them!

Many thanks to the folks over at Clickfire. They were kind enough to do an interview with me which was published this morning. Much appreciated!

DZone Founder Rick Ross Interview

I noticed a tweet from @bigokro the other day saying “Spam has invaded DZone… I wonder how they’ll deal with it.” and it made me realize that most people thankfully have no idea how much spam gets submitted to DZone every day and how much effort we put into protecting you from it.

A quick check of the database showed that we have blocked over 70,000 spam links. We typically block 100 or more each day. In addition, there are thousands of users banned for spamming and tons of domains that are blacklisted so nobody can post links pointing to them. If not for those, the number would be a LOT higher than 70,000!

Here’s a short sample of the headlines we’re talking about. These were posted yesterday (along with many dozens of other fine examples.) I think you’ll get the idea pretty quickly:

• Free And Cute Babies Pictures
• Escort vip Madrid
• Madrid Escorts
• Carl Lewis: Olympic Medals through the Vegan Diet | Vegan123
• 10 Steps to Open Your Own Consulting Business and Work from Home.
• Ladat: Cures and Remedies
• looks like a unlimited bandwidth web hosting..cool site
• Free Laptops
• Leather bar stools

Just for fun, I exported a list of the 70,452 links (titles,dates and users – no urls) that have been blocked since we launched DZone. It’s just a zipped text file, but rather large (about 1.6 mb zipped.) Feel free to download it and have a look at the outrageous junk that “SEO Experts” and outright spammers think you need to know about. It’s good for a laugh, really it is!

I’m a product of my upbringing, and (using the term loosely) I grew up in the 60′s and 70′s. It was the time of Star Trek and a real-world space program, both of which had tremendous influence in shaping my belief that the pursuit of scientific knowledge leads to good things.

The voyagers of the starship Enterprise had an excellent situation. They simply had to do whatever it was they were good at doing, and somehow resources were available to do keep doing it continuously and with little regard for cost. It’s a model that appeals to the closet utopian in me, but it’s pretty far from the day-to-day economic reality most of us live in.

Today, I listened to Tim Berners-Lee’s TED talk where he urges us all to open up our data, all data, and make it available for linked use. I love the idea of “raw data now”, but it scares me. It happens that DZone is floating on an ocean of data. In our three years online we have tracked how millions and millions of developers have used hundreds of thousands of links from tens of thousands of domains. I imagine that intriguing insights about developer trends could be drawn from this data. It might be even more intriguing if it could be correlated to open source project activity and commit rates or some similar data pool that someone else possesses. Sir Tim’s idea of exposing “raw data now” challenges us to engage in a broad experiment and find out what happens.

I’m close to taking up Sir Tim’s challenge, really I am. My desire to see what we might learn confronts my business training, which suggests that possessing information exclusively is my competitive advantage. My instincts, however, tell me not to sweat it and that things will be alright.

Your input matters a lot, and I’d like to hear your ideas about how you would want to leverage DZone’s data if we opened it up (of course, no personal data would be shared!) What new and interesting possibilities would this create? Are there steps we could take in this direction without throwing the doors wide open and inviting the world into our databases? What would you do?

I’m going to give this serious thought, and I would genuinely like to hear from you. Thanks!