Sometimes I think I should just go down to the corner store and apply for a job as a cash register clerk. Matt and I sat down to get started working yesterday after the holiday vacation and immediately discovered that we had a huge problem on our hands - a grim consequence of an earlier problem caused by forum spammers who had basically attacked our system just before Christmas. I'm so angry now that I would most likely commit a violent crime if I could get my hands on these spammers, and I'm not a violent person by nature.

It started sometime around mid-December, when a group of fairly sophisticated spammers began probing our forum system. We have always provided limited areas where unregistered “guests” could post questions and comments, and this was apparently an invitation for these spammers to post over 50,000 new forum messages pointing to every type of internet scam. Gambling, pills, lose weight, gain length, you name it - it was all there! Not only had they posted tons of garbage into the Javalobby forums, but they had also managed a cool trick of using attachments to automatically redirect visitors to various porn sites. It was a veritable buffet of everything you don't want on your legitimate website.

The day after Christmas Mike and I noticed unusually high load on one of our main servers, which tipped us off that something was amiss. After all, how could there be unusually high traffic on the day after one of the biggest holidays of the year? There was no significant traffic from any particular IP address, which is usually the first sign of someone up to no good. Monitoring our database activity revealed that a large number of new messages were being created, but they were from IP addresses all over the map. Further investigation pinpointed the forums that allowed anonymous posting, and we spent the next few hours purging thousands and thousands of useless spam messages from those areas of our system. We closed the door on postings from unregistered guests, and we were relieved to see the server load parameters fall back down into normal ranges.

That was not the end of it, however. A site like Javalobby is constantly being spidered by the major search engines, so they were quick to pick up all of these new trash messages. It had been aggravating to spend holiday time cleaning up the unwanted mess left by our guests, but the real problem didn't surface until we started going through our normal morning routine yesterday, having just returned to work from our holiday break. We generally take a look at a variety of statistics in the morning before proceeding into whatever development work we're doing. Having been out of the office for almost two weeks, we had a lot of stats to look at. It took no time to see that something was wrong - traffic was down. A little more investigation revealed the problem.

We had completely disappeared from Google's main index! If you run a website, then you know how serious a problem this is. On any given day over 10,000 visitors arrive at Javalobby as a result of Google searches, and suddenly they stopped coming! We had apparently been grouped together with the spammer's viagra and casino sites, and poof! Suddenly we no longer existed in the eyes of Google, the world's largest search engine. Countless thousands of well-ranked pages gone in a blink. Perhaps you now understand why I would commit a violent crime if I caught those forum spammers? In essence, they have wiped out strategic positioning that we took years to build.

We are, of course, doing everything we can to address the problem, but there's not very much you can do. Google doesn't send you an email to say there is a problem, and there's nowhere you can turn for definitive answers about exactly what they have found fault with. This isn't like some type of justice system where you get to face your accuser and defend yourself. Instead, you are tried and convicted without knowing a problem even exists, and then you learn about it after you are summarily executed. At that point you're left to turn, ironically, to the Google search engine to start looking for steps you can take to recover. In the immortal words of Chuck D, "It all adds up to a f**king situation."

Allan has said for years that the problem of the Microsoft monopoly would pale in comparison to the coming Google problem. Allan has an uncanny way of seeing where the real issues are. Let's not get caught up in a discussion of whether Google is a monopoly, legally or practically. I will say, however, that there's a tremendous amount of power concentrated in the proprietary Google databases. If you're not in Google, then you don't really exist on the web.

Anyway, I'm off to the corner store to fill out my job application. So long, and thanks for all the fish. I love you guys.

Rick