Recently in Spam Category

Via TUAW I found this Mail.app rule for stopping image spam, where the body of the spam message is in an image and is usually trying to get you to buy penny stocks. Because these messages use images and not actual text, they often evade spam filters.

Having a rule for Mail.app is all well and good if you use a Mac (which I do) and use Mail.app (which I don't). But thankfully you can also achieve this in Mozilla Thunderbird and it's no less difficult to do so, either. Here's how to do it.

Having spoken to others bloggers about this it appears I'm not alone in receiving a huge increase in comment spam lately. The majority of it starts with phrases like "Hello, nice site this" and "Can I share some resources with you?" followed by a big list of URLs for ringtones sites (many of which pointing to hijacked wikis and forums). Thus far they've all been blocked, thanks to my various anti-spam plugins, but they've still made it into my database and require checking every once in a while just in case a legitimate email has inadvertently slipped in there - it hasn't happened in a long time but I still like to check first. Considering I'm now getting about 100 spams a day, this often proves to be somewhat labourious.

I recently turned off trackback, which is something that no-one seems to miss so it'll stay off for the time being. I've been getting almost nothing but spam from it lately so I saw no reason to keep it on - I was getting about 1 legitimate ping for every 99 spam pings, which is just not worth it.

Unfortunately, since people are commenting here less, that figure is also applying to comments. Lately I've been barely pushing 2 legitimate comments a day, and still getting about 100 spams. By rights I should really be disabling comments too with those sorts of statistics but people are at least commenting, from time to time. It probably doesn't help that I haven't really said a lot of late so people haven't had a lot to comment on.

I'm going to upgrade to Movable Type 3.31 later on this weekend so we'll see what effect that has, if any.

One of my old email addresses gets a lot of phishing emails, and as both IE7 and Firefox 2.0 will have anti-phishing features I've been keeping them to test the filters.

Some of them are a bit amateurish, however. Take this one for example:

We are contacting you to remind you that on June 12 2006 our Account Review Team identified some unusual activity in your account. In accordance with PayPal's User Agreement and to ensure that your account has not been compromised, access to your account was limited. Your account access will remain limited until this issue has been resolved.

Wait a minute... today's only June 7th 2006, and the email was received on Monday... Do they know something I don't?

Incidentally the site had already been taken down.

Just wanted to congratulate Colin Ogilvie for posting the 7000th comment on this blog. By my reckoning, that means that the 10 000th comment should come sometime in late 2007.

Note that the ID of that comment is 8947, which means that since the comment IDs were last reset in December 2003 there have been nearly 2000 deleted comments - the vast majority being spam.

That's not as bad as trackback spam - I've had 776 legitimate trackbacks and the latest trackback had an ID of 4759. In other words, 84% of trackbacks have been spam. Yikes.

Those of you who have been reading for a few months may remember that back in October some idiot spammer used my domain's email addresses as return addresses in spam. This resulted in over 20 000 messages from servers, saying that "my" message was spam and had been rejected, or that the email address didn't exist - at times, several were arriving every minute. The problem has now gone away but it took a couple of months before my inbox returned to normal levels.

This isn't a one-off however - it happens to many people, including one Michael Pollitt who wrote about his experience of it in the Guardian on Thursday. Though the attack on him was slightly different - his name and email was used to spam guestbooks for pay-per-click spam - it still meant that he got hundreds of emails.

If you're on the receiving end of this kind of attack, using your email becomes very difficult since you have to sift through all of the failure messages - I wouldn't be surprised if I inadvertently deleted or missed some important emails during that time. It also strains your mail server and can put you in a difficult position if you don't have a very understanding host.

Unfortunately, spammers are among the worst of the earth's scum and as long as they keep on spamming people will suffer like this.

Having read this guide, I am seriously thinking about sending the following letter:

Dear Sir,

On Friday 14th Apri 2006 I received an email communication from your company, [name removed], regarding seminars that you are running in London. This email was unsolicited - I have neither heard nor made any previous contact with your company, and the email was sent to a personal email address that is not listed publicly. The email is therefore a violation of the Privacy and Electronic Communications Regulations of 2003.

I wish to recieve no further communication from your company, and demand that you remove my name, email address and all other personal information about me from your database and records. I would also request that you detail all information that you currently have on file about my identity, in line with the Data Protection Act 1998, and indicate where you found this private email address from.

Should you fail to comply with this request and further communication is received, then I shall reserve the right to pursue this matter through the Small Claims Court, as I regard these unsolicited messages as an annoyance and a nuisance.

Yours Faithfully,

Neil Turner

Though most of my spam comes from outside the UK (and a lot of it not even in English), I do get a few items from UK companies, this being an example one. Really, these companies should have no excuse. As it happens, the company in question chose to print its postal address in Bedford.

Update (29th April): Oh dear, another email from them. Now it gets interesting :) .

Has anyone else noticed lots of spam comments on high profile blogs which all point to MyNewsBot.com? I've been hit before but deleted the comment because it looked suspicious. Searching Google for 'mynewsbot' returns lots of comments by a user called 'mynewsbot' but no articles about the site itself. Some of these comments appear on high profile sites like Engadget, which Google reckons has been hit over 500 times. Overall there are nearly 64 000 pages affected.

The comments have perhaps slipped through comment filters because they do look like legitimate comments, in that they don't look obviously spammy and often relate vaguely to the subject matter. Though the site still has a Google PageRank score of 0, it has now got a lot of links to it, and yet no-one, as far as I can tell, has actually posted an entry about the site.

It's all very fishy.

Lately the trackback spam I've been getting has been quite different to before. Rather than have their spam target their own sites, they are pointing them at things like message boards and other comment threads.

These message boards have had JavaScript code posted in them which then forwards the user to the real site. In other words, the spammers are finding message boards which do not sanitise the HTML posted in them first (i.e. they accept <script> tags) and posting scripts in new threads, then spamming the URL of the thread.

It's interesting because, on the one hand, it gets around URL-based filtering. But it's also interesting because the message board page is the one that gets a PageRank boost from the site, not the actual site itself. However, the message board page, as well as having a piece of JavaScript, has lots of generated text and links to porn sites, so the PageRank effect should be passed on.

It's quite clever, if a little worrying. Maybe we need to start notifying the owners of these message boards and have them delete the threads in question, and have them tighten up security on their forms. In any case, my spam prevention mechanisms block these from getting through anyway because they also match common words used in spam, and do IP and DNS lookups.

About 6 weeks ago a spammer starting using random addresses at the domain as the return address for his canadian pharmacy spam, which resulted in me being deluged by delivery failure messages until I turned off the 'catch-all' feature in my email settings which redirected all mail sent to this domain to my mailbox.

Anyway, I'm pleased to say that the deluge has now stopped, and that the catch-all feature is now enabled again. During the time that I had it disabled, I had the messages redirected to another account which I cleared out using Webmail, so I didn't actually lose any emails not sent to my main account.

I've also resurrected an old email address that I haven't really used in a while - neilturner at myrealbox dot com. This used to be my old primary address but I've since all but stopped using it, mainly because it receives so much spam. Anyway, mail from that account is being forwarded to Gmail, and Gmail's spam filter is currently filtering about 98% of the spam that account receives. Though as yet I've only received one legitimate message from someone who hadn't updated my details.

I don't know how permanent this will be though so don't go adding my myrealbox address to your address books. neil at this domain is still my primary address.

Since redirecting any catchall mail to another account two weeks ago, my main mailbox has been largely spam-free, which has been nice. However, since I've used a number of aliases at this domain over the past few months, I wanted to check if any mail had been lost in there instead of being sent on to me.

As it happens, there was only a bit of lost mail, most of which was marketing gunk anyway. But in the two weeks since changing the mail settings I've had almost 20 000 bounce messages, with new ones coming in even today. I now have some filters set up in the Webmail program on the server so that I can delete the majority of it very quickly, but I will keep checking to see if any legitimate mail accidently slips in there.

Also, apologies if you couldn't comment on here earlier - my host's MySQL database server was being overloaded (too many connections). It's working now.

Once again, I'm getting a whole load of Mytob-infected emails from an ADSL customer using Energis. It's the variant that, rather than using its own SMTP engine, uses that of the user's ISP, making them look more like legitimate emails.

It's the last part that annoys me, because surely Energis should have some anti-virus protection on outgoing messages sent via their servers. The SMTP server I use, provided by the university, checks outgoing messages for viruses and also runs SpamAssassin over them, too. There should be no excuse for ISP-owned SMTP servers letting through viruses like this; more to the point, ISPs shouldn't be allowing infected customers to access the internet at all until they have disinfencted their machines.

SPEWS - the Spam Prevention Early Warning System - has added around 900,000 Telewest IP addresses to its blacklist, according to this BBC News article. I would imagine that this would mean that much of their customer base will be affected by this.

The action seems to be less about protecting users of SPEWS and more about making a statement about Telewest's poor record on blocking users who have compromised systems which send out junk email. Later on in the article, another source reckons that only 16,000 machines are compromised, which puts the other 884,000 people whose IPs are blocked in a difficult situation and may mean that they find it difficult to send email. I noticed that a couple of regular Telewest-using commenters here both had their IP addresses in SORBS as well.

Telewest really should be doing more about the problem, such as temporarily disconnecting customers whose systems are compromised (as ISPs like PlusNet do) and offering them more help with disinfecting their systems.

It's quite likely that I'll be on Telewest next year and based on what I've heard about them I'm not really looking forward to it.

I recently joined Project Honey Pot and have helped to catch 3 bots which have found the honey pot on this site (I may have had more but user agent filtering will have probably blocked a few).

One of the IP addresses that has made it over here seems to be responsible for sending out Gouranga messages - this being the buzzword of the Hare Krishna cult which gets plastered on various road and railway bridges over here. Weird.

It's operating from an IP address owned by Eclipse Internet, incidentally.

This IP is blacklisted. Looks like someone else who I share this server with has been sending out spam, or at least done something to warrant inclusion on Spamhaus' SBL. Thankfully, it appears the account responsible has been shut down already so hopefully it'll go away soon.

I only knew about this when trying to send an email to a user on swipnet.se - the email was rejected solely because the IP of the SMTP server appears on one blacklist. In any case, I'm using a different SMTP server for the time being.

Sounds like there was a huge trackback spam problem yesterday. Normally I get hit at least a little bit but this time around I had nothing. Either I wasn't targeted or the attack was filtered out at the server level - it's running mod_security.

For those of you looking for better protection, Jacques Distler has a MT-DBSL modification to make it use blitzed.org which is apparently better at blocking open proxies than DBSL and has fewer false positives.

Just sent this email to one of our lecturers:

Hi,

Just thought I'd let you know that the university email filter has been identifying your emails about the Computing Forums as 'junk mail' and has been delivering them to the junkmail folder and not to students' inboxes. It seems to be because you have put the addresses of the mailing lists in alphabetical order; if you were to 'randomise' them then it should be delivered normally. I know this sounds stupid but this appears to be the case. You may like to take this up with the Computer Centre.

Best wishes,
Neil Turner
Final Year Computing & Information Systems

For the geeks among you, SpamAssassin was tripped by the SORTED_RECIPS and SUSPICIOUS_RECIPS rules, which scored 4.30 and 3.00 respectively and thus sending it over the score of 6 required for an email to be marked as spam.

incidentally, the email was about a forum session entitled "Working successfully at Microsoft", presented by Graeme Chapman, a services executive in MS's Finance Sector. If you're around the University of Bradford at 2pm on Monday 7th February you'd probably be welcome - it's in the John Stanley Bell Lecture Theatre in the Richmond Building.

It's been a while since the last installment of 'Spammers are stupid' (although some recent mentions would qualify), however I just had one today that amused me slightly.

Specifically it was an item of comment spam. I have to take my hat off to the spammer because it didn't look like spam at first, until I realised that the domain he used was hot-wet-sex.org (might want to add that to your blacklist). But most interestingly, the comment had an illegal character where the 'author' had tried to include an apostrophe. Now this is, in theory, impossible because I've made the comment form enforce Unicode, whereas this was an ISO-8859-1 character. Evidently, the comment form had not been used in this case.

A quick look at my access logs confirmed this. There was several matches for the IP address used, initially browsing with a Yahoo crawler user agent (hmmm...) and then submitting the form with an AOL Browser user agent.

So, from this, we have learnt that the current generation of spambots:

1. do not respect character encoding
2. Use a Yahoo crawler user agent, even though it was not from a server in Yahoo's IP range.

Maybe I need to do some user agent filtering and block out bots that claim to be Yahoo Slurp or Googlebot, but are not being run by Yahoo or Google.

If you're using MT-Blacklist and haven't been keeping up with the Comment Spam Clearinghouse lately, then you may not have had the chance to pepper your blacklist with some useful URL patterns. These block specific terms from being used in URLs in comments on your site - so for example, an URL patten called 'ringtones' would stop someone from posting the URL http://www.allthegoddamnannoyingringtonesyoucouldeverwantandmore.co.uk/ , but wouldn't stop an innocent comment talking about them.

Unfortunately, while normal strings are added automatically via the auto-update function, URL patterns are not. So, I present my list of URL patterns that you should consider blocking. Some of them are very nasty words so they're in the extended portion of this entry.

I think you all by now know my opinion about CAPTCHAs - those images with numbers in that you have to type out to prove that you're human (more details here). They're fine if you use a browser that supports images, and that your eyesight is good enough to allow you to read them, but that does not include everyone and as such it puts form submission beyond the reach of a significant number of people. As such, I've avoided them on here.

But last night I had an idea. I could still use CAPTCHAs, but if the user was unable to see the code in the image, instead of the comment being blocked it would be forcibly moderated. That way, the spam would not get shown, but any legitimate comments could be held and then approved by a human. I could also allow approved TypeKey users to skip the CAPTCHA, on the basis that once their initial post on here has been approved it's likely that they're not going to be spammers.

SCode is, as far as I know, the only CAPTCHA plugin for Movable Type and at the moment it lacks the functionality that I described above. But maybe this 'hybrid' approach will get around the problems that CAPTCHAs have had in the past.

I've found another method for potentially reducing trackback spam, this time blocking them at the server level. The code below is for Movable Type but could be rewritten for any system that allows for pings. You'll need to be on an Apache web server with mod_rewrite enabled for this to work.

Recently I've been getting spam for an anti-spam product (oh, the irony). Naturally, having two layers of spam protection already (SpamAssassin and Thunderbird's adaptive filtering) I don't need it, which is why seeing the same bloody product being advertised does get pretty annoying, especially as by spamming me they're creating a problem and then demanding money off me to solve it - like ransom in a way.

But anyway, the latest of their emails had the following subject line:

Join the thousands who are now mail-free

So that's how it works - it gets rid of all of your email! Very clever...

If you run an email service, you know when you've become popular because some bonehead spammer launches a dictonary attack against you. And, alas, someone has done this against Gmail.

If it isn't already, make sure hq-pictures.org is on your comment/trackback spam blacklists. When I came back from Cardiff on Thursday an attack featuring this domain had just started (with over 20 pings) and this persisted right through yesterday. Thankfully after those 20 or so it was blacklisted so none of the pings actually appeared but I imagine it will have put some considerable strain on the server. The attack ceased last night with 700 pings having been filtered from that domain alone.

The domain is in Jay Allen's Master Blacklist so you should have it in your blacklists by now but if not you may want to add it to be sure. paydayl0an.com has also been a domain that has cropped up a fair amount lately, with 46 denials.

incidentally hq-pictures.org seems to be connected with various other domains that I've been hit with recently - it has a very similar IP address. As such, I've also manually added extreme-porn.org which is also hosted in that IP range in case it's used for spamming in future. The other domains all appear to be blocked by my blacklist.

In the past 3 days I've had over 1100 comment spam denials. When you consider that since I last reset MT-Blacklist in August, I've had a total of around 2800 denials, you get some idea of just how badly the server has been hammered over the past few days.

By browsing the database and sorting its output, I've been able to get a list of the strings which generate the most hits. These are:

1. pictures-movies.org (651)
2. pics-videos.net (580)
3. pictures-and-videos.com (461)
4. onlineshop.us.com (236)
5. greatnow.com (59)

There are also a number of regular expressions and URL patterns which generate many more denials.

It would appear that the first three are related - although their WHOIS data is different they all point to web sites in the same IP range.

Brad Choate has ported a Wordpress hack that prevents open proxies from commenting to Movable Type in the form of MT-DSBL. It checks the IP address used by the commenter against dsbl which maintains a list of open proxy servers. Open proxy servers are often used by spammers and flooders to get around standard IP blocking.

What will happen now is that if you use an open proxy server when making a comment, instead of the comment being posted you'll be redirected to a page on dsbl.org explaining why your comment was denied. The plugin is really easy to install - nothing more than dropping one of two files into your MT Plugins folder. One version blocks all comments made via open proxies, the other will forcibly engage MT's comment moderation.

And yes, it works fine with MT-Blacklist.

(And now back to your regularly-scheduled non-political programming)

Dave's just come back from his cruise to find that his host had disabled his mt-comments.cgi script because it was being hammered by spammers and causing heavy CPU usage on the server. Dave has MT-Blacklist installed.

I'm seeing more and more bloggers disabling comments or only leaving them open for a short time to stop comment spammers from invading. If this is a trend, then it's going to put the comment spammers in a very difficult position - they're going to fast run out of sites with open comments to spam. By drilling us into the ground with incessant spam they're biting the hand that feeds them. It's only a matter of time before that hand stops feeding them and their plan backfires.

I'm getting hit quite heavily, and in fact I'm now forcing moderation of any comments using the .info and .biz domains in URLs. You may want to do the same.

Sometime in the past week MT-Blacklist blocked its 1000th comment spam since I last did a full install in August. That's a heck of a lot of spam in just over 2 months - on average about 14 a day.

Sadly, it appears the problem is only getting worse, not better. :(

Over the past couple of days I've had a major comment spam attack from a host of domains that weren't on the blacklist. Some were legitimate but others were pure garbage - non-existant domains with seemingly random URLs. I'm wondering whether it's an attempt to poison the blacklist, or to set 'markers' to show if the blog is vunerable and therefor allow for further attacks. MT-Blacklist caught some of them, because they had too many URLs or were posted to very old entries, but others only escaped from appearing on the public side due to my forced moderation of non-TypeKey comments.

I'm always getting hit by spam, at the moment from various subdomains of eu.com, but MT-Blacklist has been silently blocking these and the first I hear about them is when an item appears in the blacklist log. This is the first time anything has foxed MTB.

As Adam and Chris have noted, a company called D2Soft technologies decided to launch its potentially interesting new site rsscache.com by scraping email addresses from feeds and spamming them. In my case, the email address used in my feeds is different to the address I have on this site so it's almost certainly been scraped.

Sorry guys, but if you're going to launch a product, spamming is not the way about it. Publishing an RSS feed is not an indication that I want to opt-in to RSS-related spam. Many people, myself included, refuse to support the spam business model, and as such you've probably lost a whole raft of customers before you've even got started properly.

My host has recently upgraded SpamAssassin to version 3, and it includes some quite cool new rules which help to combat some of the more recent spammer trends. Here's some of the new ones I've noticed:

• Message body has many words used only once
• Contains an URL listed in the SC SURBL blocklist
• Long string of long words

It's resulted in some spam getting much higher scores than it used to, which is a good thing. I wish there was an option somewhere that meant that mail over a certain score was automatically deleted - currently mail that has more than 5.6 points is marked as spam but I'd like any mail with over 30 points to be deleted outright because it's highly unlikely that it won't be junk. 5.6 gets about 95% of spam but I get the occasional false-positive so I don't want any mail above that score to be deleted.

Dear user of neilturner.me.uk,

Your account has been used to send a huge amount of junk email messages during the last week.
Most likely your computer was infected by a recent virus and now runs a trojaned proxy server.

Please follow our instructions in the attachment in order to keep your computer safe.

Virtually yours,
The neilturner.me.uk support team.

I know this message is caused by a virus but it did amuse me considering I am the neilturner.me.uk support team. incidentally this is a relatively new MyDoom variant (variant M) - when I first scanned the attachment with the July 21st definitions no virus was found, but scanning again with July 26th definitions found it. Still, I'm trying to work out why the 'instructions' for fixing my 'spam problem' would be in a screensaver.

While I haven't had any comment spam lately (must be lucky... ), I have implemented another technical fix. To post a comment, your browser now needs to send a referer. As far as I can tell, most of the robots (including FloodMT) that are used for spamming don't send referers, so this should filter them out. Legitimate commenters with referers turned off will receive a polite message asking them to turn on referers.

To implement this on your own site, create (or modify) a .htaccess file in your MT folder, and add these lines:

<Files mt-comments.cgi>
RewriteEngine On
RewriteCond %{HTTP_REFERER} ^$
RewriteRule .* /norefer.shtml [L]
</Files>

This seems to work okay as far as I can tell. You'll need to modify 'mt-comments.cgi' to point to your MT commenting script if you have renamed it, and will need to modify '/norefer.shtml' to point to the path of a file that explains the error.

Update: I've taken it off. It seems to throw errors when commenting using TypeKey. Bah.

Paul Lindner at the Everything TypePad blog has posted how TypePad has tackled the problem of comment spam. As far is I can tell, when you post a message to a TypePad blog, TypePad checks your machine to see if it is running an open proxy server. If you aren't, your comment is posted as normal, otherwise you get an error. Apparently, this blocked 20 000 spam comments on the first day of implementation alone.

So what about those of us who aren't TypePadistas? There are rumoured to be around 1.5 million machines with open proxies (often caused by backdoor trojans), so maintaining a list will be stupidly complex. OPM has a list of known open proxies and is used by systems like SpamAssassin when an email is received, so perhaps a modified mt-comments.cgi could query this and deny comments. Or the script could simply do what TypePad does and probe the machine for an open port, although my experience of logging on to IRC servers with SOCKS probing means that this may slow down the login process considerably if the machine has a stealthing firewall.

I'm sending this to the LazyWeb, just in case someone out there can think up a technical solution. Implementing something like this could really make a difference in the fight against comment spam.

Richard has a good entry about computer security, but in it, when talking about measuring the effectiveness of Thunderbird's new junk mail filter, he says:

It's rather hard to tell whether it's working at the moment though because all the spam zombies are being taken down by Sasser.

This is something I've noticed too. The amount of spam I've got over the past few days has been much lower than usual, and a lot of what I normally get is through zombiefied relays. Maybe this is a positive side effect of the worm?

Let's hope that the outbreak encourages people to upgrade their protection, like installing firewalls and whatnot. I seriously doubt it'll make much difference but we can at least cling to that hope.

I received an email this evening with the subject of 'similar audience'. It was sent to webmaster at neilturner.me.uk, which always arouses suspicion since I never use that alias for anything - when I think of 'the webmaster' I imagine a geeky guy in purple spandex from The Big Breakfast. What did the email contain?

I just visited your site, Neil's World, and thought your audience might enjoy our free online tarot readings.

I think it's quite obvious the sender didn't visit the site, or if they did they didn't spend much time here, because until this post not one entry in 1734 included the words "tarot" or "horoscope". Or "superstition" for that matter.

It's like the emails I get saying that "your site is not listed in some search engines!". I'm sure it isn't in all of them - I can't see why I'd want to be in a search engine for sites about collecting blu-tack by divorced transvestites called Bob - but I'm in all the major ones as far as I can tell and doing well too. The ones that say that 'your webmaster isn't responsible for monitoring your search engine performance' also amuse me since I am "the webmaster". Although I don't own any purple spandex.

Update: I had this one today:

I found your site http://www.neilturner.me.uk by searching boxing on various search engines, and saw you are ranking well. I thought we could help each other at no cost and with very little work.

I run a site similar to yours, and was wondering if you would like to trade links with me?

The site sold tickets to boxing matches. Now, while the word 'boxing' has appeared now and again in here, it's almost always been in the context of 'Boxing Day', a public holiday in the UK and other countries. Somehow, I don't think a student's weblog, with oodles of content, is any way similar to a site that exists purely to get commission from another boxing tickets site.

No, I had no idea what that meant either. It was the subject of an email that came to me via my Scrapie webmaster address, and it encouraged me to visit the site 'aicworld.info' for more information.

The web site doesn't look like much. To the blind eye, it's a blank page. But if you take a look at the source code, you'll actually see that a 1x1 pixel Java applet is loading in the background and is merrily downloading a file.

Suspicious, I went on to Google and found this page on Code Fish Spam Watch which gives details about this little critter. Turns out it uses an exploit in Microsoft's VM for Java to install a keylogging trojan horse virus on your computer, which is designed to pick up Passport and AOL logins, plus login details of various Australian online banks and send them to a Russian email address. The coding is quite clever because the keylogger is able to hide itself from detection using DLL hooks - it won't appear in Windows Explorer or Task Manager.

Fortunately, I don't have Microsoft's VM on here, preferring to use the official Sun/Javasoft JRE. And you should too. Although the specific flaw that this trojan exploits has been patched, Microsoft's VM is now obsolete, and is missing many Java innovations, so you should almost certainly uninstall it and switch to Sun's offering.

I've always wondered why the univeristy's copy of SpamAssassin is very forgiving to emails advertising sites that sell various penile-enhancing drugs and the like. Last night, I learned that it's because some legitimate mail within the pharmacy department contains the names of these drugs, and SpamAssassin was blocking these messages.

Darn those pharmacists.

Looks like a spammer has decided to try marking his emails with Habeas watermarks again. From the looks of things, it's the same guy as last time - I get around 3-4 emails a day that all look very similar all trying to sell me drugs that I don't need. For some reason this spammer seems to think that even though I didn't want those drugs 6 hours ago, I may well want them now so they'd better send me an email just in case. I suppose it's nice to feel wanted.

Anyway, usual advice applies: if you receive a junk message with a Habeas watermark - and SpamAssassin is one product that picks these up - then go to the report spam page on their site and copy and paste the entire email (with the all-important headers) into the box and send it in. That way, the IP addresses responsible (which appear to be home users with machines that have become zombified via trojan horse viruses) can be blacklisted and the spammer will hopefully realise that this is a bad idea - either because his email isn't getting through because it's being blocked, or Habeas find out who he is and serve him with a copyright infringement lawsuit. It'll probably be the former, but I'd like it to be the latter since the court case could prove somewhat interesting.

And to follow on from the previoius entry, I am knackered so it's off to bed I go.

I've been having a brief email conversation with the owner of SERPs, a website about a competition orginating from the alt.internet.search-engines newsgroup on Usenet. The site looks innocent enough, but it turns out the URL got onto Mark Carey's Blog Spam Database after the owner made what he claims was an on-topic post to one of Mark Carey's entries. A quick Google search over Mark's site would suggest this was the case as I came across a couple of entries where posting this URL would have perhaps been appropriate.

I've removed the URL and its .com variant as a result, but it underlines the major problem with blacklists - if an innocent domain gets caught in the crossfire it can make things really difficult. Ask any email publisher what they think about email blacklists like SpamCop and you'll probably get a general thumbs-down, since legitimate sites can get into the blacklist far too easily.

It's therefore perhaps best to double-check what you're importing. If I remember correctly Jay Allen does have some method of checking sites first, so the master blacklist should still be alright, but you may like to think twice before importing other people's lists.

Couldn't think of anything interesting to post today (it's been such a slow news day that movabletype.org got into the BlogDex top 50 for no real reason), other than this monster of a spam message. Here's what SpamAssassin had to say about it:

Got my first flood attack today - 31 comments posted to the previous entry. It would've been more, but MT managed to throttle some of them since quite a few were from the same IP address. The bulk, however, used different IPs so the comment throttling wasn't so useful in that sense. All the messages said "WINDOWS SUX HAHAHAHA LOL" followed by random gibberish. The names, email addresses and URLs were all randomly chosen too. From what I gather, it's typical of an attack using the FloodMT Python script.

Anyway, since I have MT-Blacklist installed cleaning up afterwards was pretty quick. But it's made me likely to introduce comment registration when Movable Type 3.0 comes out, or something on those lines.

In amongst all the emails infected with MyDoom, came this little gem:

or der conf irmation. yo ur or der sho uld be shi pped by Janu ary, vi a fe dex. yo ur fe deral expr ess tra cking nu mber is %RANDOM_WORD. th ank y ou fo r regis tering. yo ur u serid is: %RANDOM_WORD

It's almost as bad as the 419 scams which have "I am -----, president of ------....", where the sender forgot to add the names before sending the email.

Go and read this entry at Musings, in particular the first comment. The commenter is the owner of the site linked to as 'Crapflooders'. Posting comments like that is just asking for it, really.

And now I'm so going to get my arse spammed over here, but what the heck, it made me giggle.

In another comment spam news, earlier this morning Jay announced a new private beta of MT-Blacklist. The final version should be ready pretty soon then, which should bring in throttling and other MT2.66x features. There's also David Raynes' new Optional Redirect plugin which adds a new attribute to <$MTCommentAuthorLink$> so that you can control whether it redirects or posts the URL as normal. In theory, you could combine this with MTRegex and create a 'trusted' list of regular commenters whose URLs get posted without redirects, and then anyone else has their URLs put through the redirect script.

Phil Ringnalda, who has finally returned to the blogosphere, has posted this list of open proxy servers which are often used by comment spammers. The theory is that by blocking these it may well reduce the tide of spam - a better explanation is offered by Phil.

I've actually implemented this at the server level, using mod_rewrite to serve up this error page instead of any content if one of those addresses is used.

There's been a new, and somewhat unexpected maintenance release to Movable Type, which is now at 2.66. The new features aim to combat spamming, and include:

1. A throttling measure so that comments from the same IP address can only be posted every N seconds, where N is configurable
2. A measure to automatically ban an IP address based on an abnormal number of comments from the same address in a short period of time
3. Changed the behavior of <$MTCommentAuthorLink$> to use redirects when linking to URLs given in comments. The goal of this is to defeat the PageRank boost given to spammers by posting in the comments on a weblog

(this was taken from the official announcement)

These fixes are very much welcome, and while they won't end comment spam on their own, if everyone was to upgrade it would perhaps take away the main benefit of spamming - the pagerank boost.

Of course, this is no comfort to Kim, who has just relaunched her blog today and will now have to upgrade MT... :)

Some scum-of-the-earth spammer has been using Habeas headers in their email to get around junk email filters. For the unitiated, Habeas licences watermarks to content publishers, allowing them to mark their messages to prove that they are legitimate. Only those that comply with Habeas' terms of service can sign up, and their customers (I believe) include Lockergnome. Mail filters like SpamAssassin automatically lower the score of any email that has a Habeas header, since the email is likely to be legitimate.

Which is why an email that had all the hallmarks of a message designed to get around bayesian filters (words with punctuation inserted in them, mystery blocks of random text, random poetry, random tags etc.) came into my inbox today with a SpamAssassin score of just 0.4 - despite being received from an IP address in several blocklists, including SpamCop. So, naturally, I reported said email, and got an automated response saying that they will investigate. The response also mentioned that a particular spammer is illegally using a Habeas watermark for promoting his pharmacy sites, which looks like what I got.

Actually, as I write this I've had another email with similar characteristics, with a score of -1.9. Fortunately both were identified as spam by Thunderbird's Bayesian filter, and I've reported this email too in case Habeas can find a pattern and then take action. It's just a pity that yet another anti-spam system have been exploited.

Today my blacklist topped 1000 entries (1046 entries to be exact) - in the abscence of any updates from Jay while he heads off to the US, I've been getting updates from other sources, including most recently Long Story; Short Pier. This is all very well - with a bit of luck my blog should be pretty well protected against any scumbag attack - but it now means I have to load a 250k page every time I want to edit the list. Now while I have effectively unlimited bandwidth on this account, if comment spam does continue to be a problem for some time, then this page is only going to get larger and more difficult to use.

So, I propose two things for Jay to think about:

1. Having the initial MT-Blacklist page to be a quick-loading 'menu' page, which doesn't have all the entries on it. That way if I'm just sneaking in to update the list it doesn't have to display everything in one go.
2. Divide the entry list page into an alphabar, so all the domains starting with 'a' are on one page, the ones starting with 'b' are on another, and so on.

While I'm on the subject of comment spam, there's a post at Mezzoblue; it and the subsequent comments are well worth a read.

Update: There has been an update to Jay's master blacklist today. I only got another 3 domains from it but then I've added quite a lot of other domains recently so you may get more out of it.

This guy got bombarded by comment spam yesterday, but, on the plus side, he is sharing his blacklist file so you can add the extra domains that he added. Bear in mind, however, that importing this as is will also prevent your commenters from using words like 'sex' and 'passion', so you may like to remove those words first. It'll also stop people from being able to post domains with two consecutive hyphens (like www.my--spam--site.com) - this shouldn't be a problem since the vast majority of these domains are ethically suspect but it's something to think about.

Thanks to Mark for linking to the site in his sideblog.

Brilliant. Now even celebrities are sending me junk email:

The email in question had 30 SpamAssassin points, if it interests you. Most of them gained through falsified headers.

Mark Carey has a Blog Spam Database which includes a huge selection of extra URLs to add to your MT-Blacklist databases. Note that there were a few duplicates and some regexps which have been made obsolete by updates from Jay Allen, but I managed to add 300-or-so extra domains from it. As such, my own blacklist is now almost at 1000 entries.

Yoz Grahame's seven quick tips for a spam-free blog is also worth a look - it was written some time ago but the advice is still good.

"Your Plumber Web Site is NOT Being SEEN! WHY? NO ONE CAN FIND IT!"

Yes, folks, I'm afraid I have a confession to make. I'm not really a student, I'm a plumber called Barry from Dunstable, and this is my plumbing website.

Um, yeah. That has to be one of the more... err... 'random' spams I have lately - the fact that it was sent to the 'weblog' alias of this personal domain obviously made them assume that I was a plumber. But then, according to them, this guy's journal is really a bed and breakfast site and, perhaps more amusingly, The World Wide Web Consortium is actually a Florist.

The company, by the way is Regal Telecom, which is based in the UK. I hope they realise that by emailing me (an individual) they have just broken the law - as I understand it's now illegal for companies to send unsolicited bulk email to individuals who do not have an existing relationship with them. There was also no way of opting out of future emails. Naughty naughty, very naughty.

Update: Had another one sent to the same alias, which, interestingly enough, never normally gets hit with spam. Again, advertising SEO services (although not for my 'plumber web site' this time) but using a BTOpenworld email address as a method of contact.

Got spammed by a Polish porn spammer today. Which post did he choose? One about comment spam and some suggestions for how to stop it.

Spammers are stupid.

If you don't mind blocking all users of a particular Ukranian ISP, then you can block the entire 217.198 IP range. This ISP, or rather its users, is the cause of most of the comment spam I get, so hopefully blocking it entirely will stop that. Apologies in advance to anyone who uses this ISP but then maybe you need to write to them and tell them not to be as tolerant to spammers.

Although my university mail account has used SpamAssassin for sometime now, my everyday account hasn't had it until I changed hosts. It's been interesting to see what scores each item of mail gets, and while this latest one isn't good enough to make the high score chart, it's still pretty high:

Content analysis details:   (42.6 points, 7.0 required)

pts rule name              description
---- ---------------------- --------------------------------------------------
 4.4 DATE_SPAMWARE_Y2K      Date header uses unusual Y2K formatting
 0.3 RCVD_NUMERIC_HELO      Received: contains a numeric HELO
 0.5 REMOVE_REMOVAL_2WORD   BODY: List removal information
 2.8 SENT_IN_COMPLIANCE     BODY: Claims compliance with spam regulations
 0.1 REMOVE_SUBJ            BODY: List removal information
 0.1 HTML_70_80             BODY: Message is 70% to 80% HTML
 0.1 HTML_FONTCOLOR_RED     BODY: HTML font color is red
 0.1 LINES_OF_YELLING_2     BODY: 2 WHOLE LINES OF YELLING DETECTED
 0.1 HTML_FONTCOLOR_BLUE    BODY: HTML font color is blue
 0.1 MIME_HTML_ONLY         BODY: Message only has text/html MIME parts
 0.0 HTML_MESSAGE           BODY: HTML included in message
 0.1 HTML_FONT_BIG          BODY: HTML has a big font
 0.0 LINES_OF_YELLING       BODY: A WHOLE LINE OF YELLING DETECTED
 2.4 HTML_SHOUTING7         BODY: HTML has very strong "shouting" markup
 0.7 MIME_HTML_NO_CHARSET   RAW: Message text in HTML without charset
 3.0 FORGED_RCVD_NET_HELO   Host HELO'd using the wrong IP network
 2.8 DATE_IN_FUTURE_03_06   Date: is 3 to 6 hours after Received: date
 1.1 RCVD_IN_SORBS_HTTP     RBL: SORBS: sender is open HTTP proxy server
                            [218.89.107.18 listed in dnsbl.sorbs.net]
 4.3 RCVD_IN_OPM            RBL: Received via a relay in opm.blitzed.org
                            [218.89.107.18 listed in opm.blitzed.org]
 0.1 RCVD_IN_SORBS          RBL: SORBS: sender is listed in SORBS
                            [218.89.107.18 listed in dnsbl.sorbs.net]
 0.1 RCVD_IN_NJABL          RBL: Received via a relay in dnsbl.njabl.org
                            [218.89.107.18 listed in dnsbl.njabl.org]
 4.3 RCVD_IN_OPM_HTTP       RBL: OPM: sender is open HTTP CONNECT proxy
                            [218.89.107.18 listed in opm.blitzed.org]
 1.1 RCVD_IN_DSBL           RBL: Received via a relay in list.dsbl.org
                            [>http://dsbl.org/listing?ip=218.89.107.18<]
 1.1 RCVD_IN_NJABL_PROXY    RBL: NJABL: sender is an open proxy
                            [218.89.107.18 listed in dnsbl.njabl.org]
 2.2 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
               [Blocked - see >http://www.spamcop.net/bl.shtml?218.89.107.18<]
 4.3 RCVD_IN_OPM_HTTP_POST  RBL: OPM: sender is open HTTP POST proxy
                            [218.89.107.18 listed in opm.blitzed.org]
 1.6 FORGED_MUA_OUTLOOK     Forged mail pretending to be from MS Outlook
 1.1 FORGED_OUTLOOK_TAGS    Outlook can't send HTML in this format
 1.2 MISSING_MIMEOLE        Message has X-MSMail-Priority, but no X-MimeOLE
 0.3 UPPERCASE_25_50        message body is 25-50% uppercase
 1.1 FORGED_OUTLOOK_HTML    Outlook can't send HTML message only
 1.1 MIME_HTML_ONLY_MULTI   Multipart message only has text/html MIME parts

Some prat with a fake information site about carpets (which was actually a doorway to a site selling rugs) decided to spam me earlier on. Naturally I have removed his comments and submitted them to the Comment Spam Clearinghouse. Anyway, I got an email just now from Jay Allen, who says that there has been a marked increase in blog spam activity today.

I reckon it's the spammers assuming that a good number of bloggers will not be blogging over the festive season, so if they get their comments in now they will be there for a few days, during which time they may well get indexed by some search engines.

If you are updating over the festive period, update your blacklist now as there have been many additions this week, including a few new regexs.

On a side note, the domain that I was spammed with has also been deleted over at the ODP - I noticed it had been listed a few days ago but when I actually pasted some of the text into Google I realised that all of the content had been lifted from other sites. Additionally, the domain was registered to someone with no address, no phone number and a Hotmail account.

Anyway, Santa's almost certainly on his way, so I'd be better off in bed. I'll let you know which presents I get tomorrow - if I get any, of course. :)

Update: This morning I'd been spammed by a similar site (with similar WHOIS info) from the same IP block (a dial-up ISP), and had a blacklist denial in my event log. So 3 spams after almost a month of nothing - they're definitely up to something. Seeing as I've been hit twice by the same guy, I'm tempted to invoice him for advertising fees, as per the terms of usage.

Update 2: Another one from the same IP, this time for a Poker site. Be rest assured that the lowlife's IP block is now blocked from the site.

Looks like Lloyds TSB is the latest bank to be affected by the online fraud scam. For some reason, this time I was sent the email, even though I've never had dealings with Lloyds. That said, the email address used was one that I'd only ever used on BenHammersley.com (which proves that email harvesters do scan RSS feeds), so I was suspicious from the start.

The email was quite clever in that the text looked like plain text, except it wasn't. Therefore, what looked like a plain text link to https://online.lloydstsb.co.uk/ was actually a link to http://online-business.lloydstsb.co.uk%20%20%20%20%20%20%20%20%
20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%
20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%
20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%
20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%
20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%
20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%
20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%
20%20:UserSession%3D2f4d0zzz899amaiioiiabv5589955&userrstste%3D
SecurityUpdate&StateLevel%3DCameFrom@219.166.154.218/www/logon/
logon.ibc.htm . Removing the cuft brings out http://219.166.154.218/www/logon/logon.ibc.htm which is owned by an ISP in Japan. Neither URL uses SSL.

Interestingly I have never received any fraudulent messages for the Halifax, even though I do have an online bank account with them. Maybe I missed out.

Update: there's a Register article on the subject.

I've finally got off WebProWorld's mailing list, by entering my non-existant zip code as a space. From there I was able to unsubscribe from it and some other newsletter that I ended up on but never asked for. Naturally, this wasn't detailed anywhere on WebProWorld's or iEntry's sites - had it were, I'm sure this whole problem would never have come about.

No, actually, if I hadn't been subscribed to email newsletters without asking, then this whole problem would never have come about.

At the weekend, I had to install the RealOne player to listen to some BBC audio streams (namely The News Quiz which I'd missed, both on Friday and the Saturday repeat). Apparently, despite having no recollection of this, I indicated that I would liked to be spammed silly receive "product news, updates, and special offers".

So, when the first one of these arrived, I hit 'unsubscribe'. And, after telling me I had been successfully unsubscribed, it told me that 'it may take 3-5 days for your request to take effect'. Sorry, but WTF? What are they running the mail server on, a TRS-80? Why can't my request be dealt with now, like, well, most other mailing list providers like Lockergnome?

As it is, I got another mail from them today, although when I went to unsubscribe it said I had already been unsubscribed which is perhaps a good sign.

Oh yeah, WebProWorld are still spamming me. I've trained Thunderbird's bayesian filter to automatically send any mail from them to the trash, as I've almost given up being able to unsubscribe from them. Unless I can find some script or program that I can run which will collect mail from the alias I used for their mail and automatically bounce it back, giving the impression that the alias no longer exists.

It appears that some bots now crawl RSS feeds for email addresses, as I've just got some junk email from an address that I used on Ben Hammersley's blog. I was aware of this happening to others but since it's only been recently that I've started using alternative aliases it's not been so easy to track. The reason is that Ben includes some FOAF data in his RSS feed which includes the email addresses of those who contribute comments - while it may be useful in a metadata sense I wish he'd used the 'spam_protect="1"' attribute in Movable Type to make them less obvious. I suggest that if you have RSS feeds yourself and you run MT, then any email addresses you show on site should use that attribute.

I'm still getting junky newsletters from WebProWorld, despite trying to unsubscribe several times. Fortunately, I did use a different alias for them so I could easily just set a filter to block or bounce all mail that comes from there. If you don't want your Inbox to be taken over by huge HTML emails of little interest then I suggest you do not join, even if the forums do look interesting.

Finally, I'm still not getting much comment spam. MTBlacklist should report back to me when it blocks something, which it hasn't been, and nothing has trickled through. Of course, every time I say this I get hit by a load of it.

Actually, there was something interesting that occurred around half past four yesterday (UTC). I got 7 comments, all with the same name and email address, but with some encoded rubbish in the comments field and from 3 different IP addresses on different ISPs. Anyone else get that?

I mentioned that I'd received some unsolicited mail from WebProWorld on Friday, and despite unsubscribing to it, I'm still getting mail. It appears I'm actually subscribed to several newsletters, none of which I requested when I signed up.

All are large, HTML emails that fall foul of the 50KB truncation limit I have in Thunderbird so that I don't end up downloading big emails (most of which are either spam or virus-infected anyway, so no big loss). There is a subscription management site at iEntry.com but it doesn't want my WebProWorld password, and I was never asked to supply a 'zip code' when I joined so I can't find out what it is. I can unsubscribe to individual mailings, but only when they actually arrive. It's a major faff that I really could do without.

More to the point, WebProWorld have just alienated one of their users. I can guarantee I won't be posting in the forums again if this is how they treat their users - talk about biting the hand that feeds them.

Adam Kalsey wants you to sign his manifesto against comment spam. If having people post unwarranted links to pornographic sites, or linking to viagra stores annoys you, then add your name to it.

Thanks to Chris

Both Chris and Luke have mentioned a new site called blogspam.org, an anti-spam blog that seems to be spamming other blogs to get attention, even though it's still under construction. The messages are either emails or comment spams, written in a very generic and templated style. Now I haven't yet got one but the guy behind this certainly needs his head examining.

In the same vein, I got an unsolicited email today advertising an anti-spam tool. This is while the FTC are suing a company that is spamming people to advertise an anti-spam tool via the Windows Messaging Subsystem (not MSN Messenger). According to the FTC:

The defendants created the problem that they proposed to solve-for a fee. Their pop-up spam wasted computer users' time and caused them needless frustration.

Can't agree more. This is sheer hypocrisy.

I've been getting some really bizarre comments to my Googlism post. The author, IP and email are all the same, but the message is always different, written in the style of a Googlism.

I'm in two minds as whether this is someone with too much time on their hands or spam. My reasoning for the second idea is that this is a 'marker' post which spammers can use to find blogs where comment spam will stick out - the names are reminiscent of the bizarre names that I get email from on Hotmail, which are all spam.

Either way, the IP is blocked and comments are closed for that post. I'm actually thinking about taking a hint from Mark and closing comments on old posts. Although whereas Mark usually only allows comments on the most recent post I'll probably only close comments on those more than two months old, for example, mainly because a number of the posts are no longer relevant but also because it'll ease the comment spam problem.

Only problem is that I now have well over a thousand entries that will need the comments disabling in :-/.

Yesterday, I registered with WebProWorld.com so that I could reply to this topic about the ODP to clear up a few things (my posts are on page 3).

Today, it turns out I'm now subscribed to a daily HTML newsletter with links to new and interesting posts on the forum. This would be fair enough if:

1. I'd actually asked to receive it (I didn't, or don't remember doing so)
2. The emails weren't over 50kb a piece

Fortunately there was an unsubscribe link (which I followed) and Thunderbird truncates messages larger than 50kb to save time. Of course, I used a different alias for any correspondance from them so that I can track where any further junk from them has come from.

Another two comment spams. The first one looked innocent enough, but the name was something like 'real estate in florida' and the URL was to an estage agent. Having analysed access_log, it looks like however posted that was looking for a recent entry where they could post a comment and still look slightly conspicious. The referral was from movabletype.org - probably a good place to start because all the blogs that feature on the home page there are likely to have high pageranks on Google.

The other was posted to my Safari entry, which seems to be a bit of a magnet for spam. No referrer this time, and again, the comment would look alright if it weren't for the URL put in at the end. Amusingly the spammer refreshed the page twice to see if it had posted, which, since I hadn't approved it, it didn't.

With hindsight, I probably shouldn't have said that I only get about 1 spammy comment a week, because I've had three so far this week, all in a similar style, but from different IP addresses.

Analysis of my Apache files lead me to believe that the "Blogroll Me!" text on the front page is to blame for one of the cases, since the referer contained a Google search for that phrase. The others had the referer blocked :(.

Fellow bloggers, I'd like some feedback. How often do you guys get comment spam?

After making various changes that I've posted about in the past (see this Spam category for an archive of postings on the subject), I'm now getting perhaps 1 item of comment spam per week. Is that high, low or normal?

I'm guessing that many other bloggers get hit more than I do, but it would be nice to see what the average is.

Since I've been getting quite fanatical about it lately, there's now a "Spam" category - you can read post excerpts as HTML or subscribe to the RSS feed. All my recent entries about comment spam will be in there but the aim of the category is to cover spam in general and not just spam on blogs. That said, most of the recent stuff is about comment spam so you may find it useful.

While I'm on the subject, Feedster have launched an IP-blacklisting service for comment spammers - you can provide them with the IP and URL used and it will be added to an OPML file. Now all we need is a plugin for MT that adds these IPs automatically, although I fear that this will be futile because by the time an IP has been reported the spammer will have probably spammed a great many blogs. And there's the potential for abuse.

Great Site Folks! I have another big **** site for you which is really the #1 big **** site - check it out, its full of big **** !! here's the link: Big ****

You can guess the word I censored out. It wasn't a swear word (rather a slang term for 'mammarial glands' ) but I'm worried that if I left it as it was, comment spammers would search for it and I'd continue to be hit (since they assume that because it appears on the blog I won't remove spam and therefore I'll offer valid contributions to their pageranks).

Analysis of this latest comment spammer suggests that they are going after entries in the format http://[domain]/archives/######.html - the default for MT. Now while my more recent entries are immune, since I use /entries/ rather than /archives/ , I spent a lot of time getting the first 350 entries to redirect to their new locations. That said, they've been 302 (temporary) redirects since March, and 301 (permanent) since 31st August, and it may well be that in a couple of weeks they become 410 (gone), since I think two months is an acceptable timeframe for any search engine crawlers to fix the URLs in their databases.

So, what's my latest comment spam prevention tip? Don't use /archives/ . Use /entries/ , /posts/ or have a custom setup that doesn't involve the entry ID. That goes in hand with renaming the mt-comments.cgi file and removing common terms like "Remember Personal Info?" from entry pages and comment listings, which are my other tips.

As it is, the same guy has been hitting various guestbooks too - Google has over 1000 results for his name.

Looks like my comment spam fix isn't foolproof, since I have just got some pharmaceutical spam posted to one entry. But a quick analysis of my Apache server log revealed some interesting trends...

The IP address was 66.119.33.171, so I searched the file for this address. 5 lines came up, the first of which was most telling:

66.119.33.171 - - [08/Oct/2003:17:30:19 +0000] "GET /entries/000234.html HTTP/1.1" 200 - "http://www.alltheweb.com/search?_sb_lang=pref&cs=utf-8 &cat=web&q=%22Remember+personal+info%3F%22&avkw=fogg&o=20" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; Rogers Hi-Speed Internet)"

This shows that someone is scouring for sites with this string: "Remember Personal Info?" - many of which will be blogs. It would seem, therefore, that you could prevent this by merely changing the text to something else (ie something that not everyone else is using). It's worth a try, anyway but don't expect immediate results because the search engines will need to recrawl your site first.

While I'm on the subject, Jay Allen has another good stab at blocking the spam - might be worth keeping an eye on it.

Hi,

I visited http://www.newrecruit.org, and noticed that you're not listed on some search engines!

Sorry Stephen, it looks like I run your web site now >:-). (I'm joking of course)

Of course this email is a con - even Google gets them - and even if there are 300 000 search engines and directories out there, I wonder how many are:

• Actually relevant to the site's focus? (no point in submitting this weblog to a web directory for cheese collecting sites)
• Share their data - there are over 360 sites that we know of using ODP Data legally (and other 700 who may be using illegally).
• Actually worth submitting to? (no point in submitting to a search engine no-one has heard of nor uses)

Google's SEO page offers some very good advice, in my opinion - if you're looking for someone to do your web marketing for you, consult that first before you start making inquiries.

If there is at least 1 comment and 1 trackback to this post, then it means that my 'solution' to the comment spam worked. I won't tell you how I did it just yet, although you can probably get an idea by looking at the source code for the individual archive page for this entry.

Note that comments and trackback pings may not work on over entries in the meantime. (should be fixed now)

Update: Okay, it works, so here's how to do it:

I had this comment left today:

We live in strange times, but someday I think we will look back on all of this and marvel at how crazy it was. God, I hope so. I sure wouldn't want this insanity to become the norm.

The comment was posted to an article about Apple's Safari web browser and the initial reaction it had in the web community. It looked legitimate, if a little odd, until I noticed that the author was called 'Buy Viagra' and his URL went to a site selling that famous anti-impotence drug. The email address of 'dave@dave.net' was also somewhat suspicious.

Of course, spamming this blog is totally futile since the comments have to be approved, but try telling an automated script that. I believe, however, that there may well be a partial solution to the problem - hang in there and I'll see what I can do.

It's official... according to the contents of my junk mail, women love IT:

I'm glad to see the women of our age taking an interest in Information Technology.

Got some very strange comment spam today. The text appeared to be snagged from an FAQ on The Pennsylvania Breast Cancer and Environmental Risk Factor (PA-BCERF) project, but had random drug names with links to various online pharmacies.

None of the drugs were for breast cancer - you can probably guess what they were though. Still, it's a very odd way of posting spam.

Phil Ringnalda is offering an MT Hack which blocks Zip Code spam in post comments. I was hit by this myself in the dim and distant past, but it hasn't happened recently, so that's one reason why I haven't done this myself.

The second reason is that if the guy who is doing this realises that a lot of people are blocking him, then he'll just change the URL to something different. It's why I tend to avoid blocking IP addresses since it's easy for the scamsters to get a different one.

And the final reason is that I have comment moderation anyway, so it wouldn't make much difference. Talking of which, I've just deleted a comment from someone called John using Telewest Blueyonder as his ISP (IP: 80.194.41.211, which doesn't appear to be firewalled) who thinks the site sucks. While he's perfectly entitled to his opinion, I'd prefer something a little more constructive than that.

I've added SpywareInfo's list of Spammer IPs to my block list, in the hope that it'll keep those nasty bots out. The 403 error message has been modified accordingly to say that some IP addresses have been banned, just in case I'm blocking legitimate traffic.

Today I also joined The Harvester Project, the aim of which is to build up lists of IPs like this. While it's not foolproof, anything to stop the spread of spam is good in my opinion.

I'm slowly reading through all the blogging action that I've missed over the past two weeks (expect an (n)Echo feed soon), but Andy's post about blocking spambots led me to a useful article on the subject at Dive Into Mark. I already had pretty good protection against nasties but this should take it to the next level.

Early indications (well, tests with SamSpade) show that it's working well, although my pretty new 403 pages aren't showing since Apache encounters a 403 error when trying to get the 403 page. If that made no sense to you, then be assured that at least I know what it means. I know roughly how to fix it, but I'll need to read a few mod_rewrite tutorials first.

I could say 'Let me know if you have any difficulties accessing the site' but if course, if you were having ant difficulties accessing the site you wouldn't be seeing that message.

One side-effect of this should be a reduced bandwisth usage, which is already down dramatically, probably after I removed the image from the heading of every page. We're nearly half-way through July and I've only used a measly 186MB of bandwidth - less than 10% of my limit. Not that I'm complaining, mind.

By the way, if you want a copy of my .htaccess file, drop me an email. It goes beyond what Mark offers on his page so it should be one of the most complete out there.

Someone called Kevin Philips wants me to "Download This!". Jessica H. Cooper is asking me "How do you turn it on?". Jesse Roberts wants to know "Should we try it anyways?" [sic].

Natalie Evans tells me to "Introduce Yourself Please". Isiah Green has said "Lets work it out".

I have never met these people before, and I have no doubt that they don't even exist. They have been merely created to fool people into opening emails which look legitimate, but are actually spam. And that's why I wish Microsoft every success in suing the people who send them. If it's difficult for me to weed out legitimate mail from spam, then it must be a nightmare for other, less savvy internet users.

What shocked me is two of the people being sued live in the UK. Fools.

I've had a couple of emails from (former) readers of this blog who are somewhat disgruntled at my decision to introduce compulsory moderation of all comments, so I'd probably best explain myself.

Imagine this blog is a mural, painted on a wall. This mural is constantly being altered and enlarged, but, for the most part, it looks alright. The main artist is me, but I don't mind the public coming in and adding little bits here and there, to give the mural another dimension.

Then someone comes along and decides to graffitti all over this mural. The work of me and the others who contributed has been ruined, and so I have to go through and remove the graffiti. After all, once a wall has a little graffitti, if left there, it's often not long before more graffitti appears, and soon the whole wall is covered in it. It takes over, and suddenly the mural isn't so nice to look at.

The other problem, but perhaps not so widespread, is that occasionally some plucky businessman will come along and stick bill posters over this mural, advertising their scummy web site. They can't advertise through legitimate means, so they stick posters up on people's walls.

Back into the context of this weblog, you've probably gathered that the graffitti represents the flames and other crap that gets posted here, and the bill posters represent the spam. Now, by controlling what goes up on this mural... err... blog, I can make sure it looks nice, so that people will continue to visit it. If I let these nasties take over, I imagine the crap will just take over. And no-one likes reading crap (well, apart from Daily Mail readers ;)).

Is it an erosion of free speech? Well, no, since there was no free speech there in the first place. This is my wall... err... site, and I'll do what I like with it. That said, I welcome all comments which enrich and further the discussion, even if they are in disagreement. But graffitti will just be cleaned off and bill posters are banned.

Don't like it? Well then, get your own wall.

Have any other bloggers had a comment from 194.177.210.228? I've had three, all giving some kind of wooden complement about the site and a link to cheapflightselect.com . Looks like spam to me.

By the way, I'm implementing the comment queuing system now. It means that any posts will have to be approved by a moderator (myself and possibly Andy who semi-volunteered a few posts back), so while any relevant and friendly comments can go through, any that have nothing to do with the post or are posted only as flamebait or to make false accusations will be deleted before they are even made public.

On the plus side, I may enable anonymous comments, so you won't need to give an email address.

For some time now, I've not been able to receive emails when new comments are posted to the site, despite the settings being enabled in the Movable Type control panel to do so. So, seeing as Wednesday is my afternoon off, I decided to look into it.

The Movable Type manual's troubleshooting section suggests two fixes: correcting the location of Sendmail, or specifying an SMTP server. I was pretty sure that Sendmail would be in its default location - the server uses a relatively common RedHat setup - and adding SMTP support would require the downloading of an additonal module from CPAN.

Then I remembered some problems I was having with mail forwarding from my neilturner.me.uk account (the one on the side bar). MyRealBox rejected the email because apparently neilturner.me.uk is a source of spam (come again?), so that address now forwards mail to my university account. It turns out that it was also blocking these automated emails, on the assumption that they were spam :(. Changing the address on my profile to my university email seemed to fix the problem.

I have no idea why this domain has been branded a spammer, especially as neither SpamCop nor the three other blacklisting services have blocked it. It is yet more evidence in favour of Bayesian filtering technologies, as opposed to simple blocking lists, which can block a lot more than you can bargain for.

Just what I've always wanted.

Septic Troubles?: Want To Keep Your Septic System Running Like New For Years? (*) Try A FREE Trial of SPC Septic Cleaner (*) We're so confident in our product that we're giving it away! Click the following link for our website: (link removed)

This actually came by MSN, which is a little worrying. Fortunately it seems to be a one off - I found it in my chat logs from a couple of weeks back and it hasn't returned to haunt me yet.

By the way, thanks for all the sympathy, I really appreciate it. *hugs everyone*

And one final thing - for the person who has been claiming he had a 'beta' education in the comments, I suggest you look up the word 'beta' in a computing dictionary ;).

Today the 400th comment was posted! But unfortunately, it was spam. His IP is 66.69.244.39 .

Looking through my spam today, I can get a larger bust size, a penis extension, my hair restored, lose weight and look 10-20 years younger, all by replying to a few emails! Great!

Except I have no bust to start with, I'm not losing my hair, I'm not overweight and if I looked 20 years younger I wouldn't have been born yet. And my penis is annoying enough as it is - I'd dread to think what it'd be like if it was any longer.

Oh, and here's the obligatory link to The Penis Blog Project (do not look at it if you're at work or university). You probably know about it already, but seeing as I was discussing that area of my body it seemed appropriate.

This morning, this message arrived on my mailbox:

Hi Gorgeous,

I saw your info and thought you might be interested in chatting.
I have a couple of nice pics that you can see and also you can message
me if you want to chat. Come talk to me here ..
Thanks,

Colby

Before you start winking at me, a closer examination of the email led me to believe it is spam - it was sent to 4 people, all with MyRealBox.com email addresses. Looks like someone's been doing a dictionary attack against them.

Then I had this comment posted to one of the entries:

I am best neil, you are not. You are not even second best neil, because that is Neil Griffin!!Give up and accept defeat!!I am very good at French!!haw he haw he haw!! i eat frogs and snails and i smell of garlic!je suis francais!! j'avais une lapin.

Erm... right. Yeah. Okay.

By the way, it's "un lapin", not "une".

If you have comments on your blog, and a means of blocking IP addresses, then add 203.106.151.137 to the list. As Ben reports, someone is going around and posting spam messages in people's comments fields, and has so far hit several blogs including Kim :( . Be on your guard.

Just got a spam message with the subject Who would say no to free money?. Well, me, when I hit the delete button :).

The Bayesian spam filtering in Mozilla 1.3 is starting to come into its own now - I'm getting fewer and fewer false positives as time goes on, which can only be a good thing. Of course, I've been using it since 1.3 Alpha came out before Christmas so it's had about 3 months of coaching, but I think persistance has paid off.

Ironically, excluding my Hotmail and Yahoo accounts, the address that I get the most spam from is one that I rarely, if ever, give out. My primary email address, which I've had since August 2002, gets almost no spam, and I'm not complaining.

A fellow ODP editor just got this unsolicited advertisement via email. I think you'll agree it takes self defence a little too far...

I checked the blocking mechanism again that I implemented on Sunday using Sam Spade with 'EmailSiphon' as the user agent. Sure enough, I got a 403 error :).

I'm debating whether to change the message to something less generic. Any ideas? My current favourite is:

Access denied. Go away.

Short and sweet, but it does the job.

I checked my stats today to find 306 hits from EmailSiphon. This is an email address harvesting robot ('spambot') which takes email addresses from web sites and uses them to fill up spam list databases. Now while all email addresses on this site should be filtered in such a way that most robots will ignore, I'm still annoyed that these bots get through. So I decided to block them. >:-)

The two resources I used, from a Google search for 'EmailSiphon', were a mod_rewrite tutorial and 'a close to perfect .htaccess file'. I would have only used the second one but then even I wasn't able to get to any files, so I ditched it and took its list of user agents, but used the syntax on the first site.

Although when I tried to fetch pages using SamSpade with those user agents they still appeared, I think this is more to do with the University of Bradford's caching server.

I actually feel like writing a 'terms of usage' for this site forbidding the usage of such robots on the site - and then seeing about legal action against any company that does use them. It's an idea, anyway, and I've invoked the LazyWeb in the hope that someone can shed some light. Help, anyone? :)

BlogDex seems to be suffering the consequences of a hijacked domain. Several blogs on ubiquitus.nu now all link to the same sites selling domain names, so these all rank highly on BlogDex. I've emailed them about this.

"UltimateSports.info has contracted with a third party source to acquire your e-mail address for the purpose of sending you the latest sports information in a weekly newsletter and other valuable online offers. You'll receive great newsletter content on a weekly basis and great online shopping offers from time to time.

"We're confident that you'll enjoy our newsletter and our special offers, however, we have the highest respect for your personal privacy and will not send the newsletter or the offers if you are not interested in receiving them.

"If you do not wish to receive the UltimateSports.info newsletter and other great offers, please click below to unsubscribe from our mailing list."

And of course, the 'click below' link doesn't work. A great way to start your portal - annoy several million internet users with unwanted spam. The "highest respect for your personal privacy" bit is laughable.

Ah, spam. Despite expecting a deluge back in early March, I don't seem to be getting all that much. I got the Nigerian money scam today (which interestingly wasn't filtered since appeared to look like a legitimate email), but other than, I maybe get two messages a week. Considering my inbox has been active for 3 months and that I'm a very active netcitizen, I'd say that was good going - compare that with my last email account which was receiving more than this after about two weeks. Maybe I'm just more careful now...

It's certainly not as much as what The Guardian hinted at this morning - that article seems to suggest that people get between 20 and 50 items a day. Even in my old address, I was getting about 10 a day (which is still annoying). Which, by the way, is now inactive, since I haven't logged into it for more than 30 days. I think I'll leave it to bounce stuff for a couple more weeks before I reclaim it - then I can use it for webforms and stuff.

Got my first spam message today. Well, the first since changing my email address, anyway. And unfortunately I couldn't get BSM to work either - whether that was a problem with the email or the program I don't know, but I noticed the program's web site is no longer online. I just hope this isn't the start of a deluge. And in case you're wondering, it was one of those stupid pyramid schemes. It's now been sent into email heaven... or should I say hell?

Actually, I remember ages ago reading an article about spam, saying that rather than blaming the spammers, we should blame ourselves. Why? Well most companies won't resort to it unless there was some money to be made from it. Therefore, there must be some people out there who are actually buying from these idiots. Scary thought.