Neil Turner's Blog

Blogging about technology and randomness since 2002

Blocking the DTS Agent

If you downloaded my .htaccess file from my Blocking the RIAA entry to also block evil email harvesting bots, you may want to add this new line:

RewriteCond %{HTTP_USER_AGENT} "dts agent" [OR]

I added this just before the ‘amalden’ line and it seems to work. DTS Agent seems to be used by the Nigerian 419 scammers, according to Andy who discovered it – thanks. A test with SamSpade this morning with its full user agent string (a badly disguised IE string) confirms that this line blocks it.

According to this site, which is a good resource for user agent knowledge, the bot naturally does not obey robots.txt so you will need to forcibly block it using .htaccess.

5 Comments

  1. RIAA and Spambot blocking

    I’m giving the Blocking the DTS Agent and Blocking the RIAA .htaccess a try. Hollar if anyone experineces any troubles….

  2. *note to self: pick up Neil’s .htaccess file*
    I don’t think I should be sending any pings out, because right now my entire domain is passworded, and I’ve been doing some serious site maintenance on Revolving Duck, at the original location. So far, my MySQL db that runs RD has corrupted *twice* in the last 18 hours, and although I did manage to salvage all the entries I could, I still had to delete the db and reinstall MT. *tear* HOWEVER, RD is getting up to par, and right now I’m going nutty trying to get MTMacros Acronym Expansion working. Order should be restored in about a week and then I’ll take down the password protection. Just in case you realized I was updating and couldn’t get in. πŸ˜›

  3. Another good way of frustrating email harvesting bots is using an email encoder (plenty available free on the net) to encode your mailto links before putting them into your .html files.
    e.g.
    This mailto link —
    contact me
    will look like this after encoding and become illegible to the spambot —

  4. Of course, that would make your site somewhat inaccessible.

  5. Ihave been looking a lot at spidering technology and found myself on this site reading how to prevent remove access to stop email addresses being parsed. Two ways I use are 1.never showing the email address and storing the address in a mysql database, then when a visitor needs it he/she must verify who they are before they will get to send a message to the email address. The other is to use the PHP features that convert a string into an image, the image is pretty useless to a bot but a real visitor can see the address no problem. Also this is sure to be a short term solution as it is also fairlysimpel to revearse the image back to text. So what I do is randomise the scale of the font, making variations of font size and angle. You could go further and make the font colour very near to the background color, this would be almost impossible for an OCR application to figure out. So in a nut shell I think preventing robots visiting is a on-going task. better to protect the information from being seen in the first place. Robots are very easy towrite, email address harvesting is verysimple in fact a simple perl script of less than 20 lines of code can do it. Add this tospideringcode and you have a web side email harvester. Anyway I just noticed I am wofflingon. I hope that help you and if you need any further info send me an an icq 7529262 or yahoo msg andyf97