Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Add regular expression junk filtering to Mail.app Apps
I created a way to filter junk using regular expressions in Mail.app. I've detailed my solution on this page.'

[robg adds: This free solution uses Python and AppleScript to add regular expression filter to Mail's junk filter, thereby allowing it to catch re-written words such as "v1agra." The page even includes a nice introduction to regular expressions, explaining how to add your own pattern definitions to the junk filter files.]
    •    
  • Currently 2.25 / 5
  You rated: 1 / 5 (4 votes cast)
 
[12,745 views]  

Add regular expression junk filtering to Mail.app | 24 comments | Create New Account
Click here to return to the 'Add regular expression junk filtering to Mail.app' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Add regular expression junk filtering to Mail.app
Authored by: Anonymous on Nov 24, '03 11:53:54AM

I think you forgot to mention that the name of the disk must be changed from "Macintosh HD" to "your disk name" in the junkMatcher script.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: wscody on Nov 24, '03 01:18:07PM

Actually, in the Rules window for JunkMatcher, you can simply click on the "Choose..." button next to the blank "Run AppleScript" path, and navigate to the JunkMatcher.scpt file.

The correct pathname is automatically entered for you.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: swilcox on Nov 24, '03 02:17:05PM

This looks wonderful. I wonder, though, if it would be also possible to revise the Applescript so that we could add generic (not spam-specific) regular expression filtering to Mail. That would be awesome, but probably beyond my Applescript skill.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Nov 24, '03 02:34:05PM

I agree it'll be useful to trigger actions based on regexp. I'll see what I can do...



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Nov 24, '03 02:19:14PM

Thanks for the tip - I've changed the scripts so now they use ~/Library/Scripts/ as the place to put the temporary files (they'll be removed right away after use).

One question since I'm relateively new to AppleScript: how can you get the "/tmp" equivalent in Mac format? I know how to use POSIX path to get a Mac-formatted path into a UNIX one, but not the other way around.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: ngb on Nov 24, '03 03:48:57PM
/tmp is actually in /private/tmp, or in Mac terms "Macintosh HD:private:tmp:".

The AppleScript way to find this is to cast "/tmp" as a posix path. In this case, I'm assigning it to the variable "tmpPath":

set tmpPath to (POSIX file "/tmp")

I believe POSIX file filename can be used anywhere the file filename or alias filename constructs can be used, and conversion will happen on the fly.

[ Reply to This | # ]

Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Nov 24, '03 05:20:19PM

I believe the problem is the "on the fly" part - if I do the posix file thing, the conversion to the Mac-formatted path is done on the spot, and the result could be wrong on someone else' Mac (e.g., they might name their HD differently).



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: ngb on Nov 24, '03 08:46:32PM
If you're looking for /tmp on someone's Mac, but you don't know the name of the hard drive, then putting POSIX file "/tmp" in your script should send you to the right directory, even if interpreted on the fly. Alternately, you can get the name of the startup disk by using path to system folder and then strip filter for the text between the first and second ":".

[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Nov 24, '03 02:31:04PM

Let me also mention that there's a hidden debugging option in junkLib.py:

DEBUG=0

If you set that to 1, you'll get ~/Library/Scripts/debug.log, in which:

1. the *decoded* source of each checked message
2. the reason why it's filtered

are logged.

I know it'll be nice if there's some easy way (like a nice GUI) so that users would know what pattern makes a message junk (also a way to recycle the log file) - until I have more time...



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: FlyBoy on Nov 24, '03 03:57:32PM

A very interesting tip. I'd like to make a suggestion. The typical organization for the ~/Library/Script directory is to have subdirectories which contain the script or scripts for various tasks. It would be useful to have the script use ~/Library/Script/Junkmaster/ or something similar as the default folder, with all the python scripts and associated text files held within this folder. I hate seeing random scripts at the root level sitting in my Script Menu.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Nov 24, '03 05:17:29PM

Thansk for the suggestion - I've re-organized the files so all files are now in ~/Library/Scripts/junkMatcher, and all pattern files go to ~/Library/Scripts/junkMatcher/conf.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: c-o2 on Nov 27, '03 06:23:01PM

Being a simpleton to these things, forgive me, but... though your solution has enhanced my junk mail situation (it was getting intolerable, Mail.app just didn't get any junk), I am now receiving about 150 junk mails of the same junk mail message every hour... multiply this by new junk coming in at a rate of about 3 an hour and you can guess how many messages reside in my Junk Mail box after a couple of hours!

Like I said, now every piece of spam goes to my Junk thanks to the Applescript (great!)... but since installing it, the aforementioned is perplexing me.

One thing I noticed... in one of your screenshots, your Rule configuration screen has three rules applied. The first, I could not match... the dropdown menu after 'all' states 'Received'... I don't have the 'Received' option! (OS X 10.3.1)... and I can't figure out what character you've inputted in the text box of the corresponding first rule.

Some one show me what I am doing wrong ;)



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: blgrace on Nov 27, '03 07:24:05PM

I'm getting the same thing. It's because the same message isn't being delted from the mail server so you just keep getting it EVERY time mail.app checks mail. I'm guessing you have set yours to check every hour. Mine checks every 5 minutes. My ISP's mail server won't let me telnet in to list the offending email and delete it, so I'm having to resort to using Mail Beacon to log in and delete the messsage.
Anyone know how to fix this? I looked through the Apple Script code and I can see a possible section that needs modifying - but I have no idea on how to do it



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: blgrace on Nov 27, '03 07:28:36PM

You have to add the "Recieve" header yourself.....just scroll to the bottom of the rules list thingo and you will see "Add Header" just type in "Recieved" and click Add . . . then put a period in the "contains" field, like the screen shot. There is a link to a help page.. something like "If you can't see what's going on in the screenshot... click here"



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: c-o2 on Nov 28, '03 02:58:31AM

Yep my check every minute (just meant I receive on average (prior to JunkMatcher ;o) ) 3 spams an hour)... check my machine after a good nights sleep... 3200 junk mail messages!

Thanks for the hint with the Received thing. Will do it now.



[ Reply to This | # ]
I'm getting the same thing
Authored by: GeorgeVW on Nov 29, '03 12:29:54AM

with a Mail check every minute. In just a few hours I had close to 800 messages in the junk folder - mostly multiple duplicates of the same messages. Everything is set up as specified in the instructions. I may try re-enabling JunkMatcher and switching to an every 15 minute check to see what that does.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Nov 29, '03 12:05:01PM

To the people using POP accounts and experiencing the odd problem of re-downloading same mails: please let me know if you still have the same problem with the latest JunkMatcher (v 0.51). (in particular, did setting mail checking interval to 15 min help?) It sounds to me yet another Mail.app bug...



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: c-o2 on Nov 29, '03 12:16:06PM

I presume I using the latest version, I only downloaded it 2-days ago... yep v.051... Still doing it.

Haven't set my e-mail to 15 mins (will tentatively do so, need them faster normally).

Will report back.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: c-o2 on Nov 29, '03 04:26:46PM

Same thing on 15 min mail retrieval also... obviously less junk ;o) But the same junk.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Nov 29, '03 09:46:35PM

Ok I think I might have found what's going wrong - for people having trouble with POP accounts (re-downloading the junk mails): please open your ~/Library/Scripts/junkMatcher/junkMatcher.scpt file (double-click on it), and remove the line "delete theMsg" from the script. Let it run for a while and tell me if there are any duplicated/strayed junk messages.

The reason why "delete theMsg" is there even when there is a statement "move theMsg ..." above it, is at some point in time I found that when Mail.app moves some messages to the junk mailbox, it still leaves a copy behind in the Inbox! Since this only happens to SOME messages, there would be some messages that are really moved away and there's no message left in the Inbox to be deleted, which might trigger an error in the script. And Mail.app may refuse to engage the subsequent actions (like removing a message from your POP server) if any error happens during the rule execution...

This is my theory (and I have some limited evidence this is the case). So let's do this check folks...



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Nov 29, '03 11:10:38PM

ok it seems working, but the correct solution should be:

try
delete theMsg
end try

instead of just removing the "delete theMsg" line.

I just put up a new version (0.52). This has many other under-the-hood enhancements, and should fix this re-download-mails-from-POP problem now. Let me know if you still find problems.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: c-o2 on Nov 30, '03 03:17:15PM

Just download .52 and still the same problem after a quick check. Will do a more thorough check later on, but seems the same problem.



[ Reply to This | # ]
Add regular expression junk filtering to Mail.app
Authored by: fortepianissimo on Dec 01, '03 07:58:20PM
From now on JunkMatcher is on MacUpdate.com. The link is

http://www.macupdate.com/info.php/id/13452

You can add it to your watchlist to be informed about any future update.

[ Reply to This | # ]
No Longer on Macupdate.com, check here
Authored by: fortepianissimo on Sep 13, '04 09:45:07AM

I will no longer inform macupdate.com about any future release. Please check here:

http://www.versiontracker.com/dyn/moreinfo/macosx/22023

or directly here:

http://junkmatcher.sourceforge.net/Home/index.html#Download

for the latest version and information.



[ Reply to This | # ]