Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Filter junk mail with Entourage manually and intelligently Apps
Not everyone's using Mail, but we all have to deal with spam. I use Entourage, and its junk mail filtering leaves much to be desired, unless you sort spam by rules you create.

Over the years, I've put together a very concise, tuned list of critera to go from seeing 100% of my spam in my inbox to seeing about 5% of it or less. It consists of three rules: one to save the stuff you want from Deleted Items, one to lay burnination to the bad stuff you don't want, and one to defeat a nasty trick a lot of spammers use. I went ahead and typed them up and posted them for public consumption.

The first rule is the only complex one, because it involves adding a header to e-mail you send. This is explained in the document -- it's a way to catch spam sent to you with your e-mail address in the from field, but let your e-mail address remain in your address book, as Entourage likes it. The second rule is very, very basic, and is meant to be augmented to suit needs as they pop up. The third is also minimalist, but extremely accurate. It's essential that you make the rules in order (or arrange them that way afterwards), otherwise you'll flag your mother's yahoo.com mail as spam, and that'll make her very angry at subsequent holiday dinners.

Nothing really special, but I thought it might make some Entourage users' (and other clients, really--the rules are good for any client that supports them, I suppose) lives a bit easier. Remember that they don't delete anything; they just recolor/tag them as spam and send them to Deleted Items without notifying you.
    •    
  • Currently 2.67 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (3 votes cast)
 
[11,570 views]  

Filter junk mail with Entourage manually and intelligently | 13 comments | Create New Account
Click here to return to the 'Filter junk mail with Entourage manually and intelligently' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Another suggestion . . .
Authored by: eDogg on Mar 24, '03 10:28:06AM
Being thoroughly annoyed with spam myself, I've done some extensive research into filtering the junk out. While creating rules based on headers and certain flagged words can do a lot of good, the better solution is to use a stastical method of analysis called "bayesian filtering". For a detailed explanation of why this method is better, click here.

This may sound kinda complex, and it is, but the implementation isn't that bad. There's a project on Sourceforge called POPfile. The good news is that it's Perl-based, so it will work on any system that supports Perl (OS X included). The bad news is that the filter only currently supports POP mail, not IMAP (sorry .Mac people).

I've been using this filter for only a few weeks and it's already at 98.5% accuracy with no false positives. And I can use the same "brains" on every computer I use, since it's cross platform compatible. I highly recommend it.

[ Reply to This | # ]

How do you get this to work?
Authored by: uochris on Mar 24, '03 02:51:53PM

When I run perl popfile.pl I get

[Chris:/applications/popfile-0.18.1] mac% perl popfile.pl
Can't locate MIME/Base64.pm in @INC (@INC contains: /System/Library/Perl/darwin /System/Library/Perl /Library/Perl/darwin /Library/Perl /Library/Perl /Network/Library/Perl/darwin /Network/Library/Perl /Network/Library/Perl .) at Classifier/MailParse.pm line 15.
BEGIN failed--compilation aborted at Classifier/MailParse.pm line 15.
Compilation failed in require at Classifier/Bayes.pm line 15.
BEGIN failed--compilation aborted at Classifier/Bayes.pm line 15.
Compilation failed in require at popfile.pl line 239.



[ Reply to This | # ]
How do you get this to work?
Authored by: Quantum Panda on Mar 24, '03 05:23:10PM
There are two things you need to have installed that don't come with POPfile: (1) Perl, which is in the Developer's Tools, and (2) the MIME/Base64 module for Perl, which Apple inexplicably doesn't include in their Perl installation. This site has a link to the module, as well as good detailed instructions for installing the whole shebang and setting up a startup item to automatically run it at startup.

[ Reply to This | # ]
Filter junk mail with Entourage manually and intelligently
Authored by: Frederico on Mar 24, '03 10:40:17AM


Yikes. I can see dozen or more reasons why those filter criteria would kill tons of potentially important mail.

While I credit the author for taking some precautions that seem workable for his situation (the flagged personal header actually seems a fine idea), I strongly advise anyone else employ these rules as is with extreme caution. One may also wish to divert mail to a custom folder for long-term inspection and sorting prior to deletion. Remember, items moved to the Deleted Items folder will automatically be purged after the preset timeout in your preferences (default is 3 days, I think), so you can't wait too long to inspect, either.

This would, e.g., cause problems with poorly dated mail not in your address book, such as mail from people who may need to compose it, but do not make connections or execute sending for several days (not entirely uncommon in my life), or, given the current state of OS X 10.2.4, may simply be affected by a computer with an incorrect date.

Also, anyone who suddenly chooses to change email addresses without first notifiying you from the the current address in your book (or other methods) will very likely get snagged in this filter, particularly if they switch to Yahoo, Hotmail, AOL or the ever-growing MSN. Most people do wait to inform you until they have the new address, and this filter will prevent you from even seeing such notice in many cases.

And be very careful, too, if, like me, you're currently working on refinancing any property, working on confidential deals/situations, or anyone ever forwards you email you want to see from an address not in your address book. God forbid you ever get email from anyone unknown to you by way of referral.

With all due respect to the author, I don't think these rules -- rather, the rigid severity of the third one -- would work well for most people, and, honestly, I'm baffled as to how they work so well for the author; and, unless the author inspects each and every piece of diverted "SPAM" sent to Deleted Items *before* it gets deleted, I don't think he can state with 100% certainty it works perfectly for him, either. And, if he does inspect all that mail, what's the point of these filters?

Bottom line for many people: SPAM sucks, and you still need a degree of human interaction to avoid losing important mail. Those who depend solely on filtering, no matter how good or advanced (such as Bayesian), will undoutedly, at some point, miss an important piece of mail, and the sad part is they will never know it. Ignorance may be bliss, but is it worth the risk?

FWIW, what works for me is to filter out all known sender addresses to specific custom folders (e.g., Family, Friends,, Business, etc., and specific sub-folders therein as required), and everything else is left in the inbox for the inspection. Other strategies, such as use of special or custom addresses when registering yourself at untrusted sources so that those can be flagged as undesireable mail automatically also help. Even though I get on the order of 300-400 pieces of SPAM per day across dozens of active email accounts I must use, it takes less than a couple minutes of my day, using Entourage's built-in Junk Mail filter flagging (set to about the middle) to select and execute an AppleScript upon that forwards the email to the FCC, reports it to various abuse agencies, domain owners, etc.

Frustrating some days, yes, but moreso over content and disgust at the state of peoples morals and ethics than my loss of time.

Cheers



[ Reply to This | # ]
Filter junk mail with Entourage manually and intelligently
Authored by: Mikey-San on Mar 24, '03 11:27:57AM

Where does anything say that you have to use all of the third rule's critera?

"I'll repeat that, because it bears repeating." - Lewis Black

Where does anything say that you have to use all of the third rule's critera?

If you get tons of mail from yahoo.com, that you WANT, where the senders aren't in your address book, don't use that criterion. If you get massive amounts of spam, and don't get e-mail from yahoo.com that isn't from people in your address book, use that criterion.

It's an HTML file, not an Entourage configuration application/script that you can't change. The basic concepts are all the same: Get around the "from me to me" spam trick, make exceptions, and lay down some burnination on everything else.

I'm strict on spam. My setup doesn't touch mail I want, and my filters show it. Some are lighter, and won't use all of those. It's a "starting point", like I said.

Excuse me for assuming people who understand rules are smart enough to understand that if they don't want yahoo.com blocked, they can ignore and not use the yahoo.com filter. I should assume they're all inept and will wonder where their mail went, like you, next time.

Also, if you work in finances, etc., and you see the rules that tag "financ", and /don't/ understand that, like everything else listed there, anything with "financ" in the subject line (just like the rule plainly, clearly defines) will get deleted, you shouldn't be touching spam rules to begin with.

Also, I appreciate you calling me a liar. I really do. I use this system, and it /does/ work for me. /Also,/ I didn't state with that it works for me 100%:

"It's a good starting point for getting the spam out of your field of vision. It takes about fifteen minutes or so to set this up, but it's worth it. I see maybe 5% of all spam sent to me, with an error margin of about 0.001%.

"If you have any ideas to improve upon this, shoot 'em my way."

1. Note the two error percentages there, including the error margin phrase. I'm serious when I say that I really only see about one friendly-fire error in every thousand messages that get scanned. I've been evolving my rules (note how I use "me" there, implying a level of subjection, and how it's just a bunch of text--you can implement it however you like) for a long time, and I wouldn't say it's too far-fetched, after more than a couple of years, to become that accurate against your e-mail habits.

2. Rather than giving me helpful suggestions, as I asked, you go to town on all but the first part in a comment you have no way of knowing if I'll see (though I did, obviously). Thanks for the single sentence on the (I'd say) most interesting part of the document, the header/authentication trick. (I was quite proud of this, even though it's probably been done before.)

I don't like spam. If you don't like it, either, these concepts are easily taken into your e-mail client of choice however you like, to best fit your needs.

It's easier to criticize someone else's work than to provide better alternatives. (No, a vague reference to using Entourage's junk mail filter and AppleScript doesn't exactly count.)

Thanks for the comment.



[ Reply to This | # ]
Poking fun at myself, heh
Authored by: Mikey-San on Mar 24, '03 11:30:51AM

... And the best way to figure out when someone's using copy and paste?

"... all of the third rule's critera?"

When the same typo appears twice in a row. ;D

-/-



[ Reply to This | # ]
Filter junk mail with Entourage manually and intelligently
Authored by: shift_register on Mar 24, '03 02:39:07PM

>Wth all due respect to the author, I don't think these rules -- rather, the >rigid severity of the third one -- would work well for most people, and>, >honestly, I'm baffled as to how they work so well for the author


Well by blocking all mail from AOL, hotmail, yahoo, and MSN, that eliminates the majority of SPAM. As to "mortgage", I deal with a local mortgage company, someone I see face to face, so that is not a problem. The rest of those key words would block a lot of unwanted email.

This third rule will work real well for me.

Robert



[ Reply to This | # ]
custom addresses
Authored by: redfood on Mar 25, '03 01:21:11AM

>Other strategies, such as use of special or custom
>addresses when registering yourself at untrusted
>sources so that those can be flagged as undesireable
>mail automatically also help.

I love spamgourmet.
www.spamgourmet.com



[ Reply to This | # ]
You will need to have perl 5.8, how to auto start?
Authored by: uochris on Mar 24, '03 03:29:40PM

I downloaded and installed perl 5.8 and now it works. How do I get this to start automatically if I reboot my machine? Do I have to launch this from the terminal every time I restart my computer?

I got the installer from http://www.aaronfaby.com/downloads-jag.php#perl



[ Reply to This | # ]
How to auto start and other good info
Authored by: uochris on Mar 24, '03 06:01:35PM

I got it up and running thanks to the forums at sourceforge. This is the key thread for OS X.

http://sourceforge.net/forum/forum.php?thread_id=778129&forum_id=234504



[ Reply to This | # ]
Bayesian filtering
Authored by: supercity on Mar 24, '03 08:50:05PM
This is going to sound like an ad. I'm an Entourage user and was tearing my hair out at junk mail that outnumbered real mail by 2:1. Then I discovered Spamsieve at http://www.c-command.com/spamsieve/
This little beauty is a Bayesian filter for all major Mac mail programs (except Mail, which has its own built in). Install a few Apple Scripts into Entourage (or Eudora, whatever) and you're away. It's now achieving 97.4 per cent accuracy in moving spam messages to a separate spam folder, where I can look them over before trashing them. Can't recommend it highly enough. The best 20 bucks I've spent on shareware.

[ Reply to This | # ]
Bayesian filtering
Authored by: incongruity on Mar 25, '03 09:23:33AM

Chalk me up as another happy user of spam sieve. Bayesian filtering
is a fantastic idea for how to deal with spam...it should also be noted
(I believe) that apple's own mail.app uses a Bayesian filter for junk
email as well.

-i



[ Reply to This | # ]
Spam Seive Rocks!
Authored by: jason mark on Mar 31, '03 08:02:18PM

I agree. Spam Seive is great. Esp with the applescripts.. all you have to do is hit control-G for good mail and control-S for spam... very easy.

I also have a series of RULES that I made by hand that are run before spam seive.. these let spamsieve know what's junk even before it has to look at it.



[ Reply to This | # ]