Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Automate the download and installation of a hosts file Internet
I've written an Automator workflow that will download the MVPS HOSTS file and place it in the /private/etc folder. The workflow can be saved as an iCal plugin to run automatically, and does most of its work in the background -- it does ask for a password when it replaces the old hosts file, though.

After downloading the Automator workflow (119KB: Original hostHints mirror), open the desired workflow in Automator (one has Growl support, one doesn't) and go to File » Save as Plug-in. Select iCal Alarm, and now you can have it run once a week/month/year to help keep your Mac a little bit safer from unwanted internet intrusions. Please be aware that the workflow does not back up your old hosts file, so please be carfull when using it!

[robg adds: The workflow is relatively straightforward; it just curls the latest file from the site, renames it, and moves it into /etc/hosts. Be aware that the MVPS HOSTS file blocks a ton of sites, not all of which you may want blocked -- make sure you understand what you're getting if you use this hint.]
    •    
  • Currently 1.88 / 5
  You rated: 4 / 5 (8 votes cast)
 
[12,519 views]  

Automate the download and installation of a hosts file | 9 comments | Create New Account
Click here to return to the 'Automate the download and installation of a hosts file' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Automate the download and installation of a hosts file
Authored by: jimcintosh on Apr 27, '10 08:55:50AM
If you've added your account to the sudoers file with the NOPASSWD option, you can change the command in the fourth step to:
on run
do shell script "sudo mv ~/Desktop/hosts /private/etc"
end run
to avoid the password prompt.

[ Reply to This | # ]
Automate the download and installation of a hosts file
Authored by: Spliff on Apr 27, '10 10:20:54AM

Be careful.

My MBPro (10.6) experienced significant performance degradation when using the large MVPS HOSTS file. Websites became slow to load, and several times a day, the DirectoryService process would be pegged at over 100% CPU, slowing my computer down and ramping up the fans.

It turned out it was the long MVPS HOSTS file that was impairing my system. Since removing it, the DirectoryService problem has vanished. Now I only block the most annoying ad sites and banners with my host file.



[ Reply to This | # ]
Automate the download and installation of a hosts file
Authored by: marook on Apr 27, '10 11:31:20AM

If you really like this solution, it would be much better to create a script and run it via a launchDeamon.
Then you don't have to count on iCal (and a user logged in!) and you won't have to mess with SUDOERS and destroy the security of your system!

But it's a nice exercise in Automater stuff.. maybe.

---
/Marook



[ Reply to This | # ]
Watch out for phishing opportunities by validating entries
Authored by: PCheese on Apr 27, '10 02:07:15PM

If you choose to automate this, you should seriously consider validating the entries in the file that is being downloaded. If MVPs.org is ever compromised, it would be easy for attackers to add an entry that redirects, say, your bank's website to their phishing page. It's probably sufficient to discard any entries that do not point to 127.0.0.1.

Edited on Apr 27, '10 02:07:29PM by PCheese



[ Reply to This | # ]
to make it even more elegant...
Authored by: Christian Leue on Apr 28, '10 10:55:10AM

To avoid the little question mark icons in the rendered web pages I do the following:

1) replace every 127.0.0.1 with 127.0.0.2 in the MKVS HOSTS file (except the entry for localhost itself, of course).

2) create a pseudo-interface 127.0.0.2 with the following launchdaemon:

[code]
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.add_lo0_alias</string>
<key>ProgramArguments</key>
<array>
<string>ifconfig</string>
<string>lo0</string>
<string>alias</string>
<string>127.0.0.2</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>
[/code]

3) add a virtual host to /etc/apache2/httpd.conf and rewrite any request to serve up 1pix.gif which is a simple 1x1 pixel transparent gif image:

[code]
NameVirtualHost 127.0.0.2
<VirtualHost 127.0.0.2>
DocumentRoot "/Library/WebServer/Documents/dummy"
DirectoryIndex index.html index.php
ErrorDocument 404 /error.html
<Directory "/Library/WebServer/Documents/dummy">
Options All -Indexes -ExecCGI -Includes +MultiViews
AllowOverride None
</Directory>
RewriteEngine On
RewriteRule ^.*$ /1pix.gif
LogLevel warn
</VirtualHost>
[/code]



[ Reply to This | # ]
to make it even more elegant...
Authored by: encro on May 01, '10 04:54:09AM

You should be able to replace each instance of 127.0.0.1 with 0.0.0.0 and then not have to worry about using Apache's Mod Rewrite as it doesn't seem to render the image.

That said I've found Glimmer Blocker to be a great solution.

---
Steve



[ Reply to This | # ]
Automate the download and installation of a hosts file
Authored by: robg on Apr 30, '10 05:26:55AM
[The following was submitted as a new hint by an anonymous user; I'm running it here as a comment because it's directly related.  -rob.]



I love the fact the the hosts file is kept up-to-date, but the password prompt bothers me, and I don't really like seeing maintenance scripts like this activate while I'm working. As it turns out, this is very simple to implement via cron.
  1. Download CronniX.
  2. Click the Open button on the toolbar and type in root.
  3. Enter your admin password.
  4. Click on New on the toolbar, and then switch to the Simple tab on the sheet that pops up.
  5. I wanted this to run at 4AM every day, so I left the Minute slider alone, dragged the Hour slider to 4, and checked the other three boxes.
  6. Paste in the following command into the box at the bottom: /usr/bin/cd /etc;/usr/bin/curl -O http://www.mvps.org/winhelp2002/hosts.txt;/bin/mv /etc/hosts.txt /etc/hosts
  7. Click the New button at the bottom of the sheet, and then choose Save on the toolbar at the top to insert this into your workflow.
For those of you who are comfortable with editing the crontab from the command line, you can do it the way I originally did:
  1. Fire up Terminal.
  2. Type sudo crontab -e and enter your admin password.
  3. Press i to place vim into editing mode.
  4. Paste in the following line (replacing my time values below with your preferred time and frequency): 0 4 * * * /usr/bin/cd /etc;/usr/bin/curl -O http://www.mvps.org/winhelp2002/hosts.txt;/bin/mv /etc/hosts.txt /etc/hosts
  5. Hit the Escape key, and then type :wq to write the crontab and quit vim.
To verify your crontab, type sudo crontab -l to see it printed out at the command line.

[ Reply to This | # ]
Automate the download and installation of a hosts file
Authored by: MVasilakis on May 13, '10 08:54:26AM

If I understand this right it will whack my current hosts file. so if I have any custom entries they will get whacked. would it not be better to run a script that just adds the new addresses and modifies any changed ones?



[ Reply to This | # ]
Automate the download and installation of a hosts file
Authored by: fooljay on May 14, '10 09:39:02PM

Actually, it would be much easier if you, as a rule, disallowed any non-joint/non-collaborative modifications of /etc/hosts and instead maintained two separate files that you concatenate any time there’s a change to one or the other.

For example:

  • You maintain your own hosts file in any arbitrary but safe place (where the bigger threat is often Apple and their dreaded Software Update). Let’s say /usr/local/etc/hosts just because it’s so darn fitting.
  • You set up a script to download the MVPS hosts file periodically placing it in an equally safe but fitting place, say, /usr/local/etc/hosts-MVPS
  • Finally, you create a separate, standalone script (even as something as simple as sudo cat /usr/local/etc/hosts{,-MVPS} >| /etc/hosts) which you will use to effortlessly combine the two files and push the new content to /etc/hosts, replacing it in its entirety each time there’s an update to either MVPS’ hosts file or yours.

If you have to edit your hosts file with an regularity, there’s room for a massive amount of automation here which will not only save you time and make the setup above completely invisible but it could also keep you safe from your own oopsies.

For example, you could create a command line utility through which you can very easily view/add/modify/delete host entries without ever firing up an editor. Since shell scripts never get tired of doing boring, routine things, it could handle the whole “concatenate and clobber” process better and more reliably and tirelessly than you can and it could also serve as the conduit for MVPS’ entries into the system.

Or, to take that up a few hundred notches: You could add Git into the mix to act as sort of a hyper-powerful “caching server” with change control management capabilities. Imagine this—you set up a Git repo with three branches—“Master” (the default name), “MVPS” and “Mine”—each containing a single file: /etc/hosts. Of course, the three are very different: The one in the MVPS branch is huge, the one in the “Mine” branch is small and focused and the one in the master branch is a perfect union of the two.

With that in place, when you need to modify the host file, you modify your version checked out from your branch and then commit the changes back into Git. When the cron process downloads the latest version of the MVPS’ file, it would do the exact same, finishing off with a git commit. Both of those separate actions would immediately trigger a post-commit hook script which pulls (i.e. merges) the changes into the Master branch and then immediately performs a git update from the master branch to…./etc/hosts of course. So, your respective changes are merged and published immediately and automatically almost no extra effort on your part and neither list can ever pollute the other because because they live in alternate realities (i.e. git branches).

And, to top it all off, since Git always knows the latest state of the file, it could take a page from Tripwire’s book by “watching” /etc/hosts (and any other file you deem important enough) for unauthorized changes. Any change to a file managed by Git (on its master branch of course) would garner a swift reply including immediate restoration of the “last known good” version of the file and an urgent “WARNING! WILL ROBINSON!” alert notification to you.

Ta da… Version control, vendor change management and intrusion detection, response and notification system all rolled into one…



[ Reply to This | # ]