Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

DirectoryService high CPU utilization System
I had been experiencing extremely high CPU utilization by the DirectoryService process. Most of the time it was at 65% CPU usage, and sometimes it was up to 190%.

I traced this to my usage of the /etc/hosts file (with 16K+ entries) to combat website ads. Each entry in the /etc/host file had the form: 127.0.0.1 <hostname>.

Here is an alternative method of host redirection that reduces the CPU burden of DirectoryService to nearly nothing. Dnsmasq is a small footprint DNS/DHCP server designed for small networks, or is this case, a single machine.
  • Install a localhost version of dnsmasq. (I used MacPorts but you can download the source from the above linked site.
  • Configure the network settings to resolve DNS queries to localhost (127.0.0.1).
  • Made sure the firewall accepted the queries for dnsmasq.
  • Configured /etc/dnsmasq.conf to forward unresolved hosts out to my normal DNS servers. Also configured the /etc/dnsmasq.conf file to read my blocked ad hosts from a file.
  • Made sure dnsmasq would start at system boot.
  • Verified both normal entries resolved and ad hosts resolved correctly.
The efficient dnsmasq process does not even register any percentage, sticking at 0.0%. Afterwards, DirectoryService CPU utilization now sits at 0.0%.

[crarko adds: I haven't tested this one. There obviously are browser specific adblockers but the use of the hosts file is a time-honored method of address redirection. There is further explanation of how to use dnsmasq and some examples in this article.]
    •    
  • Currently 2.11 / 5
  You rated: 1 / 5 (27 votes cast)
 
[20,035 views]  

DirectoryService high CPU utilization | 19 comments | Create New Account
Click here to return to the 'DirectoryService high CPU utilization' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
DirectoryService high CPU utilization
Authored by: technicallyrite on Jul 14, '10 08:50:02AM
Very useful hint. Some people who block ads are ad content creators like me. The percentage of ad blockers is tiny compared to the masses that read them, so I'd say that this site and it's sister international sites are going to be just fine. Ad-free browsing is a true pleasure to those who marinate in the stuff for work.

[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: VxJasonxV on Jul 14, '10 09:43:40AM

I don't understand what hosts/DNS resolution has to do with the DirectoryService. Unless you hosts forwarded your directory server's name to an IP address that doesn't straight up reject the traffic.

I have a 10 mile long hosts file, but both at work (where I connect to a Directory Server) and at home (where I don't, but my configuration persists, obviously), I've never seen this issue.

I admit, I have seen CPU usage flare up once or twice, but we're talking for seconds at a time.

I'm glad you were able to resolve your issue, but my question stands.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: Spliff on Jul 14, '10 10:23:21AM

Same thing happened to me. With only a few entries in my host file, I had no problems. But then I used the MVPS host file (http://www.mvps.org/winhelp2002/hosts.htm) which has a ton of blocked domains. Then I noticed the DirectoryService process frequently monopolizing the CPU. Removing the majority of the host file entries fixed the problem for me.

Maybe it's a bug.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: baltwo on Jul 14, '10 12:47:11PM

Interesting. I have the same 16K entry hosts file and haven't seen any such DS CPU usage. Do note that my network is through a NAT router to the internet with three machines on the router, but no sharing between them.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: aMacUzur on Jul 14, '10 02:07:38PM

I've used this strategy for many years and it was problem-free until I updated to Safari 5 -- after which I've been experiencing the issue as described in this hint. Currently, my /etc/hosts file has over 31,000 entries. I suspect the new CPU-consumption behavior has something to do with Safari 5's pre-fetching behavior, but have not verified this. I'll be trying this hint when I have some time to deploy dnsmasq.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: leamanc on Jul 14, '10 03:01:23PM

Maybe it's just me, but I always thought 16k-plus /etc/hosts file was a kludge of a solution to a problem. In-browser adblockers are a much more effective and less resource intensive way to handle the problem. Or set up a proxy on your network that filters ads and known malware sites. Distributing and maintaining a massive hosts file just seems like a 1980s/early '90s solution to the problem.

Also, I like to run Apache, PHP and MySQL on my machines. Having all those ad servers in my /etc/hosts files slowed down page loading, and threw 404 errors all over web pages with blocked ads, unless I made a custom blank 404 error page. Once again, a kludge that slowed things down.

Now that Chrome's adblock works like Firefox's and actually stops the ads from loading (previous versions loaded then hid them), and now that Safari officially has extensions (one of which I assume is a good adblocker), I can't see any good reason to block ads via /etc/hosts.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: Anonymous on Jul 15, '10 04:42:34PM

If you frequent /etc, this is not a kludge, it's just another configuration.

Plainly you've never seen a dhcpd.conf file with more than a dozen lines...



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: leamanc on Jul 19, '10 02:24:26PM

Huge/complex/whatever config files on a server are one thing...maintaining it on your desktop seems another thing, IMHO.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: guns on Jul 16, '10 03:08:31AM

Thanks for your hint!

Apple moved moved DNS lookups to the DirectoryService daemon in Snow Leopard, and since then I've noticed the same problem with the high %CPU.

For those who say that they don't notice the hit, try refreshing 200+ feeds from an RSS reader while keeping an eye on the CPU. The drag on the system caused by DirectoryService grepping the /etc/hosts file for _every_ domain lookup is quite obvious then, but the little hits from every-day browsing also add up to quite a bit.

I tried to resolve it :), like you, by installing a local nameserver, but I installed unbound, which is a very powerful recursive DNS server. It was a bit too powerful though, and it was clear that it was optimized for extremely heavy workloads on the public internet, aggressively caching and generally not staying as quiescent as I hoped.

So I went unhappily back to the hosts file. Luckily I ran across your hint today and installed dnsmasq from source. It's perfect, and exactly what I was erroneously looking for from unbound.


You should note, though, that people who use the hosts file for adblocking should take the following steps for the best results:

  • Remove all the adblocking rules from /etc/hosts, move them to /usr/local/etc/hosts, and then add the following to dnsmasq.conf:
        addn-hosts=/usr/local/etc/hosts
    
    This prevents DirectoryService from ever scanning through the adblock list, which may still happen depending on how the DNS forwarding is set up. Dnsmasq loads the hosts file once on load (or SIGHUP) and keeps it in memory, so it is much more efficient.

  • Also, these lines should be defined in dnsmasq.conf:
        cache-size=65536
        local-ttl=86400
    
    The cache-size is the number of domains that dnsmasq will keep in its cache. 65536 is just a nice round value; it could be anything.

    local-ttl is the Time-To-Live to return for domains in the hosts file, which by default is 0 seconds!

    This is the correct behavior for people who use the hosts file as it was intended, but for people who use it to block ads, it would be sensible to have a long timeout on the query results from the hosts file. 86400 is one day. There wouldn't be anything wrong with increasing that to one week if you felt like it.

So thanks again. I'm feeling much happier with this setup.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: operator207 on Jul 19, '10 12:27:35PM

I fixed this type of problem by using BIND locally. I even have it setup to resolve certain domains using forwarders. (Helps when you have internal DNS at work, but still want to use your ad blocking setup)

null.zone.file:

$TTL 24h

@ IN SOA localhost. root.localhost. (
2010032301
86400
300
604800
3600)



@ IN NS localhost.
@ IN A 127.0.0.1
* IN A 127.0.0.1

blocked.zone.file:

zone "101com.com" { type master; notify no; file "null.zone.file"; };
zone "101order.com" { type master; notify no; file "null.zone.file"; };
etc...


Then point your DNS in Network Prefs to 127.0.0.1. If you want to query certain DNS servers for a particular domain, in your named.conf file add something like this:

zone "domain.net" IN {
type forward;
forward only;
forwarders {ip.of.dns.remote.server;};
};

I am sure there is more to it (installing BIND via Mac Ports/Fink, having it run as a service), but the above is the parts that allow you to use BIND as a ad blocker. Installing BIND is up to the user. So it securing BIND so you do not allow others to query your machine for DNS, unless you want to.

BTW, everything I have written in this post, I have gleamed in one way or another from MacOSXHints, and reading man files or docs for BIND.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: Kyuuketsuki on Jul 20, '10 12:33:36AM

I can't figure out for the life of me how to actually configure dnsmasq. Is there someone that can give a better guide? It seems this was written for people that already knew how use it (which most of us don't).

Or maybe I'm doing something basic wrong. I keep getting "failed to bind listening socket for <ipv6 address>: Address already in use"



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: sbaldiss on Jul 22, '10 10:49:44AM

yes, we need more instructions please.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: captainulf on Jul 26, '10 01:51:19AM
Has anyone tried putting the hosts information into the native DirectoryService Defaults node (or possibly a shadow node) using the dscl command? Does it make for better or worse performance than the /etc/hosts file?

You'd enter a faux host using the syntax below in Terminal.app (obviously this would need to be scripted if you want to transfer an existing 16k line hosts file)

sudo dscl localhost -create /Local/Default/Hosts/someadserver.example.com IPAddress 127.0.0.1

[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: PopMcGee on Jan 15, '11 03:33:32PM

This is just way too complicated. I attempted this but it looks like you need to be a unix sysadmin specialist with years of experience before attempting this.

I would hope somebody would offer a simpler solution, or a more clearly written tutorial.

Edited on Jan 15, '11 03:36:27PM by PopMcGee



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: FatBloke on Jul 04, '11 08:13:38AM

Hey everyone,

Here's a simple guide I put together: http://fatbloke.tumblr.com/post/7228329391/block-bad-hosts-on-a-mac-using-dnsmasq

Hope it helps.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: Hagure on Sep 26, '11 07:18:04AM

Thank you for this!

It's been a few months since I changed my hosts file, and had no idea what was causing things to slow down. I finally got fed up & searched for "DirectoryService", and found this hint. I'll get around to setting up dnsmasq when I have more time.



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: thoredge on Jul 23, '12 10:49:49AM

I experienced the same problem a couple of days. However probably of different reasons and certainly with a different solution.

I shutdown the wlan (AirPort) of the mac. The directory service then ceased to use 100%, however then the kies service (Samsung Kies - phone/pad administration tool) started to run at 100%. The solution for me was to kill that mf Kies everytime the directory service started to go amok.

This will certainly work only for a few desperate souls.

Mac OS 10.6.8



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: jabadu on Aug 14, '12 03:04:10PM

Kies was the culprit in my case as well. I've seen that software using like 100% cpu a couple of times before, but since a week or so it seem to go via DirectoryService. Killing Kies helped, so thanks for the hint!!!



[ Reply to This | # ]
DirectoryService high CPU utilization
Authored by: freedomfries on Aug 29, '12 12:03:13PM

I was having this problem with Directory Service- only while disconnected from the internet.

I solved this problem by looking at the programs that were running second high CPU. The Google Chrome Helper ! I uninstalled Chrome and no more 6000 rpm fans.

Chrome discontinued updates for OS X 10.5 and that's when I had to stop using it because it was always crashing on me. I had completely overlooked this at first with my Directory Service error but since I was no longer running Chrome, this seemed like BS.

Now I consistently use Firefox 15.0 and all is well.



[ Reply to This | # ]