Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Compile *nix programs faster multi-core Macs UNIX
If you have a multicore processor in your Mac, then instead of doing the traditional compile and install steps for a *nix program...
$ ./configure
$ make
$ make install
...use this version instead:
$ ./configure
$ make -j n
$ make install
The n in the second line is the number of jobs you want make to start. So to reduce compile time, replace n with the total number of cores in your Mac.

[robg adds: At some point, I'd like to test this with a large compile, but I haven't had the time as of yet. If you've done any testing on compile time reductions versus cores used, please post your results in the comments. I think this should work on multi-core G5s and all the Intel Macs...]
    •    
  • Currently 1.00 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (1 vote cast)
 
[19,329 views]  

Compile *nix programs faster multi-core Macs | 20 comments | Create New Account
Click here to return to the 'Compile *nix programs faster multi-core Macs' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Compile *nix programs faster multi-core Macs
Authored by: marcpage on May 25, '07 07:50:44AM

I have been trying different numbers, and it looks like number of cores + 1 works well.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: janl on May 25, '07 07:56:23AM

A few additions, -j 2/3 also can help on single-core/cpu machines because the CPU doesn't have to wait for disk reads and writes in between compiling files. Because of that, it is usually recommended to set n to cores/CPUs +1

On the other hand, not all buildchains allow parallel job execution, so if your make fails, retry without the -j option.

If you run make more than once, installting ccache speeds things a lot. For additional kicks, you can put the ccache dir on a ramdisk, if you've got some spare ram.

At last, distcc utilizes machines on your network to outsource compile jobs, just like XCode does.

Cheers,
Jan
--



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: Echidna on May 25, '07 08:00:16AM

For what it's worth, I've found that make automagically uses two (or more) cores if you are building for more than one architecture. But at that point, each build is completely independent of each other, so it makes a lot of sense to parallel-ize things.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: boredzo on May 25, '07 08:16:51AM

-j works even better on the install step. Set it to a really high number, and make will just spawn process after process after process without waiting for the process to end. You're probably not going to save much time, since the bottleneck is disk reads and writes, but it saves a little bit and it's fun to watch the terminal get slammed with all those copy commands. ☺

One way to help that would be to build the program on a RAM disk, and install it from there.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: beepotato on May 25, '07 08:31:14AM

"I think this should work on multi-core G5s and all the Intel Macs..."

Actually, it should also work on multi-processor G4s and G5s.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: eile on May 25, '07 09:17:42AM

It works quite well - for the software I am developing it gives about 70% speedup on a dual-core machine. It could be better, but I have some serial portions and dependencies in my build. As others said, submit more jobs than CPU's (I use 2x), since the compiler is often waiting on IO. Using more than 2xnCore processes does not make sense.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: krreagan on May 25, '07 10:29:53AM

I've used this option for compiling my FreeBSD systems for many years. I have found through trial and error, that the optimal value (for me at least) is n=4*CPU/Core. That is, 8 jobs for a dual core system. This is because compiling is very IO intensive so while a job is waiting for the IO to complete, another can continue. You can have several jobs waiting on a disk to complete an operation.

Because IO is so slow (relatively to CPU) several jobs tend to be waiting on IO to complete at any one time.

There are a lot of variables that will make "n" different for different systems... amount of RAM & Cache(L1,L2,L3), Bus & HD speeds... If you have lots of RAM/Cache, try upping n to higher and higher values until you don't see any increase in compile speed.

TBM



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: Ranger Rick on May 25, '07 10:47:10AM
Depends on the program, some things it helps a lot, some things very little. As others have said, sometimes -j2 is faster even on a single-core system.

Also, this won't work for everything; many make-based projects have improper dependencies and you can end up with a failed compile or worse, a junk build without knowing it, so be careful, Your Mileage May Vary. ;)

This can also be done somewhat unofficially in Fink:

http://wiki.finkproject.org/index.php/Setting_MAKEFLAGS_in_Fink

It works 99% of the time, but when it doesn't, try setting it back to -j1 before reporting problems to the maintainer. ;)

[ Reply to This | # ]
what about Fink or Darwinports?
Authored by: allanmarcus on May 25, '07 01:59:07PM

Anyone know of a way to set Darwinports and Fink to use this hint?



[ Reply to This | # ]
what about Fink or Darwinports?
Authored by: jochen Küpper on May 26, '07 09:31:35AM

By setting the MAKE environment variable?



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: brettmjohnson on May 25, '07 02:55:52PM
robg adds: At some point, I'd like to test this with a large compile, but I haven't had the time as of yet...

Actually, this works poorly for large compiles. There are 2 problems:
  1. If you specify the number of parallel jobs, make -j n, but have nested projects (make spawns other makes in sub-processes), the top level make will execute parallel targets, but the -j n is not passed on to the child make processes. In most large projects I've worked on, most of the compilation actually occurs in the sub projects, and the targets in the root project have a necessary build order that often prevents them from being built concurrently. If you call make -j without specifying a maximum number of parallel jobs, then the -j option does get passed down to the child make processes. But specifying make -j without a maximum job count has a serious drawback...

  2. If you run make -j without a maximum job count, and have a large number of files that need to be compiled, make will spawn an unlimited number of jobs. By default, Mac OS X limits the maximum number of process per user to 100. If make tries to spawn more than 100 jobs (actually 100 - number of already running processes), then the build will fail with a large number of the following error: "vfork: Resource temporarily unavailable." You can avoid this by increasing the maximum number of processes per user using launchctl maxprocs ... However, even bumping the maxprocs to 512 or 1024 has another undesirable side effect: spawning 500 or 1000 active (non-idle) processes quickly overwhelms the kernel CPU scheduler, even on my 4-core PowerMac. The result is that make spends 90% of its time context switching, not compiling.
So in my experience with large compile jobs, make -j creates too many concurrent processes, and make -j n creates too few concurrent processes. I believe make -j should be enhanced to spawn no more than 4 or 6 jobs per CPU core.

[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: woodgie on May 25, '07 03:49:21PM

That is a beautifully written, concise description of what is going on and I appreciate you taking the time to post.

It's posts like this that are utterly invaluable to learning as it gives me a nice grounding for going out to find more.

Thank you very, very much.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: jochen Küpper on May 26, '07 09:35:23AM

Set the MAKE environment variable. At least with GNU make that works nicely on all platforms I have used it (including Mac OS X;)

See the make info manual (info) as well.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: bugmenot on May 26, '07 10:10:11AM

like so:
export MAKEFLAGS="-j3"



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: jochen Küpper on May 26, '07 09:37:34AM

Well, sorry about my "random" first comment...

What's about he "-l" flag? Does that not work for you?



[ Reply to This | # ]
Cool!
Authored by: macubergeek on May 25, '07 04:54:28PM

Very cool hint!
Thanks!



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: pauldy on May 25, '07 11:15:56PM

If you use it a lot you can create a ~/.profile and put something like alias make="make -j n" in there so it just does it by default without having to type it in everytime.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: itistoday on May 27, '07 01:32:30PM

This is a really cool hint but watch out, if you're using this along with a complicated Makefile there's a chance that it won't finish the compilation because one part of the build process could depend on another having finished. However, if it does fail, it's very likely the case that simply running make again will finish the job.



[ Reply to This | # ]
Compile *nix programs faster multi-core Macs
Authored by: JWiegley on May 27, '07 10:43:51PM

I find that make -j5 saturates my dual core CPU very completely; however, it also increases disk I/O and consumes nearly all of system memory (3Gb) if C++ is involved. In those cases, make -j3 is more optimal. But both are far quicker than make -j1.

John



[ Reply to This | # ]
My experience of using RAM disk for faster builds
Authored by: Andriy on Dec 02, '07 08:00:56PM
Here is my experience of using a RAM disk to make builds faster. My setup is on Linux, but it should work with minimal changes on OS X.

[ Reply to This | # ]