Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Tune high-speed networks for faster transfers Network
Here's a little blurb from an article I found on the internet regarding OS performance tuning, written by the Pittsburgh Supercomputing Center. Note that this only really helps on data transfers on gigabit ethernet networks, though it might help latency on 100 megabit networks.
Mac OS X has a single sysctl parameter, kern.ipc.maxsockbuf, to set the maximum combined buffer size for both sides of a TCP (or other) socket. In general, it can be set to at least twice the BDP. E.g:
sysctl -w kern.ipc.maxsockbuf=8000000
The default send and receive buffer sizes can be set using the following sysctl variables:
sysctl -w net.inet.tcp.sendspace=4000000
sysctl -w net.inet.tcp.recvspace=4000000
If you would like these changes to be preserved across reboots you can edit /etc/sysctl.conf.
Read on for more detailed instructions on how to use this on your Mac.

The steps:
  1. Benchmark your connection either at a site from the article or a local fileserver, and note the data transfer speed.
  2. Try the transfer again. If its cached in ram on the server it will be much faster.
  3. Run the following commands in order, in Terminal. Type each line individually and in order, or else your TCP/IP stack will go dumb. Also, the maxsockbuf must be at least the sum of the send/recv space buffers, or TCP/IP will go dumb.
    $ sysctl -w kern.ipc.maxsockbuf=8000000
    $ sysctl -w net.inet.tcp.sendspace=4000000
    $ sysctl -w net.inet.tcp.recvspace=4000000
    
  4. Run your benchmark test again.
If you want to make the change permanent the best way to accomplish this to download Apple's Broadband Tuner. This will add the sysctl.conf file, and the line to rc. Then edit the /etc/sysctl.conf file to the above specs and reboot.

In my testing across a gigabit Ethernet network using an OS X Server with 1.5GB of RAM and two mirrored disks, I can move about 40 megabytes a second. Previously, I obtained about 15 megabytes a second. If you are on a 100 megabit network, your max bandwidth is 10 megabytes a second, so you won't see the increase.
    •    
  • Currently 1.50 / 5
  You rated: 1 / 5 (4 votes cast)
 
[49,456 views]  

Tune high-speed networks for faster transfers | 19 comments | Create New Account
Click here to return to the 'Tune high-speed networks for faster transfers' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Tune high-speed networks for faster transfers
Authored by: sblasl on Jun 19, '06 08:32:17AM

Excuse my ignorance but when I tried to install Apple's Broadband Tuner, the installer told me there was nothing to install. What would that mean?

Any chance that you could put something together for us folks that are somewhat clueless to this sort of stuff but would like to take advantage of the potential optimization?

Thanks



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: wuf810 on Jun 20, '06 12:47:51AM

Great hint. Can I confirm that the changes should be made to the server machine only.

Is there any advantage in also making the changes to client/user machines?



[ Reply to This | # ]
Negative affects?
Authored by: bcamp1973 on Jun 19, '06 09:13:38AM

How might this affect my DSL connection? I'd love to have high performance on my home LAN between my server and workstations but will this negatively affect my DSL connection speeds? It's supposed to be 3Mbs+, but obviously you never quite get the speeds they advertise...



[ Reply to This | # ]
Negative affects?
Authored by: thekurgan on Jun 19, '06 11:32:03AM

A couple of things to watch out for:

1. if you don't set maxthreshold to 0, these settings won't take effect until you have over 256 (default) sockets in use

2. if you set these windows too high, you'll get lots of retransmissions, which results in slowdowns and latency.

Netstat -s will show you several important pieces of information. I find that setting to 32768 is very good for the internet and my local transfers over 100mb ethernet.

YMMV

---
-----
If I throw a dog a bone, I don't care to know whether it tastes good or not



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: jpgetty on Jun 19, '06 10:21:31AM

Worked a charm for me. 8Mb/s up and down before, 16Mb/s up, 85 Mb/s down after.



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: Dragon76 on Jun 19, '06 10:34:05AM

I don't remember the settings for Apple's Broadband Tuner, but is this any better than that?



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: toddwilliams@mac on Jun 19, '06 11:43:49AM

I posted the article, so I figured I better register.

This will not help except on GigE LANs or on a server.

The apple broadband tuner sets a limit of 384k for both send and receive. One is set to 128k and the other is set to 256k

These settings change it to 8mb for both send and receive 4mb for each. If you have a GigE network or a heavy use xServer you could go higher with diminishing returns.

Read the article and perform a couple of google searches for High speed LAN test servers at Universities using a web100 network diagnostic tool

Todd



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: Dragon76 on Jun 20, '06 01:05:33PM

Not relevant to me, then. I just use Airport and cable modem and the broadband tuner is supposed to work for FiOS, so I am set with that.



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: neuralstatic on Jun 19, '06 11:45:55AM

fwiw: i'm getting about 48mb/s before this mod. this mod didn't improve it.

10.44 client
10.39 server
gig switch, but only cat5 wiring.



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: Slinkwyde on Jun 19, '06 08:20:03PM

The wiring is your problem. You'd need cat6 wiring.



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: Slinkwyde on Mar 29, '07 09:56:32PM

I found out much later that cat5e supports Gigabit ethernet as well. My mistake.



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: eppenberger on Jun 20, '06 03:29:13AM
The maximum throughput you can achieve between your Mac and a remote server depends on two parameters: thinnest link along the way and the round-trip time.
The first parameter is often easy, especially if the server is on your local network.
The second parameter can be measured with ping.
I said: the maximum throughput! Whatever you tune, you can't go higher than this.
Now comes the question, how to come as close as possible. This is where the TCP windowsize plays a major role. Your windowsize should be greater or equal to the bandwidth-delay product. Check the excellent document, chapter 3.1:
http://www.geant2.net/upload/pdf/GN2-05-176v4.1.pdf
There is more information on performance tuning on:
http://www.switch.ch/network/performance/

[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: regulus on Jun 20, '06 05:41:51AM

I liked this hint and reading the tutorial, thanks. With my set-up it won't help me though. Here's why.

From reading the tutorial it says that you need to calculate the BDP (bandwidth delay product). I have a gigabit ethernet network in my home. The bottleneck on my network would seem to be the firewire 400 connections for my numerous external hard drives connected to my server. All the other connections are faster than the firewire connection i.e. the hard drives, the gige network cards, the gige switch, the cat 6 wiring are all faster than the firewire connection which has a max throughput of 400 Mb/s (ideally). So to calculate the BDP you ping your server (which gives the the RTT) and multiply that by the bottleneck. Pinging my server yielded a time of .264 milli seconds. Therefore my calculation is as follows:

400,000,000 bits/s * 1 Byte/ 8 bit * .264/1000 s = 13,200 Bytes or 13.2 KB

As such the default send/receive window size is more than adequate. The tutorial even states "If the BDP is small either because the link is slow or because the RTT is small (in a LAN, for example), the default configuration is usually adequate."

Testing my speed at the default configuration I get between 22 to 32 megabytes/s from my various 10 external hard drives. Since the ideal firewire speed is 400 Mb/s which yields and ideal speed of 50 MB/s, and knowing that nothing is ideal in the real world, I think I'm OK.

Just to make sure I tested it by adjusting my parameters as suggested and saw no change in my transfer speed on my LAN. As far as my internet connection, I already get around my rated speed from my ISP as tested at broadbandreports.com. Therefore I can only conclude that this hint won't help me.

I'm always looking to tweak my speed, so if anyone sees any fault in my logic please let me know.



[ Reply to This | # ]
Caveat
Authored by: macubergeek on Jun 22, '06 02:40:13AM

The one caveat I'd add to this hint is this. Before you change kernel parameters, make sure you take note of their original settings. If the settings don't work for you, you would have two options to "roll back" the changes. You could reboot (not a good option for a production server), or you could "reset" the values back to what they were before the changes.

Run the below script to save the default settings:
[script]
#!/bin/sh
file = transfer_rates.txt
sysctl -a | grep kern.ipc.maxsockbuf >> $file
sysctl -a | grep net.inet.tcp.sendspace >> $file
sysctl -a | grep net.inet.tcp.recvspace >> $file
[\script]



[ Reply to This | # ]
jumbo frames
Authored by: cynikal on Jun 22, '06 04:12:40PM

i've read that some of the macbooks and possibly other gigabit enabled macs support jumbo frames (ethernet frames up to 9000 or so bytes each, vs the default 1500 bytes).

has anyone tried setting these up ? my powerbook has the option grayed out in the tab, and this cisco 4507 gig blade i'm connected to says it doesn't support setting the mtu:

a4507switch(config-if)#mtu ?
<1500-9198> MTU size in bytes

a4507switch(config-if)#mtu 9198
% Interface GigabitEthernet7/5 does not support user settable mtu.



[ Reply to This | # ]
jumbo frames
Authored by: hieronymus bozo on Jun 22, '06 06:41:19PM

indeed, my macbook 13 2.0ghz does support jumbo frames.

for kicks, i checked a few other machines (all running tiger or tiger server except the xserve g4, which is 10.3.9 server, fwiw) and here's what i saw:

xserve g4 solo 1.8ghz: YES, supports jumbo frames
mac mini g4 solo 1.8ghz: NO, does not
xserve g5 dual 2.5ghz: YES, supports jumbo frames
powermac g5 dual 2.0ghz: NO, does not
powermac g5 dual 2.7ghz NO, does not
powermac g5 quad 2.5ghz YES, supports jumbo frames

what are the implications of enabling jumbo frames? which switches/routers will break? which will thrive, if any, and what will the speed boost be? why does a crappy moon-era g4 xserve support them, yet the newer, technically identical mini ;-) and the second from last g5 released does not?

this functionality would be a good addition to the Mactracker app. i'll see if i suggest getting it added. in the meantime, perhaps others can post additional machines.



[ Reply to This | # ]
jumbo frames
Authored by: cynikal on Jun 27, '06 05:27:16PM

jumbo frames if you were curious, let you send larger amount of data before having to break that stream of data up because the underlying media can only send frames of a certain size.. normally ethernet has a maximum frame size of 1500 bytes, jumbo frames let you get frames up to 9000 bytes in size, thereby decreasing the overhead of having to re-pack the next amount of data in a new packet (inserting into new headers at one end, decoding those headers at the other end). When it comes to higher speed networks, such as gigabit ethernet, this might help squeeze some more performance out. Of course the switch you're connecting to has to support gigabit and jumbo frames (and in theory, it's uplink should also do the same and so on and so forth).

What i'm curious about is, what kinda performance improvement has anyone seen using jumbo frames?



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: mikebremford on Jun 27, '06 02:02:38AM

I tried this a while back and found that while I was getting slightly higher speeds on the LAN, it also affected large transfers to remote machines. We use "scp" to move files around a lot and the percentage complete scale that normally ticks up smoothly would jump up in large blocks, making it impossible to determine how long a transfer would take. Probably not a big deal for most (and it's quite possible I misconfigured something) but it annoyed me enough to put it back.



[ Reply to This | # ]
Tune high-speed networks for faster transfers
Authored by: cj69collins on Oct 01, '06 06:54:06AM

At some point in the past couple of months, I noticed an issue with uploading large binaies via FTP. Downloads were fine. HTTP uploads were OK. ASCII uploads worked. Uploads of binaries via FTP often acted as it they needed to start about 4MB in. It did not cross my mind that this hint may be the culprit. Renaming /etc/sysctl.conf solved my problem.

---
If God wished us to use Windows, he would allow us to come back from the dead...frequently.



[ Reply to This | # ]