Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Using named pipes via mkfifo UNIX
Like inline pipes (|), named pipes allow output from one application to be sent to another. Named pipes, however, allow not just stdout redirection, but also redirection of any data output that would be written to disk. I was introduced to named pipes when I was converting a Real media file to MP3, and didn't want to save the uncompressed PCM audio in between conversion while using mplayer and lame.

First, a regular pipe, |, is pretty familiar and lets you do things like this:
$ touch blah
$ echo "blah" | cat > blah
Sure, that's overkill, but sending the output of one program to another can be very usefull.

Named pipes take it a step further by abstracting the concept of the pipe within the command line to a file representation. In my example of mplayer to lame, I would have to convert the Real media (30MB) to a raw PCM file (400MB) and then into MP3 with lame (30MB). The mplayer - lame combination doesn't support the inline pipe, but you can output PCM audio in real time by playing the Real media with mplayer, and lame will accept streaming PCM audio.

The process goes something like this:
  1. Create the pipe: mkfifo aNamedPipe
  2. Start lame with the pipe as input: lame -q 2 -b 128 aNamedPipe output.mp3
  3. start mplayer with the pipe as output: mplayer input.rm -ao pcm -aofile aNamedPipe
  4. Now just wait...
Named pipes are treated like regular files and can be removed with the rm command. The above example streams what would have been a very large file directly to the input of lame; While this example uses audio, named pipes are useful for many other tasks.

I'll end with the interesting exercise of what happens when multiple processes attempt to read the same pipe. Try the following with two terminal sessions, 1$ and 2$:
1$ mkfifo pipe
1$ cat > pipe
1$ tail -f pipe
2$ tail -f pipe
The following links contain additional information that is more detailed:
    •    
  • Currently 2.80 / 5
  You rated: 3 / 5 (10 votes cast)
 
[36,939 views]  

Using named pipes via mkfifo | 10 comments | Create New Account
Click here to return to the 'Using named pipes via mkfifo' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Using named pipes via mkfifo
Authored by: fracai on Oct 27, '04 11:57:07AM
I should note that the last example really needs 3 sessions.
2 running tail -f pipe
and another running cat > pipe
try it a couple times starting and stoping each tail session

sorry for the mistake

---
i am jack's amusing sig file

[ Reply to This | # ]

Using named pipes via mkfifo
Authored by: mzs on Oct 27, '04 01:14:39PM

Named pipes are still pipes and not regular files. For example trying to seek to a particular offset in a named pipe will fail. For that reason, sometimes you can get away with named pipes, and sometimes you can't.



[ Reply to This | # ]
named pipes in the GUI?
Authored by: AJB on Oct 27, '04 05:27:00PM

Last time I checked (haven't got time now), named pipes aren't honoured in the GUI. A couple of years back I had a named pipe that had a perl script on the end waiting for someone to read it. No GUI apps could open it. :-(

I went to the trouble of sending Apple feedback about it. Anyone know if this has changed?



[ Reply to This | # ]
named pipes in the GUI?
Authored by: dantheox on Oct 27, '04 08:16:49PM

UNIX uses named pipes to implement sockets, so sometimes named pipes will show up as FTP servers in the Finder. Other times, I've seen them show up as ordinary files that would completely freeze the Finder if you looked at them funny. For example, by right-clicking on a named pipe, and going to the "Open With" submenu.



[ Reply to This | # ]
Using named pipes to convert SHN to MP3
Authored by: dantheox on Oct 27, '04 08:21:29PM
For whatever reason, the shorten command line program doesn't accept "-" as a valid output file (stdout). This prevents you from piping its output into LAME, and getting a direct conversion from SHN to MP3. To get around the problem, I've been using named pipes. Here's my shn2mp3 script:

#!/bin/sh

for file
do
        # Create a fresh pipe
        if [ -e "$(dirname $0)/stdin2mp3" ]
        then
                rm -f "$(dirname $0)/stdin2mp3"
        fi
        mkfifo "$(dirname $0)/stdin2mp3"

        # shorten won't write to stdout, so use a named pipe ("stdin2mp3")
        lame -h -m s -V 4 "$(dirname $0)/stdin2mp3" "$(dirname "$file")/$(basename "$file" .shn).mp3" &
        shorten -x "$file" "$(dirname $0)/stdin2mp3"

        # Delete the pipe once we're done
        rm -f "$(dirname $0)/stdin2mp3"
done
It's worked fine for me, but if anyone out there knows a better, cleaner, safer way to do this, I'd love to hear it.

[ Reply to This | # ]
Using named pipes to convert SHN to MP3
Authored by: hypert on Oct 31, '04 08:52:39PM
Nice hint - could be really useful, but I can't seem to get shorten to work. Here's the response in my Terminal:

Assuming raw pcm input file
LAME version 3.93  (http://www.mp3dev.org/)
Using polyphase lowpass  filter, transition band: 15826 Hz - 16360 Hz
Encoding /tmp/stdin2mp3 to ./Guthries - Careful Love.mp3
Encoding as 44.1 kHz VBR(q=4) stereo MPEG-1 Layer III (ca. 9.1x) qval=2
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA 
   200/2     (100%)|    0:05/    0:00|    0:08/    0:00|   0.9263x|1193046 h 
 32 [  0] 
 40 [  0] 
 48 [  0] 
 56 [  0] 
 64 [  0] 
 80 [  0] 
 96 [  0] 
112 [  5] %%%%%%
128 [ 47] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
160 [102] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
192 [ 39] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
224 [  7] %%%%%%%%%
256 [  0] 
shorten: No magic number
shorten: for more information use: shorten -h

I installed shorten using "fink install shorten". Where did you get your from? Any idea what magic number might be about? Despite the error response, "shorten -h" doesn't give any info about magic numbers.

Oh, not surprisingly, the resulting MP3 file is a wash of static. :-(

[ Reply to This | # ]

Using named pipes to convert SHN to MP3
Authored by: hypert on Oct 31, '04 10:43:59PM

My mistake! I used the commands above for SHD->MP3 conversion, when I meant to use the RM->MP3 conversion. Shorten was complaining because I wasn't feeding it an SHD file! Now, if I can just compile mplayer with the realaudio codecs, I'll have this working....



[ Reply to This | # ]
Using named pipes to convert SHN to MP3
Authored by: hypert on Nov 02, '04 08:58:56PM

I've spent way too much time in the last 2 days trying to get mplayer to compile for Mac OS X, encouraged by the original example in this thread of being able to convert RealAudio to MP3. (Actually, the version available in Fink compiles fine, but does not support Real codecs. The latest source files fail out of the box with a compilation error since they tries to use a "-rdynamic" option with GCC. Removing that option by hacking the config files gets past compiling, but just causes mplayer to crash later when trying to load Real files.)

I sent a private email to the original hint author (fracai), and he admitted that he based his hint on something he does in Linux.

So, the named pipes is a legitimate concept, but the mplayer conversion of RealAudio files is a bit of a "tease". If anyone else can get mplayer to properly read Real files, I'd love to know!



[ Reply to This | # ]
Using named pipes via mkfifo
Authored by: nvdingo on Nov 01, '04 04:44:06PM

I am trying to convert a non DRM WMA file to mp3 using this method.
i follow theinstructions, substituting where appropriate, and here is what (doesn't) happen.

i make the named pipe
fine no problem
i start lame, so far no problems
i have to go to another terminal window to start mplayer.
as soon as i start mplayer, the mplayer window says broken pipe immediately after outputting 'Starting Playback'
the other terminal window (the one lame was started in)
outputs a bunch of info, and then
Writing LAME tag ... Done.
and quits

Where does the fault lie?
i have never used lame before
and i don't normally use mplayer from the command line

Any ideas?



[ Reply to This | # ]
Using named pipes via mkfifo
Authored by: siah on Mar 01, '09 03:07:38PM

To make your example work, I had to use `cat pipe` in the second terminal instead of `tail -f pipe`



[ Reply to This | # ]