Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Script to list all filesystem objects with ACLs System 10.5
Access Control Lists (ACLs) are active by default in Leopard. The OS checks ACLs before it checks the standard POSIX (rwxrwxrwx) permission bits. That means it's possible to have what appears to be a secure file or directory, but a bad ACL could allow rogue processes or rogue users to gain access.

This isn't just a problem of user misconfiguration. In the last three years, I've submitted vulnerability reports to four major software vendors, because their OS X installer packages altered the POSIX permission settings (and sometimes ownership) on some important files or directories. If an installer package adds or alters an ACL (to your tax records, for example), it might go unnoticed (but could make you quite unhappy when you found out).

Unfortunately, there aren't any good tools to find ACLs, or to verify that they haven't been altered. Apple's Verify Permissions feature in Disk Utility has a history of bugs, and it's quite slow. Because Disk Utility is a closed implementation, it isn't even clear that it checks ALL filesystem objects (it appears to only check what Apple wants to check). And although it reports that ACLs exist, it isn't clear that it checks all the contents of the ACL metadata.

In an effort to quantify how many ACLs are lurking on my new Leopard install, I wrote the following zsh script. I realize I could have thrown together an actual app, or a Python/Ruby/etc script, but I prefer to prototype things like this in the shell, particularly because the shell is actually quite good at things like stream editing.

My script found over 800 objects with ACLs (out of just over 1M total filesystem objects).

I've asked the guys at the Open Source Tripwire project to add ACL metadata to the Tripwire tests. In the meantime, I may disable ACL support on my system, since it's overkill for my single-user operations, and the risk far outweighs the value to me.

About the script:

There are control characters in the script, but they show below as ^A and ^B, instead of A and B. If you're going to use this script, you should replace them. If you're using vi, you do this by typing VA and VB in place of ^A and ^B. The vi commands would be:

:%s/^A/<ctrl>v<ctrl>a/g
:%s/^B/<ctrl>v<ctrl>b/g
It is also important to leave the newlines after the line that begins:
cat ~/acls/everything_$dt | tr '"
The newlines are in the middle of single quotes, and the single quotes are there to enclose the newline character as part of the command.

You'll need to create a directory, ~/acls , to house the files this script creates, and you'll need to run the script with sudo (if you want to traverse the whole filesystem). If you only want to look at a subset of your filesystem, change the find command (from find / to find ). You can also use the -maxdepth flag for the find command, to limit how deep it searches in subdirectories. The script produces several files, tagged with the date and time:

~/acls/findacl_timer_<date and time>
This file is a summary of how long it took (3 hrs on my system).

~/acls/everything_<date and time>
The initial list of all filesystem objects, in case you want to refer to it.
 
~/acls//all_acl_objects_<date and time>.csv
A list of all the objects with ACLs, ready to be opened in OpenOffice, delimited by ^B instead of commas. (^B and ^A were chosen because they are not likely to be in a legitimate filename; commas do appear in some OS X filenames.)

If you do open the .csv file with OpenOffice, you'll need to change the import separator to ^B. To do that, you de-select the box for Comma, select Other, and use the OS X Character Palette (Code Tables, Other Encodings, 0002, Insert) to insert ASCII code 0002.

I recognize this could be done more elegantly and more efficiently. It's a quick and dirty analysis tool. Please suggest alternatives, if you know of them; I didn't find any clean way to get this data (it would have been great if there was an ACL extension to the find command, but there isn't).
 
#!/bin/zsh

if [ $EUID -ne 0 ]; then
        echo YOU MUST SUDO TO ROOT TO RUN THIS
else
        dt=`date "+%Y%m%d_%H%M%S"`
        echo Start:     `date` > ~/acls/findacl_timer_$dt
#       Traverse entire volume, printing ACL data (-e)
#       Put list in everything_$dt
        find / -exec ls -alde {} ; 2>/dev/null > ~/acls/everything_$dt
        echo Find done: `date` >> ~/acls/findacl_timer_$dt
#       Search list of everything as follows:
#       - convert all newlines to ^A (this is a way to find <newline><space>, which marks an ACL entry)
#       - convert all occurrences of ^A<space> to ^B (this removes the former newline and merges the ACL entry onto the previous line)
#       - convert all ^A back to newline 
#       Search the result for ACL entries only (a ^B, followed by the ACL number)
        cat ~/acls/everything_$dt | tr '
' '^A' | sed -e 's/^A /^B/g' | tr '^A' '
' | grep '^B[0-9*]:' | sed -e 's/ /^B/' -e 's# /#^B/#' > ~/acls/all_acl_objects_$dt.csv
        echo Done:      `date` >> ~/acls/findacl_timer_$dt
fi
    •    
  • Currently 2.38 / 5
  You rated: 5 / 5 (8 votes cast)
 
[13,389 views]  

Script to list all filesystem objects with ACLs | 21 comments | Create New Account
Click here to return to the 'Script to list all filesystem objects with ACLs' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Script to list all filesystem objects with ACLs
Authored by: Coumerelli on Aug 19, '08 09:15:22AM

The hint category is 10.5. Won't this also work on (and be good for) 10.4.x?

---
"The best way to accelerate a PC is 9.8 m/s2"



[ Reply to This | # ]
What is Repair Permissions doing?
Authored by: jscotta on Aug 19, '08 09:33:27AM

So…if this is an issue, what exactly is the "Repair Permissions" function of the Disk Utility doing? Also, is there another tool that we can use to look at the ACL's?

---
Windows because I have to. OS X because I want to.



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: dasmart on Aug 19, '08 10:28:19AM

I had to change this:

find / -exec ls -alde {} ; 2>/dev/null > ~/acls/everything_$dt

to this.

find / -exec ls -alde {} + 2>/dev/null > ~/acls/everything_$dt

otherwise i kept getting this error:

find: -exec: no terminating ";" or "+"



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: xr4ti on Aug 19, '08 11:22:21AM
The comment is correct. The backslash got dropped (I assume it disappeared in the hint posting process). The "find" line should read:

find / -exec ls -alde {} \; 2>/dev/null > ~/acls/everything_$dt

[ Reply to This | # ]

Script to list all filesystem objects with ACLs
Authored by: kirkmc on Aug 19, '08 11:56:42AM

My bad. I forgot that there's an issue with backslashes in Geeklog. I've corrected the code in the post.

---
Read my blog: Kirkville -- http://www.mcelhearn.com
Musings, Opinion and Miscellanea, on Macs, iPods and more



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: koyeung on Aug 19, '08 01:13:59PM

it should be faster to exec with \+ (instead of \;):
"This behaviour is similar to that of xargs(1)."



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: lrivers on Aug 19, '08 04:58:30PM

I'm getting "tr: Illegal byte sequence" in the terminal at the end of the run. I get a huge text file and an essentially empty csv (and the timer document).

I copied the code for the script from the page and made sure all the slashes were right and so on...



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: xr4ti on Aug 22, '08 07:08:21PM

It is faster to use '+' (much faster, as it turns out).

I avoided '+' for two reasons: 1) I wasn't sure how much more memory the '+' approach takes on a large job like this, or even if it might overrun a buffer. (It does spawn multiple processes, so it must have some decent buffer management.) 2) It produces an output that is padded with spaces, which will make the output significantly larger for a large job like this.

I've since done some testing, and found that the '+' only takes about 4* the RAM during processing (not a big deal in today's world), and seems to avoid creating commands that are too large, so it's probably the better choice (if you don't mind a resulting file that is much bigger).

The two commands did give me slightly different results. With over 1M entries to slog through, I haven't yet found the differences. But the savings in time is probably worth the larger file, so I recommend changing the ';' to a '+'.



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: xr4ti on Aug 22, '08 07:18:06PM

It sounds like something wrong with the text inside the quotes after the "tr" commands. If you're sure you changed the ^A and ^B text to <ctrl>A and <ctrl>B, it's possible it's related to your locale settings or some default in your shell.

Try experimenting with a simpler command from Terminal, like:

ls -al | tr -s '
' '<ctrl>v<ctrl>A'

and see if you get an error.



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: guns on Aug 19, '08 10:40:12AM
The hint category is 10.5. Won't this also work on (and be good for) 10.4.x?

I think I remember that ACLs were turned off on the main volume by default.

So…if this is an issue, what exactly is the "Repair Permissions" function of the Disk Utility doing? Also, is there another tool that we can use to look at the ACL's?

"Repair permissions" is just trying to restore POSIX permissions on Apple System and Application files. Like the OP mentioned, it doesn't work all that well. For instance, /var/log/secure.log should have permssions of 0600, but on my machine they are set to 0640 for some reason, a security risk. I ran disk utility which caught this error, exited with success, and never actually fixed this mistake. Furthermore, I don't think it corrects ACL errors. As far as ACL tools, Apple's ls comes with the '-e' option, which will print all ACLs on files, and Apple's chmod lets you set, edit, and delete ACLs.

--

Great post. I know that lots of vendors are lax about POSIX permissions, but I didn't know of any that added ACLs. You didn't imply that they did necessarily, but do you know of any that do? That's pretty worrying, because clearly it's possible.

Also, you may want to set the output directory to /tmp or something else that's sure to exist, or else include a 'mkdir ~/acls'.



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: xr4ti on Aug 19, '08 11:31:08AM
A great suggestion (I must have been a little too sleepy when I finished the script).

Immediately after the "else", you can add the following line:

mkdir -p ~/acls

That will create the directory, if necessary, and be silent if it's already there.

[ Reply to This | # ]

Script to list all filesystem objects with ACLs
Authored by: xr4ti on Aug 19, '08 11:37:45AM

And, on the question about vendors:

no, I don't know of a vendor that has intentionally or mistakenly set ACLs improperly. Since I've just start tracking ACLs closely, I can't be sure that the few third-party installers I've already run have altered or added ACLs to my system. I use Tripwire religiously, and it's what caught past POSIX tampering by some major vendors.

I can only say that the current crop of 800+ ACLs on my system look like they were either done by Apple, or done by Get Info before I realized how much damage Get Info can do.



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: saklad on Aug 19, '08 03:18:28PM

MacPilot has a function to recursively wipe all ACL data among many other functions. I have assumed that that will take care of the security problem. Am I right? BTW: I have nothing to do with MacPilot other than buying it to fix an ACL problem some time back.

Can someone explain when a single user Mac would want to use the ACLs?



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: xr4ti on Aug 19, '08 04:00:20PM

Wiping out existing ACLs won't prevent a rogue or badly written installer (or program with elevated permissions, for that matter) from adding new ACLs. (I've also read some warnings about wiping out all ACLs with MacPilot, I don't think the author of MacPilot intended for it to be used that way.)

As to using ACLs: I compartmentalize my data - I have different user accounts for different purposes. If I really needed these user accounts to share data in complex ways, I could go beyond the POSIX notion of shared groups and use ACLs to define read, write, inheritance, ability to change permissions, etc.

I have a fairly complex single-user system, but even for my setup, I simply don't see any reason to use ACLs.

But that's just my one opinion. I'm open to someone else saying they have a strong need for ACLs.



[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: sjk on Aug 21, '08 02:10:12PM
I have a fairly complex single-user system, but even for my setup, I simply don't see any reason to use ACLs.
Same here.

However long it takes, UNIX/POSIX and ACL file-based security will eventually be a deprecated legacy. Original UNIX file permissions, essentially unchanged except for adding an ACL layer, certainly weren't designed or intended to scale to the huge numbers of files on many current filesystems.

[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: mmnw on Aug 20, '08 12:03:31AM

Are there any reports of ACLs gone wrong? I mean, is there any additional risk beside the mentioned misconfigured POSIX permissions. If there isn't, I don't see the benefit of deactivating ACLs. Especially, since most apps only modify the POSIX permissions (I guess most developers don't even know about ACLs). The only app I know of that modifies ACLs is the Finder...

I tested the script and it revealed way more than 1000 files with ACLs. Most of them somewhere in my User Folder. I guess it is because I have many network shares, which I change quite often.

There's another point with ACLs which should be mentioned in this context. You can actually share files with people that do not have a user account on your mac (see the permission rights in the Get Info window). Actually you can share with anybody in your address book. In my understandig, this is stored in the ACLs, since this is not a standard POSIX feature. So, meddling with ACLs can break your shares.



[ Reply to This | # ]
Re: Script to list all filesystem objects with ACLs
Authored by: müzso on Aug 20, '08 03:26:43AM

The various control characters + newlines in the script make it quite sensitive (easily broken) and a bit difficult to reproduce. I've created an awk variant that pretty much does the same, but it's a lot more resistant to the usual copy&paste errors.

Here's the script (put it into something like "find_acls.sh"):


#!/bin/sh

if [ $(id -u) -ne 0 ]; then
  echo 'Tip: you might want to run this script as the "root" user (eg. via "sudo").'
fi
if [ -n "$1" ]; then
  target=$1
else
  target=/
fi
echo "Started: $(date)"
find -x "${target}" -exec ls -alde '{}' \; 2> /dev/null | awk '{if(match($0,"^ [0-9]+:")>0){if(prevline!=""){print prevline;prevline=""}print $0}else{prevline=$0}}'
echo "Finished: $(date)"

You can execute it as it is or supply a path as parameter if you want to list ACLs only in a directory. On my MacBook Pro it ran for 45-48 minutes. The "find + ls" takes the most of this, the awk script runs only in 7 seconds or so (I mean if you separate the two and do not use a pipe).

Btw. here're two nice primers on ACLS (in case you wonder what an ACL is and how to use, enable or disable it, etc.):



[ Reply to This | # ]
Re: Script to list all filesystem objects with ACLs
Authored by: müzso on Aug 20, '08 04:24:06AM
There's a case where both the original script and my awk variant "fail": if a file or directory name contains a newline and after that a space, a number and a colon. You can create such a file with a command like this:
touch "test file
 0: something.txt"
However I consider this a minor flaw and it'd be quite inefficient if one would try to write a script that considers even this special case. You'd have to create a complete file listing with "find" (using the "-print0" option to have the zero byte as a separator) and query the ACLs one by one for each file and process the output accordingly.

[ Reply to This | # ]
Re: Script to list all filesystem objects with ACLs
Authored by: xr4ti on Aug 22, '08 03:27:49PM

I haven't been a big awk fan over the years, but awk may be a better choice in this case, since it can easily access the previous line.

But this script lacks the databasing features of my original posting. In particular, this script puts each ACL entry on a new line (just like the original ls -ale printout), rather than putting them together as a single record for a spreadsheet (as in the original posting). Since ACL entries can be on multiple lines, the use of prevline in this script would have to be manipulated, and the print would have to avoid inserting newlines until the entire ACL entry was put together.

Also, I prefer to have the output from the 'find' go to a file, and run the awk/sed on the resulting file, rather than running it through a pipe. If something goes wrong before this script ends, you end up with nothing. But if you have the find output in a file, you can see if there is a file with an aberrant name or some other anomaly that caused the awk to fail.

The original posting wasn't intended for the faint of heart in the script world, but it looks like some people have managed to copy it and get it working. I built the original to put the data in a spreadsheet; there is probably a way to do that with awk, but for the time being, I'll settle with the original. Perhaps someone can work on an awk version that produces the same sort of output. (No offense and thanks for the submission!)



[ Reply to This | # ]
Re: Script to list all filesystem objects with ACLs
Authored by: müzso on Aug 24, '08 01:39:06PM

You're right: my previous script did only filter the output of find+ls and did not produce a CSV like structure.

Just for the sake of practise, I've created a pure sed variant that really creates CSV output:


#!/bin/sh

tempfile=
if [ -n "$1" ]; then
  target=$1
else
  target=/
fi

if [ -d "${target}" ]; then
  if [ ! -x "${target}" ]; then
    echo "You have no read access to the specified directory."
    exit 1
  else
    tempfilename=find_acls_csv_$(dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" ").dat
    if [ -d "${TMPDIR}" -a -r "${TMPDIR}" -a -w "${TMPDIR}" ]; then
      tempdir=${TMPDIR}
    else
      if [ -d "${TMP}" -a -r "${TMP}" -a -w "${TMP}" ]; then
        tempdir=${TMP}
      else
        if [ -d "/tmp" -a -r "/tmp" -a -w "/tmp" ]; then
          tempdir=/tmp
        else
          curdir=$(pwd)
          if [ -d "${curdir}" -a -r "${curdir}" -a -w "${curdir}" ]; then
            tempdir=${curdir}
          else
            echo 'Could not find a readable and writable temporary directory.'
            exit 5
          fi
        fi
      fi
    fi
    tempfile=${tempdir}/${tempfilename}
    if touch "${tempfile}"; then
      find "${target}" -exec ls -alde '{}' \; > "${tempfile}" 2> /dev/null
      target=${tempfile}
    else
      echo "Could not write to temporary file at ${tempfile}."
      exit 4
    fi
  fi
else
  if [ -f "${target}" ]; then
    if [ ! -r "${target}" ]; then
      echo "You have no read access to the specified file."
      exit 2
    fi
  else
    echo "The specified target path does not exist."
    exit 3
  fi
fi

if [ -n "${tempfile}" -a $(id -u) -ne 0 ]; then
  echo '"Tip: you might want to run this script as the root user (eg. via sudo)."'
fi
echo '"mode","# of links","owner","group","bytes","month","day","time","path","acls"'
cat "${target}" | sed -Ee '
:a
s#^([^/ ]+) +#\1\#*\##
ta
:b
$! {
N
bb
}
s/[[:cntrl:]]*(\n)/\1/g
:c
s#(\n[^/ ]+) +#\1\#*\##g
tc
s#\n ([0-9]+:[^[:cntrl:]]*)#!*!\1#g
s#(\n)([^[:cntrl:]]*!\*!)#\1!\2#g
s#\n[^!][^[:cntrl:]]*##g
s#\n!#\
#g
s#^([^[:cntrl:]]*!\*!)#!\1#g
s#^[^!][^[:cntrl:]]*\n##g
s#^!##g
s#((^|\n)[^![:cntrl:]]+)!\*!#\1\#*\##g
s#"#""#g
s#(.+)#"\1"#
s#\#\*\##","#g
s#\n#"&"#g
s#!\*!#\
#g
'
if [ -n "${tempfile}" ]; then
  rm "${tempfile}"
fi

It seems that the sed in Darwin is not fully following the description of it's manpages. :-( Unfortunately this script turned out to be really slow. It processed my 130MB file listing in several minutes (4-5min).

I've created an Awk script that creates a CSV output as well:


#!/bin/sh

tempfile=
if [ -n "$1" ]; then
  target=$1
else
  target=/
fi

if [ -d "${target}" ]; then
  if [ ! -x "${target}" ]; then
    echo "You have no read access to the specified directory."
    exit 1
  else
    tempfilename=find_acls_csv_$(dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" ").dat
    if [ -d "${TMPDIR}" -a -r "${TMPDIR}" -a -w "${TMPDIR}" ]; then
      tempdir=${TMPDIR}
    else
      if [ -d "${TMP}" -a -r "${TMP}" -a -w "${TMP}" ]; then
        tempdir=${TMP}
      else
        if [ -d "/tmp" -a -r "/tmp" -a -w "/tmp" ]; then
          tempdir=/tmp
        else
          curdir=$(pwd)
          if [ -d "${curdir}" -a -r "${curdir}" -a -w "${curdir}" ]; then
            tempdir=${curdir}
          else
            echo 'Could not find a readable and writable temporary directory.'
            exit 5
          fi
        fi
      fi
    fi
    tempfile=${tempdir}/${tempfilename}
    if touch "${tempfile}"; then
      find "${target}" -exec ls -alde '{}' \; > "${tempfile}" 2> /dev/null
      target=${tempfile}
    else
      echo "Could not write to temporary file at ${tempfile}."
      exit 4
    fi
  fi
else
  if [ -f "${target}" ]; then
    if [ ! -r "${target}" ]; then
      echo "You have no read access to the specified file."
      exit 2
    fi
  else
    echo "The specified target path does not exist."
    exit 3
  fi
fi

if [ -n "${tempfile}" -a $(id -u) -ne 0 ]; then 
  echo '"Tip: you might want to run this script as the root user (eg. via sudo)."'
fi
echo '"mode","# of links","owner","group","bytes","month","day","time","path","acls"'
awk '
{
  if (match($0, "^ [0-9]+:") > 0) {
    acls++
    gsub("^ +", "", $0);
    if (acls > 1) {
      buffer = buffer "\n" $0
    }
    else {
      buffer = buffer ",\"" $0
    }
  }
  else {
    if (buffer != "" && acls > 0) {
      print buffer "\""
    }
    acls = 0
    buffer = ""
    for (i = 1; i <= 8; i++) {
      gsub("\"", "\"\"", $i)
      buffer = buffer "\"" $i "\","
    }
    path = substr($0, index($0, $8) + 5)
    sub("^ +", "", path)
    gsub("\"", "\"\"", path)
    buffer = buffer "\"" path "\""
  }
}' "${target}"

if [ -n "${tempfile}" ]; then
  rm "${tempfile}"
fi
This one processed the 130MB file listing in 20 seconds. The output is a comma separated file where multiple ACLs are kept in a single field (spreadsheet cell).

[ Reply to This | # ]
Script to list all filesystem objects with ACLs
Authored by: xr4ti on Aug 22, '08 07:30:42PM

I just noticed another error in the original post, it must have been created by the blog system when it posted it.

The original post says:

"There are control characters in the script, but they show below as ^A and ^B, instead of A and B. If you're going to use this script, you should replace them. If you're using vi, you do this by typing VA and VB in place of ^A and ^B."

But it should actually say:

"There are control characters in the script, but they show below as ^A and ^B, instead of <ctrl>A and <ctrl>B. If you're going to use this script, you should replace them. If you're using vi, you do this by typing <ctrl>V<ctrl>A and <ctrl>V<ctrl>B in place of ^A and ^B."

Note that the vi commands that follow the paragraph are correct (that must be a function of how the blog system presents code).



[ Reply to This | # ]