Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!


Click here to return to the 'only if...' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
only if...
Authored by: DeathAndTaxes on Nov 11, '02 04:14:43PM

I'm sure this will only work if you can get a directory listing in your browser...IOW, if you use that same URL and you don't get an index listing, it probably won't work.

Anyway, this is the sort of thing wget excels at.

screen wget -r -l 2 -t 12 -nc --progress=bar --user-agent=Mozilla/4.5 http://whatever.site.you.want.com/the/url/path/

That will leech a single page (or index listing) and will go one level deeper.

BTW, you will probably have to download, compile, and install wget since I don't think it comes with osx standard.



[ Reply to This | # ]
only if...
Authored by: bluehz on Nov 14, '02 08:54:39AM
This works great and if you want a GUI utility use WebDevil - have used it for years. I have to say though - the wget method is MUCH faster. One note. The original code was listed as:
screen wget -r -l 2 -t 12 -nc --progress=bar --user-agent=Mozilla/4.5 http://whatever.site.you.want.com/the/url/path/
whereas in reading the man pages (GREAT INFO! - highly recommended for wget) the -l option (which stands for "level") is written as "-l2" with no space in it and the same with the -t (timeout) option - it should be "-t12". So the code would be:
wget -r -l2 -t12 -nc --progress=bar --user-agent=Mozilla/4.5 http://whatever.site.you.want.com/the/url/path/
I also - personally could not get it to work with "screen" even though I do have screen installed.

[ Reply to This | # ]