Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Scripting ftp transfers UNIX
If you often transfer the same file to/from an FTP server (or do anything repetitively with an FTP server), you can use UNIX and a simple script to automate the process. Although similar things are probably possible with grapical clients, this will teach you a bit about UNIX and a very basic shell script.

I use two such scripts on my OS X box. Since my home machine is occasinally out of OS X, that means that my family's OS X hosted website is not accessible. Before I shut down OS X, I upload a "Our server is down" page to my ISP-hosted site. When I return to OS X, I upload a "Our server is up" page. This way, our family and friends can tell easily if the site is up or not. I manage both these tasks with a double-click on an icon in the finder, thanks to these scripts.

This tutorial requires some basic understanding of the command line and an OS X text editor such as vi or pico. Read the rest if you'd like to see how to set up a simple automated FTP script.

To create these script(s), open a terminal and navigate to the directory where you'd like the script to live. For this example, I'm saving it in /Users/robg/Documents. Once in the proper directory, start an editor for the new file. The name and extension aren't that important, so pick one that you like:
vi siteup.xfr
This example script takes an HTML file from the same directory that we're working in, and uploads it to the ISP as the new 'index.html' file.

Enter the following information into the editor; see the comments following the text for and explanation of each line.
#! /bin/sh
echo Uploading the ServerUp page...
# Start of "here" document
ftp <<**
open www.yourISP.com
cd wwwdir
ascii
put /Users/robg/Documents/siteup.html index.html
bye
**
# End of "here" document
echo FTP ended - your web site is now marked as UP.
Looking line by line, here's what the script does.

#! /bin/sh
This is a shell script (as opposed to a perl script or some other type).

echo Uploading the ServerUp page...
This message will display in the terminal window.

# Start of "here"document
A "here" document is what makes this script possible. Within the here document, everything between ftp Launch FTP and tell the script to treat the following section as "here" text.

A series of FTP commands
Customize these to match your FTP site. I tell FTP to open the connection, switch to the right directory, set ASCII mode, send the "server is up" HTML page, and disconnect.

**
# End of "here" document

The end of the "here" section.

echo FTP ended - your web site is now marked as UP.
Another statement that will show up in the terminal window.

That's all there is to the script. Save the file, and exit the editor. To actually make this work, however, there are two more steps. First, you'll need to make your script executable:
chmod ugo+x siteup.xfr
This adds execute permissions to the file. The next (and last) step is the creation of a file called .netrc in the top level of your home directory. Follow these steps to create the file:
cd ~
vi .netrc
Inside the editor, insert the following lines:
machine www.yourISP.com
login yourUNAME
password yourPWORD
Obviously, replace 'yourISP', 'yourUNAME', and 'yourPWORD' with your ISP, username, and password. Save the file and quit the editor.

The .netrc file has your password in it, in plaintext. This is somewhat of a security hole, so you should immediately change the permissions on the file so that only you can see it:
chmod go-rwx .netrc
The .netrc file is needed because the FTP program will not (for security reasons, I believe) accept username and password within the context of a "here" document. Using this as a model, you can see that a number of variations are possible, for both uploading and downloading files.

Once you have the scripts, you can simply type
./siteup.xfr
into a terminal session (assuming you're in the directory that holds the script, of course), and the program will run. Alternatively, click on the file in the GUI, open an inspector window, choose the Application drop-down, click on the "A Specific Application" button, then click on the "All Applications" drop down, navigate to the Terminal program in the /Applications/Utilities folder, and select it. Close the inspector, and you can now double-click the script in the finder.
    •    
  • Currently 3.00 / 5
  You rated: 3 / 5 (4 votes cast)
 
[16,177 views]  

Scripting ftp transfers | 5 comments | Create New Account
Click here to return to the 'Scripting ftp transfers' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
A way to add username and password to script
Authored by: tcurtin on Feb 05, '03 03:46:43PM

You can actually have a script run ftp without needing a .netrc.


#!/bin/csh

ftp -nv your.ftp.site << EOF
user username password

ftpcommands
END


The key is adding the -n to the ftp command. -n tells ftp not to try to log in immediately, and lets you run the user command with username and password to do the actual login. I added the v as well to see verbose messages - you can include these in a logfile, email, etc. (Usually a good idea with a script that runs automatically - when things go wrong as they eventually probably will, you want to know about it.)

The same warning as before applies, since this file will have your password in cleartext.

Enjoy!
-Tim



[ Reply to This | # ]
A way to add username and password to script
Authored by: Mara on Apr 03, '03 08:19:35AM

Both great tips.

Now, sorry for asking a stupid question, but what if I want to upload a whole directory instead of just one file? I tried msend, but I don't get it. I need a way to script upload of the contents of a whole directory, as a form of updating the website.

Any help will be greatly appreciated.



[ Reply to This | # ]
A way to add username and password to script
Authored by: LC on Mar 05, '04 04:39:21PM
If you mean a flat directory (no hierarchy within), then you can just issue the prompt directive to turn off interactive mode, and then use mput * to transfer all files (remember to be in binary mode unless you know they're all text;)
For a directory containing hierarchy, I know when downloading that some ftp servers support "archive mode" where they will archive on-the-fly if you request a directory name using an archive suffix. But I don't know how to tell an ftp server to unpack an archive file after uploading, nor how to upload hierarchically (unless you explicitly do mkdir etc.). rdist, rsync, scp -pr, rcp -pr work though, assuming you have remote login rights;

[ Reply to This | # ]
A way to add username and password to script
Authored by: LC on Mar 05, '04 04:46:39PM
I guess wu-ftpd handles on-the-fly archiving via the ftpconversions definitions; Larry.

[ Reply to This | # ]
A way to add username and password to script
Authored by: LC on Mar 05, '04 04:56:59PM
For uploading, check this out too -- (curl does uploading;)
              In curl 7.10.8 and later, you can specify one -T for each URL on
              the command line. Each -T + URL pair specifies  what  to  upload
              and  to where. curl also supports "globbing" of the -T argument,
              meaning that you can upload multiple files to a  single  URL  by
              using  the  same  URL  globbing style supported in the URL, like
              this:
                curl -T "{file1,file2}" http://www.uploadtothissite.com
              or even
                curl -T "img[1-1000].png" ftp://ftp.picturemania.com/upload/
Larry.

[ Reply to This | # ]