rsync

See also sftp under ssh and commands.
See also rdiff -- here's a guide to rdiff-backup
See also grsync -- finally a gui!

Status

  • On Spello, rsync is running as a cron job once an hour, synching /web/ucla to merton (see CronJobs)
  • Paco is running a wajig update and download, ntpdate, a tunnel, and updatedb
Hints
  • rsync produces a dot file in the target directory while it's moving the file -- check it for progress
  • exclude files:
    • --exclude '*wmv'
    • --exclude-from=scripts/rsync-exclude
  • delete files:
    • rsync -a --delete source/ today/
  • include files:
    • this is complicated -- don't use it (disappointing)

9 August 2003 update

Backing up to the novell file server is erratic -- this is still on nssvol -- both in cp and rsync. Copying a file at a time works pretty well, using this syntax:

cp --preserve=timestamps /home/steen/grain/Research/Experiments/LarissaCohen/Cohen/2003-07-30-test.mp3 .

In addition, this rsync job internally on the netware volume works, but may have to be repeated several times to get all the files across:

steen@sigillo:/novell/nssvol/comstud/Users/steen/Experimental-data/Mona_Lisa/Backup$ rsync -aru --size-only ../../../../monalisa/ .

Note the -aur switch -- the a is for archive, which implies preserving ownership, permissions, and timestamp, and the u is for update only. The r is recursive.

1 January 2003 update

After returning from Oslo, I issued this on sigillo (don't forget to try out with n first):

rsync -rutv -e ssh /fuma/Webs/ucla/ spello:/mnt/winD/Webs/ucla/

to sync the changes made back to spello. I had also made some changes on spello, and issueed this to update files that had changed size:

rsync -rutv --size-only -e ssh spello:/mnt/winD/Webs/ucla/ /fuma/Webs/ucla/

The problem about the normal update is that all the files would have been updated because the times were slightly off. There may be a "ignore time differences of less than x minutes".

Then I updated the software directory from the repository on /vc as follows:

rsync -rutv --size-only -e ssh spello:/vc/sigillo/Software/ /fuma/Software/

This also worked fine.

Finally I tried to use rsync to update the material on the Novell file server:

rsync -rutv /mnt/winD/Webs/Panel/ /novell/nssvol/comstud/Users/steen/Webs/Control/

This worked from rsync's perspective, but for some reason user root didn't have the permissions to copy the files into the Novell file system. So a failure on this one for the moment.

Note that you may need to use -avSWz rather than -rutv to get files that have changed time but not name. Not entirely sure about this, but -rutv didn't work and -avSWzn did.

I tried out drsync, which could be helpful for keeping two computers synchronized -- but my needs are pretty simple and should be taken care of with the commands above. The cron job that passes any changes to spello on to merton is working fine.

24 November 2002 update

Here's what I used to synch the new files in winD's grain to merton on 24 November 2002:

root@spello:/merton/grain# rsync -rutv /mnt/winD/Webs/ucla/grain/ .

And this one to synch winD's crp to merton:

rsync -rutv /mnt/winD/Webs/ucla/cogweb/crp/ .

Basically, it looks like you've got it -- you could put this on a cron job and run it once an hour, and never have to think about it. The program you use to set up cron jobs is crontabs; the frontend is kcron. To give the rsynch job the right permissions, you have to start kcron as root and define the job under user root. I've set it to run the whole update once an hour -- possibly excessive, but it's really not a big job to check the files. I checked "silent" to avoid getting tons of e-mails to /var/mail/nobody. This is the command I entered, after testing it -- and I also verfied that it works before I checked the "silent" box:

rsync -rutv /mnt/winD/Webs/ucla/ /mnt/merton/

I can change this later if needed: This was supremely simple (though it required a working knowledge of rsynch), and means I can edit files anywhere and have them update. I could also set up a reverse cron job to move files the other way, in case I edit stuff directly on merton, but for the moment I'll let that go. Finally, it means I can use Amaya -- though that might kill my Dreamweaver templates again. Anyway, it means I can use Dreamweaver in VMware and not worry about the somewhat dubious connection I'm getting.

11 November 2002 update

This is what I used to backup the contents of D: to /vm

root@spello:/vm/backup# rsync -rutv /mnt/winD/ .

This produced a full copy of D: on /vm, as a safety measure during spello partitioning. Most of the material backed up has since been removed from D: -- all of the software and the Linux directory. See about integrating the software with the software on gubbio's D: partition. It could also all be written to a DVD.

July 2002 update

From CASBS:

rsync -rutv /mnt/zip/ucla/ .

This did a perfect job synching files from the zip drive to the hard drive. Note that the -n option will do a dry run, and the -t option preserves the times. Use -v also to see the files being transferred. I tested that this worked for synching the directory after a single file (this one) had been changed, and it worked beautifully.

rsync -rutv . /mnt/zip/ucla/

This synchronizes the other way, from the hard drive to the zip drive. While you are still located inside the /ucla directory under /home/steen/Documents, this command takes all new files on the hard drive and puts them on the zip disk.

rsync -avSWzn -e ssh ucla/ merton:cogweb/

This worked fine from pc-getty to merton and will most likely work fine between gubbio and merton, or spello and merton, or for that matter spello and gubbio. You can set it up as a script or a cron job. Pretty impressive stuff -- and devastatingly simple. Just sort out the effect of all the switches and set it up.

For ideas and instructions on setting up rsync as a cron job, see http://www.linuxworld.com/site-stories/2002/0108.rsync-p2.html. You may be able to get ISO images of Debian with rsync -- try something like rsync -avz --progress --stats rsync://sunsite.dk/ftp/pub/os/ linux/debian-cdimage/2.2_rev4/i386/binary-i386-1.iso

Debian

"Martin Pool, the current upstream maintainer of rsync, collected questions and answers about the integration of rsync in the Debian server network. He compiled them into a document which hopefully reduces similar threads on our mailing lists. It's pretty informal, but hopefully will be useful. Please send comments to mbp@samba.org."

Graphical frontend

There doesn't appear to be an attractive gui for rsync. WINrsync is a cross-platform graphical frontend that may be ok; get it at http://winrsync.sunsite.dk/ (the files are dated late May 2002). See also fastsync.

The "wrapper"

Check out http://hacks.dlux.hu/drsync/ -- it's called a wrapper and is a perl script as far as I can see. For a guide, see http://www.ccp14.ac.uk/ccp14admin/rsync/index.html

Backup script

A great tutorial on how to use rsync to create automatic "snapshot-style" backups. Nothing is required except for a simple script.

Configuration files

Note that the rsync daemon is not switched on on gubbio -- in /etc/inetd.conf:

# the rsync daemon
# rsync stream tcp nowait root /usr/sbin/tcpd /usr/sbin/rsyncd --daemon
#

and this in /etc/rc.config (I guess this is an alternative place to start it):

# Set this to 'yes', if you want to start the rsync server
# as daemon instead via /etc/(x)inetd.conf.
#
START_RSYNCD=no

 

 

top
Debate
Evolution
CogSci

Maintained by Francis F. Steen, Communication Studies, University of California Los Angeles


CogWeb