Backing Up Delicious With Cron and Subversion

August 10, 2005 at 11:27 am · Filed under Geek Life

Deli.cio.us has fundamentally changed the way I browse the web. Pre-delicious, if I found a web site I wanted to come back to later, I'd add it to my Favorites in an appropriate sub-folder, and then proceed never to find the link again. Favorites as folders and links just doesn't work for me. Its too hard to find things even when I keep them decently organized. That's where delicious comes in - its an online replacement for Favorites/Bookmarks that not only stores your links online but lets you "tag" them so you can find them again by keyword/topics/meme/whatever. Not only that, but you can see how other people have tagged links and find related sites. I have just a couple of  sites on my links toolbar now, where before I had hundreds.

I'm so addicted to delicious that I would be in "a world of hurt" if the site ever disappeared. So, I backup my data everyday. Here's how I do it.

My main development box runs Windows XP Pro but I also have a local ubuntu linux server that I use for file sharing, backups, development web servers, and version control using subversion. My subversion repository get backed up nightly and stored offsite weekly, so its a natural place to backup my delicious links. Plus I get the added bonus of being able to recover my links for any date in time and do diffs/comparisons (yes - that's incredibly geeky).

Delicious has an XML api that lets you download your links as a single xml file. You can use wget to save all your links to a local file like this:

wget http://[username]:[password]@del.icio.us/api/posts/all -O [local_file]

To set this up, I created a project in subversion called delicious (no trunk, branches, tags subdirs necessary), and imported the delicious.xml file.

On sidebar (my ubuntu box) I created this script in my home directory. It creates a temporary working directory, checks out the delicious project with its one delicious.xml file, grabs the new delicious xml file, commits it to subversion, and then cleans up the temp directory.

# make our temporary work directory
mkdir dtmp

# check out delicious to temp work directory
svn co svn://sidebar/stilesoft/delicious dtmp

# download our backup file
wget http://[username]:[password]@del.icio.us/api/posts/all -O dtmp/delicious.xml

# move into temp and checkin new file
cd dtmp
svn commit -m "Daily backup"

# clean up temp work directory
cd ..
rm -Rf dtmp

The script could have been simpler if we left a permanent working directory on the box and just committed the file each day, but this keeps things cleaner.

Make sure the script is executable (chmod +x backup_delicious) and then setup crontab to run it every night at 3 AM.

$ crontab -e

And add this line, substituting your home directory, then save.

0 3 * * * /home/adam/backup_delicious

Technorati tags: , ,

2 Comments

  1. Joe Grossberg said,

    August 11, 2005 @ 2:00 pm

    I did a similar thing myself, after seeing del.icio.us go down too often and getting paranoid.

    I use CVS, but close enough. (Yes, I know subversion is better; it’s on my to-do list.)

    And, FWIW I do:

    wget -q –http-user=[username] –http-passwd=[password] http://del.icio.us/api/posts/all -O /home/joegross/misc/scripts/delicious_backup.xml

    instead of:

    wget http://username:password@del.icio.us/api/posts/all -O [local_file]

    Is there any reason (not) to use the user:pass@url syntax?

  2. Adam Stiles said,

    August 12, 2005 @ 7:15 am

    Joe - I’m not sure it makes any difference which syntax you use. Neither protects your data from network sniffers.

RSS feed for comments on this post