Automaticaly publishing a website

Most of the Web hosting companies provide a FTP to publish the set of files that form the website. FTP is a really good way of transferring files between two hosts: it's simple and fast. Unfortunately that simplicity is also it Achilles heel when it comes to publishing a web site you're creating off line. The problem is that you would have to keep track of the file changes, transfer the changed files and remove any file that was deleted locally. Too much trouble for me.

The solution, or at least a solution, is a based on two concepts:

• mounting the remote ftp site in the local host;
• syncing the locally created Web site to the remote mounted file system.

To mount the remote file system I use curlftpfs, a filesystem based on FUSE which implements a user level file system over FTP.

After having the remote filesystem mouted in the local host it is easy to synchronise it using the good old rsync although I had to tell rsync to ignore errors that show up when it tries to change the time stamp of the remote files.

As I like to automate everything I created a Makefile with the following targets:

    1: mount:
    2:  curlftpfs -o tlsv1 -o user=USER FTP_HOST LOCAL_MOUNT_DIR
    3: 
    4: umount:
    5:  fusermount -u LOCAL_MOUNT_DIR
    6: 
    7: publish: ~/mnt/bitpipeline/web
    8:  rsync -recursive -verbose --progress --delete-during --ignore-errors LOCAL_WEB_DIR LOCAL_MOUNT_DIR/REMOTE_WEB_DIR

Where: USER : is the ftp user name

FTP_HOST : is the remote ftp address to use

LOCAL_MOUNT_DIR : is the directory where the remote ftp site will be mounted

LOCAL_WEB_DIR : is the directory were the Web site is built (in the local host)

REMOTE_WEB_DIR : is the remote directory that is root of the Web site file system

It might happen that your host does not support TLS, which is bad: your password will be travelling in the Net without any protection from preying eyes. If you still want to use this scheme than remove the option -o tlsv1 from the curlftpfs command.

Then the workflow would be:

1. run "make mount". You'll be asked the password for the user.
2. develop the web site (I use ikiwiki to generate the site).
3. run "make publish". This might take some time…
4. if you have more work to do go to step 2.
5. run "make umount".

The last step might fail if any process has any file descriptor open to the remote file system. Find which process (using lsof for example) and have it close that file descriptor (this might mean terminating the process). Or you can add the "-z" to the fusermount command. That will make it do a "lazy umount", meaning that the file system will be invisible to all processes except those that have a file descriptor open to it and as soon as all file descriptors are closed the file system will be properly unmounted.

To have the needed tools in my development machine (using gentoo) I had to unmask net-fs/curlftpfs.