lftp

Mirroring sites with lftp

The "mirror" command is all you need to copy/mirror a site with lftp:

$ lftp
lftp :~> connect ftp.domain.tld
lftp ftp.domain.tld:~> login <username>
lftp ftp.domain.tld:~> mirror

Here's a one liner:

$ lftp -e mirror -u <username>,<password&gt; <host>

cleanup ftp backups with bash shell script

Here's a small bash code snippet to clean up ftp backups maintained by date. Specifically, if using the SysBK to backup to an external server with just ftp access.

It should be run daily via cron and cleans up folders older than 14 days.

#!/bin/bash
# clean_bak.sh
# Cleans up old backup folders from the remote server.

USR=<user>
PSWD=<password>
HOST=<ftp.domain.tld><br />BAK_PATH=</path/to/backups>

LFTP=/usr/bin/lftp
RM_DATE=`/bin/date +%m-%d-%y -d '15 days ago'`

$LFTP << EOF
set ftp:ssl-force true
connect $HOST
user $USR $PSWD
rm -r -f $BAK_PATH/${RM_DATE}
du -h -d 1 $BAK_PATH
quit
EOF

exit 0

The du option will output the space currently being used by the backups.

NOTE: For security reasons, you should use ftps protocol to connect to the remote backup server if possible, and can force it via:

set ftp:ssl-force true

This can also be put in the "~/.lftprc" or "~/.lftp/rc" file.

lftp with TLS/SSL

lftp ftp client support tls/ssl so why not use a secure connection for ftp.

$ lftp
lftp :~> set ftp:ssl-force true
lftp :~> connect ftp.domain.tld
lftp ftp.domain.tld:~> login <username>

Insist on your host to serve up ftp with tls/ssl support so all data is secured.

If using proftpd server, tls/ssl can be configured via "/etc/proftpd.conf":

<IfModule mod_tls.c>
    TLSEngine on
    TLSLog /var/log/proftpd/tls.log
    TLSProtocol TLSv1

    # Are clients required to use FTP over TLS when talking to this server?
    TLSRequired off

    # Server's certificate
    TLSRSACertificateFile /etc/pki/tls/proftpd/server.cert.pem
    TLSRSACertificateKeyFile /etc/pki/tls/proftpd/server.key.pem

    # CA the server trusts
    #TLSCACertificateFile /etc/pki/tls/proftpd/root.cert.pem

    # Authenticate clients that want to use FTP over TLS?
    TLSVerifyClient off

    # Allow SSL/TLS renegotiations when the client requests them, but
    # do not force the renegotations.  Some clients do not support
    # SSL/TLS renegotiations; when mod_tls forces a renegotiation, these
    # clients will close the data connection, or there will be a timeout
    # on an idle data connection.
    TLSRenegotiate required off

</IfModule>

The certificate can be generated to be used on the ftp server via:

# cd /etc/pki/tls/proftpd/
# openssl req -new -x509 -days 3650 -nodes -out server.cert.pem -keyout server.key.pem

Reference:

Comment