If you are looking for the epic motorcycle journey blog that I've written, please see the Miles By Motorcycle site I put together. 
  • Ian
    File Transfers without SSH
    11/16/2016 9:03PM

    I had an interesting problem at work related to copying files from one Linux server to another.  The problem was this: I had login access to both servers and on each server I had sudo access to a service account, which does not have login access.  The files that I had to copy were owned by the service account and many of these were not readable by my own account.  Normally this wouldn't be a problem - I would sudo to the service account, tar up the files, copy that over using my login account, then untar it on the other side using the service account.  However, there were many gigs of data and not enough disk space on the server to create a tar file, even compressed.

    Netcat to the rescue!  On the receiving end, I started a netcat listener using the service account, piping the input to the tar extract command.

    nc -l 9999 | tar -xvf -

     On the sending side, I piped the output of tar's create command to the listener on the other server.

    tar c dir_to_copy | nc <receiver_ip_address> 9999

    On the receiving side, because of the 'v' (verbose) option to tar, the file and directory names go flying by as the data gets transferred.  When the transfer is complete, nc closes automatically on the receiving side.

    So that worked for the initial transfer, but I also wanted to be able to do an rsync-style update later to get any files that may have been modified, and not have to re-copy everything.  Using a slightly trickier version of the commands above, I restarted the listener on the receiver and ran the following on the sender:

    find dir_to_copy -mtime 0 -type f | tar c -T - | nc <receiver_ip_address> 9999

    I've also seen it recommended to use -print0 with the find and then --null in the tar command.

    find dir_to_copy -mtime 0 -type f -print0 | tar c --null -T - | nc <receiver_ip_address> 9999

    which is what I actually ran yesterday, but it seems to work both ways.  -print0 prints the full path of the files, but without a newline character at the end, making it look like one long string.  The --null option to tar tells it to look for this delimiter.


You must be logged into an account to post comments.