Robert's blog
Robert Važan

Fastest way to upload files to a web server

Something as simple as uploading files to a web server can turn out to be unnecessarily complicated when you consider your complete workflow.

Time is money. While uploading files via FTP tunneled through a VPN from a restricted set of clients is super secure, it doesn't do any good to my ability to get my huge backlog of stuff published. I need something faster.

The VPN is there mostly to allow me to run untrusted software like WordPress. Nevertheless, I am leaning away from the idea of running crapware on my server since bolting on the security takes way too much time.

I can just drop the VPN completely in order to make my server accessible from both of my personal computers and from my work computer without any extra configurations. This change alone will save lots of time, but it's not nearly enough.

FTP is annoying for several reasons. I am currently mirroring a folder by hand from one of my computers to the server. That makes it impossible to upload from other computers. FTP client doesn't provide me with any easy way to get the URL of the uploaded file for easy sharing.

One thing I should really do is to reverse the copy process. Upload from any computer to the server. Then use backup software to download the changes to my primary computer for backup purposes. That will remove the bureaucracy of manually maintaining the backup.

A few words about hosted services. I don't want anything on hosted services. They are crazy expensive and they often make it very hard to use custom domain and certainly very hard to switch providers.

There are various scripts around, but firstly, I am not a big fan of the PHP dinosaur, and secondly, these scripts require some non-trivial configuration to get working for a number of subdomains.

WebDAV is built into the web server, which I hope will provide seamless experience. There are two alternatives - AtomPub and CMIS - but I don't see any advantage to using them instead of WebDAV.

One problem I have with WebDAV is authentication. How to authenticate only write access while allowing all read access? I could have two URLs, one WebDAV and one public, but that again makes link sharing a nuisance with lots of accidentally posted WebDAV URLs.

This is unfortunately the only solution in case of scripts since WebDAV provides no bypass for scripts. The GET request will cause the script to run instead of downloading it. The WebDAV URL would have script interpreters disabled. Nevertheless, a lot of content I upload is script-free.

My current conclusion for server setup is to use Apache's Limit or LimitExcept directive to force all write access to be authenticated via HTTP basic authentication. I will have to publish the site via HTTPS, but that's a good idea anyway.

On client side, it seems pretty easy. Just map the WebDAV URL as a mapped network drive. There's also a specialized WebDAV client, but I see little advantage over FTP unless URLs are handled better. The tricky thing here is that I will have lots of subdomains, which poses serious configuration problems.

UNC paths can be used on Windows to access WebDAV servers. That's less complicated to setup for multiple subdomains than mapped drive. But the problem of mangled URLs remains.

It would be super easy to modify standard FTP client with an option to copy URL of the remote file to Windows clipboard. All that needs to be configured is the mapping between HTTP and FTP root folder. Unfortunately my favorite client FileZilla doesn't have this feature.

It can at least provide the FTP URL through right-click menu, which can be transformed to HTTP URL by replacing ftp://.../var/www/ with http://. This solution assumes that /var/www/ contains folders named after full domain name, e.g. /var/www/blog.machinezoo.com/.

Then there are WebDAV clients for Windows that look a lot like FTP clients except they communicate via WebDAV protocol. They might have better URL handling, but I would have to test them.

It's an open question. No solution I see is perfect. Any ideas?