Reliably uploading very large files with the Reliable Transfer Client

Its been a few months since the release of LeanServer ScaleUP, a high-performance upload engine for ASP.NET and other IIS 7.0 applications on the Windows Server 2008 platform.

One of the things that has been interesting is how many people have trouble with uploads – and how varied their challenges really are. While we initially built ScaleUP to improve upload scalability for high-traffic environments like MySpace, it soon became fairly obvious that uploads pose many other challenges – and not all of them on the server.

Today, LeanServer announces the release of Reliable Transfer Client, a cross-browser plugin that takes the web upload experience to the next level, by addressing the modern client-side upload challenges.

The Reliable Transfer Client accomplishes two major goals that have consistently impeded the upload of large files on the web (e.g. HD video):

1. Eliminate the browser upload size limits (2Gb for most browsers, and much smaller for most existing flash uploaders), allowing the upload of 5Gb,10Gb, 20Gb, or larger files.

2. Provide a highly-reliable upload experience, despite common upload errors that often cause broken uploads for larger files: client connectivity loss, closed browser, or intermittent server errors.

Both of these goals go hand in hand, since larger uploads often take a much longer time and have a higher risk of failure sometime during the upload process.

On the server, large uploads can result in excessive memory usage and disk IO, which is a problem that ScaleUP helps solve.

On the client, large uploads consume more memory, amplifying any file IO and memory usage inefficiency in the client technology (which has been the reason why flash uploaders have been mostly limited to small files). Because of this, we chose Java as the platform for RTC – having the right mix of file IO and memory control to efficiently process large files (unlike Flash 10), cross-browser compatibility (unlike Active-X and Google Gears), and widespread adoption (1).  We do have plans for Silverlight as well in the near future 🙂

Uploading a 50Gb file with LeanServer Reliable Transfer Client

The RTC was designed from the ground up to easily handle large files, reduce both client and server overhead, and provide a high degree of fault tolerance to a wide class of upload errors without losing the upload.

And if for whatever reason the upload cannot continue, the user has to option to come back and resume the upload at a later time or even from another location.  Needless to say, this makes a HUGE difference when it comes to reliably uploading large files.

Since most of our clients need to control the upload look and feel, we made RTC completely transparent to the user.  The client has no UI of its own, letting the application provide upload UI elements and use the RTC javascript API to create, start, and control multiple uploads, receive progress information, and everything else in between.

With the Reliable Transfer Client in the browser and ScaleUP on the server, we are able to turn any ASP.NET application into an upload powerhouse.  So, if you need to support large file uploads or are serious about making your upload experience better, be sure to check out the ScaleUP platform at www.leanserver.com/scaleup.

Regards,

Mike

—————-

(1) More than 75% of internet enabled browsers have Java installed, according to riastats.com.