We just reduced image propagation time on FutureGrid’s Sierra cloud at UCSD from hours to minutes! This magic comes courtesy of Nimbus LANTorrent.

We blogged about LANTorrent before: it can distribute the same file among many nodes using peer-to-peer techniques. It is available in Nimbus since version 2.6 and allows users to efficiently deploy a cluster of virtual machines based on the same image. Installing and configuring LANTorrent on the Nimbus nodes (both service and hypervisor nodes) is easy; it took only a couple of hours on Sierra (all the details are explained in the LANTorrent Configuration section of the Nimbus documentation).

Cloud computing users think on-demand availability is the best thing since sliced bread: it enables elastic computing, outsourcing for applications requiring urgent or interactive response, and reduces wait times in batch queues. But if you are a cloud provider you might not think so… In order to ensure on-demand availability you  need to overprovision: keep a lot of nodes idle so that they can be used to service an on-demand request, which could come at any time. This means low utilization. The only way to improve it is to keep fewer nodes idle. But this means rejecting more requests – at which point you’re not really on-demand… a veritable catch-22.

Amazon’s S3 is a great storage cloud service.  It provides highly available access to highly reliable storage at a price.  It has emerged as the data storage cloud de-facto standard for good reasons.  Its REST API has allowed several 3rd party software vendors to make impressive clients, both GUI and command line.

However, it is closed source and unavailable to the numerous data centers actively used for science.  Can private data cloud providers that already have substantial amounts of hardware allow their users to take advantage of the known interfaces and existing clients?  Can the Scientific community that already has access to vast amounts of computing and storage power, but not access to an expense accounts take advantage of these data storage cloud innovations?

Today we made open source Infrastructure-as-a-Service capabilities for science just a little bit better… The new Nimbus 2.6 makes your images get to the nodes a lot faster and provides capabilities allowing administrators to easily and dynamically shift resources between their tried-and-true batch scheduled cluster and a cloud — depending on where they need them more. The Nimbus Context Broker gets shiny new REST-based interfaces, which opens it up to new applications.