Crunching Numbers in the Cloud with Spot Instances

Brian McCallion Uncategorized Leave a Comment

What Would Sun-Tsu Have to Say About Cloud Computing for Financial Markets?

  1. When it comes to batch processing in the data center, no matter how many instances you have the number is always wrong. If you had more, you could run the simulation more quickly, and seize an opportunity. And at other times, you don’t need many–so you probably don’t want to pay for them when you don’t need them. Public Cloud computing for financial markets becomes inevitable when you consider on-demand from this perspective.
  2. At one point large firms perceived large fleets of compute as a competitive advantage. Today hedge funds realize the value of on-demand computing.

Hadoop Map Reduce, parallel data loads, fan-out clusters work great on Amazon Web Services.

High Performance, Real Time Data Processing Delivers on the Promise of Cloud Computing

AWS Spot Instances can save you money in ways not possible even in the largest corporate data centers

Parallel Processing
Even the largest compute or data processing job can be broken down into steps. And when these steps are delegated to a fleet of nodes for processing we become keenly aware how cloud is different.

Imagine a portfolio scenario that usually requires two weeks to run.
When run on a fleet of nodes in the cloud this work can be completed by running a fleet of 500 instances for an hour — at a far lower total cost. How is this possible?

In the Cloud compute is on demand, and costs are linear. Because you only pay for what you use, you can also use what you need. In the data center even if you maintain a stable of thousands, tens of thousands of compute nodes each of these nodes still requires care and feeding when its not employed. Further no matter what number you have, the number is finite. In the Cloud resources can be used far more efficiently.

Spot instances on AWS enable customers to purchase and run fleets of instances at 90% cost savings. It’s easier than ever to bid for and acquire fleets of spot instances at AWS.

AWS Spot Instances Fleet Management