Observations: It is so easy.

Anyone can sign up for a ‘free’ trial Cloud account, begin developing applications and pay conveniently with their credit card.

The ease of this casual approach among app developers and IT Managers is why public Cloud processing of ICT workloads has been widely evaluated and has moved beyond early adoption experiments in most Australian organisations. Cloud has become the de facto vehicle for providing and consuming ICT resources once business risk management and architectural issues have been concluded1.

For most enterprises, the chief goal and challenge should be to bring together internal, external (service providers), outsourced and Cloud costing on a consistent and comparative basis so that informed decisions on where and how to place and operate ICT workloads can be continuously made2.

All Cloud providers offer their own proprietary variations of the same core infrastructure services: computing; storage, networking and interworking with other platforms as a service (PaaS) for databases, micro-services, APIs, big data and others3.

Businesses must choose the most appropriate mix of Cloud service providers, SaaS solutions and management tools based on their usage, performance, risk and cost requirements. Using Cloud computing, developers can experiment often, fail quickly at a very low cost, and end up with worthwhile innovation as more of their ideas are readily tested on the available platforms.

Why bother about costs and billing?: Technical staff are often fully consumed with exploring and optimising their ICT Cloud environment using DevOps or traditional processes. Most see Cloud competency as a critical step in their personal development and address the new learning challenges with gusto.

Cloud vendor marketplaces for new services and tools are rapidly becoming richer, more innovative and complete. Efficiency and cost control are rarely at the forefront of developer thinking when Cloud resources appear to be so cheap, functional, easy to use and comprehensive. The most likely outcome of Cloud development pilots is that an app will be promptly delivered in a relatively short timeframe using rich ICT resources at apparently cheap costs. The billing challenge that has emerged occurs when the work goes into production using a mix of Cloud resources that may not be optimal, turn out to be more expensive than expected and find run-away popularity amongst users.

This rich and new environment has led to many instances of run-away, out of control Cloud billing, usually through novel, incomplete or inappropriate architectures, solution designs or operational implementation.

The time for particular attention to billing from both IT and business managers and auditors is now (at this point in the Cloud adoption cycle).

First costing challenge – understanding: Those with cost control responsibilities must first understand Cloud concepts, naming conventions and development practices as well as planned and actual consumption4.

All major Cloud service providers provide costing and bill analysis tools to some extent and a rich environment of third party optimisation and billing management tools is developing5.

Detailed pricing structures invite adoption – provide design and cost management challenges: In the Cloud, physical servers deliver multiple virtual machines (VM) that operate as logical servers managed by software hypervisors that manage physical resources including CPU, RAM, network bandwidth etc, between VMs for each ICT workload6.

For those enterprise and IT staff with cost control responsibilities, both the technical complexity of design choices made and frequently changed, and the numerous costing options, add up to a billing and optimisation reconciliation task that is impossible without supportive tools.

Account and billing structures: Each vendor differentiates itself from competitors by the way that it organises and commits ICT resources and bills them. In AWS’ sphere, an account is a single billing unit that can provision resources. Bills for any family can be consolidated from linked accounts to a consolidated billing account family.

The pricing structure can become endlessly complex unless individuals with cost control responsibility are in personal command of the detail and nuances of vendors’ resource and account naming, as well as bill calculation conventions and practices7.

Bill monitoring best practices:

  1. Set up agreed enterprise definitions of Cloud resources even though their use and application will vary across each service provider used.
  2. Use cost comparison (monthly) calculators and document each significant iteration of options (in a saved spreadsheet).
  3. Conduct TCO analysis of various resource use scenarios via an NPV calculation despite non-ownership (but hire) of resources. There will be too many variables possible to model options in unnecessarily minute detail.
  4. Understand how rich integration of private Cloud and enterprise apps can make billing far more complex.

Pricing, billing and cost analysis tools: Trusted Advisor reserved instance (RI) recommendations can be cross-checked and confirmed for audit with tools like Cloudyn (monitoring Hybrid Cloud costs), CloudCheckr (unified cost and security management), Cloudability (data-driven Cloud cost management), Elasticache (manage and cost AWS in-memory cache), Redshift (AWS cluster management and data warehouse), Cloudcycler (cycling Cloud resources based on time of day), Flywheel (starts and stops instances), Cloudscreener or Right Scale (helps decide which Clouds to use), GorillaStack (third party AWS cost control), Cloudcruiser (financial management across AWS, Azure, Google), ParkMyCLoud (automated schedules), ExtraHop (real-time operations analytics) or Skeddly (automated scheduling).

Other open source tools are available from Github.

Next Steps:

  • Normalise Cloud nomenclature within the designing, specifying, buying and operating groups in the organisation.
  • Tag resources with defined standard labels according to the enterprise’s use cases, projects, charge-back needs and consumption habits.
  • Trial periods should be used to confirm an understanding of billing details.
  • Measure the expected and actual baseline bill. Monitor and run alerts on exceptions.
  • Set up a billing responsibility model and cost dashboard. Design for cost.

Footnotes:

  1.  The largest Cloud providers offering infrastructure-as-a-service (IaaS) are Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), IBM Softlayer and Rackspace. Within that leading group, AWS (with >90% of global hyperscale data centre capacity), Microsoft (with widespread Enterprise Agreements) and Google dominate.
  2.  AWS has the largest number of data centre Regions, Availability Zones (AZ) and capacity. Azure has similar globally-distributed data centres while Google has fewer data centre locations and relies on low-latency networking to reach users.
  3.  AWS has led the Cloud naming conventions and jargon for over a decade on the back of its huge e-commerce operations, while Azure and Google are in fast catch-up mode.
  4.  For example, in object and block storage, AWS names object storage as Simple Storage Service (S3) and Elastic Block Storage as EBS. Azure calls object storage Block Blobs and block storage is Page Blobs.
  5.  Tool examples – AWS Cloudwatch, Cost Explorer, Trusted Advisor or Azure Pricing Calculator help realise savings by implementing best practices for provisioning resources.
  6.  Using Amazon as an example, its Elastic Compute Cloud (EC2) VMs are called ‘Instances’ with nine ‘instance families’ and up to five instance sizes for each family. S3 can have four levels of service agreement (SLAs): standard, standard – infrequent access, reduced redundancy and Glacial (for archives).
  7.  Amazon EC2 can be consumed On-demand (hourly billing), as Reserved Instances (one to three years) paid All-upfront, Partial-upfront or No-upfront with matching discounts. Scheduled Reserved Instances, Spot Instances or Fixed Duration Spot Instances.