Gartner Says Major Organisations Will Need to Grow Capacity and Performance Management Skills That Are the Foundation of Web-Scale IT
- Web-Scale IT Architectures and Techniques Are Heading for Mainstream Adoption During the Next Three Years
- Analysts to Examine the Outlook for Web-Scale IT at the Gartner IT Infrastructure & Operations Management Summit, 2-3 June in Berlin and 9-11 June in Orlando
Agile infrastructures demand that infrastructure and operations (I&O) managers acquire the performance management skills present only in web-scale IT, according to Gartner, Inc. While major organisations will need to maintain and sustain their conventional capacity-planning skills and tools, they will also need to regularly re-evaluate the tools available, and make a special effort to acquire and grow the capacity and performance management skills that are rarely evident outside the web-scale IT community.
"By 2016, the availability of capacity and performance management skills for horizontally scaled architectures will be a major constraint or risk to growth for 80 per cent of major businesses," said Ian Head, research director at Gartner. "To take advantage of web-scale IT approaches to capacity and performance management, IT architects need to fully embrace stateless application architectures and horizontally scaling infrastructure architectures."
Adding additional central processing units (CPUs), memory and storage to a monolithic server has been the traditional, vertical way of scaling up applications while capacity planning has traditionally been developed with the goal of forecasting the requirements for this vertical scaling approach. However, vertical architectures and approaches have limited scalability, making vertical architectures unsuitable for hyperscaling. For service capacity to expand seamlessly to extremely large scales, different approaches are required.
"Organisations managing such services need the ability to rapidly assign and de-assign resources to each service, as well as the ability to scale linearly and continuously as more resources are added or removed," said Mr Head. "They also need high levels of resiliency that will allow sections of the infrastructure to fail without bringing the services down. Designing the underlying infrastructure for horizontal scalability, high degrees of fault-tolerance and rapid, incremental change is a key prerequisite for effective web-scale operations."
To achieve the overall goal of an infrastructure in which services can consume capacity on an as-needed basis, the overall web-scale IT capacity planning function may be divided among two teams. The application or product team develops the applications, monitors the consumption of their services in various locations and user subsets, and requests and allocates infrastructure resources based on policy-driven utilisation triggers. The infrastructure team ensures that the overall shared physical limits do not constrain the performance of the individual services being continuously developed by the product teams.
Services constructed in this way are better equipped to scale geographically, and share multiple data centres with limited impact on user performance. Although extending service availability to new locations will carry additional burdens and requirements, horizontally scaling application and infrastructure architectures will generally be better equipped to scale geographically, because additional capacity may be used by the different applications, as required and as load patterns shift.
Where there are potentially large-scale, limited forecasts or demand history, IT leaders need to develop demand-shaping techniques to provide acceptable performance. Demand shaping enables finite infrastructure resources to maintain their vital always-on character with acceptable, if not consistent, performance experiences across the entire user base. Gartner predicts that through 2017, 25 per cent of organisations will use demand shaping for capacity planning and management, a significant increase from less than one per cent in 2014.
When deploying large-scale services, infrastructure and operations leaders need to become proficient in operational analytics tools and big data capabilities, rather than traditional capacity planning tools.
"Traditional capacity-planning tools enable I&O organisations to gather data from various sources, including monitoring tools and, possibly, business demand forecasts, then produce trending, utilisation and investment forecast information, taking into account several different scenarios," said Mr Head. "The different architectures and the huge scale of the web-scale IT organisations make traditional, highly focused tools of limited use. Demand shaping requires more and different functionality than current off-the-shelf tools provide."
Although different web-scale organisations grow and adapt their techniques to their specific requirements, a common theme is the extensive use of large volumes of operational data. Even though the infrastructures are large, the horizontal design enables clear visualisation and understanding of constraints and dependencies, such that these may be managed as the environment, loads and demands change.
In general, in-memory computing and deep analytics tools are used to extract the required information from a combination of the infrastructure monitoring tools and the instrumentation built into the applications. The resulting analytical information is used to facilitate proactive, real-time and near-real-time actions to allocate resources and manage potential bottlenecks. Similar functionality is also used to model the impact of moving workloads and to simulate the effects of potential infrastructure and application changes.
The outcome is that web-scale enterprises have developed a set of tools, practices and capabilities that enable real-time demand shaping. These operational skills and tools are unique to each web-scale organisation and so are not yet available in most end-user organisations. For example, Amazon uses Amazon auto-scaling (AAS), which measures loads in real time and allocates resources to match the loads. Netflix has found that, in its business environment, it needs to predict loads and allocate resources in advance so it has developed its Scryer tool, which forecasts upcoming loads and scales or allocates resources in advance of demand. This is coupled with its Hystrix tool, which enables it to visualise and understand latencies, and to accommodate failures and mitigate their effects on performance.
"Much of the art of achieving always-on, scalable, rapidly changing, high-performance services is a consequence of the advanced use of homegrown and customised analytics and tools by the application and infrastructure teams," said Mr Head. "These are used to shape demand in real time and to produce forward-looking capacity and investment plans to an acceptable degree of accuracy."
More detailed analysis is available in the report "Capacity and Performance Management Form the Basis of Web-Scale IT." The report is available on Gartner's website at http://www.gartner.com/doc/2690719.
This report is part of a larger Gartner research collection on web-scale IT titled "Web-Scale IT Influences the Market and Enterprises for Cloud Computing". This research spotlight can be viewed at http://www.gartner.com/document/2702717 and contains links to reports focusing on the inner workings of technologies and practices, the impact on the industry, and how IT leaders in traditional IT organisations can leverage Web-scale lessons.
Gartner analysts will take a deeper look at the outlook for IT operations trends at the Gartner IT Infrastructure & Operations Management Summit taking place 2-3 June in Berlin and 9-11 June in Orlando, Fla. Details on the Berlin event are at http://www.gartner.com/technology/summits/emea/it-operations/. More information on the Orlando event can be found at www.gartner.com/us/iom.
Members of the media can register for press passes to the Summits by contacting email@example.com (Orlando) or firstname.lastname@example.org (Berlin).
Information from the Gartner IT Infrastructure & Operations Management Summit will be shared on Twitter at http://twitter.com/Gartner_inc using #GartnerIOM.
Gartner UK Ltd
Gartner, Inc. (NYSE: IT) is the world's leading information technology research and advisory company. Gartner delivers the technology-related insight necessary for its clients to make the right decisions, every day. From CIOs and senior IT leaders in corporations and government agencies, to business leaders in high-tech and telecom enterprises and professional services firms, to technology investors, Gartner is a valuable partner in more than 14,000 distinct organisations. Through the resources of Gartner Research, Gartner Executive Programs, Gartner Consulting and Gartner Events, Gartner works with every client to research, analyze and interpret the business of IT within the context of their individual role. Founded in 1979, Gartner is headquartered in Stamford, Connecticut, USA, and has 6,100 associates, including more than 1,460 research analysts and consultants, and clients in 85 countries. For more information, visit www.gartner.com.