Market Research Logo

Sea Change Series: Orchestration Software in the Mega Data Center

Sea Change Series: Orchestration Software in the Mega Data Center

The 2017 module has 129 pages and 85 tables and figures. Orchestration Software in the Mega Data Center is used to tie a fabric architecture together that fills up an entire building with 100,000 processors and 100,001 switches. The mega data center described in the study is effective because it leverages the economies of scale. This orchestration software infrastructure study module is part of a longer study that addresses the business issues connected with data center modernization. There are 26 module parts to the larger study comprised of detailed analysis of how new infrastructure layers will work to support management of vast quantities of data.

Business growth depends on technology spending. Intelligent, automated process, not manual labor systems are what speed business growth. We have had the situation in the data center where 93% of spending is just to keep current systems running, many of those plagued with manual input. Mega data centers change that pattern of IT manual process.

The Internet has grown by a factor of 100 over the past 10 years. To accommodate that growth, mega data centers have evolved to provide processing at scale. Facebook for one, has increased the corporate data center compute capacity by a factor of 1,000, virtually eliminating much manual process. Orchestration software is a key aspect of that process. To meet future demands on the Internet over the next 10 years, companies with that capacity need to increase capacity by the same amount again while the other companies struggle to catch up. Nobody really knows how to get to increasing compute capacity by another factor of 1,000.

Business growth depends on technology spending. Intelligent, automated process, not manual labor systems are what speed business growth. We have had the situation in the data center where 93% of spending is just to keep current systems running, many of those plagued with manual input. Mega data centers change that pattern of IT manual process.

Realigning data center cost structures is a core job of orchestration software. The enterprise data centers and many cloud infrastructure operations all have similar problems of being mired in administrative expense. Containers address that issue by creating vastly more efficient operations for data center infrastructure.

According to Susan Eustis, lead author of the team that prepared the study, “The only way to realign cost structure is to automate infrastructure management and orchestration. Mega data centers automate server and connectivity management using orchestration software to manage multiple application containers. Other systems automate switching and storage, along with hypervisor, operating system, and virtual machine provisioning. “


This is the 700th report in a series of primary market research reports that provide senior executive analysis in communications, telecommunications, the Internet, computer, software, telephone equipment, health equipment, and energy. Automated process and significant growth potential are a priority in topic selection. The project leaders take direct responsibility for writing and preparing each report. They have significant experience preparing industry studies. They are supported by a team, each person with specific research tasks and proprietary automated process database analytics. Forecasts are based on primary research and proprietary data bases.

The primary research is conducted by talking to customers, distributors and companies. The survey data is not enough to make accurate assessment of market size, so WinterGreen Research looks at the value of shipments and the average price to achieve market assessments. Our track record in achieving accuracy is unsurpassed in the industry. We are known for being able to develop accurate market shares and projections. This is our specialty.

The analyst process is concentrated on getting good market numbers. This process involves looking at the markets from several different perspectives, including vendor shipments. The interview process is an essential aspect as well. We do have a lot of granular analysis of the different shipments by vendor in the study and addenda prepared after the study was published if that is appropriate.

Forecasts reflect analysis of the market trends in the segment and related segments. Unit and dollar shipments are analyzed through consideration of dollar volume of each market participant in the segment. Installed base analysis and unit analysis is based on interviews and an information search. Market share analysis includes conversations with key customers of products, industry segment leaders, marketing directors, distributors, leading market participants, opinion leaders, and companies seeking to develop measurable market share.

SEA CHANGE SERIES: ORCHESTRATION SOFTWARE IN THE MEGA DATA CENTER
ORCHESTRATION SOFTWARE EXECUTIVE SUMMARY
Orchestration Software Market Definition
Orchestration Software Used to Quickly Build Software Servers Useful for Implementing Innovation
Sea Change Series: Orchestration Software in the Mega Data Center, Amazon, Google, Microsoft, Facebook
Aim to Realign IT Cost Structure
Internet Has Grown by a Factor of 100 Over The Past 10 Years
Table of Contents
1. BUSINESS VALUE OF ORCHESTRATION SOFTWARE
1.1 Orchestration Software Market Definition
1.1.1 Orchestration Software No Waiting for IT
1.2 Orchestration Software Business Value
1.2.1 Orchestration Software Automates Data Center Infrastructure
1.2.2 Software Containers
1.2.3 Orchestration Schedulers Manage Containers
1.2.4 Orchestration Software Supports Container Automation
2. ORCHESTRATION SOFTWARE REALIGNS BUSINESS RESPONSIVENESS
2.1 Orchestration Software Supports Microservices that Drive the Business
2.1.1 Realigning Data Center Cost Structures
2.1.2 IT Relies On Replacing Virtual Machine: VM Virtualization
2.1.3 Microservices
2.1.4 Microservices Features
2.1.5 Microservices Modules
2.1.6 Difficulties with Enterprise Virtual Machines
2.1.7 Hypervisor a Difficulty
2.2 Orchestration Software Containers and Fabrics Support Business Flexibility
2.2.1 Virtual Machines Use Bare Metal, Containers Use Orchestration Software
2.2.2 Bare Metal an Inefficient Use of Compute Resource
2.2.3 Bare Metal Less Efficient
2.2.4 Robots Better than Slow And Error Prone Manual Labor
2.2.5 Orchestration Software IT processes Replace Manual Labor in Same Manner
2.2.6 Mega Data Center Orchestration Software
2.2.7 Large Fabric Mega Data Center Network Not The Kind Of Environment That Can Be Realistically Configured And Operated In A Manual Way
2.2.8 Automation of the Data Center Fabric
2.2.9 Value of Data Center Fabric
3. GROWING THE BUSINESS: IT IS THE FOUNDATION OF BUSINESS GROWTH
3.1 Google and Microsoft Positioning of Fabric and Container Orchestration Software Bring Business Growth
3.1.1 Google Shift from Bare Metal To Container Controllers
3.1.2 Google Container Controller Shift From Bare Metal In A Mixed Workload And In Nested Compute Units
3.1.3 Google Kubernetes Groups Software Containers
3.1.4 Fabric Services Inside A Container
3.2 Microsoft Orchestration Software
3.2.1 Architecting Microsoft Cloud
3.2.2 Microsoft Managed Clustering and Container Management: Docker and Mesos
3.2.3 Microsoft Azure Service Fabric
3.2.4 Microsoft Is Seen As The Overall Winner In The Move To Application Containers
3.2.5 Microsoft Dublin Cloud 2.0 Mega Data Center
3.2.6 Microsoft Data Center, Dublin, 550,000 Square Feet
3.2.7 Microsoft Dublin Center Operates at a Power Usage Effectiveness (PUE) of 1.25.
3.2.8 Microsoft Data Center Container Area in Chicago.
3.2.9 Microsoft
3.3 Mega Data Center Business Opportunities and Challenges
3.3.1 Advantages of Using Containers
3.3.2 Orchestration Software Used to Create Containers
3.3.3 Disadvantages of Using Containers
3.3.4 Advantages of Virtual Machines (VMs) as Opposed to Orchestration Software Containers
3.3.5 Container Orchestration
3.3.6 Use of Containers to Eliminate Manual Process
3.3.7 IT Pros Increasingly Turn to Chef and Puppet
3.3.8 Hardware Containers Do Not Scale
4. ORCHESTRATION SOFTWARE: BUSINESS MODERNIZATION FUNCTIONS
4.1 Orchestration Software Business Modernization Functions
4.1.1 Facebook Data Center Positioning
4.1.2 IBM Data Center Orchestration Software Automates Application Integration
4.1.3 Docker Orchestration & Docker Swarm
4.1.4 Docker Container Platform
4.1.5 Common Feature Sets For Orchestration Tools
4.1.6 Not all Orchestrators Are Created Equal
4.1.7 AWS Cloud Container Adoption Criteria
4.1.8 AWS Cloud Adoption Methodology
4.1.9 AWS Cloud Adoption Framework
4.1.10 AWS Market Leader In Cloud Computing
4.1.11 Facebook Fabric and Node are Core Structures Leveraging Software Orchestration
4.2 Examples of Mega Data Center Orchestration Software
4.2.1 Apache Mesos Orchestration Software
4.2.2 Google Kubernetes Container
4.2.3 Google Container Builder Step Toward Building Pluggable Components in a Pipeline
4.2.4 Google Programmable Access To Network Stack
4.2.5 Google Andromeda Software Defined Networking (SDN)-
4.2.6 Google Compute Engine Load Balancing
4.2.7 Google Compute Engine Load Balanced Requests Architecture
4.2.8 Google Scaling Of The Compute Engine Load Balancing
4.2.9 Google Compute Engine (GCE) TCP Stream Performance Improvements
4.2.10 Google Cloud Platform TCP Andromeda Throughput Advantages
4.2.11 Google Open Sourced Its Container Management System Called Kubernetes
4.3 Increasing the Corporate Data Center Compute Capacity by a Factor of 1,000
4.3.1 Facebook
5. DATA CENTER BUSINESS VALUE
5.1 Business Value Achieved From Mega Data Center
5.1.1 Ability To Move Fast And Support Rapid Growth At The Core Of Facebook Infrastructure Design Philosophy
5.1.2 The Right Type of Cloud: Mega Data Centers
5.1.3 AWS Has Been Able To Adapt To Change
5.1.4 Manual Labor Is Slow And Error Prone
5.1.5 Mega Data Center Orchestration Software
5.2 The Big Four in Mega Data Centers
5.2.1 Amazon, Google, Microsoft, Facebook
5.2.2 Cloud 2.0 Mega Data Center Fabric Implementation
5.2.3 Fabric and Node are Core Structures Leveraging Software Orchestration
5.2.4 Multi-Threading, Dynamic Systems
5.2.5 Oracle Multi-Threading Mega Data Center Tools
5.2.6 Orchestration Tools Manage A Cluster As A Single Deployment
5.2.7 Microservice Monitoring with Google Kubernetes
5.2.8 Docker Container
5.3 Business Value of Open Source Orchestration Software
5.3.1 Cluster Functions and Pod Benefits
5.3.2 Mesosphere DC/OS an Open-Source Project Built on Apache Mesos
5.3.3 Mesosphere DC/OS Production Containers Uses
5.3.4 Mesosphere DC/OS Orchestration Software
5.3.5 Mesosphere DC/OS Certification Compliance
5.3.6 Mesosphere Market Leadership Position
5.3.7 Mesosphere DC/OS Runs Data Services on One Single Platform
5.3.8 Cloud Computing Not Enough: Entire Warehouse Building As A Single Mega Data Center System113
5.3.9 Red Hat Ansible
5.3.10 Red Hat Ansible Architecture, Agents, And Security
5.3.11 Red Hat Ansible Advanced Features
5.3.12 Red Hat / Ansible Open Source, Agentless System
5.3.13 Red Hat Ansible Tower 3 Job Run Metrics
5.4 Orchestration Software Integrated Infrastructure Management Functions
5.4.1 Cisco Integrated Infrastructure Management
5.4.2 Cisco UCS Helps Manage Administrative Costs And Reign In Complexity
5.4.3 Mesosphere DC/OS: Mesos Features
5.4.4 Heart of DC/OS: Apache Mesos
5.4.5 DC/OS Implements Containers
WINTERGREEN RESEARCH,
WinterGreen Research Methodology
List of Figures
SEA CHANGE SERIES: ORCHESTRATION SOFTWARE IN THE MEGA DATA CENTER
Mega Data Center Depends on Orchestration Software to Control Operations, Implement Microservices
ORCHESTRATION SOFTWARE EXECUTIVE SUMMARY
Figure 1. Intelligent orchestration Used to Create Servers Used As Long As Needed and Then Disappear
Figure 2. Slow Growth Mode of Companies with Enterprise Data Centers
Figure 3. Mega Data Center Fabric Implementation
1. BUSINESS VALUE OF ORCHESTRATION SOFTWARE
Figure 4. Orchestration Software Facilitates Innovation
Figure 5. Business Innovation and Technology
Figure 6. Docker Orchestration Software Creates Containers
Figure 7. Docker Compose
Figure 8. Mesosphere Marathon
Figure 9. Google Kubernetes
Figure 10. Orchestration Software Supports Container Automation
2. ORCHESTRATION SOFTWARE REALIGNS BUSINESS RESPONSIVENESS
Figure 11. Microservices that Drive the Business
Figure 12. Orchestration Software Decreases Data Center Cost Structure
Figure 13. Files Bundled into a Container
Figure 14. Microservices: Suite Of Independently Deployable Service Modules with a Unique Process And Well-Defined, Lightweight Communication Portal: Mechanism To Serve A Business Goal
Figure 15. Microservices Distinct Features: Taxi Hailing Example
Figure 16. Microservices Market Segments
Figure 17. Microservices Modules
Figure 18. Hypervisor Virtualization Operating System Interface
Figure 19. Hypervisor Virtualization Operating System Interface
Figure 20. Virtual Machines Less Efficient Than Containers
Figure 21. Difference Between Virtual Machines and Containers
Figure 22. Bare Metal Management Replaced by Container Controllers
Figure 23. Containers vs. VMs
Figure 24. Industrial Robots Eliminate Manual Labor
Figure 25. Industry Uses Robots To Replace Manual Labor
Figure 26. Data Centers Need The Precision and Automation Similar to that Provided by Multi-Step Sequential Task Industrial Robots
Figure 27. Mega Data Center Orchestration Software
Figure 28. Single-Fabric Data Center Network Architecture
3. GROWING THE BUSINESS: IT IS THE FOUNDATION OF BUSINESS GROWTH
Figure 29. Bare Metal Presents a Lot of Extra Parameters and Metrics, Significantly More than With Containers
Figure 30. Nested Compute Units
Figure 31. Kubernetes Orchestration Software Groups Containers That Make Up An Application Into Logical Units
Figure 32. Kubernetes Orchestration Software Functions
Figure 33. Container Features as it integrates with the Service Fabric Runtime
Figure 34. Microsoft Setting Up A Secure Service Fabric Cluster in Azure using the Azure Portal.
Figure 35. Microsoft Data Center, Dublin, 550,000 Sf
Figure 36. Orchestration Software Container Area In The Microsoft Data Center In Chicago
Figure 37. Microsoft Cloud Network Features
Figure 38. Like Physical Containers on a Ship, Software Containers Bring Many Servers Densely Packed
Figure 39. Advantages of Using Containers
Figure 40. Software Orchestration Container Challenges
Figure 41. Manual Process
Figure 42. Containers Need Orchestration Software
Figure 43. Virtual Machine Data Center Management Tasks:
Figure 44. FaceBook Open Compute Project
4. ORCHESTRATION SOFTWARE: BUSINESS MODERNIZATION FUNCTIONS
Figure 45. Orchestration Software Business Modernization Functions
Figure 46. Facebook Data Center Modernization Functions
Figure 47. Facebook Altoona Iowa Cloud 2.0 Mega Data Center
Figure 48. Manual Process for Application Integration Deployment
Figure 49. Feature Sets For Orchestration Tools
Figure 50. Issues for Orchestration Software
Figure 51. AWS Cloud Container Adoption Criteria
Figure 52. AWS Cloud Container
Figure 53. AWS Cloud Adoption Framework
Figure 54. AWS Market Leader In Cloud Computing
Figure 55. Description of the Orchestration Software
Figure 56. Advantages of Using the Container Builder Cloud Architecture as a Service:
Figure 57. Google Andromeda Cloud High-Level Architecture
Figure 58. Google Andromeda Software Defined Networking (SDN)-Based Substrate Functions
Figure 59. Google Andromeda Performance Factors Of The Underlying Network
Figure 60. Google Compute Engine Load Balanced Requests Architecture
Figure 61. Google Compute Engine Load Balancing
Figure 62. Google Cloud Platform TCP Andromeda Throughput Advantages
Figure 63. IoT: Open Source IoT High Level Platform, OpenStack and Kubernetes
5. DATA CENTER BUSINESS VALUE
Figure 64. Facebook DuPont Fabros Technology Ashburn, VA Data Center
Figure 65. Cloud 2.0 Mega Data Centers Support 1.5 Billion Facebook Users Worldwide.
Figure 66. AWS Market Leader In Cloud Computing
Figure 67. In Industry Manual Labor Replaced by Robots
Figure 68. Robotics Has Phenomenally Transformed The Industrial Sector In Terms Of Manufacturing Capability
Figure 69. Spending On Manual Process Is Counterproductive Vs Automation Spending
Figure 70. Data Centers Need Orchestration Software to Provide The
Precision and Automation of Multi-Step Sequential Task Industrial Robots
Figure 71. Mega Data Center Orchestration Software Functions
Figure 72. Multiple Pathways Open To Processing Nodes In The Cloud 2.0
Mega Data Center Functions
Figure 73. Dynamic Load Balancing
Figure 74. Oracle Multi-Threading Mega Data Center
Figure 75. Mesosphere Enterprise DC/OS Orchestration Software
Figure 76. Mesosphere Customer References
Figure 77. Mesosphere DC/OS Extending Capabilities Within Container Orchestration
Figure 78. Mesosphere DC/OS Certification Compliance
Figure 79. Cloud Is Not Enough
Figure 80. Red hat Ansible Playbook Language Advanced Features
Figure 81. Red Hat Ansible Tower 3 Job Run Metrics
5.4.2 Cisco UCS Helps Manage Administrative Costs And Reign In Complexity
Figure 82. Cisco UCS Helps Manage Administrative Costs And Reign In Complexity
Figure 83. Cisco UCS Director Delivers Comprehensive Infrastructure Management and
Orchestration
Figure 84. Mesosphere DC/OS: Mesos Features:
Figure 85. Native Mesos Container Functions

Download our eBook: How to Succeed Using Market Research

Learn how to effectively navigate the market research process to help guide your organization on the journey to success.

Download eBook

Share this report