11 steps to holistic data center design

From the October, 2013 Issue of Cabling Installation & Maintenance Magazine

The steps outlined here take into account the business requirements, technology innovations, energy and operational savings.

By Michael Salvador, Belden

Designing a data center can be a daunting and complicated process. Many considerations and implications ultimately impact the cost to build, operate and scale the data center. In the past information technology (IT) departments provided an estimate of the equipment and power needs required for their various systems. Many of these estimates were either inaccurate speculations based on current needs with no forethought given to the IT roadmap and business growth, or they were overinflated based on worst-case scenarios. Facilities then used these estimates to either build exactly what was requested or to once again overinflate the design in attempts to protect themselves. This outdated process resulted in large, inefficient data centers that were costly to operate and virtually impossible to upgrade.

Understanding and planning the cable-plant technology that serves a data center is important to the facility's successful construction and operation. Deploying a modular approach and using as many standardized cabinets as possible makes the data center predictable, with better port and space use.

The rising cost to construct and operate a data center calls for a better approach to data center design. A holistic design approach provides significant benefit to data center managers, facilities, customers and the business as a whole.

A central aspect of successful holistic design involves carefully considering all of the variables with an eye on operational expenditure (opex) over the life of the data center, not just the capital expenditure (capex) to build the data center in the first place. Holistic design must also account for business growth and evolving technology, incorporating systems and innovative technologies that ensure an agile, scalable and highly robust data center that supports both current and future needs.

Successful holistic data center design requires following key steps that essentially take into account all business requirements, technology innovations, and energy and operational savings, while identifying and eliminating any ineffective decisions and operational waste.

Now, here are 11 steps to a holistic data center design.

Step 1:Including all players. In most business environments, boards of directors or C-level executives formulate a strategic business plan with goals based on customer feedback, market conditions, opportunities and corporate vision. These goals are communicated to various business units, who in turn develop plans and requirements to meet the goals. Whatever the business type, virtually all of the requirements identified to meet the business plan involve some form of IT-related service or resource that must ultimately be supported by the data center. Consequently, the data center has become a catalyst for business and a vital part of the business ecosystem.

Once each business unit identifies their requirements, IT departments review those needs and identify the technologies that enable business units to successfully carry out their plans. IT must then determine the equipment, power and space required to support the technologies. Facilities then use this information to establish their own list of requirements needed to support IT--all while keeping operational costs in check. The end result enables the business units and the strategic business plan to either succeed or fail.

Involving all players of the business ecosystem--executives, business units, IT and facilities-- in the early design phase identifies the actual requirements and future business direction needed to properly design the data center. Trusted advisors and outside design consultants and contractors should also be invited to the discussion. During this process, published meeting minutes should be distributed to all players, and recap meetings should be held at regular intervals. Primary and alternative representatives should be picked from every team, and the overall success of the project should be part of their responsibilities.

While teams can vary from business to business, following is a typical list of players involved in successful holistic data center design.

• Project sponsor: Typically this should be a C-level person (i.e. CIO or CTO) or other business- level executive.
• Project manager: Depending on the overall scope, project management should likely be this player's sole responsibility. The project manager is the overseer of meetings and the individual responsible for keeping the project and discussions on track. The project manager can be internal to the company or part of an outside architectural and engineering team.
• Business unit representatives: This could be one or multiple persons responsible for relating their business unit requirements and roadmap of future direction.
• IT technologists: These individuals can include networking (LAN and WAN) teams, server teams for each operating system, storage area network (SAN) teams and data center support teams.
• Facilities management: Any area of facilities operations and maintenance that is applicable should be included in the process, including security, real estate and property management.
• Purchasing/procurement: These are individuals who negotiate all aspects of the data center purchases.
• Outsourced services: Whether an architectural or engineering firm, individual consultant, general contractor or integrator (or a combination thereof), most organizations employ outside help at various times during the design process. Engaging these individuals depends on an organization's resources, skill level and project scope. However, to remain cost-effective, organizations need to keep markups and change orders to an absolute minimum. For larger projects, the customer may bid the project as a whole with an established percentage over cost to the general contractor, rather than dealing with separate subcontractor bids and potential surprises. This practice also puts onus on the general contractor to fully participate in the design process and understand all requirements, allowing only customer-initiated change orders to be billed and significantly lowering contingency budgets from as much as 15 percent to as low as 2 percent.
• Trusted advisors: These individuals are typically expert representatives from vendors and manufacturers who can help with the design process surrounding their specific solutions (i.e. switches, cabling, power and cooling solutions). Including these individuals in the process is at the discretion of each discipline.

Step 2:Setting ground rules. While setting ground rules may seem like a daunting, political and sometimes ugly task, it can help ensure respect for all players and professionalism that will go a long way in creating a superior product and successful outcome. Remember the following tips when moving ahead with plans for a holistic data center design.

1. All players should check their ego at the door, setting aside preconceived notions about the project and remaining open to any and all suggestions. Don't take it personally; all decisions should be based on the business needs and objectives.
2. All players should have the ability to question or provide input on any aspect of the project, including outside their realm of responsibility and expertise. Sometimes the best ideas stem from questions or suggestions from those outside of a discipline.
3. All players should have a vote on decisions. While the discipline expert should have the final say, they must be able to back their decisions with solid evidence and information. The phrase, "Because I said so," is not reason enough to override the entire team.
4. Using broad brush strokes to define aspects such as business requirements, redundancy, technology, risk adversity, and past means and methods should not be allowed.
5. All aspects are on the table to be dissected and discussed. Out-of-the-box thinking and solutions should be encouraged and discussed from all players.
6. Always consider what is important to the business and how it affects the company's bottom line, reputation and ability to meet the goals set forth in the strategic plan.

Step 3:Determining availability and redundancy. Understanding and determining the data center availability and redundancy is a critical part of the holistic design process, and one that can significantly impact cost. Critical to this is determining which of The Uptime Institute's tier ratings best suits the business needs. Tier 1 includes non-redundant capacity components with an availability of approximately 99.671 percent and 28.8 hours of downtime per year. Tier 2 includes all the capabilities of Tier 1 plus redundant capacity components with an availability of approximately 99.741 percent and 22 hours of downtime per year. Tier 3 includes the capabilities of Tier 2 plus dual-powered equipment and multiple uplinks with an availability of approximately 99.982 percent and 1.6 hours of downtime per year. And Tier 4 includes the capabilities of Tier 3 plus all components fully fault-tolerant including uplinks, storage, chillers, HVAC systems and servers with everything dual-powered, and an availability of approximately 99.995 percent with 0.4 hours of downtime per year.

Many businesses believe they need 100-percent uptime, 365 days per year, 24 hours per day--essentially Tier 4. This may or may not be true for your business; but the initial cost to build a data center at each tier must be considered. Conservative estimates put the cost increase from a Tier 1 to a Tier 2 facility at approximately 30 percent in project capex; from Tier 2 to Tier 3 represents an approximately 50-percent capex increase; and Tier 3 to Tier 4 represents an approximately 22-percent capex increase. Big business with enormous data centers often create multiple data center "halls" that each meet a specific tier classification for "right size" availability and redundancy. This tactic can save on construction costs and allows a business to achieve lower operational cost.

More detailed information on the data center tier ratings can be found by visiting The Uptime Institute at www.uptimeinstitute.com.

Step 4: Gathering requirements. When it is time to start gathering the requirements for the data center design, there are a few points to consider. First, it is important to keep an eye on what the requirements will be once the data center actually goes live--not what the requirements are today. Each business unit should communicate their three- to five-year plan for growth. IT should understand each equipment manufacturer's roadmap for this time period. In addition, anything that will ultimately be virtualized, outsourced to a service provider or moved to the public cloud should not be included in the requirements.

The next step is to create a redundancy and dependency grid for each application and system. This is an excellent tool for ascertaining the size, space, step functions for expansion, tier classification(s) and dependencies required for each application and system. The grid can get rather large and complicated, but it will provide cost saving to the company in the long run by preventing over-designing the data center. The redundancy and dependency grid can also help determine the growth of the data center and allow for a more modular approach to building the data center.

Step 5:Balancing capex and opex. When selecting the latest IT equipment or method to power or cool the data center, it is important to take an approach that balances total cost to build (capex) versus the total cost of operating the data center (opex). For example, it is not wise to select an energy-saving technology without considering the cost to operate and maintain the technology. Following are questions to answer when selecting a technology for the data center.

• What additional training or people will be required to operate the technology?
• What are the costs related to maintenance and upkeep?
• What is the lifespan of the equipment and how does it affect my depreciation expense?

Closely examining capex and opex helps to make informed, intelligent choices and if necessary, provides the reasoning for spending more up front to ultimately achieve operational savings.

Step 6: Selecting the right equipment. Determining the most efficient servers with the most efficient power supplies is an important step to holistic data center design. Examining the power consumption of the servers and finding the "sweet spot" for number and speed of the CPUs, memory and correct sizing of the power supply is part of this process.

Selecting the correct platform to support a specific application is also important. How an application uses a server must be understood to effectively balance energy savings and performance. For example, some applications are input/output (I/O) intense and may need a server that has lower compute processing capabilities. One the other hand, some applications may require more compute processing and less I/O.

Another factor is power supply size. Selecting a power supply that is too large will not be as efficient as one that is sized correctly to support the server during normal operation with some extra capacity when needed.

This same reasoning should be applied to all IT equipment. Finding the proper fit for the equipment will provide significant operational savings for the data center. Blindly standardizing on a single system without regard for energy consumption can lead to higher operational cost. Obviously, IT must also weigh cost efficiencies with the ability to support various makes and models of equipment along with technology changes and requirements moving forward.

Step 7:Designing equipment areas. Whether the data center site has been selected or not, there are several philosophies when it comes to deciding how the equipment areas of the data center should be built. As discussed previously, tier level support is part of the equation. Some key points to consider are cable-plant design and limitations in relation to switch location, business unit pods or shared pods, cabinet design and who will be allowed to perform moves, adds and changes in the data center. These points need to be tailored to meet the needs of the business.

When designing the data center, avoid exclusive cabinets and pods of equipment. Having a standardized configuration for a cabinet or pod provides benefits such as creating a specialized customer part number with a manufacturer or distributor, and ease of budgeting and ordering for future expansion. The configuration can be reviewed and adjusted when technology changes occur. It is also smart to look for ways to eliminate on-site construction labor, shipping material disposal and transportation costs by taking advantage of manufacturer or distributor services to preconfigure as many components as possible. Prior to any request for pricing, make sure to inform contractors of the use of these services or any other labor-saving technology, such as preterminated cables.

The easiest place to start is with a basic server cabinet. Using the data from the requirements-gathering phase, determine what the cabinet will look like. Considerations might include: width, depth and number of RU spaces; type of doors (single or split); cable entry (top or bottom); ability to modify the cabinet or redeploy as another type of cabinet; cable management; equipment positioning; power distribution unit (PDU) size, intelligence, type and number of outlets; cabinet color.

These same considerations apply for networking cabinets, storage cabinets and almost any other cabinet used to house IT equipment. Keeping the number of types of cabinets to a minimum can help simplify design and expansion of the data center.

Step 8:Understanding cable plant technology. Understanding cable plant technology and capabilities is a key component when designing a cable plant layout. Following are questions to answer when determining the cable plant.

• Does the business require high-bandwidth capabilities?
• What network topology design will be deployed (e.g. fabric switching or traditional switching)?
• What are the advantages and implications of using certain cable media types?
• What are the distance limitations of the data center design?
• What configuration will be used for access-level switching (e.g. top of rack, end of row, middle of row or centralized)?
• What will be required in the future?

By deploying a modular approach using as many standardized cabinets as possible, the data center becomes industrialized and predictable with better port and space utilization. Modular design involves planning for the entire space, power and cooling requirements based on the company's growth over the life of the facility. The design is broken into modules that can be added as the business requirements evolve. All shared infrastructure should be sized with the absolute final design in mind; it is important to understand that modules will need to be plugged into this infrastructure with little or no interruption of services as growth occurs. This allows for better cost efficiencies and ease of operation.

Data center infrastructure management (DCIM) is also ideal for supporting asset inventory, cable plant connectivity, power management and reporting, and "what if" scenario planning. While it can seem rather early in the process to select a DCIM package, it can be a time-consuming process to ensure that the package meets all requirements needed to serve the business. It also takes a significant amount of time to set up the DCIM templates and standard information to be tracked.

In summary, apply standardization, modularization and industrialization to as many components and pieces of the data center as possible. Understand the future applications and technology that will likely be deployed, and build in adaptability and flexibility to support them where possible. If you stay ahead of the curve and begin planning the pods or modules that will be built, it can follow for quick rollout of the IT equipment and the ability to double-check calculations prior to construction. Also try to maximize the power load within a pod or equipment area before building out the next module. This allows the facility to operate at the maximum possible energy efficiency.

Step 9:Designing the overall space. With the requirements determined, IT can then present their requirements to the rest of the team and begin discussions to finalize power, cooling and space requirements. If a space has not been selected, this information can be useful for identifying potential sites or spaces. If the space has been selected, discussions begin to review the types of power distribution, cooling, economization and containment. During these discussions, full team participation is imperative. Adjustments and compromises will occur; but when the full design is completed, every player should understand what is being proposed and built. There should be a sign-off on the design from each team leader and other key participants in the project.

Step 10:Constructing and commissioning. Regular team meetings should occur during construction and commissioning. Invite the various leaders and supervisors of all involved trades to these meetings. It is critical that the design team attends these meetings. Detailed status and problem reporting should occur to keep everyone apprised of the project, coordinate overlapping trades and provide a team atmosphere for the project. During this phase, be careful of scope creep and changes to the design. The same holistic principles should take place to resolve any design problems and construction conflicts that arise.

Step 11:Ongoing post-construction review. After the data center has gone live, it is important to continually review which techniques, technologies and strategies are performing as expected. For example, determine how successfully or unsuccessfully an energy-saving strategy is.

In reviewing, also consider what the team would do differently as the data center scales out or for the next data center build. Regular data center capacity discussion meetings are also important so that the next phases of data center construction are properly planned. These same meetings should include all of the internal team members and select outside team members as necessary. Discussion should also include status regarding any new business initiatives that can impact the data center.

The costs of building and operating a data center can quickly skyrocket if proper analysis and refinement by various bus- iness units and outside expertise are not made part of the design process. Using a holistic approach to data center design takes guesswork out of trying to determine the requirements, provides construction and operational cost savings, and offers a true sense of involve-ment to all players who directly or indirectly use the services of the data center. ::

Michael Salvador is technical solutions manager for data centers with Belden (www.belden.com).