DataCenter 101 - Documenting DataCenter - Make it Visible

By now we had an Overview of DataCenter; we learnt about Racks, Cables and its labeling too. We will go in-depth about few concepts for Power, Cooling, and HVAC in future along with learning technologies like Blade Enclosures, Virtualization and SAN Storage. But first, since now you might notices what you have in your datacenter, lets capture it first. Documenting your datacenter is the most important phase in your datacenter deployment. It’s important when it comes to Managing your Datacenter. 

I have seen people capturing Server Implementation Plans (SIP), Network Implementation Plans (NIP) and Standard Operation Procedures (SOP) for New Deployments, but where is the mapping of this information to actual physical deployments. You may an inventory available with you where you have your rack location against your server/storage/switch equipment, but can visualize your datacenter with that? In a way that I give you the information in excel sheet, and you are new to your company and something goes wrong; now I need your assistance to walk-in to the datacenter and check if the cables connections are fine for a particular setup; so you think you can do that with your current available information? Imagine the time you would lose in tracking it even though you have labels in your datacenter. You know time=$$$ right!! So I suggest, invest sometime in making it how you visual it so that you don’t have single point of failure in absence of people who deployed it.

I believe you are convinced by now that why you need to create datacenter documentation. So let’s talk about what need to be captured (suggested) in this document and what you need for that.

To Be Captured:

1.       Overview: It’s like a summary of your datacenter, how it looks. I prefer to mention who designed the document and who owns the document along with who reviewed and approved the document as important information to be mentioned in the beginning itself. It’s give a level of confidence on the document. Like I mentioned, this shows how it looks don’t jumble the position of equipment, racks and their location within. It should be like a photo as it looks.

2.       Power: I believe it’s a good practice to mention the current operating power and max requirement specification based on engineering design. You would need assistance of your local maintenance guyz to measure operating power in peak hours. Suggest you to take max noted in one week or you can take average as well. As per my experience, usually 3 phased power is used in a datacenter. In this example, R-Red, B-Blue and Y-Yellow is named as three phases and its consumption across a rack. You need to notice on the total power available in a circuit and its consumption.

3.       Network LAN: Next comes the Network connections, ISL and bandwidth usage presentation. I would suggest to make one more sheet to represent the IP subnet used in individual racks along with number of ports. This comes handy when doing a root cause analysis in case of network break-down.

4.       Storage SAN: Storage connections are usually tricky and gets messy if your datacenter goes scalable on-demand without proper planning. It is suggested to give a visualization of how storage connections are made across racks and then a logical presentation of ISL connections. Its comes handy when troubleshooting performance issues.

5.       Server Connections: I always to show case a cable connection within a rack and it should be pasted or available as reference material with critical deployments. Imagine you had to remove cable connection for maintenance or replacement of any components and cables get swapped? This is suggested for deployments in a rack.

6.       Setup Configuration: This is a brick level logical presentation of your deployments; where you map how your physical infra (Server, Storage, Network and most important Applications) exist in different layers. I have seen this in Server Implementation Plans (SIP) but not sure if everyone provide the detailed information about the deployments every-time like physical rack information, Management IP etc. Sharing some examples below:

7.       Contact Information (Disaster Run-Book): This section should always come either first or last, preferred last though. This should mention the on-call IT team members name and contact information along with respective areas of expertise like Windows, VMware, Server, Storage, Network or domain of ownership like Power, Cooling, Space etc. This comes handy and becomes a reference in case of Disaster. You can also mention Vendor support information like Contract IDs, Support Numbers, Account Managers Info etc so that you don’t have to search it everytime.

What You Need:

8.  Visio: When it comes to tools, Microsoft Visio is the best one I have seen. You would need stencils to give a real picture of your datacenter. Generic ones should be avoided or should be used only for logical explanations like routing/firewall setups, but what if you need to show actual physical connections? I have seen many usual stencils available on Visio-cafe (ask Google); rest you can ask the vendor for it, they usually provide it as free of cost. Note that there are certain open-source tools available, but they may or may not give the precision to show-case your datacenter. I may be biased, but I am just a fan of Visio.

9.   Information: Apart from tools information is most important. It may be available like a document for setups or like an excel sheet for inventory. Make sure you validate the inventory before you actually deploy it. Most important, you need to make sure this information is kept up to date.

10. Owner, Reviewers & Approvers: To make sure you have the proper information and its mapped as is; Get it reviewed from your peers (Important). See if they have any comment or suggestion or doubts before you actually publish it. What’s the use of it, if they don’t understand it. Like I mentioned before, its less technical document but more logical document. Owners are required to keep information up to date and approvers are required so that this can be use a standard reference documents and can be published.

Special Notes:

There are couple of tools available which easy your work for making automatic diagram via environment discovery like HP SAN Visibility (free), VMware Maps (free) and some other third party tools. However, you would need to consider the pros and cons that comes with these tools like support limitation, cost, credentials or access requirements etc. Adding to it no tool shows you rack placement.

Any more questions? please write back or comment here. There are more things to share.. 

Request you to join my group on Facebook & LinkedIN with name "DataCenterPro" to get regular updates. I am also available on Tweeter as @_anubhavjain. Shortly I am going to launch my own YouTube channel for free training videos on different technologies as well. 

Happy Learning!!

P.S.: I understand the example was given using small deployments, but it may look tedious when it comes to bigger deployments. Yes! It will be, if done by one man army J suggested to divide the work with respective areas of expertise or domain to make it easy, which can be later complied together.

Free Backup Professional Certification: I am sure another 'TING' happened :)

My previous post about Free Cloud certification was a big hit and got viewers from all over the world. I realized that people are interested in free learning and certification as well, since they are the parameters on which industries tends to measure our skills (though personally I don't feel that's really important, Knowledge is all that matters). Anyways..

Well, Yes!! Its Another Free Certification!! For those of you who don't know this, A free Certification is offered by VEEAM, who is a leading brand for Virtual Machine backups. All it requires is for you to go through free videos by some experts like David Davis, Elias Khnaser, Eric Siebert, David Hill etc available on this Link

This is a good source for learning about backup technology too. However, if you think you already have knowledge about backups, you can directly appear for the Exam. So as you can see its not mandate to watch all videos or complete all video sessions, though its a good refresher to revise your basics. It has some 33 questions and you need to pass with 75%, which I felt was easy to crack. You can even test your existing knowledge with Quick Test 

Just follow the link:
http://www.backupacademy.com/

Any more questions? please write back or comment here. There are more things to share.. 

Request you to join my group on Facebook & LinkedIN with name "DataCenterPro" to get regular updates. I am also available on Tweeter as @_anubhavjain. Shortly I am going to launch my own YouTube channel for free training videos on different technologies as well. 

Happy Learning!!

DataCenter 101 - Labeling Your DataCenter

      Labeling of the data center is as important as Cabling in DataCenter, since deploying a DataCenter may be easy, however troubleshooting it can be a mess when you need to track the cables. A Label gives the information about the equipment. So lets discuss some points to remember about Labeling:

·      A Label should always have a clear font with bright background to make it readable.

·      Always make 2 labels for the equipment, so that it can be pasted on both rear & front side of the equipment, where one on of them should be pasted near its management port.

·      During cable connectivity between equipment, label the cable immediately on both ends, to avoid future hassle. Doing Labeling after deployment can be tedious and may require double effort in bigger infrastructure.

·      Labeling a cable (Power, LAN, FC, FCoE, FC ISL, LAN Uplinks) can be done before deploying the cable as well if location of the other equipment is too far. You can mention the equipment’s location along with other details over the label.

·      Hostname of the equipment (fixed) like of Switches, can be made in a format where we can track its location as well. This can help tracking switch in datacenter, rather maintaining a separate inventory for switch location.

Special Format Tips

·     

For Blade enclosure, mention its Hostname and Management IP in its label. Create two labels, so that it can be posted in both rear & front side of blade enclosure, which can help anyone to track the equipment.

·      Labels of cables going from a switch should point to the other end of the cable. In this scenario, always put two labels on each side of cable, one label should mention the device its connected to (switch name, IP), other should mention the location/port number# in the switch. The more details you will provide, the more easy it will be for anyone to walk-in datacenter & perform troubleshooting in case of any issues.

·      Label for a power socket should point to its respective transformer/phase say TXXPXX

·      Label for Storage should have its name, management IP and its WWN. All ports in storage should be labeled properly pointing to its target switch IP/Port as well. Cable labels can be in format of say SN-IP-PRT e.g. S30P24ISL, where it points to Switch30 Port 24, and it also says it’s a “ISL” port for FC. In case of Storage roles of a port can be “Mgmt”, “Host port”, “Disk Port” or “Remote Copy” and even “Misc” based on services available in array.

·         

A question might arise in mind that why do I need label since every equipment has its own serial number pasted over it. Well, as per my experience, it would be really difficult when you have to track server by its serial number specially in a time of crisis or server failure, which need to be troubleshoot locally or physically. You can keep a record where it shows you the location of server as its Floor#-Row#-Rack# e.g. 02F2407, however a rack may have many other equipment of equipment with same kind of configuration, look and feel. To keep things easy, its always good to have a label, so that whenever you have any issue, you can look into your database or records (can be an excel sheet), go to the location of the rack and looking at the label without wasting time searching a serial number, you can start your work.

Special Label

·      You can post “warnings” on server playing vital roles in datacenter or which are very critical as per its role internally or for the customer. You can also mention its “role” is required. These kind of labels are very handy especially when you are working in a R&D environment or Labs, where a user might be required to walk-in lab or perform some tasks as per project requirements.

·      You might be required to paste the “asset tag” bar code on the equipment as well.

·     You can even post a printout of the visio/logical architecture of connectivity in between equipment or keep it handy so that it would be easy to track the point of failure and resolve the issue ASAP.

·   In case of small project, a project can keep a visio diagram which shows the exact connectivity in data center for their equipment, which will have its port, IP, uplink, location etc mentioned in the diagram itself along with OS and other minute information like firmware versions as well. This can be helpful for R&D Labs or environments.

Pasting the Labels

·      Labels should be pasted on equipment where its visible and clear

·    While pasting the label on an equipment, one of the two label should be pasted near its management port.

·      Paste label on both rear & front side of the equipment.

·     For cables, pasting a label is tricky and should be done with Caution. If it’s a LAN Cable, Patch Cord or Power cable, it can be pasted completely along its length.

    Special Note:

·    You can use special equipment like label printing machines or also called as Label Printer, which gives you neat, laminated, formatted labels. Sizes available for labels can be 12MM & 24MM, however Color of the label may vary from vendor to vendor. You may Google it to see how it looks in case you haven’t seen one.

·      Usually most preferred colors are “White” & “Yellow” base with Black Ink.

·    12MM is used for cables, Switches, Rack mount server of space using up to 1U/2U, while 24MM will be used for equipment like Blade enclosure, Storage arrays etc, wherever we have equipment which consume space more than or equal to 4-8U.

·     You can also use Cable Numeric Markers for Patch Cord cables (see picture attached)

·     Keep the overview of data center handy. You can create Visio diagrams of each rack with its location and show exact or logical connectivity of equipment inside it. You can even mention about ISL/Uplinks going from or to the rack. 

 Any more questions? please write back or comment here. There are more things to share.. 

Request you to join my group on Facebook & LinkedIN with name "DataCenterPro" to get regular updates. I am also available on Tweeter as @_anubhavjain. Shortly I am going to launch my own YouTube channel for free training videos on different technologies as well. 

Happy Learning!!

DataCenter 101 – DataCenter Cabling Strategies

In my last article about Rack Fundamentals, we discussed about how the racks to be chosen for deployment. Once are racks are deployed, and equipment are stacked over it, we need to interconnect them via Cables. Cabling is the key activity when it comes to building of a Datacenter. It does take huge amount of efforts and need specialists to do this activity. It may sound cabling is just plug-in of cable from source to target ports, but is that it?

Designing your DataCenter and implementing its core components is most crucial part to handle when it comes to Managing your DataCenter. This includes all the data cables i.e. your LAN Network Cables (Copper, Fiber) or SAN cables (Copper, Fiber) and Power Distribution Cables.

I don’t think we need to go back to the history about when the cabling of DataCenter was taken into considerations. However, they do play a vital role to understand your Network Topology. Still, I would be giving you an overview just to refresh about the kind of cables we usually use in DataCenter.

Overview of Cable & Connectors:

·         Network: CAT5 of Category 5 is used for 100MB connection, while Cat 5e support 1G and Cat6a supports 10G connection. For Network LAN Copper Cables RJ45 is the common connector type, which  is supported for Data Speed max up to 1GBPS. You might have seen this with your Broadband connection or office Ethernet connection. For 10G LAN Network, Fiber channel cables are mostly used for connectivity. However, there are 10G Copper cables available, mostly seen used with blade enclosure modular switches. SFP+ are factory installed with 10G copper cables. Even for storage array connectivity within system i.e. from controller to JBOD, mostly 10G Copper SAS cables are used.

·         Storage: FC cables are used mostly with Storage SAN connections. The most commonly used are 2G/4G/8G/16G connections. For 1G/2G Yellow color, 2G/4G Orange color, 8G/10G Aqua colors while for 16G Dark Blue color cables are used. FC cables are further connected to respective speed SFP+, note that these SFP+ are backward speed compatible. However If a high speed cable is connected to lower speed SFP+ it will work in respective speed. I have also seen people using certain SFP+ which are used to connect RJ45 cable with 10G ports.

·         Power: For Power Distributions, usually 6A/12A cables are used with server, storage & Switches. 12A cable is mostly used with Blade enclosures, High End Servers or with an Enterprise Storage device. Just to recognize, 6A cables are with oval curve and usually seen along with Desktop computers as well, while 12A are rectangular in shape and is uniquely identified.

·         Misc Cables: These are equipment specific or management cables like Serial cables, KVM  or ILO cables etc

Now I am not getting into the specifications of each cables or the cables to be considered like LC/SC, OM1/2/3/4 etc,  that would be a  separate chapter all together, which I would cover when I will start discussing about Storage in future blogs. However if you are curious Wiki can help you. I am also not covering how to make these cables or how to  connect these cables. For this, I would be making a video shortly in my Video learning series.  

For Power Distribution:

·      Power from Large UPS or Transformers is taken and stepped down to receive correct voltage and distributed along panel mounted  in the columns

·      Power is then further distributed from columns to rack mounted power distribution units via Whip (to avoid raised floor expenses).

·      Each equipment will be then further wired to these PDU for connection based on their requirements.

·      Remember Amp X Voltage = Power; so as the Amp increases or decreased, the gauge of the wire needs to be increase or decrease to accommodate the change.

·      Considering the High Availability, always plug-in both the PDU of equipment to different phases or power feeds

Note: Load per rack or circuit load need to considered while wiring these hardware

Cabling Layout Architecture:

·      Remember the old school network topology you studied while learning about Networks and we learned how network can be designed with Star, Bus, Ring and Tree topology. Well, for data centers, a Star topology is used.

·      Many factors need to be considered while deciding a layout for power distributions e.g. size of the datacenter, kind of equipment, budgets etc

·      Usually power distribution panels are installed at the end of the row from where is further distributed across racks in that row. This approach is easy to manage and less expensive as well. Also, it breaks whole circuit into multiple points which avoids a bigger failure in case of any incident.

Data Cabling Practices:

·      Follow National Electric Code as Standard for deployment

·      Test cable before deployment.

·      Prefer Over-head Deployments with cabling trays; not suggest if constant change is expected since not scalable. Note they should have sufficient distance away from sprinklers.

·      Don’t  tie the cables with Rack

·      Don’t over tight the length of the cable, use the right length required. Don’t keep them lose either. Use cable guidance closets for the deployment. Cable guiding trays and their management devices reduces downtime by avoiding human error and over-heating.

·      Cables connected via Patch Panels installed over (every) rack. These patch panels are designed and deployed in initial phase of DataCenter deployment.

·      Cable back-bone in a rack  should be preferred non-scalable. However, it is a best practice to wire as much as possible with latest cable standard during initial deployment.

·      Cables should be located behind the  walls and should not be in open

·      Most important, once the cables are deployed, Label them. I would be writing a my next article regarding best practices for cabling.

Caution: Do consider the scenarios while designing when your patch cable will fail.

Any more questions? please write back or comment here. There are more things to share.. 

Request you to join my group on Facebook & LinkedIN with name "DataCenterPro" to get regular updates. I am also available on Tweeter as @_anubhavjain. Shortly I am going to launch my own YouTube channel for free training videos on different technologies as well. 

Happy Learning!!

Meet Your DataCenter - An Overview

There has been a buzz about Cloud computing, Virtualizations, etc, but whats behind all this? Its your DataCenter. The very first thing  you need to keep in mind is your DataCenter should be Scalable, flexible, adaptable and most important cost-effective. You need to plan even the unseen demands on resources like electrical power and cooling, as it can increase costs and lead to performance issues if not fulfilled or even outages.

The physical layer of datacenter is the base upon which all network, storage, servers resources sits, further which gives platforms to business critical applications or even platforms like virtualization or cloud computing.

DCPI or DataCenter Physical Infrastructure is a one layer below the computing later i.e. our servers, storage and network. It involves the following components:

·    Power: It consists of multiple components which make the power distribution possible across datacenter. These components are  - The Generators, Batteries, UPS, Serge Protectors, Transformers, Distribution Panels and Circuit Breakers. This is the only components which hasn’t changed much over decades, now I am not referring to the ways we generate them or monitor them but the components itself haven’t changed much. When originally the power distribution components were designed, their principles were not actually meant for IT infrastructures, which lead to the challenges as mentioned below. It is clear that why integration of components of the power system started to move towards pre-engineered and manufactured solutions like Modular and Scalable components.

·   Cooling: It involves the system which removes or overcomes the heat from the datacenter. This involves chillers, cooling towers, condensers, duct work, pumps, piping and rack level distributions. Just like Power, evening cooling system haven’t emerged or changed much over these years which lead to the same  challenges as faced in Power. There has been a change in how cooling was done in past few years, though this technologies have not been much adapted yet in market.

·    Racks: Racks are the cabinet which holds the equipment. These racks should be considered and designed during design phase of datacenter.  The old standards for Racks were IEC 297-3 for mechanical dimensions while EIA 310 for  rack mounting of Telecomm and IT equipment. These days mostly TIA941 has been considered as Standard. Mostly 42U racks are considered as standard and all equipment including server, storage or network are designed  to kept in standard racks.  A detailed article is already published on this blog as “Rack Fundamentals"

·      Physical Security/Fire Protections: This includes physical security devices at rack and datacenter level e.g. keys (commonly used), biometric, codes or access cards while Fire protection system includes smoke detectors, clean agents or heat detection sensors. These days they are mandate when it comes to standardizing the datacenter. NFPA (National Fire Protection Association) established in 1896, imposes standards, codes, research, training and education worldwide to reduce fire and other hazards scientifically. NFPA-75 is the standard for the protection of electronic computer or data processing equipment.

·        Cabling: This is most when it comes to designing and managing datacenter, this includes all data cables and power cables within datacenter. Some best practices for cabling are:

o   Overhead deployments to make sure there is

o   Underfoot deployments

o   Rack installation with guidance cables

o   Do not tie or hang cables

o   Testing of cables before installation

o   Most important, Label these cables to trace back.

o   Power cabling best practice are available I National Electric Code, under continuous load cables

o   Cables should not be undersized or stretched

o   Use cable raise rather preferring raised floors

 A detailed article is already published on this blog as “Cabling Strategies” 

·     Management: Managing the datacenter includes Building Management System (BMS), Network Management System (NMS), and Enterprise Management System (EMS), Element Managers and other software and hardware. A detailed article is already published on this blog as “Managing your Datacenter” in three parts

·         Services: To perform several operations in DataCenter, many services are required to support DCPI which includes – Consulting and design, Installation, Maintenance, Monitoring and Decommission services.

Integration of these aspects are taken care in the Design phase of the DataCenter. Usually, its all about a Plan, which would be Constructed and then Deployed. Be it any component in your computing layer, storage, server or network, they cannot work efficiently till your DCPI works fine. Adding above, there are three more important components which together with DCPI make things work for you. It’s the:

·    People: Well most Important you need to people which appropriate skills and levels of expertise to run the datacenter

·      Process: It includes the Standard Operation Procedures, which should be well documented as well. A detailed article is already published on this blog as “Documenting your Datacenter” 

·     Information Technology: which includes hardware and software used for data processing, storage and communication

Note that more detailed articles will be published on above topics, this was just an overview.  In my next article, we will start looking at individual components of DataCenter.

Any more questions? please write back or comment here. There are more things to share.. 

Request you to join my group on Facebook & LinkedIN with name "DataCenterPro" to get regular updates. I am also available on Tweeter as @_anubhavjain. Shortly I am going to launch my own YouTube channel for free training videos on different technologies as well. 

Happy Learning!!

DataCenter 101 – Rack Fundamentals

You might have heard a lot about “Rack-stack” while planning a new deployment, but did you got chance to have a view of it. You might see them everyday, but did you noticed what’s its type or what is missing or what can make it better while planning its life cycle? In my last article I provided an Overview of DataCenter. Now, Lets walk through DataCenter Racks.

Racks and Enclosures are the building blocks of DataCenter, which manage the real estate space inside them, where all equipment are kept. There are two types of enclosure used in datacenter.

1.       The 19 Inch Standard – Its designed based on EIA-30 standard to ensure the physical compatibility between  racks and equipment mounted in those racks. The is the mostly commonly used rack world wild. Every rack has size units as “U”, where 1U=1.75 inches. Usually 42U is the standard rack size. Dimensions for these racks  are : 17.72 inches gap between side rails or called as Enclosure opening  to mount equipment; while the width between the mounting holes is nearly 18.31 inches; thus the overall minimum width for front panel is nearly 19.03 inches. These racks have Square Nut mounting Rails which come along with the equipment.

2.       The Earth quake Standard – International Building Code  and Euro-code specifies how these racks should be bolted to floor where there is a high risk for earth quakes ; while NEBS and ETSI mentions the rack alignments enforcement for these racks.

Based on type, racks can be further divided into two categories:

1.       Open Frame Rack – Two Post Frame or called as Rely racks holds equipment which can be mounted front or center. It is used mainly for lightweight low cost equipment like switches; however they offer No security, No airflow control, low weight capacity and low stability. Four Post Frames give support for both Back and Front mounted equipment, which makes them more versatile. These are used for servers, storage and network equipment. It is physically stronger and hence can support heavy equipment.

2.       Enclosures: These are advanced rack containment system with front and back doors, side panels and a roof. These advance racks provide forced air flow and hence enhanced air cooling capability. These are mostly used as Server Enclosures. In past years, they have been getting deeper to support dense equipment. Though usual size is 42U, but they can be as tall as 48U, however while deploying 48U racks you need consider overhead equipment like fire safety sprinklers or cable guidance. Network enclosures are usually 48U since they need wide space for cables and master switches, which can accommodate upto 2000 cables approx.

  

3.       Wall Mount Enclosures: Now there is a special category of enclosure, small in size, which conserves floor space and provide neat installations. I haven’t seen them in practical use though, still counts as a category.

Note: Non-standardized racks can lead to high costs since they may not accommodate the equipment designed for standard rack. It is always suggested to buy Vendor Neutral Standard racks, which comes with guarantee to support all vendor equipment.

Rack Accessories: Based on the requirements, many rack accessories can be deployed along with rack, which includes KVM monitors, Cable Management kits, PDUs, Cooling fans etc.

Floor Plan

While designing your Racks, you need to think about Availability. In-sufficient air flow or power can damage your equipment. Even if the cables are stretched, they can get damage or unplugged. Remember your floor plan and rack spacing is most important while designing your datacenter. It take cares of the HVAC (Heating Ventilation and Air Conditioning). (image found from google in one of the blog, using it to explain). However, make sure your don’t compromise on the physical security of the rack access. Cold asile should be 4 ft wide while hot should be 3 ft.

How to do Effective Air Flow in Racks:

• Use blank panels to prevent heat exhaust from being recirculated. So Don’t through your blank panels, many people fail to realize their importance.
• Use of Net doors in back and front panel, though they come with design now
• Implementation of Air Distribution Units (ADU) which are commonly mounted at the bottom of the rack are designed by EIA-310 19 inch rack standard. These units pulls the cold air and supply them directly to the rack.
• Implementation of Side Air distribution Units, which gets installed above the rack, provides side-to-side air flow.
• One more scalable solution available is Air Removal Units, which just removes the hot air coming from the equipment.

PThats all I had to share about the Racks, I hope after reading this article you would be visiting your Server rooms, datacenter or labs to see what kind of equipment do you use. 

Any more questions? please write back or comment here. There are more things to share.. 

Request you to join my group on Facebook & LinkedIN with name "DataCenterPro" to get regular updates. I am also available on Tweeter as @_anubhavjain. Shortly I am going to launch my own YouTube channel for free training videos on different technologies as well. 

Happy Learning!!