This data center site assessment was prepared for Adobe Systems in 2006, presented as delivered.
ADOBE
Data Center Hosting Site Assessment
June 8, 2006
Presented by: Mark Nichols, Global Telecom Solutions
Assessment of Sterling Network Systems Data Center (SNSDC)
Phoenix, Arizona
Table of Contents
Executive Summary
- 1.0 — Scope of Assessment, General Observations, and Limitations
- 2.0 — Environmental Management
- 3.0 — Power
- 4.0 — Security
- 5.0 — Telecom Infrastructure
- 6.0 — Internal Infrastructure
- 7.0 — Support Services
- 8.0 — Site Maintenance and Certification
- 9.0 — Building Ownership, Management and Overview
- 10.0 — Conclusion
Appendix
- Exhibit 1 — Location Map
- Exhibit 2 — Scalable Cooling of North Wind Project
- Exhibit 3 — Natural Disaster Risk Locations
- Exhibit 4 — Electromagnetic Diagram
- Exhibit 5 — SNS Data Center Security Policies and Procedures Overview
- Exhibit 6 — Daily Maintenance Checks
- Exhibit 7 — Carriers Available
- Exhibit 8 — Published Cross-Connection Form
- Exhibit 9 — Remote Hands – Layer One
- Exhibit 10 — Operations and Maintenance Schedule
- Exhibit 11 — About SNS
- Exhibit 12 — Phoenix, AZ, “The Ideal Location for Your Enterprise Data Center”
- Exhibit 13 — Shipping Policy
- Exhibit 14 — Conference and DR Room Policy
Executive Summary
This report contains an assessment of the capabilities of the Sterling Network Services Data Center (the “Site”) located in Phoenix, Arizona to determine if the Site can meet Adobe’s requirements and is suitable for housing Adobe’s technology platforms.
The report was developed by:
- Physical inspection of the Site
- In-person interviews of the Site management team
- Investigation of Site vulnerability as reported by government agencies
During our two day physical inspection of the Site, we reviewed the following areas of services, and the design parameters to which Sterling builds out a Customer’s individual Data Center Suites. The following are the highlights of the key service areas reviewed:
- Environmental Management: The critical systems for air conditioning designed with N+1 redundancy and diverse fail-over in event of interruption. Fire suppression is the most commonly preferred system using dry pipe, multi-zone water sprinklers with VESDA sensors, and dual event activation and isolation. Leakage control and particulate management were found to be consistent with industry best practices.
- Power: The Site is located on the same electrical grid serving key government buildings as well as the operations center of the commercial power service provider’s headquarters (APS). In the unlikely event of commercial power interruption to Site, uninterruptible power generation is engineered in a N+1 topography and service delivery can be maintained for a minimum of 72 hours of run time before refueling diesel storage tanks is required.
- Security: The facility is managed with a 24×7 on-site Security team with strict policies and procedures for containing visitors to their designated spaces. The use of CCTV, electronic keys and biometric scanning devices are used throughout the Site. Additional customer-specific security apparatus can be implemented with approval by Sterling management.
- Telecom Infrastructure: The Site has access to the largest telecommunications carriers in the United States, including nine of the top ten best peered domestic internet backbones. The existing fiber capacity from telecommunications service providers appears to greatly exceed current Adobe requirements.
- Internal Infrastructure: The Site is constructed of re-enforced concrete cast in place walls. No additional seismic bracing has been added to the Site as risk of seismic activity is considered low. However, we recommend that data cabinets be anchored to the raised floor as a precaution.
- Support Services: Sterling currently provides visual inspections, mounting and installation of hardware and powering machines as part of its service offering. Given that Adobe plans to have on-site employees to maintain Adobe systems, the Site support services provided by Sterling should be adequate.
- Ownership and Management Team: The Site is run by George Slessman, the Site General Manager. Based on our three hour interview, Mr. Slessman has developed an atmosphere in the facility of a dedication and professionalism.
With regards to the contract terms by and between Adobe and Sterling, we suggest that as part of the agreement that the Suite design parameters and specifications be identified in an attachment and that Sterling be bound to provide the Services consistent with or better than the stated parameters and specifications.
Our overall evaluation of the Site is consistent with industry best practices for high availability data centers and Telco central office standards. Based on the visit to the subject property and our discussion with Sterling staff, there are no indications that the property cannot support Adobe’s business needs.
1.0 — Scope of Assessment, General Observations, and Limitations
This report describes an assessment of the capabilities/attributes of the Sterling Network Services (“Sterling”) Data Center (SNSDC, hereinafter referred to as “the Site”) located in Phoenix, Arizona (see Exhibit 1: Location Map), and determine whether it meets Adobe’s requirements and is suitable for Adobe’s technology platform.
The due diligence and data collection activities to support this report were performed by Mark Nichols and Michael Flaherty and included a two day physical review of the Site on May 15–16, 2006. This assessment is not an independent verification of the information collected; i.e., statements made by Sterling were assumed to be factual and physical systems observed were assumed to be operational.
During the two day visit the following general observations were made:
- All Sterling staff members observed were professional and courteous
- Customer Data Center Suites were clean and free of debris
- Shared Customer work spaces were clean and free of debris
- Movement of mechanics and trade craftsman was orderly
With respect to the physical infrastructure, the commercial relationship is a lease of space with a set of environmental specifications that are provided on a service level basis. By way of example: a customer of Sterling is not purchasing/renting a 750 kilowatt generator for electrical power back-up, but is rather leasing 5000 square feet of data center space with 100 watts of power per square foot with 100% availability. Given the above, references or statements of system specifications may apply to typical configurations as opposed to the exact configuration that would support the Adobe space.
Sterling states that performance to SLAs will be represented and warranted in the agreement and that Sterling will stipulate to a design specification in the agreement as a contractual obligation with right to provide an equal to or better than specified design throughout the life of the agreement. Archstone Consulting recommends that Adobe ensure that the stipulation of the design specification is a provision in the agreement.
Sterling did not fulfill all requests for information made by Archstone Consulting. The unfulfilled data requests were either determined by Sterling to be too invasive or burdensome to fulfill. These generally related to security items or actual maintenance logs and reports related to system maintenance or site operations.
2.0 — Environmental Management
Sterling has implemented telecom carrier grade or Tier 4 (Uptime Institute) class climate control, fire suppression and leak detection facilities to protect customer assets. Telecom carrier grade (Central Office grade) systems are considered to be the most highly reliable and available systems standards.
Climate Control
Climate control in the individual customer data center suites (hereinafter referred to as “Suite”) at the Site is provided by a dual loop chilled water plant. The water plant consists of three (3) rooftop chillers manufactured by Marley (recently acquired by STX Corp.) and York (recently acquired by Johnson Controls). The roof top chillers are arrayed in redundant paths (N+1 design) providing chilled water to the computer room air conditioning (CRAC) units within each Suite for heat removal and air flow.
Servicing the chiller plant are primary and secondary pumps (N+1 design) that circulate through two (2) diverse closed loop supply and return water systems connected to two (2) 50,000-gallon chilled water storage tanks. In the event of service interruption to either of the two main chillers, the chilled water tanks have an estimated 20 minutes of chilled water capacity to cover the time required to bring the redundant chiller on-line.
The closed loop system and on-site chilled water storage maintain chilled water capacity to satisfy 72 hours of operations. In the case of the utility fed water supply being interrupted, the Site has external water refilling capability and supply contracts from multiple contractors that allow for indefinite operation.
Though not in production at this time, there are two (2) twenty-four inch (24″) chilled water feeders entering the Site on the mezzanine floor (one level below ground) that are intended to deliver additional chilled water from an Arizona Power Supply (APS) municipal project called “North Wind” (see Exhibit 2: Scalable Cooling of North Wind Project).
Within each data center zone temperature control is provided by diverse and N+1 redundant Liebert (recently acquired by Emerson) or Stulz CRACs being remotely monitored with Netbotz remote power management (RPM) and power quality management (PQM) systems. Each of the Suites are managed to maintain ambient temperatures between sixty-eight and seventy-two degrees (68–72) Fahrenheit. Space below the twenty-four inch (24″) raised floor is reserved for cooling and ventilation air flow. In the event that additional data center ventilation or cooling is required to lower ambient temperature levels, perforated tiles are added on an as needed basis. To maintain consistency of Suite temperature and cooling efficiency, Sterling performs a “hot spot” inspection with infrared testing equipment every 90 days around all conduits, electrical panels, and PDUs. The infrared testing is also a general preventive maintenance process as excessive heat is a precursor to general device failure.
Suites are maintained at 38–50% relative humidity to mitigate static. Static is often caused by equipment in operation within the data center when humidity is between 30–35%.
Fire Suppression
Fire suppression throughout the Site (sans basement fuel storage which is via non-toxic foam) is provided by a dual-alarmed, dual inter-locking and multi-zoned, dry-pipe, popped-head, water-based system. Fire suppression activation occurs when at least two of three monitored elements cause an alarm. The monitored elements are: heat at sprinkler head (190 degree trip), Forward Looking Infrared (FLIR) detection, and Very Early Smoke Detection (VESDA which is a particulate sensory system), and sensors that sample the air (including inside ventilation plenums). VESDA sensors were observed in all Suites visited. Additionally, fire suppression can be triggered by manually pulled alarms.
For the fire suppression system to trip, multiple and at least two (2) cross-linked events (“symptom detections”) must occur. Fire suppression is localized at the event point only. Currently, there are eleven (11) fire zones within the Site which includes one for each Suite. Any water discharge will be sprinkler-head specific, which will limit the potential for damage caused by over-spray. The fire suppression system may also be activated and de-activated manually. Manual alarm pull tabs were oddly missing from view within the telecom “Meet-Me” room and this observation was brought to the attention of Sterling.
In the event of a fire alarm, Sterling operations staff will respond within two to four minutes depending on the exact location of the event. In the event an emergency response is required by the City of Phoenix Fire Department, Battalion Unit 1 would respond and is 1.27 miles from the Site. Battalion Unit 1 consists of five (5) additional stations in the area.
The Site facilities engineer is the dedicated liaison to the City of Phoenix Fire Department.
Leakage Control and Detection
The building is of concrete floor construction and all floors are waterproof sealed to prevent water damage as well as release of particulates. All systems (such as CRACs and Generators) which contain liquids throughout the Site are placed on collection trays with rope leakage detection systems. Additionally, rope leakage detection is placed under the raised floors. In the event of detection of any fluids, the leak detection system will sound an alarm to the network operations team (NOC) so that the issue can be addressed. In the cases where water is detected from a detector within proximity to a critical system, the leak detection system will shut down the offending mechanism in addition to sounding an alarm to the NOC.
The roof was resurfaced with a twenty (20) year system in 1998. There are no vertical penetrations of the roof. All roof penetrations are covered with housings and actual conduits that penetrate the roof start horizontal and then turn vertical prohibiting direct precipitation in any conduit system.
Telecom cable vaults are waterproofed sealed from exterior penetration and contain a drain as a safeguard.
Particulate Management
The Site is constructed primarily of steel-reinforced concrete. Concrete is known to flake and release particulate dust into the environment. Suites have been bead blasted and sealed to mitigate the inherent potential contaminant of the building material.
Contaminations are also introduced by foot traffic from customers, contractors and operations persons who enter a Suite. Each Suite has Tacky Mats® installed at the entrances to capture contaminants and dirt that could be imported into the Suite from shoes and wheeled carts. All tacky mats are replaced every six (6) hours during the “Ops Walk”.
Particulates can also be introduced by cardboard packaging containers. During the Ops Walk, particulate matter is policed and Suite tenants are notified in writing of any refuse materials that could introduce particulates.
Disasters and Other Risks
Most natural disasters are related to weather patterns (tornado, cyclone, hurricane, storms and flood), but some are geological (volcano, earthquake, tsunami and tidal wave). Given Arizona’s distance from the Atlantic and Gulf coasts, Phoenix is virtually immune to hurricanes. While hurricanes also form in the Pacific off the coast of Mexico, these storms usually travel from east to west, away from the United States. Occasionally, such storms curve back to the northeast — in 2003, Hurricane Marty brought tropical storm force gusts and flooding to parts of Arizona after moving up the Gulf of Mexico — but such events are rare.
According to the Federal Emergency Management Agency (FEMA), during the last ten (10) years, flooding and wild fires were the sole causes of the Major Disaster Declarations within the state of Arizona. Due to the Site’s inherent metropolitan location, the risk of wild fire will not be addressed.
The closest flood plane to the Site is approximately nine (9) miles away and originates from the Salt River. The last recorded flood of the Salt River impacting Phoenix was in 1965. Total dollar cost was estimated at $10 million; however damage to the Site specific location is unknown. The recurrence interval is rated at 10–50 years (U.S. Geological Survey Water-Supply Paper 2375, 591 p.). Flood events, by default, have unforeseen levels of impact. The Site does have subterranean critical systems infrastructure that could be impacted by the volume and speed of flood waters.
In the last 100 years, there have been only two earthquakes with epicenters in Arizona with magnitudes greater than 5.0 — a magnitude 5.7 earthquake in 1992 and a magnitude 5.3 event in 1993 — both of which occurred in the northern part of the state. In 1852, a magnitude 7.0 quake occurred on the California–Arizona border near Yuma, though there is some doubt about both the exact epicenter and magnitude since it was not recorded using instruments. While performing due diligence in the public domain, the following evaluation was discovered and an excerpt follows:
PHOENIX COMMUNITY EARTHQUAKE HAZARD EVALUATION
MARICOPA COUNTY, June 13, 1994Prepared For: STATE OF ARIZONA
Department of Emergency and Military Affairs
DIVISION OF EMERGENCY MANAGEMENT
FEMA/NEHRP
Federal Emergency Management Agency
Cooperative Agreement No. AZ102EPSAThe rate of seismic activity in the Phoenix area has historically been very low. However, the Phoenix area is affected by earthquakes occurring in northern Mexico or California. The most recent earthquakes to occur in the Phoenix area were located in the Cave Creek in 1974 (M 2.5 and 3.0). The magnitude 7.2 northern Sonoran earthquake of 1887 resulted in ground shaking in the Phoenix area that triggered rock falls. Growth of urban areas has spread population onto alluvial plains steep slopes that are often the most hazardous areas in earthquakes. A repeat of the 1887 Sonoran earthquake results in greater damage to Arizona’s population. The Phoenix area is in the Basin and Range geologic province where reactivation of faults has with devastating results in the past (M 7.2, Mexico). The nearest mapped potentially active faults to the Phoenix area are the Sugarloaf and faults located about 40 to 43 miles northeast of Phoenix, respectively. The largest credible earthquakes that could occur on these faults are about magnitude 6.75. The earthquake hazard to the Phoenix area is significantly lower than other portions of the state such as Yuma or the Flagstaff–Grand Canyon region. However, because of these risks, Arizona is now designated by the Emergency Management Agency National Earthquake Hazards Reduction Program as a “High Risk” state for earthquakes.
The earthquake hazard to the Phoenix area is significantly lower than other portions of the state such as Yuma or the Flagstaff–Grand Canyon region. However, because of these risks, Arizona is now designated by the Federal Emergency Management Agency National Earthquake Hazards Reduction Program as a “High Risk” state for earthquakes.
Sterling provided a document during the visit entitled Natural Disaster Risk Locations (see Exhibit 3: Natural Disaster Risk Locations) that outlines the minimal natural disaster risks in the Phoenix area. The document was created by ComputerSite Engineering Inc. Because of the low risk of seismic damage as perceived by Sterling, no additional seismic safeguards beyond the original building design have been deployed. Sterling stated that many Customers choose not to anchor cabinets within their Suites. However, as a precaution, we recommend that Adobe anchor cabinets.
The largest natural hazard facing Phoenix is thunderstorms. Of the perils associated with severe thunderstorms, lightning mitigation (lightning occurs in all thunderstorms) is necessary to protect electrical systems. Sterling has installed lightning rods on the rooftop that terminate at the Grounding Tree located in the basement to neutralize the impact of lightning strikes.
Given the relative low risk of natural disasters, terrorism is probably the highest risk to the Site as well as to the other numerous high-technology companies in and around Phoenix. Still, when compared with other major metropolitan areas, such as New York, San Francisco, Chicago or Washington D.C., the relative threat of terrorism is quite low.
Electromagnetic Interference
Electromagnetic waves, which destroy or cause malfunctions of systems by transmitting strong electromagnetic waves from the area around a data center, are now considered an important operational issue. Inherent Site EMI mitigation are the two (2) interior eighteen to twenty-four inch (18″–24″), steel reinforced, concrete walls (dividing the building into four (4) vertical quadrants separated by floor plates) that break-up electromagnetic frequency (EMF). Sterling does not allow mechanical or electrical systems to cross paths in building conduit and risers, thus eliminating origins of EMI from poor systems topography. Sterling performs harmonic testing quarterly to ensure EMF levels are within range of Power Quality Management (PQM) guidelines. Sterling provided a diagram to identify EMI mediation (see Exhibit 4: Electromagnetic Diagram).
Electromagnetic security measures are important to protect electronic devices from eavesdropping or electromagnetic frequency (EMF) attacks. At this time, the Site is not hardened against EMF vulnerabilities.
Building Permit Process
For Site and Suite improvements, Sterling maintains an annual “open” building permit that allows for immediate building and tenant improvements under $100k. By maintaining the open permit, Sterling can commence construction at will and follow-up with the regulatory approval and inspection processes in arrears.
3.0 — Power
Commercial Power Feed
The primary source of power to the Site is provided by the Arizona Power Service (APS), a subsidiary of Pinnacle West Capital Management Corporation (NYSE:PNW), from the Palo Verde Nuclear Generating Station (PVNGS) located 55 miles west of Phoenix. According to the APS website, the PVNGS has been the largest power producer of any kind in the United States since 1992. The PVNGS is operated by APS and is owned by a consortium of seven utilities in the southwest. APS owns a 29% stake in the enterprise. According to the Pinnacle West website, Pinnacle is expected to invest nearly $5 billion in new capital during the next five (5) years to meet the future needs of the second-fastest growing region in the country.
The Site’s electrical service consists of three (3), N+1 redundant 2,000 kVA 480-volt transformers providing 16,000 amps of electrical service. Physical entrance of power to the Site is serviced by two (2) diverse and dedicated subterranean points of entry (the North and East vaults) connected to the APS “SPOT” grid fed by multiple sub-stations and has a history of zero outages. The SPOT grid is represented by Sterling to be the most reliable grid available from APS. Since September 11th, 2001, APS will not release specific information about the grid configuration. However per Sterling, the grid is constructed in a loop fashion with five substations and only three substations are required to operate to maintain the grid’s nominal operation. Additionally, Sterling stated that only seven (7) buildings are currently on the grid and these include the State Capital, State Supreme Courthouse, Federal Building, and APS Command Center for Nuclear Power. Sterling has a third diverse and dedicated subterranean point of entry (the South vault) with an estimated 10,700 amp capability planned for operational availability in the first quarter (Q1) 2007.
An alternate source of commercial power is not available as State of Arizona utility regulations do not permit a secondary provider in a single market.
Uninterruptible Power Supply
The Site’s electrical service is connected to redundant automatic transfer switches (ATS) manufactured and maintained by Circle D. The ATS equipment is connected to an architecture of N+1 redundant uninterruptible power systems (UPS) manufactured and maintained by Liebert and MGE. Adjacent to the UPS are battery strings for short term stand by power until the generators can be activated and brought on-line. Inspection of the battery strings revealed the batteries were in new or like new condition without any visible corrosion or stress. SNSDC represented that the batteries are replaced within 24 months as per the manufacturer’s suggested specifications. The UPS servicing each customer Suite can be delivered in either an N+2 or an N+1 power distribution unit (PDU) architecture. PDU capacity may be negotiated with each Customer to meet and or exceed the service level agreement (SLA). Battery power is designed to provide a minimum of five minutes of power at maximum per square foot output including environmental systems for the Suite.
The UPS is supported by redundant on premise diesel generators from 750kW to 1.5MW dedicated to each Suite. The generators are sized to support all ensuite power requirements including all environmental and security systems within the Suite. Additionally, the same type of UPS system exists to support the Site’s command, control, security and environmental systems. Additionally, Sterling stated that the generators are interconnected in an N+1 redundancy architecture so if necessary one Suite’s generators can support various other Suites.
In the event that additional power is required to support the Site there are electrical “taps” for street level portable generator remote power feeds. The “taps” are networked throughout the Site electrical systems in a distributed architecture to all generators within each data center location.
Generators are maintained with an internal temperature of approximately 120 degrees Fahrenheit for system readiness. The “cold” (admittedly an oxymoron as the generators are maintained by warming devices) start specification to online power generation is within 45 seconds.
The Site is licensed for 80,000 gallons of fuel storage though only 20,000–25,000 gallons is maintained at the Site in individual 2,000 gallon tanks. Due to the inherent degradation of diesel fuel over time, Sterling either burns or recycles the fuel bi-weekly through testing. Sterling stated that 20,000+ gallons of fuel will maintain 72 hours at full run capacity for all generators. The current practice is to only run the generators at 50% of capacity, thus doubling run time. Fuel refilling is contracted with two (2) fuel delivery providers; one is Giant Diesel which has data center facilities at the Site. The storage tanks can be refilled via two (2) diverse exterior fuel tubes.
A request to review the fuel storage area was denied for security reasons. Sterling represented that the fuel is maintained in the basement of the facility and access to the basement is highly restricted to protect the facility from distribution of knowledge that would be perceived to be valuable to terrorists.
Customer Hand-off
Power is conditioned via power distribution units (PDUs) located within dedicated customer areas. The PDUs are deployed in an N+1 architecture and power is available up to 275 watts.
DC Power Optional
DC power is also available based on specific customer requirements. Sterling currently has an inventory of DC power converters on-premise for customer installations. During the inspection, DC power was viewed as being provided to a minority of customers.
Grounding System
Due to the basement security limitations, we were not allowed access to inspect the Site grounding system. Sterling stated the ground system was designed by and procured from Lucent Technologies and that it is a “branch tree” design and is the single grounding system for the entire site with tested resistivity of less than 3 ohms. The core of the design is a 60′ tungsten rod driven to the bedrock. Due to the dry soil conditions of Arizona, the ground rod is surrounded by a slurry material that is then kept moistened to maintain the integrity of the electrical ground. Sterling requires that all customers be connected to this system. During our inspection, the grounding system was prevalent throughout the data center facility and encased exclusively in highly visible and distinctive green shielding.
Simulation Testing
To test the overall site integrity, Sterling conducts power failure simulations. Customers are provided seven (7) days prior written notice for all power failure tests. Sterling conducts the following tests on regularly scheduled intervals:
- Each Suite UPS system is tested every ninety (90) days via simulated interruption of commercial power
- An annual primary power building power failure for the entire facility which simulates total loss of commercial power to the Site
- Each generator is tested bi-weekly for 4 hours
4.0 — Security
The Site is constructed with poured in place twenty four inch concrete walls. The exterior building has simulated windows for aesthetic reasons. No signage other than the address exists indicating the building’s purpose. Behind each window is the building concrete wall. There is no direct line of sight from the exterior of the building to any office or Suite space. All exterior doors are locked and require card key access (even to the visitor’s lobby) or entry to be granted by security. The exterior portions of the building which are considered sensitive (such as telco cable vaults including portions of the roof) are surveilled by security cameras including the two steel roll up gates for delivery sallies. When any delivery sally is opened it is manned at all times by security. The south delivery sally is due to be sealed later this year. Inside the north delivery sally are concrete vehicle barriers to prevent a vehicle from penetrating the interior of the building and compromising security. All street manhole covers which are exclusive to the building and under Sterling’s control are tack welded in place.
Visitors to the building are required to show a governmental picture identification or current employer identification. Contractors are required to show a valid work order for a current customer. All visitors are provided with blue bracelets signifying escort required and contractors with appropriate rights are given yellow bracelets. Customers may badge their independent contractors as employees if they so choose provided that the Site screening protocol is followed.
Entry beyond the visitor’s lobby requires electronic card key access and all elevator operation requires card key access. Entry into Suites or other sensitive areas requires both card key and finger print scan. Entrance to Suites and other sensitive areas are surveilled by video cameras and certain areas are monitored with motion detection systems (see Exhibit 5: SNS Data Center Security Policies and Procedures).
The entire interior of the building is “walked” every six hours by operations personnel and a checklist is followed to document various readings and conditions throughout the building (see Exhibit 6: Daily Maintenance Checks).
5.0 — Telecom Infrastructure
The SNSDC has network neutral access to almost every major telecommunications carrier in the United States, including nine of the top ten best peered domestic internet backbones (see Exhibit 7: Carriers Available). Sterling stated that all major IXCs have designated the “Meet Me Telco Room” as a Point of Presence thereby reducing Customer “local loop” charges and eliminating a typical single point of failure. Sterling did indicate some difficulty with AT&T regarding the POP designation.
Sterling provides cross-connections from the telco Meet Me room to the Customer’s Suite within 48 hours of receipt of “CrossConnection Form” (see Exhibit 8: Published Cross Connection Form). The standard SNSDC cross-connection service includes dedicated cabling, testing of the physical layer, tagging of the circuit and termination to designated customer premise equipment (CPE).
Sterling stated that the following major carriers were present in the building:
- AT&T (legacy)
- Sprint
- MCI (legacy)
- Qwest
- SBC (legacy)
- Level3
- WilTel
- Cox Business Services
- Internap
- Global Crossing
- Citizens / ELI
- Time Warner Telecom
- XO Communications
- AboveNet
- Broadwing
- Cogent
- ON Fiber
- Telesphere Networks
- Systems Solutions
The following dark fiber providers are also present:
- AGL Networks
- SRP Telecom
- AboveNet
- Level 3
- WilTel
- 360 Networks
SNSDC stated that each carrier has between a dozen to several dozen fiber strands entering the building in a ring architecture through the diverse entrance facilities. A review of the telco entrance vaults was not permitted for security reasons; however the diverse fiber access conduits entering the Meet Me room were clearly identifiable. The fiber optic cables entering the Meet Me room were shielded within armored casing and delivered high above-head to secured carrier cages.
During the inspection of the telecommunications “Meet-Me” room, the carrier infrastructure present (though unmarked) contributed to the impression that the statements regarding carrier capacity were consistent with physical communications systems on premise.
Sterling indicated that Nextel wireless access is poor in the Site. Considering the ubiquitous reliance on Nextel for operations team communications within the communications industry, this obvious oversight should be remedied. Additionally, cellular service from T-Mobile was rated at only one (1) bar out of a possible five (5) inside the Site conference room located on street level. Customers considering use of cellular technology for emergency communications should create a work around for the apparently limited wireless communications in advance of tenancy.
Site rooftop communications appear to support alternate methods of fixed wire line services. Sterling stated that the building is zoned and municipally permitted for satellite, microwave and radio antennae and supporting rooftop hardware. An eighty (80) foot radio tower is mounted on the rooftop currently and appears to be able to facilitate point-to-point dish technology. There is abundant available “real estate” on the roof top to assemble a large satellite earth station with an unobstructed line of sight to the southern hemisphere.
6.0 — Internal Infrastructure
The Site is located on a complete municipal block consisting of an approximate foot print of 80,000 square feet (SF). The Site consists of five (5) equal floor plates for a cumulative Site size of 400,000 square feet (SF). The Site topography includes two floors that are below grade level (basement and mezzanine floors), a street level floor (Floor One), and two floors that are above street level (Floors Two and Three).
Sterling stated the Site is built with eighteen to twenty-four inch (18–24″) steel reinforced, cast-in-place, blast resistant concrete walls and flooring. Visual inspection during the Site tour was consistent with Sterling’s statements.
Sterling maintains a network operations center (NOC) on a twenty-four hours a day, seven days a week, three-hundred and sixty-five days a year schedule (24x7x365) to monitor and maintain the systems that support the data center Suites as well as the building itself. The systems include security, cooling (and associated water, fire suppression, and power). The NOC team is located on the ground floor of the Site in an area contiguous to the management offices.
During our inspection, we were shown a Suite on the third (3rd) floor that is undergoing an interior build-out for re-location of the NOC team. Coincidentally, the third (3rd) floor is also the proposed floor for the Adobe Systems data center space. Having the NOC team of a data center provider contiguous to a Customer’s data center facility would be an attractive convenience and minimize response time to any in-Suite alarms.
The Sterling NOC uses the Netbotz 500 monitoring and management appliance for the critical systems infrastructure throughout the Site.
The following are the standard features of a Suite:
- Twenty-four inch (24″) raised floors with 1,000 lb PSF minimum load rating
- Cable ladder racking for LAN topography and shielded power tray mounts
- All ventilation entry delivered through raised floor tiles
- Dedicated PDUs and CRACs with VESDA sensors
- Magnetic card and biometric (fingerprint) security systems
- Suite walk-through performed every six (6) hours by Sterling NOC technician
- Suite is managed as a discrete cell for all environmental and power systems (failure of a Suite’s systems will not affect another Suite)
- Tile pullers are for the sole use of Sterling (customers are not allowed to move or modify tiles)
- Suites will not be labeled with customer names
Currently, there is office space available in two locations within the Site:
- 15×18 feet on the third (3rd) floor located approximately 40 feet from the proposed Suite
- 10×10 feet (3 staff spaces available) on the first (1st) floor located near the building entrance
7.0 — Support Services
Sterling provides support services to its Customers. These services are generally not technical services. Services provided are:
- Shipping, Receiving and Storage of Goods
- Telecom Cross-connects and Service Extensions
- Remote Hands
Sterling does not provide any support services that require the configuration of equipment or software; i.e., no key pad, keyboard, or “mouse” manipulation. Requests for service are initiated via a form (see Exhibit 9: Remote Hands – Layer One).
Customer Delivery and Staging Area
Sterling will receive, log, and notify Customers of equipment deliveries and provide thirty day storage of customer equipment on the mezzanine level (a non data center floor) free of charge (small packages are stored in the NOC). Customers are responsible for moving packages into the Customer Suite.
The segregation of the staging environment to the mezzanine (non data center Suite space) floor represents several best practices:
- Minimizes the vulnerability of particulate contamination from equipment packaging (Styrofoam, cardboard, plastic, paper, etc.)
- Limits the access of non qualified personnel to the elevators and general access Suite floors
- Minimizes potential damage to Suites from activity associated with unpacking and assembly of devices
- Maximizes the use of premium (expensive) Suite space
Telecom Cross-connects and Demarc Extension
Sterling will, on a fee basis, provide cross-connect and service connection into the customer’s Suite (see Exhibit 8: Published Cross-Connection Form). Extension of the telecom demarc would be up to the individual carrier.
Cross connect services available are:
- Analog
- T1/DS-1
- 100 Base T
- DS-3
- Fiber Multi and Single Mode
Remote Hands
As previously stated, Sterling remote hand services do not include configuration. Sterling will provide:
- Physical installation of devices to mounting hardware
- Power-up machines and power recycles
- Plugging and unplugging of cables
- Visual inspection and feedback of status lights
- Board swap services
8.0 — Site Maintenance and Certification
Sterling was requested to provide Site maintenance and testing logs for all power, HVAC, security, and fire suppression systems. Sterling declined to provide the above citing confidentiality and burden as the reason. Based on the totality of our assessment we do not feel this should prejudice any Customer’s decision to select the Site for occupancy. Sterling did state that upon request and during final contract negotiations, weekly, bi-weekly, monthly, quarterly, and annual maintenance details would be provided.
Sterling stated that all systems critical to Site operation and the operational integrity of the Customer’s Suite are maintained to the manufacturer’s specifications. This includes any requirements for the system to only be serviced by either the manufacturer or an authorized service provider.
The Site is not currently SAS 70 certified but is scheduled to complete its SAS 70 certification Type I and Type II by the end of 2006. No client, to date, has been denied SAS 70 compliance because of data processing located at the Site.
Sterling currently does not possess any ISO 9000 certifications and does not plan to seek any such certifications.
Sterling stated that they are in compliance with approximately 155 of the 160 parameters of the Uptime Institute. Non compliance issues were items such as 24 inch raised floors instead of 30 inch and complete physical separation of A & B power systems.
Sterling states the 2004 federal OSHA audit contained only a single remediation requirement which was ear protection available immediately outside of generator spaces.
9.0 — Building Ownership, Management & Overview
Ownership
The Site was formerly owned and operated by the Arizona Republic Press newspaper. In 1999, Sterling Network Services (Sterling) was formed by principals of Sterling Capital, Ltd., who purchased the Site. To qualify the monetary stability of Sterling and subsequently the Site, we requested a financial statement to be provided for review. Our request for financial information was denied.
Sterling represented that after Site acquisition, Sterling immediately began a $25 million investment upgrade towards renovation and installation of carrier grade or high availability building infrastructure systems improvements and renamed the Site “the Downtown Phoenix Technology Exchange” which appears to be a synonym for the Sterling Networks Services Data Center (see Exhibit 11: About SNS). As part of the marketing collateral for the Site, Sterling distributes a presentation reflecting their views on the benefits of locating your data center in Phoenix. The presentation is attached (see Exhibit 12: Phoenix, AZ, “The Ideal Location for Your Enterprise Data Center”).
Building Management
The Site management team is led by the Sterling Managing Partner, George Slessman. During our visit to the Site, Mr. Slessman spent more than three (3) hours with us to contribute answers to our more highly detailed Site questions. We believe that Mr. Slessman is very qualified and clearly capable of managing a high availability Site and the types of services provided to Sterling Customers.
Given the nature of Sterling’s business and the amount of construction type activity that occurs in the Site, municipal relations are important for continuous and unencumbered operation of the Site. The Site has an “open” building permit which allows for construction to start without a specific permit. All building code (except for fire code) compliance is managed by the architectural firm of Peter Lendrum and the firm performs the internal compliance management function. Sterling has not been subject to any fines, infractions, or notice with respect to any type of regulatory infraction, breach, infringement, code violation, or contravention, nor has Sterling been subject to or party to any penal action from any governmental or quasi governmental authority as of this date.
The building engineer acts as the fire code compliance officer and oversees the annual fire inspection.
Tenancy
Sterling distributed the following Site Customer (tenant) list:
- Sony Pictures
- Starwood Resorts
- Blue Cross
- Blue Shield
- Giant Industries
- Go-Daddy Software
- Freescale
- Fidelity
Sterling also stated that Toyota, APS, and USA Today were all customers as well as some bank card processors and mutual fund companies. Sterling stated that there is less than one percent (1%) customer loss and the cause of such was the customer entering bankruptcy or liquidation. Furthermore, Sterling stated that they have not lost a customer due to a failure to meet an SLA.
Tenant Improvements
Tenants often require improvements within their Suites or dedicated areas that require a high degree of specialization and should only be performed by contractors. Tenants may use the contractor of their choice as long as the contractor requests and subsequently acquires approval from Sterling prior to commencement of performing any physical alterations to the Suite and or premises. To add a contractor to the approved list, tenant can have contractor apply with Sterling directly. Sterling also has a list of pre-approved contractors from which Customers can choose.
Shipping Policy
Sterling has a shipping policy and it is attached (see Exhibit 13: Shipping Policy).
Conference & Disaster Recovery (DR) Room Policy
Sterling has a conference and DR room available for customer use which is governed by the attached document (see Exhibit 14: Conference and DR Room Policy).
Vacancy, Estimated Absorption and Planned Expansion Facilities
At the time of our Site inspection, Sterling stated they could provide greater than five-thousand (5,000) square feet of Suite space on the third (3rd) floor to Adobe.
10.0 — Conclusion
Based on the visit to the Site and our discussions with Sterling staff, there are no indications that the Site (including the management and maintenance procedures of the Site) cannot support Adobe’s business needs. Our overall evaluation of the Site is that it is consistent with industry best practices for high availability data centers and telecom central office standards.
Appendix — Exhibit 14: Conference and DR Room Policy
Sterling Network Services, LLC — Meeting Space (dated 5/5/2006)
Purpose: To establish a procedure for reserving various common amenity spaces, including the War Room, Conference Rooms, 120 phone room and Cubicles.
Scope: This policy applies to all direct customers of Sterling Network Services, LLC.
Policy: All requests to reserve space in the Sterling Network Services conference rooms, War Room or cubicles need to be done through Jennifer Hinnant at jhinnant@sterlingnetwork.com. Listed below are the rates for the different spaces we have on site. All reservations will be on a first come first serve basis.
| Location | No Charge | ½ Day | Full Day | Week | Month |
|---|---|---|---|---|---|
| Conference Room (6–8 people) | 2 hours | 100 | 200 | 800 | 3000 |
| War Room | 3 hours | 300 | 600 | 2400 | 9400 |
| Phone Room | 2 hrs | NA | NA | NA | NA |
| Cubicles | 1 day | No Charge | No Charge | 400 | 1600 |
- Phone and internet service included.
- Cross connects to any data center or carrier available at published rates per Cross connect request form.