Undergoing continued growth in
2006, it was clear that additional future growth was on the horizon for the
Vanderbilt University Medical Center (VUMC). VUMC is a comprehensive healthcare
facility in Nashville, TN, dedicated to research, biomedical education, and
patient care. Its campus includes The Vanderbilt Clinic, which has space for
more than 100 ambulatory specialty practices of the Vanderbilt Medical Group.
Because together they provide around-the-clock acute care to over a million
patients annually, an uninterruptible power supply is crucial.
More than that, it’s required by
The Joint Commission on the Accreditation of Healthcare Organizations. According
to that accreditation body, each healthcare facility must have an emergency
power testing program that includes generator load testing and Emergency Power
Supply System (EPSS) maintenance. Standby power generation systems must be
tested monthly at 30% load for 30 minutes.
Per code, power systems must
operate around the clock with full redundancy, providing switchover of power
within 10 seconds, in the event of power interruption. A combination of utility
and generator backup power is required for compliance in achieving proper
redundancy, with multiple generators feeding loads versus loads dedicated to a
single generator.
Plan in Place
Jon Ross, project engineer for
Smith, Seckman & Reid Inc., was designing the new tower addition at VUMC at
the same time that the new medical research building was beginning construction.
The research building required new emergency generators that were to be located
on the roof of the building for lack of other options. The new hospital tower
project, which was in the early design stage, would also require more capacity
than the existing hospital emergency generators had available. Ross says that
the early design plan was to add another generator plant to serve the hospital
tower.
 |
Photos: John Ross
Because Vanderbilt's other equipment was already Cat, there was strong desire
to go with Cat Gensets. |
 |
A sixth unit is expected in 2009, in addition to replacement of two aging
gensets. |
Realizing that both the research
building and the new tower required more generators, Ross presented an idea to
then-head of space and facilities at Vanderbilt, Fred DeWeese, initiating a
collaboration to consolidate power across the medical campus. “We looked at the
whole campus for ways to consolidate,” explains Ross. In drafting his plan, he
considered the load requirements across campus. “There were 18 generators across
campus, some of which were paralleled.”
In the past, Vanderbilt had added
generators as needed for projects without a comprehensive plan for serving the
entire medical campus. Ross says it’s not an uncommon approach. “Some take a
longer look at infrastructure and spend money to consolidate. Others use the
money given for a project and make do. Vanderbilt is kind of like that.”
Compounding the problem is the
fact that Vanderbilt has an urban campus, with buildings spread out over a large
area, making consolidation difficult. “Obviously, our preference was to put
everything in one area, but we couldn’t,” adds Ross. He thought that with the
large facilities being constructed and the requirement for significant added
generation, it was time to “step back” and look at a campus solution, not just a
project solution.
In determining a plan, Ross says
many considerations were taken into account. “We tried to decide where power
comes from for new projects. How much equipment could we get in? What else could
happen? Should we put in more than we need now—for future growth? Since we know
growth will continue, we needed to think long-range and set up a plan to
accommodate growth.”
A new generator facility couldn’t
be built within existing buildings because of spatial limitations, but needed to
be near the existing electrical infrastructure. There were other constraints:
noise, exhaust fumes, and fuel storage. In addition, Ross thought ahead to
maintenance, particularly for emergency equipment.
The resulting plan that was
proposed included building a new emergency power plant big enough to serve not
only the new hospital tower, but also the entire existing hospital and clinic.
That made the three existing 1,200-kW emergency generators that originally
served the hospital, available to use at the research building. Since the
existing generators were actually located between the hospital and the research
building, the cost of redirecting the generator capacity to the new building
would be more than offset by not having to buy generators for the research
building.
Switching—and
Sharing—Gear
The collaboration of Vanderbilt,
Smith, Seckman & Reid and a local Caterpillar dealer concluded in The
Vanderbilt Clinic Highbay Power Plant project in 2006. Things moved quickly,
with a presentation in June, followed by coordination in July, and design the
following month.
“We spent about six months on
design and one year on construction,” reports Ross.
Equipment packages were purchased
in November and December 2006 for the $6 million project—a budget increased from
the original $4–5 million. “We proposed a 9-MW plant in the Highbay area or
parking deck across from hospital,” he explains.
Despite the fact that the
anticipated load for the existing hospital, clinic, and new tower could be
served with a 6-MW plant, additional capacity was planned for future
consolidation of smaller, individual generators on campus.
Vanderbilt wanted “n+1” capacity
for the new plant, where the code required essential load is “n” and the “1” is
a spare generator. Ross explains, “The n+1 design philosophy was important to
hospital maintenance, because they can have a generator down for maintenance and
know they always have enough capacity to serve the code-required essential
loads.”
The team agreed to use the spare
generator to run a chiller in the new tower basement when it wasn’t required for
essential loads. The generators were designed for 4160 V to reduce the
distribution costs from the new plant and to be able to serve the 4160-V
chiller.
 |
Photos: John Ross
The collaboration concluded in The Vanderbilt Clinic Highbay Power Plant project in 2006. |
 |
The hospital received the newer generators, while its three 1,200-kW generators were relocated to the
Medical Research Building, which required no additional capacity. |
Although the new tower Ross was
working on was integrated into the hospital, they had to add separate generators
because it was not integrated. “We didn’t need all that space for the tower, so
we used it for generators for the hospital,” he says. “That left for enough
space to eliminate extra gen.”
The final plan to provide reliable
redundant standby power with automatic switchover capability during power
interruptions required four Cat 3512B 1,500-kW generator sets with customized
switchgear. According to a Caterpillar publication, “Each Cat generator set
produces 4,160 volts for transmission to 480-volt substations.” The new
components were designed to use 15-kV breakers, matching the existing wiring at
Vanderbilt. Not only did that integrate the new system better, but it also
improved the versatility of power distribution.
Ross explains that because the
majority of Vanderbilt’s equipment was already Cat, there was a strong desire to
go with Cat gensets. Besides, he notes, their pricing was better. Although all
hospital transfer switches were Russelectric, Caterpillar proposed their ISO
paralleling gear as a packaged solution for the plant. Vanderbilt chose to go
with the Caterpillar solution for this project.
With the addition of the new
building, the hospital’s emergency branch loads totaled more than one 1.5-MW
generator. Per the National Electric Code, power for the emergency branch loads
has to be available within 10 seconds. “We couldn’t guarantee that one generator
would be online and a second paralleled and closed in within 10 seconds, so we
designed the switchgear with two separate paralleling buses with equal
generators connected, and divided the emergency branch distribution between the
busses,” says Ross. “A tie breaker was designed to allow the two busses to be
synchronized together after all generators were connected to their respective
busses, so the system operates as a single paralleled system once it is
running.”
 |
| Photo: Eaton Corporation |
The double-ended parallel
switchgear with parallel ties across the break allows each bus to come online
within the mandated 10 seconds, so that the total emergency branch load is
powered within the code required time, then synchronized. It’s not unique, he
adds, but it is one way to meet the code requirement for emergency power for
essential systems.
The metal-clad switchgear with
Powerlynx 3000 control, enhanced operator interface with touch-screen controls
and remote PCs for monitoring and control allow tracking of power output and
distribution, as well as routine maintenance, in real time from a remote PC. The
switchgear is compatible with the existing Automatic Transfer Switch interface
and is used during mandated testing to simulate a power supply shutdown and
switch over to full output within the required 10 seconds.
By taking the entire hospital and
clinic emergency load onto the new generator plant, not only did they ensure
that the hospital had reliability in the new emergency plant with added capacity
for chillers when necessary, but they freed up the older hospital generators for
use at the Medical Research Building (MRB). “The old hospital generators are
serving the MRB,” elaborates Ross.
The budget-saving move meant that
the hospital, which increased both its square footage and load, got the newer
generators, while its three 1,200-kW generators were relocated to the MRB, which
required no additional capacity. The MRB project installed new feeders to the
existing generators instead having to install a generator plant on the roof of
the new building.
The fact that the existing 1200s
were located under the plaza that separated the hospital and the MRB helped in
making the decision, Ross believes. “We had a lot of problems figuring out the
MRB generators. Since the 1200s were actually in a location accessible to the
MRB, it made a lot of sense to put the money for new generators into the
hospital and let the MRB utilize the existing ones.”
In the end, the MRB acquired good
gensets that are easily serviceable just below grade—as opposed to being placed
on the roof—and the hospital got better reliability with the new generators.
“The hospital has a critical need, so it made more sense to put the new
generators there,” he says. “This plan allowed us to consolidate some, without
putting any on the roof.”
There’s even room for growth. Ross
indicates that two spaces in the Highbay area are left. One is intended for
backfeed to eliminate two or three generators in the parking deck, although he
says it hasn’t happened yet because the hospital keeps adding load. He says
Vanderbilt is adding a fifth generator in its first expansion. Chris Buckner,
with VUMC’s Office of Space and Facilities Planning, told Caterpillar that he
expects a sixth unit in 2009, in addition to replacement of two aging gensets.
 |
Photos: John Ross
"We view technology as an asset to facilitate even better care for hospital patients." |
 |
Caterpillar proposed their ISO paralleling gear as a packaged solution for the plant. |
Challenging Moves
Although the project won awards
and plenty of notice, Ross says it wasn’t always easy. “Noise was an issue,” he
states.
On the urban campus, the 11-story
brick and glass hospital isn’t far from the clinic. The team was concerned that
the noise from the generators that are located at street level and facing the
hospital would bounce between the buildings, Ross explains, disrupting the
patients and staff. Compounding the problem is the fact that air is drawn in
from the parking deck, with radiators blowing it out at street level.
To mitigate noise issues, they
installed sound attenuators on the radiators and intake louvers to reduce the
amount of noise going into the garage and out to the street. In addition, he
says they used oversized radiators to account for slowing the air down for a
full heat exchange.
“We utilized an existing elevator
shaft to route the exhaust piping to the roof of the garage, so the fumes were
not a problem,” he continues. “Smoke, heat, noise—no problem. Standing on
the sidewalk in front of the generators, the passing buses are louder than the
generators. It’s been a wonderful success.”
Essentially, Ross says there were
“no hitches; it was planned pretty well and fell into place.” There were a few
tweaks along the way, however. The generators fit well in plain view, with a
“nice maintenance area,” but, because height was an issue, they had to offset
the silencers, tightly wrapping the piping. The parallel gear resides in a
glass-walled room adjacent to the generators. They had to work with the city to
address the fire ratings due to an aboveground fuel vault with three
10,000-gallon tanks in a containment basin.
More coordination was needed when
they determined the best route for the distribution to the hospital was through
a project that was joining two package decks. “We evaluated three routes,”
recalls Ross. “There were all kinds of utilities running in the surrounding
areas.”
Running two feeders across the
street to the double-ended emergency unit sub-station required excavation. But
that wasn’t the only aspect that necessitated precise timing. When 26 transfer
switches were swapped from the old to the new generators, they were moved over a
couple weekends in December 2006. “It was a big feat!” exclaims Ross. “Through
much planning and preparation before the weekend shutdowns, the team reconnected
all transfer switches, tested all control signals, and put them back in
operation without a problem. The patients and staff experienced no disruptions.”
In fact, he says, it was not a
true shutdown because there was no loss of power.
Passing the Test
Buckner told Caterpillar that the
University did “extensive testing of each generator and each transfer switch,
because we couldn’t have a situation where the hospital wasn’t covered. In fact,
we conducted a continuous 36-hour run test of all four generators synchronized
to the board after each of the generators was extensively tested individually,
so that we could simulate an extended power curtailment from our local utility.”
They also conducted four different shutdowns to individually test “each and
every start signal and each load shed from the 31 transfer switches associated
with the power plant.”
Confidence is running high, thanks
to successful test results and a system that’s simple to operate, test, and
maintain. “Everything is going according to plan,” summarizes Ross. “The first
phase was to get it in place and get the loads transferred. Everyone’s happy:
It’s very quiet and the new generators are dedicated for patients.”
Carolinas Hospital—Growth and
Compliance
VUMC isn’t the only medical
facility trying to keep up with its own growth. Carolinas Hospital System (CHS)
in Florence, SC, moved into a multi-million dollar medical campus in 1998, but
continued to experience growth and increased power loads. Serving patients in a
nine-county area with state-of-the-art equipment, CHS provides acute care,
cancer treatment, cardiac care, emergency/trauma services, maternity care, and
an array of specialized rehabilitation programs.
 |
Photo: Eaton Corporation
The Blade UPS server room is an integral part of Non-Destructive Testing. |
The need to meet The Commission’s
full power redundancy and standby power requirements and Health Insurance
Portability and Accountability Act (HIPAA) requirements, as well as accommodate
increasing information technology (IT) demands and achieve greater reliability
and continuous availability of its IT systems, meant that it was time to bolster
its power protection efforts in two critical areas: its patient health
information servers and its primary distribution cabinet.
“Our initial UPS implementation
was not designed to accommodate the entire data center,” explains Ray Graham, IT
consultant for Carolinas Hospital System. “Power protection is a vital part in
maintaining our technology infrastructure.”
Thus, they sought a solution that
would address their immediate data center requirements as well as future
expansion plans.
Although code requirements for
essential systems and emergency backup power for critical and life safety are
clear, IT systems requirements are less so. However, with the typical commercial
customer experiencing four to 15 outages per year, it was obvious to Graham that
CHS’ IT infrastructure was susceptible to daily power quality issues such as
surges, sags, load fluctuations, and other power interferences. “We view
technology as an asset to facilitate even better care for hospital
patients,” he says. “In a hospital
environment, downtime must be avoided as critical equipment and patient
information is needed around the clock.”
A proactive plan for power
protection and management strategies to protect IT systems, diagnostic imaging
equipment, clinical labs, and monitoring support systems against a full range of
problems was implemented.
Powering Up
The hospital’s data center serves
1,800 users and houses 50 servers, generating and storing large volumes of
critical patient health care information. When CHS relocated in 1998, its
uninterruptible power system was already approaching capacity. In 2005, the data
center added two additional units to power its servers, but it wasn’t long
before those Powerware 9125s were also near capacity.
To ensure protection for its data,
hospital administrators hired Jones Engineering, an engineering contractor based
in South Carolina, to determine the appropriate specifications needed. They
identified several criteria, such as total system scalability, battery runtime,
and redundancy requirements, before ultimately recommending Eaton’s Powerware
BladeUPS solution.
The deciding factor, Graham
states, was the system’s modular design and built-in bypass capability. “With a
modular system, there are no downtime requirements, and when additional capacity
is needed, you simply plug in another module,” he adds.
Success with previous Powerware
products confirmed their choice. Eaton’s Powerware BladeUPS is the industry’s
most energy-efficient, rack-based, three-phase system, specifically designed and
optimized for today’s high-density computing environments.
Another reason CHS opted for the
Powerware system relates to additional features not available with other UPS
systems. CHS liked the integral network-monitoring feature, which allows e-mail
notifications about power events to be sent in advance of cutting servers over
to backup power. The UPS is programmed to distribute pager and e-mail
notification for both critical and major power events and allow the IT staff to
examine the load to determine how many more devices can be added before reaching
capacity.
 |
Photo: Eaton Corporation
Remote visual inspection is possible with Data Module installation. |
Plugged in
Two weeks from placing the order
in January 2007, CHS installed two 12-kW Powerware BladeUPS modules, with the
capability to store five modules in a single 19-inch rack, for a total of 60-kW.
Because the two modules are in redundant mode, staff can perform full
maintenance on any module without interruption of conditioned power to their IT
equipment.
Once the Powerware BladeUPS system
was installed, staff began migrating its critical servers. Additionally, eight
servers containing primary patient information systems were also transferred.
Plans are in place to remove the old UPS and power the entire data center with
the Powerware BladeUPS. The two Powerware 9125 units are also scheduled to be
moved to the hospital’s main distribution closet, to provide power protection
for core networking equipment that supports hospital connectivity to all remote
sites. “We are now using UPS systems in our two most critical areas—patient
health information servers and our primary distribution closet,” reveals Graham.
“This strategy is essential, because the Powerware BladeUPS now bridges the gap
between loss of utility power and locally generated power. We have complete
confidence that we will not experience power problems that harm our computers,
data, or affect patient care.”
Eventually, everything in its data
center will be transferred to the Powerware BladeUPS, including clinical,
radiology, and facility applications—including accounting and payroll
applications—as well as all infrastructure systems, networking components in the
data center, and current domain support systems. An additional six servers will
be added to support that transition and implementation of an entirely new health
information systems application.
Carolinas Hospital depends on the
UPS to maintain its critical healthcare systems. “Downtime procedures are a last
resort, because we cannot prolong the amount of time required to provide care
and services to patients,” explains Graham.
Downtime
also increases costs, while simultaneously decreasing patient satisfaction.
“Anything we can do to prevent that is a plus.”