In Case of Emergency Break Glass

In case of emergency, break glass

What if you completely lost the internet for 2 days? You woke up one morning and your phone was just a phone. You couldn’t check your email, read the news, or stream a video. You couldn’t contact your boss or coworkers because you don’t have their phone numbers. How rudderless would you feel? How panicked? Now imagine this scenario at a university level, with 50,000 students and thousands of faculty and staff who rely on the internet every day for their work, studies and research. And imagine the outage lasting 5 days, instead of 2.

On November 16 and 17, the Division of Information Technology and the UW–Madison Police Department stepped through this fictional — but very plausible — scenario involving a massive cybersecurity breach that resulted in the decision to disconnect UW–Madison from all internet service providers. During this simulation, members of distributed and central IT worked together to discuss how eliminating internet access would affect their areas, what changes they could make to adapt to the outage, and how they could keep faculty, staff, students, and the community aware and informed.

“These exercises play a crucial role in fostering relationships,” says Reed Erickson, emergency management specialist with UWPD, “as well as with deliberating on procedures, pinpointing gaps, and formulating solutions to enhance our response capabilities, preparedness, and resilience in the face of real incidents or emergencies.”

Make a plan

If UW–Madison experienced a cybersecurity breach so massive it had to shut off all internet access, the effects would be far reaching. It would potentially affect course registrations and grades, data analysis in research labs, medical students making rounds, and even activities as simple as buying lunch in the cafeteria. Everything would grind to a halt. One example that was raised during the simulation involved freezers containing research specimens. Currently, these freezers are connected to an online system that monitors the temperatures and alerts staff if the temps slip out of range. Without internet access, this system would require staff members to observe and record the freezer temps around the clock.

It’s one thing for internet access to be down for 24 hours. It’s another thing for it to be down for 5 days, like in November’s exercise. How would this constant vigilance take its toll on personnel? What measures would need to be put in place to ensure everyone is getting rest and maintaining a work-life balance? These were also questions that the group explored in depth.

“It’s important to perform these exercises to practice and gain an understanding of how to respond in a crisis situation,” says Todd Shechter, chief technology officer with DoIT. “But this simulation, although it derived from a technical situation, was not geared towards a tech response. Our focus was on the people-side and how to keep everyone informed, safe, and healthy.”

Move forward

No one knows exactly how they’ll respond to a threat or desperate circumstance. No one knows until they experience it firsthand. But with forethought, practice and understanding, a terrible situation can be managed effectively. In the case of November’s exercise, some departments updated their Continuity of Operations Plans, some refined their list of employee phone numbers, but everyone left the table with a set of shared best practices, a sense of community, and the knowledge that no matter the catastrophe, UW–Madison and the people within it are resilient and will find a way to keep moving forward towards success.

Thanks and recognition

Additional simulations will be organized in the future. However, we would be remiss to not mention the hard work that was put into producing this one. We would like to recognize the following individuals for their grace, knowledge and collaboration:

  • Todd Shechter, chief technology officer, Division of Information Technology
  • Matt Prough, information technology director, College of Letters & Science
  • Miguel Garcia-Gosalvez, information technology service management associate director, Systems Engineering & Operations, Division of Information Technology
  • Jennifer Sutherland, business continuity manager, Systems Engineering and Operations, Division of Information Technology
  • Suzanne Brandt, director of emergency management, UW–Madison Police Department
  • Robert Maier, emergency management specialist, UW–Madison Police Department
  • Reed Erickson, emergency management specialist, UW–Madison Police Department
  • And last but not least, the 60+ IT professionals from across campus who came together for these simulations. Thank you all.