Noel O’Grady, Rewriting the rulebook: Learnings from how the data centre industry has reacted to COVID-19

Noel O’Grady, Director of Irish operations at business risk and resilience consultancy Sungard AS.

The data centre industry is arguably facing its largest ever crisis with the onset of the COVID-19 pandemic. 

In-house teams have had to quickly build out their ability to support remote workers on a scale never seen before. Commercial providers have had to tighten access to facilities globally, decide which staff are essential to keep onsite, and reach for their pandemic contingency plans – if these even existed. 

Elsewhere, data centre managers have had to embark on mass disinfection of premises and assess access following both staff and visitors testing positive for the virus. 

The last few months have been a huge learning curve for many, and here are three key lessons we have learned.

Noel O’Grady, Sungard Availability Services
Photo Iain White / Fennell Photography

 

Lesson 1: Be prepared to scale quickly for remote working demands

As our worlds of work and play shifted from office blocks and business parks to living rooms and kitchen tables across the globe, society has been highly dependent on internet services and supporting equipment working as expected.  

 

Infrastructure teams with responsibility for making sure organisations and consumers stay online have worked hard to expand network bandwidth and computing muscle in data centres. The most frequent bottleneck created in many cases has stemmed from the overnight enablement of vast remote workforces. Many company networks simply weren’t set up to meet demand at such scale. 

 

Organisations need to provide staff with private, secure corporate network access from home. VPN capacity has had to ramp up and conversations about readying the infrastructure to support it have had to happen quickly. Companies must now prepare to ensure employees can access a company network from their own machines at home too – there will have been many cases where people hadn’t taken work equipment home with them, and IT were unable to access buildings to get it to them.  

Technical understanding of the expanded capacity to support remote application access became a boardroom issue overnight as well. This will only be a good thing for companies moving forward. Senior executives will have no choice to overlook infrastructure. They will have to understand and monitor it like never before. 

Lesson 2: Pre-define ‘essential’ staff

Evaluation of who was ‘essential’ was another thing organisations had to determine quickly, and this expanded to the data centre industry. Essential workers had to be divided up. Some kept onsite and others sent home to become reserves. There had to be no risk of cross-contamination between them.  

Some senior-level facility and site managers were asked to work from home as well, as they possessed knowledge and experience covering multiple types of lower-level employees, such as line engineers, technicians and operations staff. 

The outcome of this crisis will be that data centre organisations should now formally plan and document who is essential to keep onsite during such a state of emergency, and who are the backups that should remain at home. This should be clearly communicated at regular intervals, with tests run to confirm the process works as expected.

 

Lesson 3: Disinfect premises post-positive infection reports and have strict access rules

There have been some international reports of staff and visitors who have set foot in data centres testing positive for COVID-19 at a later date. In response, the affected facilities have had to be thoroughly cleaned and disinfected by a specialist cleaning contractor. Data centre operators will now have to be crystal clear about the measures they will take in the event of such an incident and what they will do if a member of staff or a visitor falls ill whilst onsite. 

In order to maintain effective social distancing, some facilities have made all non-essential staff work from home on a mandatory basis. Clients are also being actively discouraged from visiting and encouraged to use engineers to complete physical tasks wherever possible. 

All deliveries and visitors should be handled by isolated security staff following very specific guidelines. Employee health has to remain the priority and having these measures in place requires a high level of adaptability on behalf of workers, so the businesses and the industry as a whole needs to be very clear.

If a data centre facility does have to be evacuated temporarily, whether it can stay online without staffing depends very much on its design, and is something that must be accounted for in planning and operations. Those with a low level of redundancy face obvious risks if left unmanned.

Rewriting the rule book

Data centre operators around the world have been forced to make big adjustments in light of COVID-19, with some of the areas mentioned above likely to become a necessary standard post-crisis. 

Historically, most business continuity plans for data centres were based on local scenarios, where ‘acts of God’ wreaked havoc on one place. Rarely had many considered that one place being the entire planet. 

 

This pandemic has shown us the speed of disease spread and highlighted the importance of a quick response. For the most part, data centres across the world are still functional, with our increasingly digital infrastructure showing a considerable level of resilience. However, it remains to be seen which contingencies will become the new norm and which won’t result in any permanent changes. 

How can Irish SMEs prepare for the economic impact of coronavirus? Noel O’Grady, Director, Sungard Availability Services

As national governments continue to halt mass events and impose limits on international travel, it seems that COVID-19 continues to pose the threat of causing a global pandemic. Neil McDonnell, Chief Executive of the Irish SME Association (LSME), warned Irish businesses to prepare themselves for the potential disruption the virus may bring to Ireland’s economic landscape. Mr McDonnell advised businesses to consider not only the short-term impact of the virus, such as the unavailability of staff due to sick leave, changes in customer behaviour and restrictions on international travel, but also the longer-term impact on global supply chains. Ireland has increasingly become reliant on Chinese manufacturers with some Irish SMEs sourcing up to 40 per cent of stock from China. He offered a stark warning, commenting that “[if] you don’t have the working capital to hit pause, you’re in trouble,” 

Coronavirus has already caused massive disruption across the globe, a prominent example being the GSMA’s decision to cancel Mobile World Congress 2020, one of the largest international trade conferences in the world. 

Noel O’Grady, Sungard Availability Services
Photo Iain White / Fennell Photography

Some SMEs may have already decided that employees must work from home to reduce risk of infection, believing current business continuity plans will be adequate to continue operations as normal. But a wide-scale outbreak like coronavirus requires more than a business continuity plan or an on-the-fly decision to have everyone work remotely; after all that doesn’t work for all roles in all businesses.

How coronavirus will affect organisations

From a business continuity perspective, breaking news, government advisories and actions, and changing economic impacts make viral outbreaks a moving target. In order to protect the availability of mission-critical processes and operations throughout the uncertain times, Irish SMEs must be prepared for a variety of disruption scenarios on a rolling scale of severity and impact. 

The first priority must be to protect the people within an organisation. During an outbreak, businesses must provide a safe working environment for personnel and visitors. According to Mr McDonnell, P2P, or person-to-person, organisations such as retail, hospitality, healthcare or delivery must be especially cautious: “Where [P2P] contact is unavoidable, you are going to have to make contingency plans for backfilling, such as talking to agencies about where you’ll get similar labour and that sort of thing.” This may result in businesses needing to temporarily close, especially for SMEs for whom staff shortages can cause the biggest impact. 

Upon learning that someone in the facility has been taken ill, organisations should take precautions to limit active virus exposure, which might mean restricting access to certain locations and possibly disinfecting all work and rest areas where the person taken ill may have visited. Depending on the both size of the workplace facility and the ability for staff to work remotely, companies should consider establishing work area recovery spaces to bridge any gap in operations caused by workplace unavailability.  

How coronavirus will affect customer bases

Businesses should be aware of how viral outbreaks can lead to changes in customer behaviour and demand for certain products and services. Some companies, like the makers of surgical masks, might see product demand surge, while others, like retailers, might see store traffic dry up. Banking customers might switch to using drive-up windows or online transactions to reduce P2P interaction. Increased online transactions in particular can lead to a dramatic escalation of calls to customer services in lieu of people seeking in-store interactions.

Anticipating these disruptions, businesses should consider what can be done in the present to continue meeting customer commitments in the future. Service disruptions can be very damaging to the experience of both consumers and businesses alike, therefore potential gaps in services need to be planned for. For example, manufacturing organisations may wish to accelerate production now to ensure the production meets demand when a pandemic causes disruption to critical supply chains.

How coronavirus will affect communication 

Whether there’s an illness within the ranks, general concern over social interaction, school closures, or another event, businesses must assess the impact of employee absence or staff working from home. How will customer demand be met? How will surges in IT help desk call-ins from remote workers be handled? Are employees adequately trained on remote working or set policies temporarily authorising overtime or accelerated schedules? Having the answers to these questions ahead of disruption will pay dividends when an actual crisis arises. 

Businesses must also communicate clearly. Communications protocols should be set with personnel to make clear how an organisation will share advisories and actions, i.e. when and how employees will be kept in the loop about facility closings and other changes. Organisations also need to be clear about what time-recurring updates will be available and how employees can access them, whether it’s through a website, a hotline number or a preferred channel. And should the unfolding situation call for immediate notification, a means to communicate time-sensitive urgent messages is essential.

While it’s important to spread awareness of the strategies in place to ensure business continuity, assigning an individual with the role of communicating critical messages during a viral outbreak can help lead others quickly to safety and resolution. Look to the head of human resources for the workforce and workplace elements. The operational aspect to pandemic preparation and response should fall under the COO.

Start or refresh your pandemic plan now

The potential local and global economic impact of coronavirus cannot and should not be underestimated, with the Irish stock exchange recently plunging by more than 4.3 per cent, its biggest drop since the Brexit referendum result in 2016. Ultimately, a pandemic plan addresses workforce, workplace, vendors, and customers. Crafting or updating one will make employees feel more comfortable, and ensures companies are better equipped to handle customer needs despite an outbreak that impacts business. 

Irish SMEs should conduct a comprehensive risk assessment at regular intervals to gain access to the information they need to avoid the potential economic, operational and psychological disruption that coronavirus can create. Assessments help protect staff, workplaces, third parties and anyone else who comes into contact with an organisation. At the end of the day, having protections put in place ahead of the fact is a far more effective approach to ad-hoc recovery efforts, the consequences of which can be devastating. 

Preparing Irish businesses for increasingly extreme weather events.Noel O’Grady, Sungard AS #Weather #ICT #SungardAS

The impact of extreme weather events on business performance, both positively and negatively, should not be underestimated. Abnormal weather is the difference between observed weather and its normal value, which is typically calculated using the 30-year average.  With the news that Ireland can expect plunging temperatures and extreme storms to become a more regular occurrence, the question of how companies can best prepare and respond should be an increased priority for business owners. 

 

Production, supply, demand and operations can all be impacted by extreme weather. Abnormal weather patterns offer a unique form of disruption for companies and, in recent years, organisations in the fields of energy, water, telecoms, transport, retail, leisure and agriculture have all been affected in a variety of ways. Small businesses are the most vulnerable, with there being no simple way of telling how long the weather-related disruption can last. Prolonged periods of disruption can lead to financial distress, triggering shortfalls in sales, store closures and even job cuts.   

 

Thanks to climate change, the frequency and intensity of abnormal weather patterns have increased too, with a shift to warmer temperatures only set to further the threats. Financial losses caused by adverse weather that didn’t seem material enough for companies to concern themselves with 10 years ago, are now examined and managed by weather-based financial instruments that seek to cover the risk to which businesses are exposed. As recently as October 2019, the Irish government were forced to send out an orange weather warning in preparation for ex-hurricane Lorenzo, proof that the time has now come when businesses can no longer fail to have an effective back-up plan in place to limit the negative impacts of a disaster.   

 

Failure to plan 

 

The European Economic Area records that, over the period 1980 – 2017, total reported economic losses caused by weather and climate-related extremes in the member countries amounted to approximately EUR 453 billion. In 2018, Sinn Féin proposed a bill to make it compulsory for employers to close its place of business during a red alert weather warning. It stated that in the event of a place of work being subject to a red weather warning, an employer would be required to “close such place and make reasonable measures to inform all employees to stay away from it for the duration of that warning.”  

 

Organisations must review assumptions about the risks they face because of their direct influence on both organisational resilience and the bottom line. Key to deciding how to move forward is making an accurate assessment of the level of risk a business is willing to accept, and taking a multi-tiered approach to assess and plan for a range of threats.

 

Advice for organisations to consider today 

 

To best address extreme weather events, companies should consider the following: 

 

  • Get a good weather forecast. The quality and range of forecasts has improved markedly in recent years.
  • Evaluate how to react to a disruption of this nature – organisations should look at completing an assessment of business impact, risk and overall resilience. 
  • Ensure ICT is robust yet agile – businesses must ensure an ICT service continuity plan is in place, and that it extends to disaster recovery scenarios. 
  • Prepare operations for action accordingly – this should consider business continuity processes, training and awareness processes for staff at all levels of the business, and vendor/supplier risk management too. 
  • Develop crisis leadership competencies within the executive team – this can include training, executive coaching and coached crisis scenarios.   
  • Foster a mutually supportive network of suppliers and stakeholders – examine market-wide crisis exercises and identify best practice. Also assess supply chain resilience. 

 

A growing trend 

 

Environmental risks have grown in prominence over the last decade and we can expect this trend to continue. Findings from the World Economic Forum’s 2019 Global Risks Report send a clear message to companies, with extreme weather events, natural disasters and climate change all highlighted as top concerns. 

 

As tempting as it might be to think “that’ll never happen to us”, understanding and investing in a business continuity plan, and business interruption insurance can greatly support recovery. The latter can be an invaluable safeguard for a business, for example allowing for the reimbursement of ongoing expenses and lost gross profit while a permanent business location is being repaired. 

 

Disasters can happen when least expected. Companies might not be able to prevent the storm, but they can take steps to handle the situation and keep operations up and running with minimal disruption. This is what the most resilient companies do, and it’s why they’ve stayed afloat while others have not.