Handing over the keys of on-call coverage to another group can present challenges, especially when the group is overseas. But offshore teams can offer multiple benefits as well.
As someone who has worked in technology for almost three decades, I’ve had to deal with being on call 20 of those years. It goes with the territory in IT, but there must come a time when globalization steps in and alleviates some of the burden to free up veteran staffers for more critical project work.
SEE: TechRepublic Premium editorial calendar: IT policies, checklists, toolkits, and research for download (TechRepublic Premium)
On-call issues generally involve break/fix emergencies involving hardware and network failures, capacity issues and access problems, three of the most common problems I’ve faced. These have widespread impact but are generally simple enough to solve if responsible staff have the right knowledge, tools and permissions.
However, it can be challenging addressing these problems when roused from a deep slumber at 2 a.m., traveling on a weekend with limited remote access or juggling family life with a sudden on-call emergency that just interrupted everything.
This is why I was relieved to begin a process of transitioning on-call coverage to an overseas team. Not only does it reduce stress and allow time for more meaningful endeavors, it will allow both my American team and our offshore team to work during business hours. Our night time is their day time, meaning they’ll be able to handle emergencies during their business hours when they’re still fresh, and we’ll have uninterrupted sleep. The light at the end of the tunnel is nigh.
These are the top 10 goals we fleshed out in order to make the transition work successfully.
1. Identify key priorities
The first area to address is exactly what you want the offshore team to do; handle low-level requests (such as password resets) along with on-call issues, maintain maximum system uptime, work with external customers, engage in monitoring and other goals for this team to support the business.
Priorities will vary depending on the team, their knowledge level and the goals of the organization, so get these laid out in a concrete fashion.
2. Identify responsibilities
This is different but related to No. 1: establish the actual steps the offshore team will take to identify its scope of responsibility. Perhaps overseas workers can reset existing passwords but not create user accounts or change group memberships. Perhaps they can identify what’s using up disk space causing a critical alert to page people, but not actually remove files or otherwise change the file system (see No. 9).
It might be that they can determine which process has hung a server but not actually reboot the server or restart the process (also see No. 9). Lay out exactly what is expected of the offshore team since this information will be needed for No. 6.
3. Identify vendors, and ensure 24/7 support
Many on-call issues involve vendor support, and you’ll need to provide the team with the vendor contact information, support agreement details, any accounts needed for vendor support access and every other element needed to get support as quickly as possible.
Also, make sure the vendor provides 24/7 support or at least support during the timeframe the offshore team will be handling on-call issues.
4. Identify key players
Gather a list of the team members who will be handling these duties as well as any supervisor- or management-level people and specify them as such. They should be part of a standard email distribution list or Teams group or channel (or some other similar communication platform) dedicated to communication with them.
5. Identify and provide the access and tools needed
Obviously the horse pulling the cart here will be a remote access methodology such as a VPN for the offshore team to connect to company resources or cloud-based accounts for related access to business environments.
Create the internal accounts as needed—make sure each individual has their own account—then for internal or external accounts you must determine what groups and privileges they will need in order to perform their duties. Apply the settings using groups where possible so you don’t have to elevate privileges on a per-user basis.
SEE: The future of work: Tools and strategies for the digital workplace (free PDF) (TechRepublic)
If using temporary offshore workers, establish when their contracts will end and set the accounts to expire on that day.
6. Build runbooks
Runbooks are training documents which lay out exactly how to do what needs to be done. Whether it’s restarting devices, clearing out disk space or working with user accounts, your runbooks should be as detailed as possible and contain step-by-step instructions for analysis, resolution and verification. The amount of effort you put into these runbooks will pay massive dividends down the road.
Don’t just include how you do these things, but why, so they understand the nuts and bolts of operations.
If you’re going to include commands that can be copied and pasted into another window such as a remote desktop or SSH session, ensure in the first place that copy and paste is allowed (some companies block or restrict this for security reasons) and ensure your commands are in plain text. Characters in unicode, such as in a Word or Teams document or website with hidden formatting, can wreak havoc when pasted into a “text only” environment by introducing unwanted elements you can’t see, which cause commands to fail.
7. Train and shadow
Go over the runbooks with on-call personnel who will be responsible for these duties and take the time to help make sure they get it right with one-on-one guidance until they’re ready to fly, so to speak. Again, the time and effort you put in here initially will ensure you’re freed up for other duties and obligations elsewhere down the road.
8. Establish future trainers
This one will also help you stay focused on alternate priorities besides on-call tasks. If possible, identify and establish key leads in the offshore team who can then train new staff to transfer their knowledge.
The best-case scenario is that you engage in a “one and done” training session so they can pick up the ball and run with it, freeing you up from having to be tethered to offloading this work again and again when new staff arrives on the offshore team.
9. Establish escalation points and schedules
The simple fact is you may not—and probably won’t—be able to offload every emergency issue to offshore teams. The stuff they can’t handle, or should not touch, will likely have to go to an onshore in-house team, and that’s as it should be.
Establish what they should escalate to in-house staffers and when, and implement schedules for escalation so that these staffers are aware when they may be on the hook for answering such escalation calls.
Make sure to establish schedules for the offshore team and let them know when they will be responsible for addressing these issues. Will it be during their working hours or will they be on 24/7? The decision is based on your organizational needs and agreement with the offshore provider.
10. Update documentation as appropriate
Any given implementation is always going to need care, feeding and revision down the road. Make sure every living element of your offshore on-call transition is subject to modification as needed to ensure continued success going forward.