When I finished up at the gym earlier this week, I found an urgent text on my phone from a friend with a medical practice. It seems that the PC running her patient accounting system was unbootable. She was not too panicked, because she had two different methods of backing up her data. I arrived to find the hard drive dead, and sadly, both backup methods had stopped working months ago. The PC was relatively easy to get running, but the data was gone.
Many organizations have spent large amounts of time and money developing complex disaster recovery plans. Despite all of that planning, an organization can easily be disrupted by the loss of a single key PC or server, a problem most disaster recovery plans fail to address.
This overlooking of key PCs often happens as a result of "IT bypass." In other words, a group or department get impatient when the IT team cannot meet their needs quickly enough, so they do their own thing, leaving "invisible" devices which gradually become mission critical. It can also happen in smaller organizations because a system quickly gets setup to resolve an immediate issue, and soon becomes a necessary resource without anyone realizing it.
In either case, someone remains responsible for the smooth operation of all company systems, even when they don't know about them. This may seem a bit unfair, but it is reality. An organization cannot be fully prepared to response to a disaster without accounting for all such key systems.
So, how does the responsible organization take responsibility for these invisible systems? Many satisfy themselves with having a policy against such systems. This may help management sleep better at night, but usually accomplishes little else.
Thankfully, there are tools, within the reach of even small companies, that help to identify and manage systems centrally. The major categories are:
Asset management - A wide variety of cloud and premise products are available which can inventory and monitor devices connected to a network. These products track hardware configurations, software installations and attached peripheral devices. They can report the addition of a new PC, and can even note that a database is in use on the device. Simply tracking the PC inventory centrally, and making note of new devices as they are added, can help to ensure that no system comes in under the radar unnoticed.
Centrally-monitored backups - As my friend learned the hard way, an unmonitored backup process cannot be relied on to keep running. There are, however, numerous cloud and premise backup products that facilitate central monitoring. The IT department or a key manager can quickly ensure that all workstations and servers are being backed up as intended.
On a related note, many organizations don't worry about workstation backups because they require that key files be stored on a server. I have yet to actually find an organization without some key files on individual workstations, however.
Centralized malware products - Sophos reminded us in an article last week that this is the 15th anniversary of the infamous "love bug" virus. It was annoying, and consumed a vast amount of Internet resources, but was thankfully easy to remediate. These days, we deal with complex malware such as "ransomware", that can encrypt all files on a PC, and hold them hostage pending a large payment. By using a malware product for which the software's status on each workstation can be monitored centrally, such issues can be avoided or minimized.
Coordinated patch management - Trying to make sure that all systems in an organization have the appropriate patches is an extremely difficult task. To further complicate this, Microsoft has announced that they are doing away with "Patch Tuesday," thus leaving deployment timing up to the end users. Fortunately, central systems are available, often in conjunction with asset management systems, that can push and monitor security patches via a central console.
Used together, the tools listed above, and a number of other resources, can help avoid IT bypass, and risk of unmanaged key systems can be minimized.