OT Security Made Simple podcast: pragmatism in the OT SOC
On "OT Security Made Simple" I talked to Klaus Mochalski about building OT SOCs. One point from it is worth repeating here. In OT security monitoring, the tools above all create visibility. On that basis the security team and the OT team take the decisions together.
Where an IT SOC would isolate a host or lock an account, OT often has neither option, because it is technically impossible or operationally forbidden. You do not switch off a controller running a live process because an alarm lights up. When a new asset appears on the network, it can be an attacker with a Raspberry Pi or an unannounced maintenance visit. The first action is to check the maintenance calendar, contact the customer, and watch the new device for suspicious activity in parallel. A foreign device that is not part of the process can be taken off the network safely once port scans, brute force and exploits come together and the customer has agreed. A controller running a process cannot.
Build an OT SOC backwards
The starting point is incident response. Which incidents should be detected, what needs a playbook, for which asset, which finding, which action, who decides. The use cases follow from that. Plenty of use cases that raise alarms nobody acts on are worth nothing. Only then does the data requirement follow: which log sources matter and have to be connected, on the host and in the network. In OT this chain does not run in one direction only. What data is reachable at all decides which use cases can be built.
Getting at that data is laborious in practice. Often there are no logs at all, or they are produced but never collected, because no log collector was ever planned for. Some switches cannot do port mirroring, and some protocols are not recognised or logged by the sensors in the first place.
Passive monitoring at field level is the normal case. We do not install an agent on a PLC or a protection device, but we watch their network traffic. In an IEC 61850 substation, MMS and GOOSE run on the station bus and sampled values on the process bus. GOOSE and sampled values are layer 2 multicast and do not leave the local bus. Hang the sensor up in the DMZ only and you never see the fast protection communication.
Check first, then act
Attack detection in OT depends on two things: signatures and a baseline of what normal looks like. One incident showed me why they belong together. Our sensors reported exploits and malware in bulk. The first reading in the team was that the plant was thoroughly infected and had to come off the network. The analysis showed it was the backup job, which had downloaded the signature updates for the sensors themselves. The sensor had recognised itself by its own signatures.
In IT a false positive is not serious. In OT the reaction weighs more than the false positive. If the playbook says "shut down on mass alarm" and you follow it to the letter, a false positive does real damage. What helped was context. The baseline knew this host as a regular update client, and we could download the PCAP and look for ourselves. That is what visibility means: being able to check before you act. Sensors that only report "attack X detected" and hand out no raw data are worthless at that moment.
Start small
If you run an existing plant and want to retrofit monitoring, do not try to solve everything at once. The most effective first step is the smallest one. Onboard a few systems, write a few use cases, set up the first playbooks. Alongside that, settle the process questions that count on the day: who may do what, who escalates to whom, what happens after 18:00 when the day shift has gone.
And then practise. Tabletop before red team. The SOC and plant operations need the same picture of what is possible, permitted and sensible. When an incident happens, very few people know what they are allowed to isolate and what has to be agreed with the customer first. I see that again and again. It is not a decision you want to make for the first time while something is burning.
The full conversation with Klaus Mochalski is in the current episode of "OT Security Made Simple".