Speaking of #ICSCybersecurity… I think I’ve gotten a lot of followers on social justice and general IT lately. As a little re-intro, what I do for a living is respond to and investigate hacking of critical industrial infrastructure like power, water, manufacturing, and transportation. Stuff that doesn’t look like computers but often is today. I’ve been doing it for over a decade and a half, If that’s ever something you want to know more about, AMA and I’ll do my best to answer your questions.

@hacks4pancakes At one point there was a best practice of air gapping those industrial systems from the internet. Is that still the case? As a practical matter, are systems like that air gapped? And does air gapping as actually implemented help defend against attacks, and why or why not? Anything else about air gapping?

@hacks4pancakes how often are you seeing privacy impacts in the utilities space (smart meters and similar, plus upstream systems) when you go in to investigate?

I've tested a handful and I've got a 100% hit rate on "this thing is a privacy problem", but I'm not sure if that's representative or just how the dice rolled.

@hacks4pancakes what's your general big-picture strategy for coming up with recommendations to improve security in networks where all the PLCs and middleware are cleartext and open to manipulation by design? I always feel at a bit of a loose end with it, 'cos ipsec/TLS are no good (handshake RTT latency or any delivery issue = timely alarm delivery failure = safety hazard = hard nope), and I feel like I end up defaulting to "fix the middleware RCEs I found, firewall stuff, consider VLANs".

Keep me bookmarked if anyone ever starts saying anything about “hacking the grid” or “cyber Pearl Harbor” so I can come over and smack someone upside the head, with credentials.

@gsuberland you get why it’s that way by design. Basically, good passive monitoring and architectural controls.

@hacks4pancakes good to know, thanks. I should probably push more detailed recommendations than I have been on that front (monitoring is something I've been slacking on in general; I'm behind on it).

any pointers on specifics I should direct them towards, beyond the basic "suricata and snort are free and well supported"? are there any good baseline rulesets for ICS environments? any guides you tend to poke clients towards? (I appreciate that this subject is an entire job's worth of complexity)

@gsuberland sadly, most IT / enterprise security monitoring solutions like ET rules do very little with lower level industrial protocols or device manipulation, but something is better than nothing.

@hacks4pancakes @gsuberland fwiw, I concur on this. Even industrial-specialized monitoring systems aren’t going to be all that great out of the box for a random environment and basic architecture along with asset inventories are far more crucial to start with.
Some systems basically need a customized SIEM that’s kept tuned to the specific environment month to month to actually give any value on monitoring how it runs just because not all systems even manage meaningful logs.
For perspective, there are actively used code bases where the lower levels don’t have the concept of a user. Fundamentally they have all the cybersecurity and access control of a manual valve you’ve got your hand on.

@whereisthespai @hacks4pancakes yeah I'm thinking I should probably just tackle it from the perspective of having any view at all beyond the most basic AV/EDR, since the orgs I generally work with don't have the security maturity to be spinning up and operating a full custom SIEM with all the trimmings. I do tend to recommend pushing middleware alerts to off-site syslog so they've got some backup in case of ransomware or wipers.

@gsuberland @hacks4pancakes honestly just getting most places being onboarded to the point where they know what they have, where it is, and why they have it has been the biggest consistent payoff I’ve seen.
Then getting them to where they know how the things actually do what they do is a whole other kettle of fish. But that also all feeds into being able to benefit from stuff like monitoring systems and IR/professional services retainers.

@gsuberland @whereisthespai @hacks4pancakes I have Seen quite sensible requirements from shipyards handed to contractors/Tier 1+ manufacturers. But most of this falls with the essential non-existence of physical security while a ship is in harbour.
With land-based OT, these do a lot of good research: https://www.tha.de/en/Computer-Science/HSA-innos.html

@wamserma @whereisthespai @hacks4pancakes the IMO also created new regs as of January 2023 to try to ensure remote monitoring has sufficient isolation and security assurance.

@gsuberland pretty much never. Sorry for the bad news.

@david42 I see maybe two airgaps a year doing it daily outside of nuclear and defense. They’re mostly a myth now and have been for at least a decade. Too much efficiency and reduced staffing gained by networking. We even see cloud now. Vendors demand connections to provide warranty support.

@hacks4pancakes I could imagine a setup with a physical airgap that can be connected for maintenance and then disconnected, but I guess anything like that requires hands and time, and sounds like the industry is "optimizing" that out.

@david42 correct. I have seen remote access that’s connected and disconnected but it’s rarer all the time. It’s mostly just a normal DMZ between IT and the process environment, and those are as good as you make them.

@hacks4pancakes @david42 this kind of setup (OT and IT network separated) is supposed to be the standard in the marine space but I have yet to find a case where there wasn't something bridging the gap. a lot of the time it's intentional (remote monitoring) but it's also not uncommon that a contractor plugged a patch lead between the switches and then forgot to remove it. all the networked gear is serviced on-site by the vendor so rigid change control is borderline impossible.