Genchi Genbutsu: The Toyota Principle Cyber Risk Quantification Needs

8 MIN READ

The chalk circle

At the Toyota Motor Company in the 1950s, Taiichi Ohno had a habit that made his engineers uncomfortable. He would take a new manager down to the factory floor, draw a chalk circle on the concrete, and tell them to stand inside it. Then he would leave. Hours later he would come back and ask what they had seen.

The answer he wanted was never "the process is running correctly." It was a specific, textured observation about a bolt that arrived a fraction late, or an operator whose reach was awkward, or a fixture that vibrated once in every ten cycles. The point of the chalk circle was that no report, no dashboard, no meeting could substitute for standing in the actual place and watching the actual thing. Ohno gave this practice a name that became one of the most enduring principles of the Toyota Production System: genchi genbutsu — go to the actual place, look at the actual thing, get the actual facts.

Seventy years later, cyber risk quantification has drifted almost entirely away from it.

Most CRQ practice today happens at the spreadsheet, not the substation. And for one specific category of asset — the unscannable ones — that drift is why so many risk registers are quietly wrong.

What Ohno was really saying

Genchi genbutsu is often mistranslated as "go and see." That undersells it. The literal reading is closer to "actual place, actual thing." Ohno's insistence was that the representation of a system — the spec, the report, the compliance evidence, the KPI — abstracts away the variance that decides whether the system fails.

The variance is where the risk lives. And the variance almost never survives the transit from the shop floor to the report.

Toyota's engineers were expected to walk the gemba (the actual workplace) before making any decision that mattered. Not to look at charts about the gemba. Not to interview someone who had been to the gemba. To go there themselves, watch, ask questions of the people doing the work, and only then form a hypothesis about what needed to change. The reason the practice endured is that it repeatedly caught problems the reporting layer had smoothed over.

The parallel to cyber is uncomfortable. A CVSS score is a representation. A vendor risk questionnaire is a representation. A compliance attestation is a representation. Each one abstracts away the variance in the actual place, and each one becomes the input a risk register is built on. The register looks defensible because it aggregates hundreds of representations. It is not defensible if the underlying representations have systematically lost the variance that decides whether an incident happens.

Scannable assets — the illusion of gemba

For scannable IT — laptops, cloud workloads, standard servers — the scanner is a passable substitute for the gemba walk. It touches the actual thing, reads the actual configuration, records the actual patch level. The abstraction happens at a level the risk analyst can inspect. If the scanner reports OpenSSL 1.1.1w on a public-facing host, that fact is verifiable and the loss scenarios it enables are computable. FAIR-style quantification works reasonably well here because the inputs are grounded in the actual place, even if a human never went there.

There is a whole category of assets that scanners cannot touch — and for those assets, the desk-based CRQ workflow silently substitutes representations for reality, and the risk register inherits every abstraction error the representations contain.

Unscannable assets — where the chalk circle is not optional

The unscannable category is not small. It contains most of what boards actually worry about.

These are the assets with the highest consequence of compromise and the least rigorous risk data. A ransomware attack on the corporate wiki is embarrassing. A ransomware attack on the SCADA layer of a water treatment plant is a public safety event.

For every one of these categories, there is no scanner substitute for genchi genbutsu. The information the CRQ model needs — patch level, network topology, physical isolation, credential hygiene, incident history, operator familiarity with recovery procedures — lives in exactly two places. It lives in the site. And it lives in the head of the person who has been working at the site for fifteen years and knows every quirk of the equipment.

If you want a defensible risk figure for these assets, you have to go and get it from those two places. That is genchi genbutsu, re-stated for cyber. It is not optional. It is the only source of ground truth.

What a cyber gemba walk actually looks like

Walking the cyber gemba is not glamorous. It looks like a few very specific practices, each one repeatable.

Sit next to the operator during a normal shift. In an OT control room, ninety per cent of the risk-relevant information is what the operator does without thinking. Which alarms they dismiss automatically. Which sensors they trust and which they do not. Which panels have physical labels that contradict the digital layout. That behaviour is the operational reality of the system. It never appears in an audit report.

Read the incident logbook, not just the incident database. Every operational site has a paper or shared-document logbook — shift handovers, unusual events, near-misses, workarounds that got left in place because nobody had time to fix them properly. This is where the low-frequency, high-magnitude scenarios hide. A near-miss two years ago that never became a ticket is a data point the ALE model would otherwise miss entirely.

Trace one attack path physically. Pick a hypothetical intrusion vector — a compromised engineering laptop, a poisoned firmware update, a rogue USB — and walk the physical path an attacker would follow to reach a consequence. Every door, every network segment, every jump host, every trust boundary. What you find is almost always different from the network diagram.

Interview the field engineer, not just the CISO. The CISO has a defensible model of what the CIRMP says. The field engineer has a defensible model of what the equipment does. Both are necessary. The gemba walk is the second one; it is the one most desk-based CRQ skips.

Record what nobody wrote down. Every OT engineer carries tacit knowledge about failure modes that vendor documentation has never captured. "That model of PLC has a memory bug if you leave it running past ninety days." "This substation has a physical bypass that engineering doesn't know about." "We had a similar incident in 2019 but it never got escalated." That knowledge is the highest-value data source in the entire CRQ pipeline, and it lives nowhere except in the practitioner's head.

Structured expert elicitation is genchi genbutsu at scale

The obvious objection is that this doesn't scale. Ohno could stand in a chalk circle at Toyota because there were a finite number of factory floors. A CISO responsible for hundreds of unscannable assets across dozens of sites cannot personally walk each gemba every quarter.

That is where structured expert elicitation earns its keep. The point of a well-designed elicitation process is to industrialise the gemba walk. Instead of relying on a single manager to visit every site, you turn each site's practitioner into a calibrated participant in the risk model. Their tacit knowledge — the same knowledge Ohno's chalk circle was designed to surface — enters the model as a probability estimate on a specific scenario, tied to a specific asset, aggregated with other practitioners' estimates, and tracked over time so the accurate ones carry more weight.

Done properly, this is genchi genbutsu with better economics. Every practitioner walking their own gemba, contributing what only they can see, into a single quantitative model that respects the variance. It has three properties that a spreadsheet-only CRQ cannot match.

First, it captures tacit knowledge before it walks out the door. The retirement of one senior OT engineer at a critical infrastructure operator can vaporise a decade of unrecorded risk knowledge. Structured elicitation makes that knowledge institutional before it is lost.

Second, it produces auditable disagreement. When two experts on the same site give very different estimates on the same scenario, that disagreement is itself a signal — either the site has hidden complexity or the experts are drawing on different information. Both are worth investigating. A single-expert or committee-consensus process buries this signal; structured elicitation surfaces it.

Third, it improves. As scenarios resolve — either through incidents or through explicit resolution windows — the calibration data compounds. Practitioners who are consistently accurate become more heavily weighted. Practitioners who are consistently over- or under-confident are corrected. The model gets better as the organisation uses it, in a way a spreadsheet fed by static questionnaires cannot.

Five things to do this quarter

The bottom line

Ohno did not draw the chalk circle because he distrusted his managers. He drew it because he understood that the representation of a system, no matter how carefully built, cannot substitute for the actual place and the actual thing. Every industrial discipline that took his insight seriously — lean manufacturing, six sigma, human factors, high-reliability organisations — built practices to keep the gemba walk institutional.

Cyber risk management is one of the last high-consequence disciplines that has not. For scannable assets, it does not have to; the scanner is a proxy gemba walk. For unscannable assets, the proxy does not exist. The choice is between doing genchi genbutsu on cyber — through structured elicitation of the people who actually run the systems — or attesting to a risk figure that is a representation of a representation of a representation.

Boards, regulators, and insurers are getting less patient with the second option. The Enhanced CIRMP Rules, APRA CPS 234, DORA, and SOCI enforcement are all, in effect, asking for evidence of the first. That evidence looks like calibrated expert input tied to specific assets and specific scenarios, aggregated into a quantified loss distribution the board can defend.

Ohno would have called it obvious. He would also have pointed out that seventy years is a long time to keep re-learning the same lesson.

The chalk circle is still available. The substation is still there. The people who know the actual thing are, for now, still willing to tell you what they see. Ask them.

Related reading

• Taiichi Ohno — Toyota Production System: Beyond Large-Scale Production (Productivity Press, 1988) — the primary source for genchi genbutsu.
• CyQuantiFi — Nine Asset Classes. Five Hazards. One Number Your Board Now Has to Defend.
• CyQuantiFi — Everyone's Quantifying Cyber Risk Now. Almost No One Is Checking the Inputs.