Maintenance
Can you trust an operations case study without questioning the author?
Operations case studies get useful when you can question the author: eleven questions to send, what a non-answer tells you, and how to ask without giving offense.
What to take away
- The author is almost always reachable. Eleven short questions tell you more than the write-up alone.
- Ask about the gapswhat failed first, what else changed, who did the daily work.
- A vague answer is information. It usually means the number was never measured, not that the author is lying.
- Any answer describes a different operation, so the outcome should be a small test, not adoption.
The eleven questions to send
Send one message, numbered, and say one-line answers are fine. Most authors reply in writing.
I read your write-up on [the change] at [site]. We are looking at something similar at [our site]. Questions below, one-line answers fine, whichever are quickest. I will send back what happens if we try it.
- What did you change, and when did it start?
- How was the improvement measured, and who recorded the numbers?
- What else changed in the same periodheadcount, volume, equipment, season?
- What did you try before this that did not work?
- Who did the daily work, and how were they chosen?
- How many people, lines or shifts took part, and how many were left out?
- Is the figure a result you can still measure, or a target set at launch?
- What did it cost in money and staff hours, and who absorbed that?
- What did you stop doing to make room for it?
- What would you do differently if you ran it again?
- What would you need to see here before you believed it would work?
Eleven questions to send
- What did you measure before, and how was it defined?
- Over what period, and did that period contain anything unusual?
- What else changed at the same time?
- What did you try first that did not work?
- Who ran it, and what did they do that is not in the write-up?
- Is it still running, and what does the measure look like now?
- What broke that you did not expect?
- Who would this not work for?
Two questions that will not get you anywhere
"Do you think it would work for us?" They do not know your operation, so they will be polite. You get an encouraging answer that carries nothing.
"What is the return on this?" Almost nobody has an honest figure, and asking pushes them to build one that then travels with their name on it.
Asking without it sounding like an audit
Talking to the author is one thing. Talking to the people who do the work is another, and that skill is covered in how to interview for operating processes without wasting anyone's time.
- Say what you are considering, first. A question with context reads as interest. Alone it reads as scrutiny.
- Ask about failures as history. "What did you try first" is easier to answer than "what went wrong".
- Offer the trade. Promise to report back, which turns a request into an exchange.
Reading the answers
The form of the answer tells you how much weight the write-up carries. Reconstructed numbers look the same in a slide deck as recorded ones, and the difference shows only when you ask who wrote them down.
| Answer | What it means | Next move |
|---|---|---|
| Before and after figures, recorded weekly | The measurement was designed | Trust the rest of the account more |
| We did not measure it before | The improvement is an impression | Do not repeat the figure; test it |
| Nothing else changed | Usually nothing deliberate changed | Ask about headcount, volume, season |
| A long account of what failed first | The author was paying attention | Weight the transfer claim higher |
How to read the answers
What they say
- Precise definition
- Measurement was designed
- Did not measure before
- Improvement is an impression
- Nothing else changed
- Rare, usually deliberate only
- Long answer to Q4
- They were paying attention
- Silence on Q11
- Written to conclude
What it means
- Precise definition
- Raise credibility
- Did not measure before
- Cannot support a number
- Nothing else changed
- Follow up on staffing, volume
- Long answer to Q4
- Good sign
- Silence on Q11
- Polished past the edges
What to do
- Precise definition
- Did not measure before
- Nothing else changed
- Long answer to Q4
- Silence on Q11
Documents rarely hold invented numbers. They hold edited-out failures and rewritten timelines, the shapes collected in operations case studies mistakes: the traps and the way around them.
Example: the vague answer and the follow-up
You ask question 3: what else changed in the same period. The author replies: "Nothing significant changed."
That is vague, and it is not dishonest. It usually means the author is counting deliberate changes only, not a supervisor leaving or a seasonal peak.
Follow-up: "Was headcount the same on the first and last day? Was any new software installed? Was the quarter busier or quieter than the one before?"
Leave "significant" alone and ask for countable facts. That habit is what healthcare quality work calls root cause analysis, where the value sits in the questions asked.
When there is nobody to ask
Published accounts often have no reachable author, and industry write-ups often have no author in the sense that matters. Mark each of the eleven answers present, absent or vague.
What the test looks like, sized to two weeks, is in operations case study framework: turn one claim into a two-week trial. A document that answers three of eleven is a claim, not a source.
Where only successes get written up, the collection misstates the odds even when every document is accurate, the documented shape of publication bias.
What to do with the answers
Write the answers into your decision file in the author's words, dated. A verdict loses the material you will want in six months, when someone asks why you did this.
Judging whether a result travels is a probability estimate. Estimates of that kind get pulled around by the most recent example you saw, a bias set out under risk assessment. Answers do not settle the estimate on their own.
For which accounts repay a second read, see the operations case studies worth studying twice.
Then decide one of three things: run a small test, drop it, or ask one more question. Adopting outright is not on that list, however complete the answers, because they describe a different operation.
