https://en.wikipedia.org/wiki/Stanislav_Petrov#Aftermath

> Petrov underwent intense questioning by his superiors about his judgment. Initially, he was praised for his decision.[2] Colonel-general Yuri Votintsev, the then-commander of the Soviet Air Defense's Missile Defense Units, who was the first to hear Petrov's report of the incident (and the first to reveal it to the public in the 1990s), states that Petrov's "correct actions" were "duly noted".[2] Petrov himself states he was initially praised by Votintsev and promised a reward,[2][22] but recalls that he was also reprimanded for improper filing of paperwork because he had not described the incident in the war diary.[22][23]

> Petrov has said that he was neither rewarded nor punished for his actions.[24] According to Petrov, he received no reward because the incident and other bugs found in the missile detection system embarrassed his superiors and the scientists who were responsible for it, so that if he had been officially rewarded, they would have had to be punished.[2][24][22][23] He was reassigned to a less sensitive post,[23] took early retirement (although he emphasized that he was not "forced out" of the army),[22] and suffered a nervous breakdown.[23]

Building the right incentives around that can be tricky, those incentives need to ensure the highest levels of management aren't themselves disincentivising their directs & their departments from surfacing pain & problems - but it's also pretty common for people to mask those signals purely out of a well-intentioned desire to help. It's important to coach people on the idea that in large group sizes, it's more efficient to let certain kinds of problems play out and not be so reactive to them.

Too many companies ground their performance incentives & processes around oversimplified ideas that don't match the reality of human behaviour

When things come up with other teams, you’ll have a catalog of tasks that were done to show why you didn’t have the same issue. The work was done, just at a better time to avoid downtime.

Often, 'leaders' make mistakes and people below suffer the consequences. It is important to let these leaders deal with the pain caused by their decisions from their cluelessness about how things work.

Nothing ever breaks. - "What are we paying you for?"

Management can choose their burden.

We've become too comfortable, since actual toil is no longer seen in the company: Manufacturing is overseas, customer support is overseas, logistics is an afterthought with established guarantees. Thus we want the mild weather and smooth meetings. If your engineering team is too smooth, maybe you should already branch out to help other related but "struggling" teams to get your hands dirty and noticed.

But when someone comes up with something simple but effective, it always looks so obvious in retrospect.

The director then clearly advised that they should use the complicated way because that's how you get published: not because you're clever, but because your solutions sound complicated.

It resonates perfectly with your comment and it's an unfortunate reality that most people don't bother for beautiful solutions and praise complicated processes. That's how we neded up with bureaucracy, probably :D

  > elegant solutions

And of course that's true. We push for speed, absent of direction, while praising velocity. To be honest, at this point I'm disappointed the engineers gave up and just started becoming business people.

"This is what my regular customers pay me. If I hired one of my friends or relatives I see it as my duty to pay them at least what they are worth, this is the way you raised me."

I believe this to be true btw. If someone is really your friend, you want them to do well and that means you pay what they usually get or you don't bother them and get someone else.

Also, telsa self-driving. yes, we know about the greatly publicized accidents, or the tweets of the founder, but the avoided incidents not so much.

https://www.boeing.com/company/bios/kelly-ortberg

Originally it was engineers from the top down, but over the last 15-20 years those leaders with engineering backgrounds have retired and been replaced by non-engineer MBA's. And the more I look around, the more I see that as a common trope is the US.

I’ll ask for something preventative or that otherwise hardens our systems. They ask “is it a need?” and I’ll say something like “we can function without, but that means we have a 5-10% chance in the next 6mo of having a major failure and embarrassing ourselves in front of a live audience in the thousands as well as our client.” They then decide how much that risk is worth to them, and whatever they decide is kind of out of my hands at that point. If the thing I warned them of comes because they didn’t pay for it, I can point to the receipts (though I’ve never had to, we’re small enough people remember those conversations).

60% of the time they just get what I need maybe? But ultimately it’s about CYA. Tell them what’s up, tell them what the solution is, tell them what the consequences are if they don’t do the solution, and make them decide.

Again this obviously depends on company culture and structure, but I can’t imagine on the only person who can do this!

I don't think that's it. Emergent problems require attention and action from leadership, who in turn can make the problem visible to higher ups. This creates signal, and positive feedback when the problem is fixed or mitigated.

If the problem doesn't exist to begin with, there is no signal. Managers don't get to show their fast-acting skills, and there are no heroics to speak of.

So ultimately poorly maintained and managed projects who deliver fixes for problems of their own doing create a perverse incentive, whereas no one is lauded or promoted for doing normal day-to-day things.

Speaking from experience, this does nothing. If you're at a company that is okay with average performers, then absolutely, 100%, fix all the bugs in advance, make the system rock solid and stable, prevent downtime, be a good engineer.

If on the other hand if you're at a company where 10% of people must get stack ranked and PIP, or at a company where "meets expectations" actually means you're going to get the stick, and you're supposed to be "redefining" expectations every year ... then yeah, don't do anything preventative. The optics are better when you take the 3am on-call and fix the issue (that you secretly knew in the first place would happen some time in the future in your coworker's code, and already knew how to fix -- but don't actually fix it until it surfaces). Be the savior that the VPs praise in the next meeting, that's your insurance against the PIP.

They set the rules of the game, you just play the game. The rules were their choice. They could have chosen different rules.

("I have made this longer than usual, only because I have not had the time to make it shorter.")

Blaise Pascal

  H: So, Watson.
  W: Hmm.
  H: You do not propose to invest in South African securities?
  W: How on earth do you know that?
  H: Now, confess, you are utterly taken aback.
  W: I am!
  H: I should make you sign a paper to that effect.
  W: Why?
  H: Because in a few minutes you will say it is all so absurdly simple.
  W: I should say nothing of the kind!
  H: You see, my dear Watson, it is not really difficult to construct a series of inferences, each dependent upon its predecessor and each simple in itself. If, after doing so, one simply knocks out the central inferences and presents one's audience with the starting point and the conclusion, one may produce a startling, though possibly a meretricious, effect.
  H: I can tell by an inspection of the groove between your left forefinger and thumb, that you have decided not to invest your small capital in the gold fields.
  W: I can see no connection.
  H: Very likely not; but I can quickly give you a close connection.
  H: Here are the missing links in the very simple chain: You had chalk between your forefinger and thumb when you returned from the club last night. You put chalk there when you play billiards, to ease the cue. You never play billiards except with Thurston. Now, Thurston, you told me, four weeks ago, had an option on some South African security which expired in a month, and which he desired you to share with him. Your checkbook is locked in my drawer, and you have not asked for the key. So, you do not propose to invest your money in that manner.
  W: How absurdly simple!
  H: Quite so. Every problem is absurdly simple when it is explained to you.

Note that there is also the flip side of the coin, people who spend all their time worrying about things that never happen, so it's not like you can just reward a defensive attitude things are more complicated than that.

If online comments are anything to go by, I'm not alone.

If you're in the Bay Area and you can get a Sonic fiber connection, I would highly recommend them over AT&T/Comcast/etc.

The difference is how it was communicated. Most non-Tech/non-infrastructure-people got no clue about these things. If they know you're battling the demons of plumbing on their behalf they will thank you, if you're the weird guy that has smeared dirt in the face and is seen once a week while the plumbing fails ever so often, guess what.

That means even if the problems and their fixes remain the same, the communication around them really matters. Tech people can be extremely bad with this. And if we're talking IT it is really the plumbing that holds the company together.

This is a pain signal. Some IT dude saying things are crap in every meeting is not.

Personally, I only rehire people from projects that went smoothly, not ones where I had to make the urgent phone call.

Teams that "just work" are highly valued. They clear up my attention for other things.

If someone is constantly playing the hero, I see that as incompetence. If the boss can’t see that, they are also incompetent. I have no respect for “leaders” who don’t know how to get out of the firefight.

I’ve made some high profile appearances, working 18 hour days on 4 day long outages, from vendor issues I was no part in causing. I figure that gives me some good will on playing hero without willingly creating problems for myself. I’m too old to manufacture stress for the optics.

For what it’s worth, with the right boss, I have had proper reporting work. Everything ran smooth and work was relaxed. My boss would regularly tell me I should take 3 months off because we were so far ahead of everyone. He would occasionally get bored and lob a grenade into the works to cause some chaos, but since everything else was running so smooth we were able to sort them out and keep going. People who couldn’t explain what they were doing were always getting yelled at and assumed to be doing nothing.

It’s not a problem in this industry, it’s a problem everywhere.

> I'm not sure how we solve this, other than having management come from engineering.

You mean the engineers who are causing the chaos you’re complaining about?

Engineers aren’t some magic group of people who know better than others - we’re just as fallible as other people.

"Accounted for X situation" "Added gaurdrails to protect against Y"

When working as a business analyst i have to do this sort of thing all hte time or else id get no credit for half my work

Obviously the only winning move here is not to play. Things like stack ranking are a perversion and no amount of compensation would be worth working for a company like that. If you choose to play, you're complicit in the moral abomination.

It’s extremely advanced technology, though, and most CEOs would rather rent seek / camp than give up some decision-making power (and very few are even aware it’s possible).

My colleague in the IT department had one idea, replace our commercial certificates with Let's Encrypt and drop the EV requirement. In total he'd stand to get a bonus of a little over €2000. He never got the money, because things like that was part of his job apparently.

I disagree with the implied idea here that "engineers are better managers". The solution is to have good management, not to assume that "engineers are better managers". I have seen good and bad managers, and in both groups there were engineers and non-engineers.

More often than not it is some IT dude observing network crap-out once a month, performing analysis, noticing an upward trend and then saying in every meeting that things are crap and there will be issues twice a week in some time.

> If you're an underfunded IT department and your network has an issue twice a week, you will get that funding.

More often than not, if the IT department is already neglected they will not get that funding. Things will be delayed until the crap outs eventually actually happen twice a week and then some external heroic consultants will be hired to fix the issue underfunded IT department "could not".

Which means that everyone is playing the game to not be cut.

Yeah, but then I wouldn't have been able to pay for my healthcare. A certain toxic company's health insurance paid for my care, though. Prior to joining said toxic company I'd be racking up $6000+ in healthcare bills a year with shitty startup-sponsored insurance.

After 2 years, it was decided I didn't play the hero well enough though, and ended up having to leave. I work for a less toxic company now, but the next time I need a heart-related surgery (likely in ~5-10 years) I'll join a toxic company in the months leading up to pay for it.

The rules of the US, I guess.

> I’m too old to manufacture stress

My point was less about manufacturing artificial stress. I don't do that. But many times I see issues in coworkers' code. If the company will value and praise me for catching and fixing them early, then by all means I'll do that. But if fixing issues in the codebase early for prevention only gets me criticism of "you haven't met expectations, we expect you to exceed expectations every performance cycle" then hell, I don't feel like fixing anything proactively. In that world I'd rather be the hero that fixes it when it surfaces, that's more likely to nail the rating.

In the long run, IME, you'll be recognized either by management or your peers if you keep doing that over and over again.

That's what the moneys for ...

But I'm not sure the author of this thread works in such a place. In that case the game is different.

In the case where the "urgent midnight fix" is important, it's necessary to promote the visibility of your (just working) team. If visibility is the game, then be visible.

You know how test-driven-dev was always "write the test first"? In that environment a test is always written before any code.

Well in the "ticket closing" scenario it's important to open a ticket, regardless of how trivial, for every code action taken. For every meeting attended. For every scenario dodged. If tickets are the way to score then write tickets.

If "being a hero" is the valuable thing, then be a hero. Be prepared to champion your team every chance you get. Every time you interact with management stress the emergency you just fixed (before it became an emergency.) Tomorrow do it again with the next thing.

Management needs visibility. Be visible. I know, this seems stupid and beneath you. But that's why they call it a job, not playtime.

I will say my motivation for helping other people avoid issues has dropped. If they want to make problems for themselves, they can. Me helping them hasn’t worked so far, so maybe some sleepless nights will be a better teacher.

I had a former boss call me Brent after reading the Phoenix Project. That made me step back and stop helping so much. Everything seems worse, but whatever… if that’s what they want.

And people ask why I hate humans.

― Dorothy Sayers

At some point in one's early single-digit they learn that touching hot stuff hurts. They start to avoid stuff that they know is hot, but still come in contact with hot stuff accidentally. Later they learn techniques minimizing probability of touching hot stuff even by accident. By the time one reaches twenty or so, the only times a person burns themselves is really by being way too reckless.

> Like it or not, sometimes the best thing for an organization isn't to just fix every problem and prevent it from bubbling up; it needs to be treated like a learning opportunity for org leadership, which means sending the pain signals upward before just repairing it.

Should we accept that management as a whole is in general more clueless than your average teenager? The "learning opportunity" should, ideally, happen exactly once, realistically once in a very rare while.

> It's important to coach people on the idea that in large group sizes, it's more efficient to let certain kinds of problems play out and not be so reactive to them.

You are conflating two things here, I guess. Yes, some "problems" are not worth to be fixed proactively or at all, but that has very little to do with group sizes, it's a "simple" cost-benefit tradeoff. As groups grow the left hands tend to become increasingly unaware of what the right is doing and that is the primary reason why we have management class in the first place.

The problem OP raises is attention span of the metaphorical gold fish in the management layers. Even if a department does everything in their power to communicate impending problems, do risk weighed cost-benefit analyses, get proactive treatments pre-approved by higher management, the same higher management forgets the risks and costs savings once they have been mitigated, effectively incentivizing firefighting. Some teams gradually fall into eternal firefighting and burn out, others start manufacturing fires to get rewarded. The biggest problem is that it is nearly impossible to tell the two apart.

more often than not, many things in the business are on fire and underfunded at the same time. you can get recognition for your work without the problem being permanently solved the right way, and it may not result in more funding but peopel will think of you for new opportunities that pop up later as someone who is reliable.

if you dont think the recognition will happen and youre just burning out solving these problems then stop solving them. new problem pops up thats outside your job description, its not your problem. generally though if youre working for someone like that anything you do is a lose-lose

Which resulted in top students 'undervaluating themselves' and bottom students 'overestimating themselves'. Or under/overvaluating a random future variable that they don't have knowledge of, at least.

The original DK paper actually shows a positive correlation between the guesses and the test results: students are generally aware how they are among their peers, and smarter students guessed higher than studetns with less time to study on their hand.

This being said, the 'DK effect' is something people talk about, and it might exist, and it might be perceived by people. It's just that the original DK paper does not support it.

* another lesser talked problem with the DK paper is that people don't actually believe the answers they give, because the question is nonsensical.

If someone just takes a test, they won't think that "I'm sure I'll end up the 24% this time". Even if they are forced to anwser this question, even then they won't believe it, because that's not how random and future work. People are generally aware of about where they will perform (with positive correlation, in fact the original DK paper shows it) but they are not aware of results of specific, random future events, and they are not claiming that they know the results of specific, random future events, or believe it in their hearts.

DK paper tries to frame them as they were actually believing this, but they are not.

More to read at: https://en.wikipedia.org/wiki/Dunning-Kruger_effect

For a more broad example than IT cost center stuff, you can look at how some large companies go through cycles of arrogance with their customer bases, launch a product that fails, and then are humbled enough to try and pivot and earn good will back. Microsoft is always somewhere in this cycle for instance. The organization can never really learn this lesson permanently and will "regress" from time to time based on financial pressure or greed or some other impulse.

I'd generally point to econometrics and statistics applied to business. The key activity is causal inference and then the context determines the mix of econo vs. stats required to help the org make high-quality decisions to increase output or make it more lucrative or higher-quality.

It seems reasonable to assume that for some group of intellects they are not smart enough to know how not smart they are. There is no definite boundary where this effect is either on or off, therefore there are probably some gradations to this awareness as you climb up the intelligence ladder.

Another way of putting it: if dumb people had more insight they would cease to be dumb.

If the policy is wrong and needs to be more specific, pay it out this time and change the policy. Don't just break your word.

The policy they think they've implemented is stupid. "Save money in someone else's department" is just going to create a ton of anger as people rush to step on each other's toes, and then those people have to constantly re-justify all the decisions they've made.

It's absolutely brain-dead.

With access to the SSL certificates.