Validation Through Intimidation		Date	2023-02-01
		Tags

This post is a short one to start the year and is about an an­ti-­pat­tern that I have ob­served within sys­tems en­gi­neer­ing pro­jects. I sin­cerely hope that this pat­tern is not en­demic, though I sus­pect its more preva­lent in the dis­ci­pline than we’d like to ad­mit. This an­ti-­pat­tern is a pos­si­ble mech­a­nism to the preva­lence of project cost & time over­runs and com­plex sys­tems of­ten be­ing a dis­ap­point­ment to their cus­tomers. I call this an­ti-­pat­tern Val­i­da­tion Through In­tim­i­da­tion.

Val­i­da­tion

First­ly, val­i­da­tion is the name given to the process of en­sur­ing that the model that the en­gi­neer­ing project has (be that the re­quire­ments or the de­sign) is an ac­cu­rate and pre­cise enough de­scrip­tion of—or so­lu­tion to—the cus­tomer’s prob­lem in the en­vi­ron­ment. This is op­posed to ver­i­fi­ca­tion which is the process of en­sur­ing that our de­sign is cor­rect against the in­ter­nal re­quire­ments. A sys­tem can pass all ver­i­fi­ca­tion tests but still be in­valid if we have mis­un­der­stood the cus­tomer needs (or they have changed dur­ing the course of the project be­neath our un­der­stand­ing). Where ver­i­fi­ca­tion is a process in­ter­nal to a project to en­sure co­her­ence, val­i­da­tion is ul­ti­mately ex­ter­nal as it re­quires in­put from the stake­hold­ers and knowl­edge of the con­tex­tual en­vi­ron­ment of the sys­tem un­der de­sign. Ver­i­fi­ca­tion is there­fore the in­duc­tive and de­duc­tive process of en­sur­ing in­ter­nal co­her­ence of project where val­i­da­tion is the ab­duc­tive process of en­sur­ing co­her­ence be­tween the project and the wider en­vi­ron­ment.

Val­i­da­tion can only be con­ducted in con­junc­tion with the cus­tomer. Only the cus­tomer truly knows their own de­sire and a pro­duc­t/ser­vice can only truly be con­sid­ered valid once the cus­tomer needs are ful­filled in re­al­i­ty. How­ev­er, cus­tomers are rarely have a sin­gle co­her­ent need. A set of cus­tomer re­quire­ments will be con­structed from a set of de­sires fil­tered through each other which may end up with a bit of a con­tra­dic­tory mess or, worse, a re­quire­ment set that does­n’t re­flect the en­vi­ron­ment and needs on the ground.

An ex­am­ple of this is sys­tems en­gi­neer­ing tools pro­cure­ment in large com­pa­nies. The only peo­ple who have the au­thor­ity to pro­cure the ex­pen­sive tools that sys­tems en­gi­neers need are those who are usu­ally re­moved enough from the sys­tems en­gi­neer­ing re­al­ity that they do not have the knowl­edge re­quired to un­der­stand the sys­tems en­gi­neers’ needs. The sys­tems en­gi­neers ex­pressed needs are there­fore gar­bled or dis­re­garded by these man­agers and the tool that is fi­nally pro­cured does not meet the needs of the en­gi­neers who will ac­tu­ally use it.

Or­gan­i­sa­tional Pres­sures

The above prob­lem is com­pounded by the de­sires of the man­age­ment chain within a cus­tomer—a­gain, of­ten the cus­tomer does not have enough in­for­ma­tion (whether through in­com­pe­tence or the law of req­ui­site va­ri­ety) to make a fully in­formed de­ci­sion on a pro­cure­ment. Some­times that in­for­ma­tion is clouded by ad­ver­tis­ing pro­pa­ganda (after all, no one is ever fired for buy­ing IB­M…). Tra­di­tion­al­ly, cus­tomers tend to com­mit large sum of money in an ini­tial con­tract ne­go­ti­a­tion with the sup­pli­er. The re­quire­ments in this ne­go­ti­a­tion are the ones that the sup­plier will com­mit to even if the cus­tomer needs are likely to change in the fu­ture.

There­fore, an ini­ti­a­tion of a project al­ways comes with the risk that the sup­plier can­not meet your needs and you ef­fec­tively lose that large chunk of cash. There is noth­ing wrong with this—as the old say­ing goes “There’s noth­ing wrong with be­ing wrong but every­thing wrong with stay­ing wrong”. In­di­vid­u­als who are man­ag­ing the project on the cus­tomer side though are of­ten pres­sured into “s­tay­ing wrong” due to the sunk cost fal­la­cy.

The sunk cost fal­lacy is a well doc­u­mented phe­nom­e­non of hu­man be­hav­iour. As quoted in Wikipedia:

Peo­ple demon­strate “a greater ten­dency to con­tinue an en­deav­our once an in­vest­ment in mon­ey, ef­fort, or time has been made.”

Ba­si­cal­ly, we are ir­ra­tionally pre­dis­posed to put more time, ef­fort and money into projects that we’ve al­ready put a lot of re­sources into de­spite the fact that the best course of ac­tion in these cases would be to abort the pro­ject. One cause of this is the fact that abort­ing a fail­ing project is tan­ta­mount to ad­mit­ting the project is fail­ing. We don’t like be­ing wrong, even if, at the point where the in­cor­rect de­ci­sion was made, there was in­suf­fi­cient in­for­ma­tion for us to know that the project would fail. This think­ing may ap­ply to the per­son mak­ing the de­ci­sion to sign a con­tract with a sup­plier or their man­age­ment, in which case, they may be pres­sured to con­tinue with that sup­plier at risk to los­ing their job—heav­ily dis­ci­pli­nar­ian work cul­tures of­ten in­cen­tivise peo­ple to hide prob­lems from their su­pe­ri­ors and lead to the Ther­mo­cline of Truth.

In­tim­i­da­tion

I of­ten jok­ingly de­scribe the phe­nom­e­non of Val­i­da­tion by In­tim­i­da­tion to my col­leagues us­ing the ex­am­ple of lock­ing your cus­tomer rep­re­sen­ta­tive in a room with co­pi­ous cof­fee but no food or ac­cess to the toi­let, only let­ting them out once they’ve signed off the val­i­da­tion pa­per­work. The truth is, the pres­sures on the cus­tomer rep­re­sen­ta­tive from their own or­gan­i­sa­tion to sign off the pa­per­work are pow­er­ful enough to force their hand.

Un­for­tu­nate­ly, sys­tems en­gi­neer­ing or­gan­i­sa­tions may be aware of this fact and can play it to their ad­van­tage. Given the or­gan­i­sa­tional pres­sures that ex­ist on the cus­tomer and there­fore the fact that these projects will most likely be ac­cept­ed, big projects are in­cen­tivised to re­duce en­gi­neer­ing ef­fort and max­imise prof­it.

This is es­pe­cially true if the or­gan­i­sa­tion in ques­tion has a mo­nop­oly over the pro­duc­t/ser­vice, is a large donor to the po­lit­i­cal party in power or has a pow­er­ful union that will pres­sure the gov­ern­ment into buy­ing from them. Think of the hold that a ship­build­ing com­pany may have over a gov­ern­ment if its the only ship­builder in the coun­try and hap­pens to be the only large em­ployer in a city that votes for the party in pow­er.

This is a monopoly-­monop­sony di­alec­tic, nei­ther can buy from nor sell to any­one else, the qual­ity of re­quire­ments, prod­ucts and ser­vices suf­fers and the re­sent­ment be­tween the two or­gan­i­sa­tions con­tin­ues to grow.

The cus­tomer is left feel­ing help­less and ul­ti­mate­ly, the end users have to work around their de­sires not be­ing met. Here are some ex­am­ples: G4S se­cu­rity at the 2012 Olympics; the Gen­eral Dy­nam­ics Ajax APC; The UK COVID Test and Trace sys­tem de­liv­ered by Serco amongst oth­ers; the list goes on. I would put money on these com­pa­nies con­tin­u­ing to re­ceive con­tracts in the fu­ture de­spite these fail­ures.

Pos­si­ble so­lu­tions

As the an­ti-­pat­tern de­scribed here is mostly a man­i­fes­ta­tion of the sunk cost fal­la­cy, one pos­si­ble so­lu­tion is to build a just cul­ture within your or­gan­i­sa­tion and spread aware­ness of the sunk cost fal­la­cy. Pro­cure­ment teams and man­age­ment need to be able to feel that they can cut the or­gan­i­sa­tion’s losses when in­for­ma­tion arises that shows them that a project is fail­ing. This way of work­ing should be en­cour­aged and mea­sures should be put in place to en­sure that the or­gan­i­sa­tion can work around hav­ing to abort the plan.

A sec­ond pos­si­ble so­lu­tion is for the sup­plier to work ac­cord­ing to em­piri­cism and keep fre­quent com­mu­ni­ca­tion with the cus­tomer. As with some process ap­proaches like Scrum, the cus­tomer is en­gaged to val­i­date an evolv­ing rep­re­sen­ta­tion of the fi­nal pro­duc­t/ser­vice in fre­quent re­views. This cre­ates a val­i­da­tion con­trol loop which can steer the project to­wards the cus­tomer needs with each re­view. Bal­ance needs to be found be­tween the length of time it takes to de­velop a re­view­able rep­re­sen­ta­tion in the project and the fre­quency of which the cus­tomer’s needs change to en­sure that this loop is­n’t dri­ven at res­o­nance. This ap­proach can also be sus­cep­ti­ble to Gaffer Daz­zling where a sup­plier keeps a sep­a­rate, un­rep­re­sen­ta­tive ver­sion of the project go­ing to mis­in­form the cus­tomer.

What do you think? Does the above ex­pla­na­tion fit with your ex­pe­ri­ences? Do you think the pro­posed so­lu­tions are prac­ti­cal, work­able or just plain wrong? Please leave any thoughts that you have in the com­ments be­low!

Site proudly generated by Hakyll | Theme based on Oskar Wickström's Monospace Web.