Chapter Five. Knowledge and Information Problems
in the Large Organization
Although the analysis of information problems in large organizations has been refined
in many constructive ways, little special insight is required to recognize their bare
existence. They have been acknowledged even by corporate managers:
One of the executive vice-presidents of the Union Carbide Corporation...
remarked in a private conversation that he and his colleagues "had no idea how to
manage a large corporation." He said they simply did not know enough of the
corporate workings, nor did they know what to do even if a clear problem was
identified.1
A. The Volume of Data
From the beginning of organization theory as a distinct discipline, numerous writers
have remarked on the central feature of information problems: the sheer volume of data
to be processed within organizations, and their inadequacy for doing so.
Herbert Simon introduced the concept of "bounded rationality." Although human
behavior in an organization was "intendedly rational," he said, it was only "limitedly so";
there were limits to an individual's "ability to make correct decisions" in an organization:
The limits of rationality have been seen to derive from the inability of the human
mind to bring to bear upon a single decision all the aspects of value, knowledge, and
behavior that would be relevant.2
In a later work he argued that "traditional economic man" was ill suited to the theory
of organization. It was necessary to "take account... of the empirical limits on human
rationality, of its finiteness in comparison with the complexities of the world with which
it must cope." The "principle of bounded rationality," stated in so many words, was that
The capacity of the human mind for formulating and solving complex problems is
very small compared with the size of the problems whose solutions is required for
objectively rational behavior in the real world--or even for a reasonable
approximation to such rationality.3

1

Barry Stein, Size, Efficiency, and Community Enterprise (Cambridge: Center for Community Economic
Development, 1974), p. 49.
2
Herbert Simon, Administrative Behavior (New York: The Free Press; London: Collier-Macmillan
Limited, 1945, 1947, 1957), pp. xxiii-xxiv, 39, 108.
3
Simon, "Rationality and Administrative Decision-Making," in Simon, Models of Man: Social and
Rational (New York, London, Sydney: John Wiley & Sons, Inc., 1957) p. 198.

Friedrich Hayek's groundbreaking article on distributed (or idiosyncratic) knowledge,
"The Use of Knowledge in Society," was written in the context of the ongoing socialist
calculation debate, and directed primarily at the inability of state central planners to
replace the price mechanism as a system for processing information. But it is also highly
applicable to similar attempts by central planners within the corporation to replace the
market with hierarchy.
If we possess all the relevant information, if we can start out from a given system of
preferences and if we command complete knowledge of available means, the problem
which remains is purely one of logic. That is, the answer to the question of what is
the best use of the available means is implicit in our assumptions. The conditions
which the solution of this optimum problem must satisfy have been fully worked out
and can be stated best in mathematical form: put at their briefest, they are that the
marginal rates of substitution between any two commodities or factors must be the
same in all their different uses.
This, however, is emphatically not the economic problem which society faces....
The peculiar character of the problem of a rational economic order is determined
precisely by the fact that the knowledge of the circumstances of which we must make
use never exists in concentrated or integrated form, but solely as the dispersed bits of
incomplete and frequently contradictory knowledge which all the separate individuals
possess. The economic problem of society is thus not merely a problem of how to
allocate "given" resources--if "given" is taken to mean given to a single mind which
deliberately solves the problem set by these "data." It is rather a problem of how to
secure the best use of resources known to any of the members of society, for ends
whose relative importance only these individuals know. Or, to put it briefly, it is a
problem of the utilization of knowledge not given to anyone in its totality.4
Hayek's list of assumptions in the first paragraph, by the way, sound remarkably like the
neoclassical model of the firm as a simple "production function," with the most efficient
combination of factors determined by technical considerations. His allocation of "given"
resources, likewise, foreshadows the concept of "allocative efficiency," as opposed to "xefficiency," which we examined in Chapter Two.
He went on to apply his concept of distributed knowledge more specifically to the
production process, coming up something much like Michael Polanyi's "tacit knowledge."
Of course, Hayek in turn was anticipated by Chester Barnard, who wrote about the
"know-how" or "behavioral knowledge" which was "necessary to doing things in concrete
situations" but was "not susceptible of verbal statement."5 At any rate, Hayek wrote:

4

Friedrich A. Hayek, "The Use of Knowledge in Society," The American Economic Review, Vol. 35, No. 4.
(Sept. 1945), pp. 519-20.
5
Chester Barnard, The Functions of the Executive (Cambridge: Harvard University Press, 1938), p. 291.

...a little reflection will show that there is beyond question a body of very important
but unorganized knowledge which cannot possibly be called scientific in the sense of
knowledge of general rules: the knowledge of the particular circumstances of time
and place. It is with respect to this that practically every individual has some
advantage over all others in that he possesses unique information of which beneficial
use might be made, but of which use can be made only if the decisions depending on
it are left to him or are made with his active cooperation. We need to remember only
how much we have to learn in any occupation after we have completed our theoretical
training, how big a part of our working life we spend in learning particular jobs, and
how valuable an asset in all walks of life is knowledge of people, of local conditions,
and special circumstances....
If we can agree that the economic problem of society is mainly one of rapid
adaptation to changes in the particular circumstances of time and place, it would seem
to follow that the ultimate decisions must be left to the people who are familiar with
these circumstances, who know directly of the relevant changes and of the resources
immediately available to meet them.6
Something very like Hayek's idiosyncratic knowledge was described by Michael
Polanyi as "tacit knowledge." The basic rules of an art, he said, are useful only when
integrated into a practical knowledge of the art which is gained by experience; otherwise,
they are mere maxims.7 The practical knowledge, in many cases, cannot be reduced to a
verbal formula for transmission.
An art which cannot be specified in detail cannot be transmitted by prescription,
since no prescription for it exists. It can be passed on only by example from master to
apprentice. This restricts the range of diffusion to that of personal contacts, and we
find accordingly that craftsmanship tends to survive in closely circumscribed local
traditions...
It follows that an art which has fallen into disuse for the period of a generation is
altogether lost. There are hundreds of examples of this to which the process of
mechanization is continuously adding new ones. These losses are usually
irretrievable. it is pathetic to watch the endless efforts--equipped with microscopy
and chemistry, with mathematics and electronics--to reproduce a single violin of the
kind the half-literate Stradivarius turned out as a matter of routine more than 200
years ago.8
A great deal of technique cannot be reduced to a verbal formula because it is

6

"The Use of Knowledge in Society," pp. 521-22, 524.
Michael Polanyi, Personal Knowledge: Towards a Post-Critical Philosophy (New York and Evanston:
Harper & Row, Publishers, 1958, 1962), p. 50.
8
Ibid., p. 53.
7

unconscious, based on an acquired feel for the tools in one's hand, and built into one's
muscular memory like the technique for riding a bicycle.9
This is of real practical importance for industry. The great technical research
laboratories of modern industry, seeking to apply scientific method to the analysis of
production techniques, first faced the daunting task of reducing the knowledge of
traditional craft production into a form they could understand--i.e., "of discovering what
actually was going on there and how it was that it produced the goods." In the case of the
study of cotton spinning in the 1920s, "most of the initial decade's work on the part of the
scientist will have to be spent merely in defining what the spinner knows." "...[E]ven in
the modern industries the indefinable knowledge is still an essential part of technology."10
If it makes sense to keep production decisions as close as possible to direct
knowledge of the production process (or "that decisions be made by those closest to
where the real work is being done," as Jeffrey Nielsen put it)11, then the worker
cooperative would seem to be the ideal form of organization for aggregating knowledge.
More generally, as we shall see later, top-down systems of authority present inherent
knowledge problems because those with direct experience of the matter under
consideration must follow policies made by those without such direct experience; and
those making the policies must base their decisions on information which has been
distorted by several rungs of hierarchy between those with the process-knowledge and
those with the power.
Continuing in the last-quoted passage, Hayek elaborated further on the kinds of
idiosyncratic knowledge involved in the production process:
To know of and put to use a machine not fully employed, or somebody's skill which
could be better utilized, or to be aware of a surplus stock which can be drawn upon
during an interruption of supplies, is socially quite as useful as the knowledge of
better alternative techniques....
Is it true that, with the elaborate apparatus of modern production, economic
decisions are required only at long intervals, as when a new factory is to be erected or
a new process to be introduced? Is it true that, once a plant has been built, the rest is
all more or less mechanical, determined by the character of the plant, and leaving little
to be changed in adapting to the ever-changing circumstances of the moment?
The fairly widespread belief in the affirmative is not, so far as I can ascertain,
borne out by the practical experience of the business man. In a competitive industry
at any rate--and such an industry alone can serve as a test--the task of keeping cost

9

Ibid., pp. 61-62.
Ibid., p. 52.
11
Jeffrey Nielsen, The Myth of Leadership: Creating Leaderless Organizations (Palo Alto, Calif.: DaviesBlack Publishing, 2004), p. 8.
10

from rising requires constant struggle, absorbing a great part of the energy of the
manager. How easy it is for an inefficient manager to dissipate the differentials on
which profitability rests, and that it is possible, with the same technical facilities, to
produce with a great variety of costs, are among the commonplaces of business
experience which do not seem to be equally familiar in the study of the economist.12
This is quite close to what Barry Stein wrote on the the importance, cumulatively, of
incremental changes in the production process, which might well have a greater effect on
productivity than simply building a new factory with the latest generation of equipment.
And as Stein pointed out, in largely the same terms as Hayek, the workers directly
engaged in the production process are, more than anyone else, possessed of the
specialized knowledge of how to tweak the process in order to improve productivity.
The kinds of specialized distributed knowledge described in the paragraph above are
also, in the hands of labor, a source of enormous agency problems. The possession of
idiosyncratic knowledge can be parlayed into considerable information rents. As we shall
see in later discussions of on-the-job labor struggle, the special knowledge of workers can
be used by workers to slow down work or hamper the profitability of the enterprise, in
ways which it is almost impossible for management to adequately monitor or assign
blame. Likewise (as we shall see in Chapter Nine), such idiosyncratic knowledge
drastically degrades (in its lack) the performance of "replacement workers," and involves
enormous costs of replacing and training a new labor force in the event of a strike or
lockout. Many right-wing libertarians, hostile to organized labor, are fond of asserting
that unions would be powerless without the use of force to exclude non-union workers.
But in a free market, the disruptive potential attending localized knowledge over the work
process, and the transaction costs of replacing workers, would be (especially given the
incomplete nature of the labor contract and the resulting pervasiveness--in Williamson's
words--of bargaining) a great source of bargaining power.
Of course, the fact remains that the individual with idiosyncratic information is
himself ignorant of much of the larger environment within which he operates. The price
system, Hayek wrote, is ideally suited to coordinating the information dispersed among
many such individuals. The individual, with his specific knowledge of time and place,
"cannot decide solely on the basis of his immediate surroundings. There still remains the
problem of communicating to him such further information as he needs to fit his
decisions into the whole pattern of changes of the larger economic system."13
Fundamentally, a system where the knowledge of the relevant facts is dispersed
among many people, prices can act to coordinate the separate actions of different
people in the same way as subjective values help the individual to coordinate the parts
of his plan.14

12

"The Use of Knowledge in Society," p. 522.
"The Use of Knowledge in Society," pp. 524-25.
14
"The Use of Knowledge in Society," p. 526.
13

If this is a point for the market system against state planning, it is also a point for the
market system against the internal hierarchy of the corporation. Everything Hayek says
about the ability of planners to do an adequate job of aggregating distributed information
in the economy at large applies to the ability of management to aggregate distributed
information within the large corporation. Everything Hayek says about the calculation
problems attending the replacement of the market by administrative decisionmaking in a
centrally planned economy applies, equally, to reliance on administrative decisionmaking
within the centrally planned corporation (about which more in Chapter Seven).
Oliver Williamson described the worker's power over the production process resulting
from distributed knowledge in terms very like Hayek and Polanyi:
Almost every job involves some specific skills. Even the simplest custodial tasks
are facilitated by familiarity with the physical environment specific to the workplace
in which they are being performed. The apparently routine operation of standard
machines can be importantly aided by familiarity with the particular piece of
operating equipment.... In some cases workers are able to anticipate the trouble and
diagnose its source by subtle changes in the sound or smell of the equipment.
Moreover, performance in some production or managerial jobs involves a team
element, and a critical skill is the ability to operate effectively with the given
members of the team....
...[T]ask idiosyncracies can arise in at least four ways: (1) equipment
idiosyncracies, due to incompletely standardized, albeit common, equipment, the
unique characteristics of which become known through experience; (2) process
idiosyncracies, which are fashioned or "adopted" by the worker and his associates in
specific operating contexts; (3) informal team accommodations, attributable to mutual
adaptation among parties engaged in recurrent contact but which are upset, to the
possible detriment of group performance, when the membership is altered; and (4)
communication idiosyncracies with respect to information channels and codes that are
of value only within the firm.15
But he used this, believe it or not, as an argument for hierarchy. Idiosyncratic knowledge
(or "task idiosyncracies"), he said, were a kind of asset specificity which led to small
numbers bargaining problems in the market, which in turn could be solved by replacing
the market with hierarchy. That's rather odd, considering that hierarchy tends to make
idiosyncratic knowledge less usable by reducing the control by any one individual over
matters under his direct observation.
It's odd, as well, because Williamson admits elsewhere that the very same task

15

Markets and Hierarchies, Analysis and Antitrust Implications: A Study on the Economics of Internal
Organization (New York: Free Press, 1975), pp. 5, 62-63.

idiosyncracies that result in small-numbers exchanges in the market persist as information
rents from impacted knowledge within an organization. For example, the process of onthe-job training, by which incumbents in possession of idiosyncratic knowledge are
expected to pass it on to new employees. The danger, Williamson says, is that the
incumbent employees "will hoard information to their personal advantage and engage in a
series of bilateral monopolistic exchanges with the management...."16 And this is not
always just the moral equivalent of price-gouging--it can also be legitimate self-defense,
in an environment where the interests of workers and management are often diametrically
opposed, and workers' knowledge can be used against them. As Dave Pollard points out:
Employees hoard rather than sharing knowledge, including knowledge that could
yield innovation, to protect their position and rank in the company.
Employees rarely volunteer new ideas, fearing ridicule, retribution, being ignored,
or having credit for the idea stolen by their boss if it succeeds.17
Although Williamson doesn't mention it, the same phenomenon occurs even when a
plant is fully staffed, to the advantages of an incumbent workforce against a management
that might attempt to replace it during a strike. One of the information rents of
idiosyncratic knowledge, in the case of collective bargaining, is the costs of training
replacement workers without the cooperation of the striking incumbents, and the long
learning curve during which productivity will be seriously degraded by the lack of the
incumbents' idiosyncratic knowledge. We will examine this problem at greater length in
Chapter Nine.
B. The Distortion of Information Flow by Power
Our consideration of Williamson's treatment of information rents in a hierarchy,
immediately above, suggests another problem with information in the large organization.
In addition to the basic problems caused by the sheer volume of data and the inability of
hierarchical organizations to process it, information problems are complicated by power
relations within the bureaucracy.
As early as 1932, F. C. Bartlett published a study of serial reproduction of information
that had a strong bearing on the transmission of information in a hierarchy. The
experiment was a fancy version of the child's game "telephone," where a bit of
information is repeated around a circle from person to person and comes out
unrecognizable at the end. In this case, a line drawing of an owl was transformed by
serial reproduction into "a recognizable cat." Bartlett drew the conclusion:

16

Ibid., p. 63.
David Pollard, "A Prescription for Business Innovation: Creating the Technologies that Solve Basic
Human Needs (Part Two)" How to Save the World, April 20, 2004
<http://blogs.salon.com/0002007/2004/04/20.html>.
17

It is now perfectly clear that serial reproduction normally brings about startling
and radical alterations in the material dealt with. Epithets are changed into their
opposites; incidents and events are transposed; names and numbers rarely survive
intact for more than a few reproductions; opinions and conclusions are reversed-nearly every possible variation seems as if it can take place, even in a relatively short
series. At the same time the subjects may be very well satisfied with their efforts,
believing themselves to have passed on all important features with little or no change,
and merely, perhaps to have omitted unessential matters.18
The same principle is illustrated by this ancient piece of office humor, endlessly
circulated by the Samizdat of photocopier and Internet:
In the beginning was the Plan.
And then came the Assumptions.
And the assumptions were without form.
And the plan was without substance.
And darkness was upon the faces of the workers.
And they spoke among themselves saying, "It is a crock of shit and it stinks."
And the workers went unto their Supervisors and said, "It is a pail of dung and we cannot
live with the smell."
And the supervisors went unto their Managers saying, "It is a container of organic waste
and it is very strong such that none may abide by it."
And the managers went unto their Directors, saying, "It is a vessel of fertilizer and none
may abide by it."
And the Directors spoke among themselves, saying to one another, "It contains that
which aids plant growth and it is very powerful."
And the Vice Presidents went to the President, saying unto him, "This new plan will
actively promote growth and vigor of the company with very powerful effects."
And the president looked upon the plan and saw that it was good.
And the plan became Policy.

18

F.C. Bartlett, Remembering (New York: Cambridge University Press, 1932), quoted in Oliver
Williamson, Economic Organization: Firms, Markets, and Policy Control (New York: NYU Press, 1986),
p. 35.

And this is how shit happens.

Oliver Williamson saw the experiment as a lesson on the distortion of information
within a hierarchy. On that subject, he wrote:
Communications distortions can take either assertive or defensive forms.
Defensively, subordinates may tell their supervisor what he wants to hear; assertively,
they will report those things they want him to know.... Distortion to please the
receiver is especially likely when the recipient has access to extensive rewards and
sanctions in his relations with the transmitter, as in up-the-line communication in an
administrative hierarchy.... The cumulative effects across successive hierarchical
levels of... adjustments to the data easily result in gross image distortions... and
contribute to a limitation of firm size....19
(Or rather, under state capitalism, because of limits to the competitive ill effects of such
distortions, they contribute to the low levels of efficiency typical of the dominant firms.)
Other thinkers have made similar observations, but drawn more radical conclusions
from them. For example, Kenneth Boulding wrote:
Another profitable line of study lies in economic sociology, in the analysis of the
way in which organizational structure affects the flow of information, hence affects
the information input into the decision-maker, hence affects his image of the future
and his decisions, even perhaps his value function. There is a great deal of evidence
that almost all organizational structures tend to produce false images in the decisionmaker, and that the larger and more authoritarian the organization, the better the
chance that its top decision-makers will be operating in purely imaginary worlds.
This is perhaps the most fundamental reason for supposing that there are ultimately
diminishing returns to scale.20
R. A. Wilson also remarked on the informational problems of hierarchies. For
Wilson, the distortions that occur as information is filtered through a hierarchy result not
just from errors of replication, but from systematic distortion in a particular direction.
Information is distorted by power relationships within a hierarchy.
….in a rigid hierarchy, nobody questions orders that seem to come from above, and
those at the very top are so isolated from the actual work situation that they never see
what is going on below....21
Every authoritarian logogram divides society, as it divides the individual, into

19

Markets and Hierarchies, pp. 122-23.
Kenneth Boulding, "The Economics of Knowledge and the Knowledge of Economics," American
Economic Review 56: 1/2 (March 1966), p. 8
21
R.A. Wilson, The Illuminatus! Trilogy, p. 388.
20

alienated halves. Those at the bottom suffer what I shall call the burden of nescience.
The natural sensory activity of the biogram--what the person sees, hears, smells,
tastes, feels...--is always irrelevant and immaterial. The authoritarian logogram, not
the field of sensed experience, determines what is relevant and material…. The person
acts, not on personal experience and the evaluations of the nervous system, but on the
orders from above….
Those at the top of the authoritarian pyramid, however, suffer an equal and
opposite burden of omniscience…. They must attempt to do the seeing, hearing,
smelling, tasting, feeling and decision-making for the whole society.
But a man with a gun is told only that which people assume will not provoke him
to pull the trigger. Since all authority and government are based on force, the master
class, with its burden of omniscience, faces the servile class, with its burden of
nescience, precisely as a highwayman faces his victim. Communication is possible
only between equals. The master class never abstracts enough information from the
servile class to know what is actually going on in the world where the actual
productivity of society occurs…. The result can only be progressive disorientation
among the rulers.22
A civilization based on authority-and-submission is a civilization without the
means of self-correction. Effective communication flows only one way: from mastergroup to servile-group. Any cyberneticist knows that such a one-way communication
channel lacks feedback and cannot behave "intelligently."
The epitome of authority-and-submission is the Army, and the control-andcommunication network of the Army has every defect a cyberneticist's nightmare
could conjure. Its typical patterns of behavior are immortalized in folklore as SNAFU
(situation normal—all fucked-up), FUBAR (fucked-up beyond all redemption) and
TARFU (Things are really fucked-up). In less extreme, but equally nosologic, form
these are the typical conditions of any authoritarian group, be it a corporation, a
nation, a family, or a whole civilization.
Proudhon was a great communication analyst, born 100 years too soon to be
understood. His system of voluntary association (anarchy) is based on the simple
communication principles that an authoritarian system means one-way
communication, or stupidity, and a libertarian system means two-way communication,
or rationality.
The essence of authority, as he saw, was Law — that is, fiat — that is, effective
communication running one way only. The essence of a libertarian system, as he also
saw, was Contract — that is, mutual agreement — that is, effective communication

22

Ibid., p. 498.

running both ways. ("Redundance of control" is the technical cybernetic phrase.)23
You know I think I began to realize the danger of hierarchy and developed the
snafu principal about communication when I was working for the second largest
engineering firm in the United States. I listened to the engineers bitching all the time
about how the financial interests wouldn’t let them do any of the work that seemed
really important for them to improve their output. And I was reading William
Faulkner's Go Down Moses, which is still one of my favorite novels, and there was a
sentence in there which was like a mini satori for me. And the sentence goes: "To the
sheriff, Lucas was just another nigger and they both knew that; to Lucas the sheriff
was an ignorant redneck with no cause for pride in his ancestors, nor any hope for it in
his prosperity. But only one of them knew that." And I suddenly realized, yeah, every
power situation means the people on top are not being told what the people on the
bottom are really noticing. Then I could see how this applied to this engineering firm.
And then how it applied to corporations in general and so on.24
Or as Hazel Henderson quoted Bertram Gross, "organizations are devices for
screening out reality in order to focus attention on their own specific goals."
....they regularly intercept, distort, impound, or amplify information, structuring it for
their own needs and channeling employees' efforts toward their own goals...
A person with great power gets no valid information at all.25
Hierarchy, by impeding the exchange of information, works against the very purposes
for which cooperative, group production is undertaken in the first place. According to
Peter Blau and Richard Scott, the superiority of group over individual production results
from three factors:
(1) the sifting of suggestions in social interactions serves as an error-correction
mechanism;
(2) the social support furnished in interaction facilitates thinking; and
(3) the competition among workers for respect mobilizes their energies for
contributing to the task.26

23

Robert Anton Wilson, "Thirteen Choruses For the Divine Marquis," from Coincidance - A Head Test.
<http://www.deepleafproductions.com/wilsonlibrary/texts/raw-marquis.html> Originally published in The
Realist.
24
Lance Bauscher, Utopia USA interview with Robert Anton Wilson. 22 Feb 2001
<http://www.deepleafproductions.com/utopialibrary/text/raw-inter-utopia.html>.
25
Hazel Henderson, "Coping With Organizational Future Shock," Creating Alternative Futures: The End
of Economics (New York: G.P. Putnam's Sons, 1978), p. 225.
26
Peter M. Blau and W. Richard Scott. Formal Organizations: A Comparative Approach (San Francisco:
Chandler Publishing Co., 1962), p. 121.

Hierarchy interferes with these tendencies. It reduces social interaction and support. It
also sets up barriers to mutual respect by reducing the lower-ranked individual's potential
for acquiring respect, and the higher-ranked individual's respect for the performance of
those at lower rungs of the hierarchy. The inflated importance of recognition by superiors
also weakens the importance of mutual esteem among peers.27 Finally, hierarchy distorts
the error-correcting function of interaction, by increasing the perceived costs of correcting
the errors of a superior.28
These interferences by hierarchy in the information-aggregating process are affirmed
by Melville Dalton's case studies of corporate management. He cites the social insularity
of engineers and other staff officers, at a factory he studied, and their coolness toward the
foremen and line supervisors. Some foremen, in interviews with the author, stated their
habit of avoiding the management cafeteria, despite an interest in the work problems
being discussed there, because of the aloof attitude of the engineers. Such differences in
status, Dalton observed, "discourage easy informal ties between staffs and many middle
and lower line supervisors," and "prevent staff people from getting close to
situations...."29
Nevertheless, from the perspective of senior management, these information problems
were viewed as a necessary evil justified by the benefits to social control. Dalton quotes
an administrator of a factory division:
Coming right from college, the boys can never have any first-hand experience
with workers, so it's easy for them to get management's viewpoint. You can see that a
man coming up from the shops will have a worker's viewpoint. He may not be able to
shake it off, but go around the rest of his life appearing to have management's view of
things....
....In addition to paying for their salaries many times over, these young engineers
are grateful for their jobs and have nothing but management's interest in mind.30
The sheer cost of this phenomenon--the failure of those in authority to take advantage
of production workers' distributed knowledge, because of exclusion or fear--is suggested
by an example from consultants Ken Cloke and Joan Goldsmith:
In a large entertainment industry organization, the leader of an operations unit came
up with a sweeping plan to increase the quality of customer service delivered by his
department. He met with middle managers to share his plan and ask them to
communicate it with staff who directly interacted with customers. Not only did his

27

Ibid., p. 122.
Ibid., p. 123.
29
Melville Dalton, Men Who Manage (New York: John Wiley & Sons, Inc., 1959), p. 94.
30
Ibid., p. 99.
28

direct reports communicate in clumsy, half-hearted, inconsistent ways, they failed to
give clear directions or assign responsibilities for results. Staff with direct
responsibility for customer service were resentful that they had not been consulted or
asked for their ideas beforehand and consequently interpreted his message as a
criticism of their work. Because middle managers had no direct relationship with
customers, they were unable to lead or participate in problem solving with staff, and
everyone went off in separate directions, leading to a decline in customer service.
Six; months later, we were asked to help the organization address this failure. In a
meeting with the division as a whole, employees were asked to communicate their
thoughts about how to solve the problem. At first, they were reluctant to speak in
front of their managers, but in the safety of small groups, they put forth several
proposals, which they integrated into a single plan to reorganize the entire division.
Their plan involved moving every manager into a direct service delivery role and
creating a new structure in which the entire staff was organized into self-managing
customer-oriented teams.
Their proposal was so bold and convincing that it was clear that nothing would
change in customer service if they did not implement it. After a lengthy process of
debate, dialogue, and conflict resolution, they overcame the determined resistance of
managers who felt they were being demoted, and the proposal was approved by
leadership and brought to life by employees. There was no downsizing of managers,
who were assigned to new, challenging, and ultimately more satisfying positions
working directly with customers and teams.
Because employees participated fully in the process, they felt highly motivated
and responsible. Teams were trained in consensus and collaborative processes and
were empowered to immediately implement solutions to the problems they faced.
Once managers had direct experiences with customers and staff were given adequate
trust, authority, and responsibility for producing results, they identified a number of
successful ideas for improving customer service....
...Empowered staff were given authority to solve customer problems directly or
enlist the services of others and provide "one stop shopping" for customers, who no
longer had to wade through layers of bureaucracy to solve their problems. Results
included improved customer satisfaction, elevated morale, and substantial savings by
removing unnecessary layers of management.31
Information is also distorted by the fact that the end-user of the information relies on
information sorters so far removed from him as to have little idea of what will be useful
and what will not. Lester Thurow, for example, observed

31

Kenneth Cloke and Joan Goldsmith, The End of Management and the Rise of Organizational Democracy
(New York: John Wiley & Sons, 2002), pp. 7-9.

[W]ith the onset of the new information technologies, ordinary bosses could
implement what extraordinary bosses had always preached. Bosses could do a lot
more bossing....
To do so, however, one had to build up enormous information bureaucracies.
Information could be gotten, but only at the cost of adding a lot of white-collar
workers to the system....
To the boss, more information seems like a free good. He orders it from
subordinates, and the cost of acquiring it appears on the budgets of his subordinates.
Subordinates in turn can neither refuse to provide the requested information nor know
if the information is valuable enough to justify the costs of its acquisition....
Essentially, both bosses and subordinates are imprisoned in standard operating
procedures that create an institutional set of blinders.32
Not surprisingly, this leads to a huge glut of useless information, as bureaucracies
generate the maximum level of information input to make themselves appear useful and
to insure themselves against blame, with little idea of what is useful and what is not. At
the same time, management makes decisions to suit its own interests, but justifies them
by genuflecting toward the information. Thus information becomes a legitimizing
ideology.
Martha Feldman and James March found little relationship between the gathering of
information and the policies that were ostensibly based on it. In corporate legitimizing
rhetoric, of course, management decisions are always based on a rational assessment of
the best available information. And in the neoclassical view of the firm as production
function,
information is gathered and used because it helps to make a choice. Investments
in information are made up to the point at which marginal expected cost equals
marginal expected return.33
This [conventional] perspective on decision making leads to some simple
expectations for information utilization. For example, relevant information gathered
for use in a decision will be examined before more examination is requested or
gathered; needs for information will be determined prior to requesting information;
information that is irrelevant to a decision will not be gathered.

32

Lester Thurow, Head to Head: The Coming Economic Battle among Japan, Europe, and America (New
York: William Morrow, 1972), pp. 171-72, in David M. Gordon, Fat and Mean: The Myth of Managerial
Downsizing, p. 76.
33
Martha S. Feldman and James G. March, "Information in Organizations as Signal and Symbol,"
Administrative Science Quarterly 26 (April 1981), p. 182

Studies of the uses of information in organizations, however, reveal a somewhat
different picture. Organizations seem to deal with information in a different way
from that anticipated from a simple reading of decision theory.34
Feldman and March did case studies of three organizations, and found an almost total
disconnect between policies and the information they were supposedly based on:
Considerable information was gathered by the organizations involved in the
decisions. Considerable information was sometimes volunteered by other
organizations. There was little systematic relation between the time of receiving the
results of a study and the time of making a decision. There was no obvious consistent
relation between the finding of studies and the decision made. Information was
gathered. More information was sought. Information was considered. But the link
between decisions and information was weak.
...The literature reports phenomena that can be summarized by six observations
about the gathering and use of information in organizations....: (1) Much of the
information that is gathered and communicated by individuals and organizations has
little decision relevance. (2) Much of the information that is used to justify a
decision is collected and interpreted after the decision has been made, or substantially
made. (3) Much of the information gathered in response to requests for information is
not considered in the making of decisions for which it was requested. (4) Regardless
of the information available at the time a decisions is first considered, more
information is requested. (5) Complaints that an organization does not have enough
information to make a decision occur while available information is ignored. (6) The
relevance of the information provided in the decision-making process to the decision
being made is less conspicuous than is the insistence on information. In short, most
organizations and individuals often colelct more information than they use or can
reasonably expect to use in the making of decisions. At the same time, they appear to
be constantly needing or requesting more information, or complaining about
inadequacies in information.35
Feldman and March did their best to provide a charitable explanation--an explanation,
that is, other than "organizations are systematically stupid."36 "Systematically stupid"
probably comes closest to satisfying Occam's Razor, and I'd have happily stuck with that
explanation. But Feldman and March struggled to find some adaptive purpose in the
observed use of information.
They began by surveying more conventional assessments of organizational
inefficiency as an explanation for the observed pattern. First, organizations are "unable...
to process the information they have. They experience an explanation glut as a shortage.

34

Feldman and March, p. 172
Feldman and March, p. 174.
36
Feldman and March, p. 174.
35

Indeed, it is possible that the overload contributes to the breakdown in processing
capabilities...." Second, "...the information available to organizations is systematically
the wrong kind of information. Limits of analytical skill or coordination lead decision
makers to collect information that cannot be used."37
Then they made three observations of their own on how organizational structure
affects the use of information:
First, ordinary organizational procedures provide positive incentives for
underestimating the costs of information relative to its benefits. Second, much of the
information in an organization is gathered in a surveillance mode rather than in a
decision mode. Third, much of the information used in organizational life is subject
to strategic misrepresentations.
Organizations provide incentives for gathering more information than is optimal
from a strict decision perspective.... First, the costs and benefits of information are
not all incurred at the same place in the organization. Decisions about information
are often made in parts of the organization that can transfer the costs to other parts of
the organization while retaining the benefits....
Second, post hoc accountability is often required of both individual decision
makers and organizations....
Most information that is generated and processed in an organization is subject to
misrepresentation....
The decision maker, in other words, must gather excess information in anticipated
defense against the possibility that his decision will be second-guessed.38 By
"surveillance mode," the authors mean that the organization seeks out information not for
any specific decision, but rather to monitor the environment for surprises. The lead time
for information gathering is longer than the lead time for decisions. Information must
therefore be gathered and processed without clear regard to the specific decisions that
may be made.39
The incentives Feldman and March discussed so far all seem to result mainly from
large size and hierarchy. The problem of non-internalization of the costs and benefits of
information-gathering by the same actor, of course, falls into the inefficiency costs of
large size. The problem of post hoc accountability results from hierarchy. At least part of
the problem of surveillance mode is another example of poor internalization: the people
gathering the information are different from the ones using it, and are therefore gathering
it with a second-hand set of goals which does not coincide with their own intrinsic

37

Feldman and March, p. 175.
Feldman and March, pp. 175-76.
39
Feldman and March, p. 176.
38

motives.
The strategic distortion of information, as an agency problem, is (again) the result of
hierarchy and the poor internalization of costs and benefits in the same responsible actors.
In other words, the large, hierarchical organization is "systematically stupid."
The authors' most significant contribution in this article is their fourth observation:
that the gathering of information serves a legitimizing function in the organization.
Bureaucratic organizations are edifices built on ideas of rationality. The cornerstones
of rationality are values regarding decision making....
The gathering of information provides a ritualistic assurance that appropriate
attitudes about decision making exist. Within such a scenario of performance,
information is not simply a basis for action. It is a representation of competence and a
reaffirmation of social virtue. Command of information and information sources
enhances perceived competence and inspires confidence. The belief that more
information characterizes better decisions engenders a belief that having information,
in itself, is good and that a person or organization with more information is better
than a person or organization with less. Thus the gathering and use of information in
an organization is part of the performance of a decision maker or an organization
trying to make decisions intelligently in a situation in which the verification of
intelligence is heavily procedural and normative....
Observable features of information use become particularly important in this
scenario. When there is no reliable alternative for asserting a decision maker's
knowledge, visible aspects of information gathering and storage are used as implicit
measures of the quality and quantity of information possessed and used....40
In other words, when an organization gets too big to have any clear idea how well it is
performing the function for which it officially exists, it creates a metric for "success"
defined in terms of the processing of inputs.
This adoption of extrinsic measures as proxies for real productivity, when the
organization is incapable of measuring productive work, extends far beyond the specific
task of information-gathering. When (as Paul Goodman put it in a quote below)
management is incapable of knowing "what a good job of work is," a proxy measure must
be found. One such false metric is "face time," as opposed to actual work, as blogger
Atrios observed:
During my summers doing temp office work I was always astounded by the culture of
"face time" - the need to be at your desk early and stay late even when there was no

40

Feldman and March, pp. 177-178.

work to be done and doing so in no way furthered any company goals. Doing your
work and doing it adequately was entirely secondary to looking like you were working
hard as demonstrated by your desire to stay at work longer than strictly necessary.41
One of Atrios' commenters, in considerably more pointed language, elaborated:
If you are a manager who is too stupid to figure out that what you should actually
measure is real output then the next best thing is to measure how much time people
spend pretending to produce that output. Of course you really should know what the
output you should measure really consists of. If you don't know that then you are sort
of forced into using the time spent measurement.
But in fairness to management, it's not the stupidity of the individual; it's the stupidity
of the organization. All large, hierarchical organizations are stupid. Some exceptional
individuals are better than average at compensating for the stupidity of the organization,
and securing--through a combination of Herculean effort and genius--some minimal
performance in spite of it; but it's a feat comparable to driving a car with the emergency
brake on. (The problem may also result from management being too smart. In many
cases, management adopts an irrelevant metric because maximizing it has the incidental
effect of promoting their own bureaucratic interests, whereas maximizing a more relevant
measure might require the diversion of resources that management would prefer to devote
to empire building and self-dealing--but that's a matter more suited to later chapters on
agency problems and managerialism.)
More importantly, though, the gathering of information provides reassurance that
management decisions are rational and based on the best possible information, and
therefore secures acquiescence to management authority:
Belief in the appropriateness of decisions, the process by which they are made,
and the roles played by the various actors involved is a key goal of a social
structure.... Ritual acknowledgement of important values celebrates a shared
interpretation of reality.... Thus, requesting information and assembling it are ways of
making social life meaningful and acceptable....
....Decisions are orchestrated so as to ensure that decision makers and observers
come to believe that the decisions are reasonable--or even intelligent. Using
information, asking for information, and justifying decisions in terms of information
have all come to be significant ways in which we symbolize that the process is
legitimate, that we are good decision makers, and that our organizations are well
managed..42

41

"Face Time," Eschaton blog, July 9, 2005
<http://atrios.blogspot.com/2005_07_03_atrios_archive.html#112049256079118503>.
42
Feldman and March, pp. 177-178.

The way that information-gathering, in Feldman and March's analysis, serves to
engineer the acceptance of decisions as "legitimate," bears a striking resemblance to the
tendency in American political culture for citizens to rally around the government even
(or especially) in catastrophic wars, on the assumption that "they have access to
information that we don't." I've witnessed it myself in the workplace, unfortunately. A
coworker of mine in a VA hospital where I used to work frequently rallied to the defense
of the MBA types' clueless decisions, on the grounds that "they went to school and took
special classes to make decisions."
It's remarkable how often professional decisionmaking bureaucracies, supposedly
privy to almost unlimited information, are blinded by groupthink and institutional
cultures, while those outside the decision loop with far more modest amounts of
information are able to get a clearer picture of reality simply by subjecting the
bureaucracy's unquestioned assumptions to the test of common sense. The tendency of
insular decision-making circles toward over-optimism, and refusing to take possible
negative or untinded consequences into consideration, was the subject of Irving Janis'
Group Think.
Robert Jackall, in Moral Mazes, described the legitimization of management
decisions in terms similar to those of Feldman and March, albeit from a much more jaded
perspective. Jackall referred to
the difficulties of assessing to what extent functionally rational devices actually are
used in making decisions, particularly by higher-ups. Vocabularies of rationality are
always invoked to cloak decisions, particularly those that might seem impulsive when
judged by other standards.43
Such vocabularies of rationality are invoked, especially, in the face of management
policies that an outside observer might perceive as examples cynical self-dealing:
...just after the CEO of Covenant Corporation announced one of his many purges,
legitimated by "a comprehensive assessment of the hard choices facing us" by a major
consulting firm, he purchased a new Sabre jet for executives and a new 31-foot
company limousine for his own use.... He then flew the entire board of directors to
Europe on a Concorde for a regular meeting to review, it was said, his most recent
cost-cutting strategies.44
Feldman and March sum up their attempt to rescue the corporation from charges of
"systematic stupidity":
It is possible, in considering these phenomena, to conclude that organizations and

43

Robert Jackall, Moral Mazes: The World of Corporate Managers (New York: Oxford University Press,
1988), p. 75.
44
Ibid. p. 144.

the people in them lack intelligence. We prefer to be somewhat more cautious. We
have argued that the information behavior observed in organizations is not, in general,
perverse. We have suggested four broad explanations for the conspicuous overconsumption of information. First, organizations provide incentives for gathering
extra information.... Second, much of the information in organizations is gathered
and treated in a surveillance mode rather than a decision mode.... Third, much of the
information in organizations is subject to strategic misrepresentation.... Fourth,
information use symbolizes a commitment to rational choice.45
But in fact, their own argument proves that the organization is systematically stupid,
in terms of its own official rationale for existing in the first place. We have already
argued that the first three explanations all involve excessive size and hierarchy, poor
internalization of the positive and negative results of decisions by decisionmakers, and
the separation of knowledge from authority. As for the fourth, organizations elevate the
collection of useless information into a legitimizing ideology, and substitute a symbolic
metric for genuine rationality, because they are systematically stupid. The health of the
organizational apparatus (in the same sense as Bourne's "war is the health of the state")
supplants the organization's original purpose for existing. The consumption of inputs is
redefined as an output of the organization. But if the organization is viewed as a means
to an end, rather than an end in itself, then the emperor has no clothes.
And it is hardly necessary to attribute stupidity to individuals in order to explain the
functional stupidity of the organization. Because of the pathologies of large size and
excessive hierarchy, the organization in effect provides a sort of invisible hand
mechanism by which individuals, by maximizing their utility in a rational manner given
the environment of incentives, collectively promote inefficiency and irrationality. That,
to my thinking, is the very definition of "perverse."
As if to verify this assessment, Feldman and March go on to describe the
circumstances in which conspicuous over-consumption of information is likely to occur:
The kinds of information behavior noted here should be more common in
situations in which decision criteria are ambiguous than in situations where they are
clear, more common where performance measures are vague than where they are
precise, more common when decision quality requires a long period to establish than
when there is quick feedback, more common where the success of a decision depends
on other decisions that cannot be predicted or controlled than where a decision can be
evaluated autonomously.46
In other words, it is more common in situations where the organization is so large that
nobody has any clear idea of what's going on, what other people are doing, or what the

45
46

Feldman and March, p. 182.
Feldman and March, p. 183.

purpose of action even is. It is more common in situations where decisionmaking
authority is removed from those in direct contact with the problem, who are most capable
of directly assessing what needs to be done and monitoring the results of action, and
given to those separated from such knowledge by several rungs of authority. It is more
common in situations where authority flows downward, with each rung of hierarchy
interfering with those below who are better informed, and receiving orders from those
above who are even more clueless, until one reaches Boulding's "completely imaginary
world" at the apex of the pyramid.
But I repeat, the systematic stupidity of the large, hierarchical organization is perfectly
compatible with the individual competence of those making it up. The inefficiencies of
size and hierarchy are such, as Paul Goodman wrote, that nobody could do an effective
job of running it:
Assume, for the sake of analysis, that the top-direction of a very large centralized
corporation is very wise and devoted to the goal of the organization. Nevertheless,
being one man or a small group, top-management does not have enough mind to do an
adequate job....
Top management cannot be departmentalized. A manager cannot restrict himself
to policy, but must be the final judge of application to doubtful and new cases as well.
If because of pressure of time unique cases are treated as routine, a manager's expert
judgment is useless.
....A policy is decided, and to make sure that it is understood and correctly
executed, it is simplified and a procedure is standardized. In a large organization such
standardization is essential.... But of course the standard misfits every actual
instance.... It is almost impossible for the best procedure to be used except
clandestinely, or for the best man to be employed unless he goes through unusual
channels....
Subordinates tend to become stupider more rapidly and directly, simply because
they cannot learn anything by exercising initiative and taking responsibility.
Stultification occurs acutely when a man is bright and sees a better way to do
something, but must follow a worse directive.47
Although the organization may contain a great deal of expertise and skill, severally,
the individual's expertise is useless when nobody has authority to apply it directly to a
problem on his own initiative. The whole is less than the sum of its parts. To quote
Goodman again:
When the social means are tied up in such complicated organizations, it becomes

47

Paul Goodman, People or Personnel, pp. 76-79.

extraordinarily difficult and sometimes impossible to do a simple thing directly, even
though the doing is common sense and would meet with universal approval, as when
neither the child, nor the parent, nor the janitor, nor the principal of the school can
remove the offending door catch....
....[A]s the feeling of powerlessness spreads, there is a deep conviction that
"Nothing Can Be Done" because of the machinery that has to be set in motion, even
when the problem or the abuse is simple and something can easily be done.48 (For an
example of this, see the Appendix to this Chapter Six, "Toilet Paper as Paradigm.")
One reason the individual skills and competencies of the organization's members are
unusable is that idiosyncratic knowledge is poorly, or not at all, transferrable. The
production worker is second-guessed by management who are not only less qualified than
he to judge competence in his line of work, but less qualified to make the framework of
rules within which he practices his specialty.
In my opinion, the salient cause of ineptitude in promotion and in all hiring
practices is that, under centralized conditions, fewer and fewer know what is a good
job of work.... Just as there is reliance on extrinsic motives, there is heavy reliance
[in the large university] on extrinsic earmarks of competence: testing, profiles,
publications, hearsay among wives, flashy curricula vitae. Yet there is no alternative
method of selection. In decentralized conditions, where a man knows what goes on
and engages in the whole enterprise, an applicant can present a masterpiece for
examination and he has functional peers who can decide whether they want him in the
guild....
There is no test for performing a highly departmentalized role except evidence of
playing a role and of ability at routine skills. Inevitably, the negative criteria for
selection become preponderant... and so the whole enterprise becomes still stupider....
In brief, as those who judge--colleagues, consumers, the electorate--become stupid,
management also becomes stupid. So after a while we cannot maintain the
assumption that in established firms top-management can be wise and capable.49
As increased division of labor within the organization leads to functions being
stovepiped, it takes a progressively longer time for those in one department to receive the
knowledge they need for their own functioning, and progressively more overhead and
time are consumed in aggregating inputs from the different departments for making an
organization-wide policy. And because the knowledge of one specialty is poorly
reducible or summarizable, the person in one department is a poor judge of exactly what
information or other inputs another department needs, and a generalist senior manager is

48
49

Goodman, People or Personnel, pp. 88, 91.
Goodman, People or Personnel, pp. 83-84.

a poor judge of the inputs he is aggregating from the departments below. In the case of
education,
there is an immense increase in the number of administrators themselves. With
centralization, standardization, and "efficiency," the ratio of teachers to students may
fall. But the ratio of administrators in the population will rise perhaps even more than
proportionately....
My guess is that the more "efficiently" the academic machine is run, the more
expensive it is per unit of net value....50
Goodman, taking the example of Columbia University, estimated the cost per capita if
students hired instructors directly and paid market rents on the buildings, and found that
actual tuition charges were
four times as much as is needed to directly pay the teachers and the rent! This seems
to be an extraordinary mark-up for administration and overhead.51
As we saw in Part One, the large corporation survives, in spite of internal
diseconomies from information problems, because it is insulated from the competitive ill
effects of inefficiency. Because of the greater inefficiency costs of hierarchy, as R.
Preston McAffee and John McMillan argue, the hierarchical firm can only exist in a
monopoly market, and the length of hierarchies varies inversely with the competitiveness
of markets. The firm must be a net beneficiary of monopoly, so that the inefficiency costs
of hierarchy can be subsidized by rents drawn from the rest of the economy.52
Private information creates a cost of operating a hierarchy, which becomes larger
as the hierarchical distance between the information source and the decision maker
increases. When information about a firm's capabilities is dispersed among the
individuals in the firm, production is inefficient even though everyone behaves
rationally. Because hierarchies need rents in order to function, a firm with a long
hierarchy may not be viable in a competitive industry.53
Rents... are the lubricants that make it possible for a hierarchy to function.... [I]f
larger firms mean longer hierarchies, then potential rents must be present for a large
firm to be viable. Thus firms are small because the industry is competitive.54
Do monopolies produce above minimum cost, causing a welfare loss beyond the

50

Goodman, The Community of Scholars, pp. 241-242.
Goodman, The Comminity of Scholars, pp. 241-242.
52
R. Preston McAffee and John McMillan, "Organizational Diseconomies of Scale," Journal of Economics
& Management Strategy, Vol. 4, No. 3 (Fall 1995): 399-426.
53
Ibid., p. 399.
54
Ibid., p. 402.
51

thoroughly explored allocative inefficiencies? Conversely, does competition force
minimum-cost production? Generations of economists have believed that
competition provides the discipline needed to induce managers to make relatively
efficient production decisions. Adam Smith said that monopoly is "a great enemy to
good management, which can never be universally established but in consequence of
that free and universal competition which forces everybody to have recourse to it for
the sake of self-defense."55
In an oligopoly market, the typical firm can afford to be inefficient and bureaucratic
because all the firms in the market share the same institutional cultures, the same
management assumptions, and the same conventional patterns of organization. Indeed,
when the state's subsidies and protections for large size cause large size to be typical in a
given market, the typical firm cannot be otherwise than inefficient.
The proliferation of useless information, described by Feldman and March above, has
a synergistic relationship to the expansion of bureaucracy. Lloyd Dumas argues that the
proliferation of paperwork and new forms within a bureaucracy leads to the creation of
new managerial positions to take the burden off of existing managers; but these new
managers simply generate even more paperwork. And this paperwork, in turn, is used to
justify the hiring of more administrative personnel--and so on.56 This, Dumas says, is the
basis of Parkinson's Law.57
Conclusion and Segue
In addition to all this, information problems are a necessary precondition for agency
problems. The information problems of the large organization are such that those lower
in the hierarchy are usually desperate to make those at the top aware of how things really
are; nevertheless, as we have already seen Williamson suggest, the strategic witholding
or monopoly of information by agents is a source of rents against the principal.
We have already seen this, in the discussion above of the information rents attending
idiosyncratic knowledge. The "assertive distortions" that Williamson referred to fall on
the indistinct boundary line between information and agency problems. Self-dealing
from information rents is a form of opportunism. And more generally, assymetrical
information is central to most agency problems. As we quoted Williamson in the
Introduction to Part III, opportunism wouldn't be a problem without bounded rationality;
otherwise it would be possible to rule it out ahead of time with comprehensive
contracting covering every possible contingency.

55

Ibid., pp. 414.
Lloyd Dumas, The Overburdened Economy: Uncovering the Causes of Chronic Unemployment,
Inflation, and National Decline. (Berkeley, Los Angeles, London: University of California Press, 1986),
pp. 65-66.
57
C.N. Parkinson, Parkinson's Law, or the Pursuit of Progress (London: John Murray, 1958).
56

The basic agency problem as resulting from information asymmetries, as Paul
Milgrom and John Roberts described:
...some of the information that is important for the organization to make good
decisions is not directly available to those charged with making the decisions.
Instead, it is lodged with or producible only by other individuals or groups that are not
empowered to make the decisions but may have a direct interest in the resulting
outcome.58
Or to put it the other way around, what Williamson calls "information impactedness" is
more a problem of opportunism than of bounded rationality. It is
mainly attributable to the pairing of uncertainty with opportunism. It exists in circumstances
in which one of the parties to an exchange is much better informed than is the other
regarding underlying conditions germane to the trade, and the second party cannot achieve
information parity except at great cost--because he cannot rely on the first party to disclose
the information in a fully candid manner.59
The reason why outsiders are not at a parity with insiders is usually because outsiders
lack firm-specific, task-specific, or transaction-specific experience.60

This results, as we shall see in our chapter on managerialism, in rents accruing to
corporate management at the expense of outside investors. But same principle, inside the
corporation, limits the managerial hierarchy's effective control over the productive labor
force.

Appendix 5A: The NHS's IT Program as an
Example of Systematic Stupidity61
[From Alex at Yorkshire Ranter:]
"The inspiration to digitize this far-flung bureaucracy first surfaced in late 2001,
when Microsoft's Bill Gates paid a visit to British Prime Minister Tony Blair at No.
10 Downing St. The subject of the meeting, as reported by The Guardian, was what

58

Paul Milgrom and John Roberts, "An Economic Approach to Influence Activities in Organizations,"
American Journal of Sociology, Supplement to vol. 94 (1988), p. S156.
59
Williamson, Markets and Hierarchies, p. 14.
60
Ibid., p. 31.
61
Alex Harrowell, "HOW NOT TO Build a Computer System," Yorkshire Ranter, November 19, 2006
<http://yorkshire-ranter.blogspot.com/2006/11/hownotto-build-computer-system.html>. The quotes are
from "UK Dept of Health: Prescription for Disaster," Baseline Magazine, November 13, 2006.
<http://www.baselinemag.com/c/a/Projects-Management/UK-Dept-of-Health-Prescription-for-Disaster>

could be done to improve the National Health Service. At the time, much of the
service was paper-based and severely lagging in its use of technology. A long-term
review of NHS funding that was issued just before the Blair-Gates meeting had
concluded: "The U.K. health service has a poor record on the use of information and
communications technology—the result of many years of serious under-investment."
Coming off a landslide victory in the 2001 general election, Blair was eager to
move Britain's health services out of technology's dark ages. Gates, who had come to
England to tell the CEOs of the NHS trusts how to develop integrated systems that
could enhance health care, was happy to point the way. "Blair was dazzled by what
he saw as the success of Microsoft," says Black Sheep Research's Brampton. Their
meeting gave rise to what would become the NPfIT."
[Alex helpfully comments, "Couldn't they have introduced him to Richard Stallman?"
And remaining helpful, he adds boldface to the project's key failures:]
After a February 2002 meeting at 10 Downing St. chaired by Blair and attended
by U.K. health-care and Treasury officials as well as Microsoft executives, the NPfIT
program was launched.
In quick order, a unit was established to purchase and deliver I.T. systems
centrally. To run the entire show, NHS tapped Richard Granger, a former Deloitte
and Andersen management consultant. Granger signed on in October 2002 at close
to $500,000 a year, making him the highest-paid civil servant in the U.K., according
to The Guardian.
In one of his first acts, Granger commissioned the management consulting
company McKinsey to do a study of the massive health-care system in England.
Though the study was never published, it concluded, according to The Guardian, that
no single existing vendor was big enough to act as prime contractor on the
countrywide, multibillion-dollar initiative the NHS was proposing. Still, Granger
wanted to attract global players to the project, which meant he needed to offer up
sizable pieces of the overall effort as incentives....
The process for selecting vendors began in the late fall of 2002. It was centralized
and standardized, and was conducted, Brennan and others say, in great secrecy. To
avoid negative publicity, NHS insisted that contractors not reveal any details about
contracts, a May 2005 story in ComputerWeekly noted. As a byproduct of these hushhush negotiations, front-line clinicians, except at the most senior levels, were
largely excluded from the selection and early planning process, according to
Brennan.
[Alex summarizes the failures:]
First of all, letting the producer interest poison the well. Microsoft execs, eh? The big

centralised-bureaucratic proprietary system vendor Microsoft was permitted to
influence the whole process towards a big centralised-bureaucratic proprietary system
from the very beginning. This occurred at a time when Health Secretary Alan Milburn
was constantly railing against "producer interests" blocking his "modernising
reforms". This was code for the trade unions that represented low-waged nurses and
cleaners, and the British Medical Association that represented doctors. Can anyone
spot the difference between the two groups of producer interests? One of these things
is not like the other..
The managerialists inevitably called on a management consultant to run the show
- as we all know, we are living in a new world, and the status quo is not an option, so
nobody who actually knew anything about the NHS, hospitals, or for that matter
computers could be considered. (Granger failed his CS degree.) With equal
inevitability, he called on management consultants to tell him what to do. The great
global consulting firm McKinsey duly concluded that only great, global consulting
firms could do the job.
Choosing which ones was clearly a job only central authority could undertake, and
the intervention of the press, the unions, competitors or elected representatives would
only get in the way, so the whole thing vanished behind a cloud of secrecy. Secrecy
enhances power. It does this by exclusion. The groups excluded included the doctors,
nurses, technicians and administrators of the NHS - which means that the canonical
mistake, the original sin of systems design was predetermined before the first
requirements document was drawn up or the first line of code written. Secrecy
specifically excluded the end users from the design process. There are two kinds of
technologies - the ones that benefit the end-user directly, and the ones that are
designed by people who think they know what they want. They can also be described
as the ones that succeed and the ones that fail. Ignore the users, and you're heading for
Lysenkoism.
Among the "problems" of the NHS system was that most hospitals had their own
computer systems, developed either by small IT firms or in-house. The contracts
stated that each of the five new regional service providers and the "spine" (BT) would
have to replace them, design a single regional system, but also maintain "common
standards" nationally. The sharp will spot the contradiction. If you have common
standards for information exchange, why can't you have them within the region as
well as between regions? Why do you need the regional system at all? Why do you
need the big global consulting firm - standards, after all, are for everyone, from
Google to the hobby programmer cranking out a few lines of Python or such. In fact,
almost all developments in computing in the last 10 years have been in the direction
of separating levels of abstraction. It doesn't matter if the web server runs Linux and
the database Windows Server if they both speak XML at the application layer.
This was actually recognised for some purposes. The NHS bought 900,000
desktop licences for MS Windows and further commissioned Microsoft to develop a

common interface for the NPfIT, thus ensuring that any common interface would be
proprietary and unalterable except by Microsoft. But no-one seems to have thought
through the implications of common standards. Instead, the contracts specified that
the old systems must be torn out and the data transferred to the new, thus adding a
huge sysadmin nightmare to the costs.
Trying to keep down the costs, iSoft outsourced the development to India. But the
Thomas Friedman dream of hordes of crack coders as cheap as chips showed some
flaws - specifically:
the programmers, systems developers and architects involved didn't comprehend
some of the terminology used by the British health system and, more important,
how the system actually operated, the CfH conceded.
Neither did IDX's developers working with Microsoft in Seattle know anything
about the NHS. This choice, like the secrecy, ensured that no NHS institutional
memory would be available to the developers. So, 100 medics were shipped off to the
coder farm to explain. Naturally, this effort to fix fundamental architecture problems
by tinkering just added complexity and cost, as Pareto's theory of the second best bit.
Eventually, one of the regional systems contractors decided to take iSoft's off-theshelf product and hack it into something vaguely suitable, and another walked away.
IDX and GE Healthcare's product was so dire that even BT couldn't make more than
one implementation work in two and a half years, and then sacked them.
But, there is no sign any of this will affect policy whatsoever. Instead, the
managers content themselves with intermediate statistical targets (apparently they are
installing 600 N3 lines a month, a rather poor performance for any normal ISP),
rigged definitions (the deal with Microsoft is said to have saved £1.5 billion compared to what? certainly not open-source..) and bully rhetoric about feeding the
slower huskies to the faster ones (I am not joking). The inevitable signs of failure,
meanwhile, emerge - it doesn't work.
"As an example, in July, mission-critical computer services such as patient
administration systems, holding millions of patient records being provided by the
CSC alliance across the Northwest and West Midlands region, were disrupted
because of a network equipment failure, according to the CfH. As a result, some
80 trusts in the region were unable to access patient records stored at what was
supposed to be either a foolproof data center or a disaster recovery facility with a
full backup system. Every NPfIT system in the area was down for three days or
longer. Service was fully restored and no patient data was lost, the CfH says.
That was not the first such failure. In fact, in the past five months more than 110
major incident failures having to do with NHS systems and the network have been
reported to the CfH, according to ComputerWeekly."

But, of course, the users are lying and everything is wonderful.
"The CfH responded in an e-mail to Baseline: "It is easy to misinterpret the
expression 'major incident.' Some of these could have been, for example,
individual users experiencing "slow running." We encourage reporting of
incidents, and we are open and transparent about service availability levels,
which we publish on our Web site."
Perhaps they'll put the chocolate ration up there too.

