|
The Cathedral and the Bazaar
Extended content
|
- Anyway, in a world of cheap PCs and fast Internet links, we find pretty consistently that the only really limiting resource is skilled attention.
- Investors are still thinking through the consequences of reinventing the software industry as one with an explicit focus on service rather than closed intellectual property, and will be for some time to come.
- The behavior of retailers when a vendor folds is very revealing. It tells us that they know something the vendors don't. What they know is this: the price a consumer will pay is effectively capped by the expected future value of vendor service (where "service" is here construed broadly to include enhancements, upgrades, and follow-on projects). In other words, software is largely a service industry operating under the persistent but unfounded delusion that it is a manufacturing industry.
- The reason this is a serious issue is that both the pool of users and the pool of talent available to be recruited into open-source cooperation for any given product category is limited, and recruitment tends to stick. If two producers are the first and second to open-source competing code of roughly equal function, the first is likely to attract the most users and the most and best-motivated co-developers; the second will have to take leavings. Recruitment tends to stick, as users gain familiarity and developers sink time investments in the code itself.
- In computer hardware, where freedom reigns for both suppliers and consumers alike on a global scale, the industry generates the fastest innovation in product and customer value the world has ever seen.
- For purposes of examining the software market itself, it will be helpful to sort kinds of software by how completely the service they offer is describable by open technical standards, which is well correlated with how commoditized the underlying service has become.
- In a future that includes competition from open source, we can expect that the eventual destiny of any software technology will be to either die or become part of the open infrastructure itself. While this is hardly happy news for entrepreneurs who would like to collect rent on closed software forever, it does suggest that the software industry as a whole will remain entrepreneurial, with new niches constantly opening up at the upper (application) end and a limited lifespan for closed-IP monopolies as their product categories fall into infrastructure.
- The free market, in its widest libertarian sense including all un-coerced activity whether trade or gift, can produce perpetually increasing software wealth for everyone.
- If you're really ahead of the game, plagiarism is a trap you want your competitors to fall into!
- One thing I understood from the beginning is that the press almost completely tunes out abstractions. They won't write about ideas without larger-than-life personalities fronting them. Everything has to be story, drama, conflict, sound bites. Otherwise, most reporters will simply go to sleep — and even if they don't, their editors will.
- Yes, the success of open source does call into some question the utility of command-and-control systems, of secrecy, of centralization, and of certain kinds of intellectual property. It would be almost disingenuous not to admit that it suggests (or at least harmonizes well with) a broadly libertarian view of the proper relationship between individuals and institutions.
- Similarly, to be a hacker you have to get a basic thrill from solving problems, sharpening your skills, and exercising your intelligence. If you aren't the kind of person that feels this way naturally, you'll need to become one in order to make it as a hacker. Otherwise you'll find your hacking energy is sapped by distractions like sex, money, and social approval. (You also have to develop a kind of faith in your own learning capacity — a belief that even though you may not know all of what you need to solve a problem, if you tackle just a piece of it and learn from that, you'll learn enough to solve the next piece — and so on, until you're done.)
- Nobody who can think should ever be forced into a situation that bores them.
- The ARPAnet/PDP-10 culture, wedded to LISP and MACRO and TOPS-10 and ITS and SAIL. The Unix and C crowd with their PDP-11s and VAXen and pokey telephone connections. And an anarchic horde of early microcomputer enthusiasts bent on taking computer power to the people.
- Users are wonderful things to have, and not just because they demonstrate that you're serving a need, that you've done something right. Properly cultivated, they can become co-developers.
- Open-source development breaks this bind, making it far easier for tester and developer to develop a shared representation grounded in the actual source code and to communicate effectively about it.
- The organization of the software and the organization of the software team will be congruent.
- If you want the most efficient production, you must give up trying to make programmers produce. Handle their subsistence, give them their heads, and forget about deadlines.
- We're proving not only that we can do better software, but that joy is an asset.
- Joy, humor, and playfulness are indeed assets.
- The verdict of history seems to be that free-market capitalism is the globally optimal way to cooperate for economic efficiency; perhaps, in a similar way, the reputation-game gift culture is the globally optimal way to cooperate for generating (and checking!) high-quality creative work.
- There is a critical difference (Ryan observes) between saying, "I'm giving you this reward because I recognize the value of your work", and "You're getting this reward because you've lived up to my standards." The first does not demotivate; the second does.
- If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource.
- The history of Unix should have prepared us for what we're learning from Linux (and what I've verified experimentally on a smaller scale by deliberately copying Linus's methodsNote 12). That is, while coding remains an essentially solitary activity, the really great hacks come from harnessing the attention and brainpower of entire communities. The developer who uses only his or her own brain in a closed project is going to fall behind the developer who knows how to create an open, evolutionary context in which feedback exploring the design space, code contributions, bug-spotting, and other improvements come from from hundreds (perhaps thousands) of people.
- Another vital factor was the development of a leadership style and set of cooperative customs that could allow developers to attract co-developers and get maximum leverage out of the medium.
- The Linux world behaves in many respects like a free market or an ecology, a collection of selfish agents attempting to maximize utility which in the process produces a self-correcting spontaneous order more elaborate and efficient than any amount of central planning could have achieved. Here, then, is the place to seek the "principle of understanding". The "utility function" Linux hackers are maximizing is not classically economic, but is the intangible of their own ego satisfaction and reputation among other hackers. (One may call their motivation "altruistic", but this ignores the fact that altruism is itself a form of ego satisfaction for the altruist). Voluntary cultures that work this way are not actually uncommon; one other in which I have long participated is science fiction fandom, which unlike hackerdom has long explicitly recognized "egoboo" (ego-boosting, or the enhancement of one's reputation among other fans) as the basic drive behind volunteer activity. Linus, by successfully positioning himself as the gatekeeper of a project in which the development is mostly done by others, and nurturing interest in the project until it became self-sustaining, has shown an acute grasp of Kropotkin's "principle of shared understanding". This quasi-economic view of the Linux world enables us to see how that understanding is applied. We may view Linus's method as a way to create an efficient market in "egoboo" — to connect the selfishness of individual hackers as firmly as possible to difficult ends that can only be achieved by sustained cooperation. With the fetchmail project I have shown (albeit on a smaller scale) that his methods can be duplicated with good results. Perhaps I have even done it a bit more consciously and systematically than he.
- Many people (especially those who politically distrust free markets) would expect a culture of self-directed egoists to be fragmented, territorial, wasteful, secretive, and hostile. But this expectation is clearly falsified by (to give just one example) the stunning variety, quality, and depth of Linux documentation. It is a hallowed given that programmers hate documenting; how is it, then, that Linux hackers generate so much documentation? Evidently Linux's free market in egoboo works better to produce virtuous, other-directed behavior than the massively-funded documentation shops of commercial software producers. Both the fetchmail and Linux kernel projects show that by properly rewarding the egos of many other hackers, a strong developer/coordinator can use the Internet to capture the benefits of having lots of co-developers without having a project collapse into a chaotic mess. So to Brooks's Law I counter-propose the following: Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one.
- The case can be put less negatively: where network effects (positive network externalities) dominate, open source is likely to be the right thing.
- It may well turn out that one of the most important effects of open source's success will be to teach us that play is the most economically efficient mode of creative work.
- The Lockean logic of custom suggests strongly that open-source hackers observe the customs they do in order to defend some kind of expected return from their effort. The return must be more significant than the effort of homesteading projects, the cost of maintaining version histories that document "chain of title", and the time cost of making public notifications and waiting before taking adverse possession of an orphaned project. Furthermore, the "yield" from open source must be something more than simply the use of the software, something else that would be compromised or diluted by forking. If use were the only issue, there would be no taboo against forking, and open-source ownership would not resemble land tenure at all. In fact, this alternate world (where use is the only yield, and forking is unproblematic) is the one implied by existing open-source licenses. We can eliminate some candidate kinds of yield right away. Because you can't coerce effectively over a network connection, seeking power is right out. Likewise, the open-source culture doesn't have anything much resembling money or an internal scarcity economy, so hackers cannot be pursuing anything very closely analogous to material wealth (e.g. the accumulation of scarcity tokens). There is one way that open-source activity can help people become wealthier, however — a way that provides a valuable clue to what actually motivates it. Occasionally, the reputation one gains in the hacker culture can spill over into the real world in economically significant ways. It can get you a better job offer, or a consulting contract, or a book deal. This kind of side effect, however, is at best rare and marginal for most hackers; far too much so to make it convincing as a sole explanation, even if we ignore the repeated protestations by hackers that they're doing what they do not for money but out of idealism or love. However, the way such economic side effects are mediated is worth examination.
- Amabile goes on to observe that "The more complex the activity, the more it's hurt by extrinsic reward." Interestingly, the studies suggest that flat salaries don't demotivate, but piecework rates and bonuses do. Thus, it may be economically smart to give performance bonuses to people who flip burgers or dug ditches, but it's probably smarter to decouple salary from performance in a programming shop and let people choose their own projects (both trends that the open-source world takes to their logical conclusions). Indeed, these results suggest that the only time it is a good idea to reward performance in programming is when the programmer is so motivated that he or she would have worked without the reward! Other researchers in the field are willing to point a finger straight at the issues of autonomy and creative control that so preoccupy hackers. "To the extent one's experience of being self-determined is limited," said Richard Ryan, associate psychology professor at the University of Rochester, "one's creativity will be reduced as well." In general, presenting any task as a means rather than an end in itself seems to demotivate. Even winning a competition with others or gaining peer esteem can be demotivating in this way if the victory is experienced as work for reward (which may explain why hackers are culturally prohibited from explicitly seeking or claiming that esteem).
- Indeed, it seems the prescription for highest software productivity is almost a Zen paradox; if you want the most efficient production, you must give up trying to make programmers produce. Handle their subsistence, give them their heads, and forget about deadlines. To a conventional manager this sounds crazily indulgent and doomed — but it is exactly the recipe with which the open-source culture is now clobbering its competition.
- If (as is generally accepted) over 75% of a typical software project's life-cycle costs will be in maintenance and debugging and extensions, then the common price policy of charging a high fixed purchase price and relatively low or zero support fees is bound to lead to results that serve all parties poorly. Consumers lose because, even though software is a service industry, the incentives in the factory model all work against a vendor's offering competent service. If the vendor's money comes from selling bits, most effort will go into making bits and shoving them out the door; the help desk, not a profit center, will become a dumping ground for the least effective employees and get only enough resources to avoid actively alienating a critical number of customers.
- Part of the answer certainly lies in the fact that using software does not decrease its value. Indeed, widespread use of open-source software tends to increase its value, as users fold in their own fixes and features (code patches). In this inverse commons, the grass grows taller when it's grazed upon.
- In the absence of money compensation, think "It's not worth submitting this fix because I'll have to clean up the patch, write a ChangeLog entry, and sign the FSF assignment papers…". It's for this reason that the number of contributors (and, at second order, the success of) projects is strongly and inversely correlated with the number of hoops each project makes a contributing user go through. Such friction costs may be political as well as mechanical. Together I think they explain why the loose, amorphous Linux culture has attracted orders of magnitude more cooperative energy than the more tightly organized and centralized BSD efforts — and why the Free Software Foundation has receded in relative importance as Linux has risen.
- An almost equally important payoff of open source is its utility as a way to propagate open standards and build markets around them. The dramatic growth of the Internet owes much to the fact that nobody owns TCP/IP; nobody has a proprietary lock on the core Internet protocols. The network effects behind TCP/IP's and Linux's success are fairly clear and reduce ultimately to issues of trust and symmetry — potential parties to a shared infrastructure can rationally trust it more if they can see how it works all the way down, and will prefer an infrastructure in which all parties have symmetrical rights to one in which a single party is in a privileged position to extract rents or exert control. It is not, however, actually necessary to assume network effects in order for symmetry issues to be important to software consumers. No software consumer will rationally choose to lock itself.
- Brook's Law: "Adding more programmers to a late project makes it later."
|
Homesteading the Noosphere
Extended content
|
- Not until the Linux explosion of early 1993–1994 did pragmatism find a real power base. Although Linus Torvalds never made a point of opposing RMS, he set an example by looking benignly on the growth of a commercial Linux industry, by publicly endorsing the use of high-quality commercial software for specific tasks, and by gently deriding the more purist and fanatical elements in the culture.
|
The Art of Unix Programming
Extended content
|
- Top-down tends to be good practice when three preconditions are true: (a) you can specify in advance precisely what the program is to do, (b) the specification is unlikely to change significantly during implementation, and (c) you have a lot of freedom in choosing, at a low level, how the program is to get that job done.
- Transparency is therefore more than an esthetic triumph; it is a victory that will be reflected in lower costs throughout the software's life cycle.
- There is a flip side to this. In the Unix world, libraries which are delivered as libraries should come with exerciser programs.
- CSV (fields separated by commas, double quotes used to escape commas, no continuation lines) is rarely found under Unix.
- Use # as an introducer for comments. It is good to have a way to embed annotations and comments in data files. It's best if they're actually part of the file structure, and so will be preserved by tools that know its format. For comments that are not preserved during parsing, # is the conventional start character.
- When you feel the urge to design a complex binary file format, or a complex binary application protocol, it is generally wise to lie down until the feeling passes.
- One of the many consequences of the exponential power-versus-time curve in computing, and the corresponding pace of software development, is that 50% of what one knows becomes obsolete over every 18 months.
- Web pages get bogged down in the dispute over whether the reader or author should control the appearance.
- When the superior programmer refrains from coding, his force is felt for a thousand miles.
- Tools that look glossy but shatter under stress are not good long-term value.
- One of the main lessons of Zen is that we ordinarily see the world through a haze of preconceptions and fixed ideas that proceed from our desires. To achieve enlightenment, we must follow the Zen teaching not merely to let go of desire and attachment, but to experience reality exactly as it is—without the preconceptions and the fixed ideas getting in the way.
- Rushing to optimize before the bottlenecks are known may be the only error to have ruined more designs than feature creep. From tortured code to incomprehensible data layouts, the results of obsessing about speed or memory or disk usage at the expense of transparency and simplicity are everywhere. They spawn innumerable bugs and cost millions of man-hours—often, just to get marginal gains in the use of some resource much less expensive than debugging time. Disturbingly often, premature local optimization actually hinders global optimization (and hence reduces overall performance). A prematurely optimized portion of a design frequently interferes with changes that would have much higher payoffs across the whole design, so you end up with both inferior performance and excessively complex code.
- To design for compactness and orthogonality, start from zero. Zen teaches that attachment leads to suffering; experience with software design teaches that attachment to unnoticed assumptions leads to non-orthogonality, noncompact designs, and projects that fail or become maintenance nightmares.
- To design the perfect anti-Unix, write an operating system that thinks it knows what you're doing better than you do. And then adds injury to insult by getting it wrong.
- Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features.
- Are the individual functions in your modules too large? This is not so much a matter of line count as it is of internal complexity. If you can't informally describe a function's contract with its callers in one line, the function is probably too large.9 9 Many years ago, I learned from Kernighan & Plauger's The Elements of Programming Style a useful rule. Write that one-line comment immediately after the prototype of your function. For every function, without exception.
- As with buildings, it's easier to repair superstructure on top of a solid foundation than it is to replace the foundations without trashing the superstructure.
- When you see the right thing, do it—this may look like more work in the short term, but it's the path of least effort in the long run. If you don't know what the right thing is, do the minimum necessary to get the job done, at least until you figure out what the right thing is. To do the Unix philosophy right, you have to be loyal to excellence. You have to believe that software design is a craft worth all the intelligence, creativity, and passion you can muster. Otherwise you won't look past the easy, stereotyped ways of approaching design and implementation; you'll rush into coding when you should be thinking. You'll carelessly complicate when you should be relentlessly simplifying—and then you'll wonder why your code bloats and debugging is so hard.
- Mixing languages is better than writing everything in one, if and only if using only that one is likely to overcomplicate the program.
|
|
