White Trash Software Engineer

Lord of the Files

2011-08-28T13:35:00.001-07:00

I started this blog after failing to generate any interest for a book proposal I submitted to a myriad of publishers specializing in either academic or technical publications. This was back in 2007, after returning to North America from four months living and traveling through sub-Saharan Africa. That jarring experience motivated me to try taking a couple of papers I had published in Software Engineering Notes and expanding them into a book about the human factors of programmers (instead of programming). In hindsight, the working title (White Trash Software Engineer) may have scared away the academic publishers, while the lack of technical content may have scared away the other publishers. Falling through the cracks has been somewhat of a recurring theme in my professional endeavours; as someone who likes to use both sides of his brain and favours interdisciplinary research, it can be tough to categorize me and my message. Or to find an appreciative audience.

The company that I started (Backstage) back in 1996(!) was acquired in September of 2010 by RealNetworks, and is now a studio in their GameHouse division. Backstage didn't really take off until I quit trying to be an academic and started devoting all of my attention to growing a company in 2007. It really is true that you can't be a part-time entrepreneur.

This blog now contains excerpts from the book Lord of the Files: Essays on the Social Aspects of Software Engineering, self-published in 2011. Files, not Flies. Most of the chapters in the book started out as entries in this blog. The gratifying experience of running Backstage through its emergence as a successful social gaming company also found its way into the book.

People like to belong to communities that both identify and set them apart from the rest. My book provides an alternative to both the geek culture that dominates computer science, and the default counter-culture of open source software development and companion zealotry exhibited by the likes of Richard Stallman and Linus Torvalds. Lord of the Files while embodying a punk esthetic, was written for the intelligent programmer with a sense of humor, social skills, and a cultural center that extends well beyond Star Trek, an ugly pony tail, and “Windows sucks”. If that resonates, this could be your brand; a badge of passive aggressive frustration with the pigeon-hole that society has reserved for us and its promulgation from within by a small minority. I am not a socially retarded, illiterate eunuch. I am angry but compassionate; disappointed but optimistic; divorced but in love. I can do and teach. I am good at what I do, but refuse to do it for free. I wrote this book because nobody was speaking for me. Not even Jaron Lanier.

Other than my editor (the quite remarkable Grenfell Featherstone, who helped me tone down the private angst and find a better title), the first person to read my book in its entirety was my girlfriend's mother. It made her laugh, and she tells me she genuinely enjoyed the book. As a psych nurse, I think she liked how much of my emotions and feelings come through in each of the self-contained essays.

According to Grenfell, I wrote a book that speaks to a wider audience than just programmers. However, I hope it finds an audience with programmers, not because I need the money or the fame, but because I think the message is sorely lacking in our industry. Namely, that to be a good software engineer, you first and foremost need to be a good person.

Buy it today at CreateSpace, or Amazon.com

Join the discussion on Facebook.

Epilogue: redemption through introspection

2010-01-18T13:10:00.000-08:00

Address your inner injuries before a lack of self-esteem destroys your prospects for career advancement.

Sometimes I fear that my ex-wife wants me to continue to be sad, lonely, and give her all my money. In other words, she wishes we were still married.

When a marriage begins to fail, and you no longer feel safe in sharing your true self or feelings with your partner, it can seem like the loneliest place on earth. If you were single, you could reach out to others in an appropriate manner, but when you are married, you are constrained. The result is a train wreck of secrecy, and a search elsewhere for the fulfillment and validation that you no longer experience with your spouse. Not surprisingly, pornography, cybersex, gambling, drugs, alcohol and other forms of self-medication become increasingly attractive. When you are single, you can simply join a running club.

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Want more? Buy it today at CreateSpace.

Join the discussion on Facebook.

Pandemonium reigned

2008-11-16T10:52:00.000-08:00

This book is my midlife crisis.

I got divorced at the age of 40. Leaving a bad marriage enabled me to enjoy a crisis of identity that led to the success of Backstage, which in turn prompted me to write this book.

But why write this book now, at this age and stage of my life? I used to be able to write to entertain, but then I went to grad school and learned to write in a formal, academic style. I wondered if I had permanently dulled those neural pathways that were involved in writing goofy short stories when I was younger, especially the one that ended up giving the kids in my high school something else to make fun of other than my intense social awkwardness, lankiness, glasses, and braces?

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Want more? Buy it today at CreateSpace.

Join the discussion on Facebook.

What the bleep should we know!?

2008-11-08T12:02:00.000-08:00

You can work for me if you can answer one simple question.

David Parnas was once asked what he thought was the most often overlooked risk in software engineering. His answer was “incompetent programmers.... One bad programmer can easily create two new jobs a year. Hiring more bad programmers will just increase our perceived need for them. If we had more good programmers, and could easily identify them, we would need fewer, not more.”[1]

That was harsh. As usual, though, he is correct.

At this point in my life I think I must have interviewed over 300 candidates for various positions in software development. I have hired dozens of co-op students, and made the hiring decisions on countless permanent positions. I have also fired three people. I only had employees at Backstage from 2005 to 2010, so why all this experience in hiring and firing? Somewhere early on in my career it became known that I was good at the difficult job of appraising the technical skills of applicants. Because of this skill, my previous employers asked me to become intimately involved in the process of assessing the talent of potential employees.

The technical screening interview consists of a twenty minute conversation where I attempt to verify that the applicant is competent, and that his or her resume is, in fact, truthful. Depending on the position we were hiring for, I had managed to boil the interview down to one simple question—answer it, and you could work for me. It was rather disheartening how many computer scientists were unable to answer it correctly.

This excerpt from the forthcoming book Lord of the Files is brought to you by Thought Pilots. Join the discussion on Facebook.

[1] Nancy Eickelmann, “ACM Fellow Profile: David Lorge Parnas,” ACM SIGSOFT Software Engineering Notes, 24 (3), 1999.

Nobody ever got laid for buying IBM equipment

2008-11-02T23:55:00.000-08:00

Where are all the babes? The dearth of women in our industry is examined from the perspectives of the female reproductive imperative and the systematizing male brain hypothesis.

I chose to major in computer science for two reasons: money, and I was good at it. As a teenager in the early 1980s, everyone kept telling me “computers—now that’s where the future is headed!” and “computers is where the good jobs will be!” I felt a bit like Dustin Hoffman’s character in The Graduate, only computers had displaced plastics as the one word to remember. While I didn’t end up seduced by a cougarific Mrs. Robinson, I did notice that the advertised salaries for computer programmers in the weekly careers section of the Vancouver Sun would invariably make my girlfriend very horny. The promise of lots of money was getting me lots of honey.

But to be honest, I really wanted to be a physicist with expertise in acoustics so that I could grow up to design concert halls. However, in my first semester at university I got a D in physics and an A in computer science, and my path was chosen. Since people like to do what they are good at, and since I am people, I stuck with computer science. It was a good fit for me because I enjoyed it, and it seemed incredibly easy. If there was ever an academic pursuit that was uniquely designed for the way my brain worked, it was computer science.

I once asked my class at Camosun College why they had chosen to major in computer science. This was the year 2001, and I had become slightly exasperated with trying to teach data structures and algorithms to a group of adults who did not seem to share my love for the topic. Indeed, many of my students were only there because the job prospects had seemed so promising just two years earlier. The best response I received to the question was from a young man sitting in the back row, who sarcastically exclaimed “I’m here for the babes!”

Why are there so few women in computer science? For many universities, this seems to have replaced does P = NP as the single greatest research challenge facing computer science departments throughout North America. I respectfully submit this essay as my take on the situation. Cautiously, and without judgement, I think there are two leading causes:

1. the lack of attractive male mating partners in our industry is at odds with women’s biological imperative for reproduction: women are just not attracted to repressed, unkempt, socially awkward, high functioning nerds with Asperger’s; and,

2. computer science is particularly suited to those endowed with a systematizing brain, which women tend not to have, but is usually found in men—especially the repressed, unkempt, socially awkward, high functioning nerds with Asperger’s.

Taken together, these two reasons conspire to ensure that women choose not to pursue careers as software engineers.

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Want more? Buy it today at CreateSpace.

Join the discussion on Facebook.

White Trash Software Engineer

2008-10-18T21:48:00.000-07:00

Building large software products is not easy. If you want to persevere long enough to be successful at it, I have found that it helps if you were raised with conditional love, fear failure, and lack self-esteem

My mom’s favourite cinematic moment is the scene from An Officer and a Gentlemen when Richard Gere, with newly minted officer’s commission and freshly-pressed white uniform, marches across the factory floor to sweep Debra Winger off her feet. The film ends with the promise of a life of love and romantic adventure for Winger’s character as she follows her man through a series of exotic overseas air force postings. My mother’s fondness for this film is likely because it mirrors her own revisionist take on events leading to her marriage to my father. The only problem is that my dad—a budding Sapper in the Canadian Military Engineers—sabotaged his commission when he told his commanding officer to fuck off during his officer training in Wainwright, Alberta. As a result, love lifted mom and dad up where they belonged, which was apparently the same place they had started from: Chilliwack

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Want more? Buy it today at CreateSpace.

Join the discussion on Facebook.

A Seven-Layer Hierarchy of Careers in Computer Science

2008-10-16T21:10:00.000-07:00

If your first job as a software engineer isn’t all you had hoped for—good! Starting out at the bottom isn’t necessary for you to learn the business; it’s so you can learn some humility. You are no better than anyone else.

I once offered a job to a guy who described himself as a “creative generalist.” We were looking for a project manager, but I offered him the position, anyway. Ultimately he refused the offer when I balked at his request for a MacBook Pro. I assumed the much cheaper Dell PC we already had could compute the critical path on a GANTT or PERT chart just as well as any Mac. He assumed I was an asshole for refusing his request.

In retrospect, my prima donna detector should have gone off when he described himself as both creative and a generalist. Don’t get me wrong, creativity is an imperative skill if one wants to thrive in any software company, particularly one that builds games.[1] But a generalist? It’s great to be good at many things, but being great at one thing is even gooder. Play to your strengths and specialize, kids. Nobody needs a generalist.

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Want more? Buy it today at CreateSpace.

Join the discussion on Facebook.

[1] Creativity is an often overlooked quality in good programmers. In 1956, IBM’s first recruitment campaign for computer programmers was a print ad that targeted people who enjoyed algebra, music composition, and games, and who had lively imaginations. While not exactly the stereotype of today’s computer nerd, it is a strikingly accurate set of criteria for the type of person who would be considered creative, logical, and likely enjoy programming. For more on this fascinating story, see Nathan Ensmenger, “Building Castles in the Air: Reflections on recruiting and training programmers during the early period of computing,” Communications of the ACM, 54 (4), pp. 28-30.

Your favourite methodology is eXtremely gay

2008-10-14T22:00:00.000-07:00

Good software is not an emergent property of the rules that govern the decentralized interactions of average programmers. Instead, good software results from good programmers, regardless of the methodology. Keep your software process from getting in the way of your best programmer.

When programming, you can never have too many comments. No, wait… when programming, you can never have too many jelly donuts. I always get those confused. The former is an example of a coding standard, which can be part of a programming methodology. The latter is an example of a software development process (albeit one that includes “now we will eat some jelly donuts” as a phase in the life cycle). In this chapter we discuss the difference between a software development process and a methodology, and why neither really matters.

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Want more? Buy it today at CreateSpace.

Join the discussion on Facebook.

All we really need to know about software engineering is in the film Office Space

2008-10-05T13:49:00.000-07:00

One of the difficulties with teaching software engineering to a class of undergraduates with very little industrial programming experience is trying to convey why some of the material covered is relevant and important. For example, the importance that designing for change plays in software engineering is difficult to explain to a group of students who have never experienced the joy and frustration of maintaining a program written by someone else. After emphatically waving my arms in front of the chalk board for twelve weeks, yet feeling unable to teach my students everything they would need to know about software engineering, I would screen the film Office Space on the last day of class.

The 1999 Mike Judge film Office Space chronicles the efforts of software engineer Peter Gibbons to find peace with the role of work in his life. One of the few artifacts of popular culture (Douglas Coupland’s Microserfs being another notable) to feature a software engineer as the protagonist, Office Space – despite its enduring mass appeal and broad comedic base – embodies many subtle gems of wisdom regarding the craft of software engineering.

The following rules are all found to varying degree in the plot and characters of the film.

design for change
software engineering is a social activity
an untested program does not work
not everyone gets to write video games
Brooks’ Law
you will be expected to work long hours

Design for Change
We are all taught as undergraduates that maintenance is the dominant phase of the software development life-cycle. Our responsibility as software engineers is thus to design systems that are easy to modify without introducing new bugs; i.e., to design for change.

Office Space is set in 1999 at fictitious Dallas computer firm Initech where Peter, like most software engineers, is employed maintaining an existing software system. In particular, Peter is busy fixing the Y2K bug in bank software. More than eight years on, many of us in the industry have mostly forgotten about the incredible person-effort expended on fixing all that code, which obviously was not properly designed for change. The film thus serves as an object lesson in how not to design systems. It is also a historical reminder of what the Y2K bug was all about, even if Peter’s attempt to explain it to his love-interest Joanna is aborted when he intuits her waning interest. Even without reminders such as these, the Y2K crisis is likely to remain a footnote in the history of computer science for one of two very different reasons:

it was a such a monumentally anti-climactic non-event, suggesting the scope of the problem was severely overestimated by industry observers (and practitioners); or
it was our finest hour – a triumph of modern software engineering that averted a global catastrophe.

Figure 1. Samir, Peter, Michael, and Tom in cubicle hell at Initech. Whoever decided that cubicles were an ideal way to layout offices should be forced to live out an eternity stuck in a cubicle in the middle of an expansive floor pinned beneath the incessant glare of fluorescent lighting. The demoralizing effects of these wall dividers, which offer little in the way of privacy and noise reduction, are legendary. Given that software engineering requires teamwork, and cubicles are an impediment to effective communication, they should be rejected outright.

Software Engineering is a Social Activity
Software engineers write modules, classes, and libraries that are used by other software engineers in the construction of larger systems. Teamwork is thus an essential component of the process. Moreover, these systems are constructed to meet the requirements of end-users who belong to a cultural group that is usually decidedly outside that of software engineering. Interaction with this group (the customer) is an important part of the development process.
Two significant problems with this reality are reflected in the film:

software engineers as a whole – a point vociferously made by Business Analyst Tom Smykowski while explaining what it is he does at Initech – are not good with people; and,
these people with the poor social skills (like Milton) are our teammates.

It is difficult not to sympathize with Smykowski, the first to be downsized during the ongoing business process re-engineering of Initech. Like Dan’s father in Microserfs, middle-aged Smykowski has woken up to an industry that sees him as irrelevant. When asked during his interview with the efficiency consultants (the Bobs) “what would you say you do here,” Smykowski defensively explains that he takes the specs from the customers to the software people, “because engineers are not good at dealing with customers.” This stereotype of the socially challenged software engineer seems the raison d’etre of the modern day Business Analyst, whose sole job requirement it often seems is simply – as Smykowski bellows ironically – to “have people skills!” Smykowski’s plight should serve as a reminder of the need to maintain relevancy: people skills alone are not enough.

Unlike Smykowski, Milton appears so completely socially retarded as to be unbelievable. To those of us in the industry, however, people like Milton are not an aberration. Indeed, some of the best programmers I’ve ever known share many of Milton’s personality traits, which are likely related to Asperger’s syndrome and a healthy dose of obsessive-compulsive disorder (OCD). These psychological afflictions are considered by some to be an advantage when it comes to the kind of thinking required to excel at software engineering.

Figure 2. Tom Smykowski, Business Analyst. Learn to love your Business Analyst, for they have the unenviable task of pinning down and documenting the customer specifications (i.e., specs). You can always blame the Business Analyst when the system you built works according to spec, but does something stupid.

While Peter’s patience is merely strained by Milton, others who do not share his neural processing style openly mock Milton and his OCD. In particular, Initech vice-president Bill Lumbergh delights in asking Milton to move his desk simply because of the unease it causes him. Likewise, in a scene reminiscent of the alpha-male who steals the geek’s ball and claims it was his in the first place, Lumbergh’s commandeering of Milton’s red Swingline stapler is a gesture that seems predicated only by his distaste for Milton as a person. While not endemic, this kind of manipulation and bullying of the more socially challenged amongst our peers is an unfortunate practice witnessed far too frequently.

Untested Programs Do Not Work
Peter’s colleague Michael Bolton – the classic passive-aggressive nerd – feels both exploited and disrespected by his employer despite his contributions to the firm’s intellectual property. Michael’s revenge is enacted when he unleashes a virus on the credit union mainframe designed to slowly steal fractions of pennies during interest calculations. However, the program is put into production without the opportunity for testing.

As Stroustrup succinctly puts it, “a program that has not been tested does not work.” To Michael’s dismay, his virus has a bug that results in a conspicuous amount of money to go missing, resulting in the likelihood of its detection. He afterward explains to his co-conspirators Peter and Samir that he must have had the decimal point in the wrong place, and that he always makes some stupid little mistake like that. In truth, all software engineers make stupid little mistakes like that, all the time. In fact, it’s often a complete shock to me that software works as well as it does given how:

massively complex systems have become;
escalated the corresponding attention to detail required to maintain these systems has become; and,
how easy it is to introduce run-time defects through simple typos and editing errors that escape compile-time detection.

This is why we test, and especially why we regression test after making even the most benign changes.

Not Everyone Gets to Write Video Games
Part of the apparent lack of job satisfaction indicated by the employees of Initech likely stems from the boring application area: banking software. We all want to work on the cool stuff, like games, artificial intelligence, virtual reality, porn websites… but let’s face it: the majority of software engineering jobs are in mundane application areas (like finance and corporate IT) that involve straightforward information storage and retrieval.

It begs the question, is Peter simply bored? Is Peter’s career crisis merely caused by underutilization and a coincident lack of motivation? The Bobs certainly feel that Peter’s problem is that he hasn’t been challenged enough, believing him when he explains “it’s not that I’m lazy, it’s that I just don’t care.” Software engineers should strive to find work environments and application areas that they find at least mildly interesting. If they don’t, they risk the predicament Peter explains to the Bobs: if your only motivation is the fear of losing your job, you will only work just hard enough to not get fired. As discussed in a previous post, software engineers instead should learn to assess their own work interest and its effect on their productivity.

Brooks’ Law
In his classic monograph on software engineering, Fred Brooks describes the phenomenon whereby adding manpower to a late software project only serves to make it later. This counterintuitive result, which has become known as Brooks’ Law, is due to many factors, including the increased overhead of communication, training, coordination, meetings, etc.

A possible corollary to Brooks’ Law is that within a software firm, individual programmer productivity is inversely correlated to company size. The larger the company, the less empowered programmers feel, or need to be; ineffectiveness can be hidden within the bureaucracy. Conversely, small companies engender ownership: employees see the direct consequences of their actions. I refer to this (purely anecdotal) observation as the Brooks Effect. Initech is a case study in the Brooks Effect. While interviewing for his own job, Peter admits to the Bobs that “in a given week I probably only do about 15 minutes of real, actual work.” The rest of the time is spent on such mundane activities as meetings, documentation, going for coffee with teammates, zoning out, filling out time-sheets (the infamous TPS Reports that serve as a running gag throughout the film), asking yourself “is this good for the company?”, playing Tetris, and eating birthday cake with coworkers.

The parody of the ubiquitous birthday celebration is recreated with particular precision in Office Space. From the uninspired singing to the disingenuous show of appreciation, no detail is left unskewered. In reality, once a company reaches the size of 50 employees, there is a statistical likelihood of one birthday cake event per week. For some inexplicable reason, this celebration is usually scheduled right before an afternoon code review. The comatose state that invariably follows can only partly be blamed on the resulting insulin shock.

Case in point, in my one year of work for Bell-Northern Research, as one of over 4,000 employees I managed to check-in to the code repository an underwhelming 16 updates to the DMS-100 digital telephone switch software. During that time span, however, I did participate in over 30 birthday celebrations and three farewell luncheons (not including my own).

You Will Be Expected to Work Long Hours
Peter’s boss Bill Lumbergh is a slightly exaggerated caricature of the prototypical alpha-male, and the film’s most memorable character. Either a programmer who rode his social skills through the management track to the top of his game, or an exemplar of the Dilbert Principle, the always coffee-sipping Lumbergh seems primarily concerned that his employees accurately complete their TPS Reports.

Lumbergh, who Peter believes represents “all that is soulless and wrong” doesn’t hesitate to ask Peter to “go ahead and come in [Saturday].” The key phrase is “go ahead”… as if to suggest that Lumbergh is doing Peter a favor by letting him work over the weekend. Software engineering – like many other professional positions such as law and medicine – is an occupation that can demand Herculean effort and extended work hours, particularly as deadlines approach. The overtime is rarely ever paid, despite efforts by jurisdictions such as California to legislate against this practise. But the legality doesn’t seem to matter, anyway, as many software engineers crave the intellectual calisthenics that programming offers, and will work overtime (or after-hours on open source projects) irrespective of compensation. In this regard, Peter is an anomaly.

When Peter fails to show up to work on Saturday, Lumbergh launches a never-ending stream of telephone calls to his home in an effort to locate and chastise his AWOL employee. This is my favorite scene in the film, not only for the comedic element of Lumbergh’s distinctive greeting that begins each of his 17 recorded messages, but because it echoes my own personal experience with two different employers. While employees everywhere have likely dealt with the inequity of the boss who parks the Porsche in the only reserved spot, the frantic cell-phone calls, pages, voice messages, instant messages, and urgent email seems particularly endemic to the life of the software engineer on the mission critical path.

Figure 3. Lumbergh asks Peter to work over the weekend. There is obviously a dress code at Initech (stated or implied) that forces all the men to wear a tie, even though they spend their day at a desk writing code. What on earth does wearing a tie have to do with programming? Why not specify that all software engineers have to wear a funny hat or wig? The tie is as necessary as a hat or wig, especially given (as Smykowski claimed) the software engineers are not allowed to meet with the customers. They made us wear a tie when I worked at IBM’s software development lab in Toronto (and made us start work at 8:42 AM because of the unionized manufacturing plant next door), except for one dude who was such a compiler genius that he got away with wearing leather from head to toe. As an aside, you know your career is progressing nicely if you get to exert more control over your wardrobe with each new position or employer. At Backstage we adopted a “pants optional” dress code because ass sweat is greatly reduced by the absence of pants. But be warned: people will make assumptions about your socio-economic status based solely on how you are dressed. The doorman Theo at my apartment building in Cape Town thought I was just a couch potato who passed his time surfing. He had judged me based on the way that I dressed, and had no idea I had a doctorate and was a president of a company. While the official uniform of software engineers is the unbuttoned dress shirt and khakis, I prefer sweat pants: comfortable, but apparently guaranteed chick repellant. “It’s a good thing you are married, because you’ll never get laid dressing like that” suggested my queer eye, Canterbury Dave from Smuggler’s Cove Pub, where I spent too much of my time as my marriage disintegrated riding the spiraling crashing waves of impotent despair from my usual seat at the bar. Of course, married or not, I wasn’t getting laid, anyway. At least the bartender used to put on the alternative 80s station on the satellite whenever I was there. But I digress.

Is It All That Bad?
Towards the end of the film, Joanna counsels Peter that it is okay to be uninspired by his work, because “most people don’t like their jobs.” This is one moment where the film rings hollow, for in my experience, software engineers love their work. We may not like our boss, and the meetings, and the politics, but at the end of the day, there is nothing else we would rather be doing.

As I ease into my 40s, Peter’s rhetorical question posed to Michael and Samir is particularly poignant: “what if we’re still doing this when we’re 50?” Personally, I sincerely hope I’m still doing this when I’m 50. As Lumbergh would say… yeah, that would be great.

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Want more? Buy it today at CreateSpace.

Join the discussion on Facebook.

END NOTES

I am amazed at how accurate the depiction of the software development process and software engineers is in Douglas Coupland’s Microserfs (Harper Collins: 1995) and his sort-of sequel JPod (Harper Collins: 2006).

Any textbook on software engineering will claim that maintenance is the dominant (longest) stage of the software life cycle. The seminal paper on how to design programs so they are easier to change is Parnas, David L. (1972): “On the criteria to be used in decomposing systems into modules,” Communications of the ACM, 15 (12), pp. 1053-58. In my experience, the adoption of early daylight savings in March of 2007 caused more software outages than the Y2K rollover.

Silberman, Steve (2001): “The geek syndrome,” Wired, 9 (12) discusses the tendency of computer scientists and engineers to exhibit autistic traits (in particular, Asperger's syndrome), and the strange cluster of autistic children born from couplings of techies in the Silicon Valley.

One of the best books on programming ever written is Stroustrup, Bjarne (1991): The C++ Programming Language (Second Edition), Addison-Wesley, from which I quoted page 380.

Brooks’ Law is from Fred Brooks Jr., The Mythical Man-Month: Essays on Software Engineering, Addison-Wesley, 1975.

The Dilbert Principle suggests that in many companies it is the worst employees who are promoted to management, because the company cannot afford to move the productive employees out of their current positions. Scott Adams wrote The Dilbert Principle: A Cubicle's-Eye View of Bosses, Meetings, Management Fads, and Other Workplace Afflictions (Harper Business, 1996).

The requirement to pay overtime to all software engineers was later overturned by California law S.B. 88 which exempts jobs that require skill and proficiency in the theoretical and practical application of highly specialized information to computer systems analysis, programming and software engineering.

An earlier version of this essay appeared in Software Engineering Notes, volume 30, number 4, 2005.

The programmer life cycle

2008-10-04T21:54:00.000-07:00

Legendary Canadian businessman Jimmy Pattison was famous for his ruthless policy of firing the poorest performing monthly salesperson at each of his many car dealerships. I could never do that at my own company – Backstage Technologies, a casual game studio and sometimes contract software engineering firm – because excellent programmers like the guys who work for me are extremely difficult to replace. But that doesn’t mean I don’t rule with an iron fist. In fact, I think there should be a Japanese-style TV reality show set at Backstage entitled Iron Fist. Each weekly episode of Iron Fist would conclude with me, adorned in iron gloves, whaling on the nuts of the employee with the fewest accumulated lines of code in production. Unfortunately, most weeks that would be me.

Why is my productivity waning, and why isn’t the amount of work that my employees do constant? Most textbooks on software engineering make at least passing reference to the notion of programmer productivity. The conventional wisdom is that an average programmer completes a paltry 500 debugged lines of code (LOC) per month. To students and professionals alike, this number seems absurdly low. The standard set of explanations for this phenomenon includes, among other things, the communication overhead of working in teams (i.e., Brooks’ Law), and the time allotted to non-programming activities like requirements gathering, documentation, and design.

These explanations tell only part of the story. While it has long been accepted that productivity rates can differ by an order of magnitude between the best and worst programmers, little has been said regarding the reality that work rates of the same programmer can likewise differ over time. Not unlike the software systems they construct, programmers follow a predictable life cycle. However, the programmer life cycle is not comprised of distinct activities (like the software development life cycle, which we discuss below) but rather by emotional stages that directly affect and predict productivity. The sequence of stages is characterized by an initial six month period of intense interest, at which time productivity rates are indeed much higher than the oft-quoted 500 LOC/month average. After a short period of volatility, the programmer then enters a prolonged phase of steadily dwindling interest, resulting in productivity rates that mimic the average. Each time a programmer switches employers or begins a significantly new project, the life cycle starts anew.

This conjecture is based purely on my experiences and observations while working as the senior software engineer for two successful Internet startups (abebooks.com in Canada, then Prophet.Net in the United States). The basic premise of the proposed relationship between stages of the programmer life cycle and productivity is the simple observation that employees are most productive when interest and satisfaction in their jobs is at its highest.

The Software Development Life Cycle
According to Webster, a life cycle is the series of stages through which something (as an individual, culture, or manufactured product) passes during its lifetime. In the case of a software product, the usual stages of its life are: requirements, design, implementation, testing, and maintenance.

Briefly, the requirements stage is when the product is conceptualized as a set of requirements, which are documented by a business analyst. This is what the product must do. The design stage is when a software architect imagines how the software will be built by decomposing the system into a set of interacting subsystems (database server, application server, web server, etc.), which in turn are decomposed into their constituent modules (database tables, transactions, stored procedures, classes, processes, threads, etc.). The architecture of the system is usually described by a set of directed graphs, where the nodes are the subsystems and the arcs denote relationships (e.g., “calls”, “stores”, “reads”). This is how the product will work. The implementation stage consists of database administrators and programmers realizing the design as a working program. The testing stage is meant to ensure that the program meets the requirements. Once this has been determined, a version of the program is released to its user community. Finally, the maintenance stage is where the program is changed into new releases that fix bugs that were not discovered during the testing stage, and new versions that add additional features.

Figure 1. The waterfall model of the software development life cycle.

Part of the goal of software engineering is to apply what is known as engineering rigor to the process of programming, the belief being that if it’s good for building bridges and toaster ovens, it should work just as well when writing software. Since engineers love linear project plans that involve milestones in the form of diagrams and documents, the first attempts at applying this concept to software development led to the infamous waterfall model of the software development life cycle. The waterfall model (see Figure 1) imagines each of the five stages of the life cycle as a self-contained step, where the output of each step falls only into the next. The model does not allow for backtracking or for information to flow in any direction but down. The obvious analogy to shit rolling downhill is left as an exercise to the reader.

The Stages of the Programmer Life Cycle
Everything we do as software engineers falls under one of these five categories that comprise the software development life cycle. The effectiveness with which we undertake these tasks is governed by where we are in the programmer life cycle (see Figure 2).

The programmer life cycle is comprised of six stages:

Euphoric
Productive
Irreplaceable
Resentful
Bored, and
Unproductive

While this particular life cycle model is perhaps most likely to apply to highly productive individuals (so-called star programmers) working in the 24/7 world of web development and e-commerce, it is my belief that there are fundamental truths in its structure that render it applicable to many software development situations and domains.

Euphoric. The first stage involves the start of a new job or significantly different project. The programmer is stimulated by both the newness of the environment and the challenges ahead. In many cases the programmer’s euphoria is fueled not only by these changes, but also by the coincident escape from a previous work situation that had become routine or was underutilizing his/her talents. This introductory phase is quite short in duration, lasting only as long as necessary to acclimatize to the new surroundings, learn a new development environment, and become familiar with the domain.

Productive. Once acclimatized, the programmer’s work interest reaches a peak. Because of this acute interest, productivity is at its highest. This stage – which lasts about six months – is when the programmer develops or takes ownership of some mission-critical software system. This coincides with steadily rising value to the organization.

Figure 2. Productivity as a function of stages of the programmer life cycle.

These first two stages constitute the “honeymoon period.” The next two are best described as the “volatility phase.”

Irreplaceable. Management soon recognizes that the programmer has become a valuable commodity. The programmer’s prestige within the organization is at an all-time high. As such, significant increases in compensation and fringe benefits are doled out in an effort to keep the programmer from ever possibly consider leaving. The golden handcuffs are sealed with the ubiquitous stock option grant, or the promise of one. The programmer feels on top of the world. Unfortunately this will not last.

Resentful. Management – sensing a sudden asymmetry in the employer-employee relationship – begins to bear resentment that a single individual (the programmer) is now responsible for the ongoing success or failure of the venture. Fearing a loss of control, management begins to act in a manner suggesting ownership of the programmer’s time and space. Symptoms of this malaise include requiring the programmer to carry a pager, work weekends, install broadband connectivity at home, and never under any circumstances take holidays. The programmer starts to receive email at all hours of the day from trigger-happy monitoring software reporting false alarms, and from managers requesting new features and emergency bug fixes.

In return, the programmer develops feelings of resentment towards management. This is exacerbated by management’s policy of rewarding the programmer’s competency not with bonuses and time off, but with additional workload and responsibilities. The first signs of complacency begin to appear in the programmer’s workplace attitude.

This unstable time of mutual resentment is necessarily short-lived, as emotions run too high for the process to carry on for more than a month or two. The working relationship can implode during the resentful stage, particularly if volatile personalities are involved. In the worst case, the programmer quits; the additional workload coupled with the stress of being irreplaceable yet resented becomes too much to take. However, in most cases the resentful stage merely settles into an equilibrium of mutual need: management’s need for the star to carry on keeping the software running, and the programmer’s need to be a star.

Bored. The post-resentment equilibrium sees the programmer’s activities shift more towards ongoing maintenance, consultative meetings with management, and internal knowledge transfer to other programmers and customer support staff. Because the initial challenges of the new project, environment, and technologies have all been met, the intellectual stimulation has dropped. This leads to boredom. Coupled with the excessive mental context switching demanded of the new activities, the programmer’s productivity (as measured by LOC/month) experiences a significant drop. Despite the tedium, however, this stage can last indefinitely provided the productivity remains above the minimum expectation level given the programmer’s current remuneration.

Unproductive. Like a manic depressive who goes off his medicine because he misses the occasional euphoric episode, or a love junkie addicted to the adrenaline rush of the first six months of a new relationship, the programmer is unlikely to remain in a state of boredom forever: something has to give. The change is triggered by a slide into the unproductive stage, characterized by the programmer working on his/her resume and visiting job sites on the ‘Net, while management views the programmer as “coasting,” overpriced, and expendable. One of two outcomes is inevitable: the programmer finds a new employer, or management moves the programmer to a significantly new role or project. Either way, the life cycle starts again.

Conclusion
This life cycle model should serve as a cautionary tale to both programmers and managers. The lesson for the programmer is to be aware of each stage and its effect on productivity levels, for ultimately one’s success as a software engineer depends on one’s perceived productivity. By recognizing the symptoms of boredom leading to unproductiveness, the programmer can proactively search for remedies, usually in the form of a frank discussion with management, or seeking out new projects and technological challenges.

Conversely, managers must understand the causes and effects of this life cycle in order to combat high levels of attrition and declining productivity. To get the most out of the organization’s stars, managers must avoid the resentment trap by resisting the temptation to over-burden the irreplaceable programmers with additional responsibilities lest they kill the golden goose. Instead, managers should look for challenges that will keep the stars at their peak performance level.

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Buy it today at CreateSpace.

Join the discussion on Facebook.

END NOTES

The waterfall model is due to Winston Royce, “Managing the Development of Large Software Systems,” Proceedings of IEEE WESCON 26, 1970, pp. 1-9. Ironically, Royce proposed the waterfall as an example of how not to build software systems.

Brooks’ Law, which states that adding manpower to a late software project only serves to make it later, is from Fred Brooks Jr., The Mythical Man-Month: Essays on Software Engineering, Addison-Wesley, 1975.

The first study to find significant differences in programmer productivity is Sackman, H. H., W. J. Erikson, and E. E. Grant, “Exploratory experimental studies comparing online and offline programming performance,” Communications of the ACM, 11 (1), 1968, pp. 3-11. Brooks himself estimates that good programmers are 5 – 10 times more productive than ordinary ones.

The 500 lines of code per month average is cited in Van Vliet, H., Software Engineering Principles and Practice (Second Edition), John Wiley & Sons, 2000, p. 175.

An earlier version of this essay appeared in Software Engineering Notes, volume 29, number 3, 2004.

I, Programmer: the Three Laws of Software Engineering

2008-10-04T20:54:00.000-07:00

When I taught the third year undergraduate course on software engineering at the University of Victoria (UVic), I began by asking students if they could define the topic of study, software engineering. Many of my students were in a formal engineering degree program; others were computer science majors within the Faculty of Engineering. Surprisingly, they struggled to offer a satisfactory definition of what it was they were studying.

So we start with engineering, which I take to be the building of useful tools for humanity. It follows that software engineering is the building of useful software tools for humanity. But isn’t that just programming? How is software engineering different from computer science?

A common public misconception is that computer scientists design and build computers. Designing computer hardware is actually the job of computer engineers. Computer science is about software: the mathematics of computation, the science of abstraction, and the art of programming. The vast majority of computer science graduates end up in careers as software engineers, writing practical software applications for other people to use.

Fred Brooks suggests that the scientist builds things in order to study, whereas the engineer studies in order to build things.[1] In order to build software systems, the engineer must study computer science. Lots of computer science. Thus the reality is that most practicing software engineers are computer science graduates—not formally trained engineers.

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Want more? Buy it today at CreateSpace.

Join the discussion on Facebook.

[1] Frederick P. Brooks, “The Computer Scientist as Toolsmith II,” Communications of the ACM, 39 (3), 1996, pp. 61-68. He further postulates that by any reasonable criterion, computer science is not a science—it is engineering.

Software engineering is a social activity; forget that and your career is lost

2008-10-04T17:59:00.000-07:00

Software engineering is a social activity; forget that and your career is lost.

Not everyone can be a software engineer. It requires significant proficiency in mathematical reasoning, systems thinking, perseverance, and intelligence to build information storage and retrieval systems using today’s abstract programming techniques.

Thus, the goal of computer science curricula is to endow students with the requisite mathematical maturity and problem-solving skills to become good software engineers. However, nothing is typically done to instill students with the requisite plain old maturity needed to succeed in the emotional minefield that is the modern office.

In his influential book on the C++ programming language, Bjarne Stroustrup says that programming is a human activity, and all is lost if this is forgotten.[1] While I don’t know that all is lost if we forget this, I will say that software engineering is a social activity, and ignoring this fact is a recipe for career failure.

This excerpt is from the book Lord of the Files, published by Thought Pilots.

Buy it today at CreateSpace.

Join the discussion on Facebook.

[1] Bjarne Stroustrup, The C++ Programming Language (2^nd ed.) (Addison-Wesley, 1991), p. 363.