Saturday, October 18, 2008

The white trash software methodology

My mother's favourite scene from An Officer and a GentlemenThe White Trash Software Methodology is to hire the smartest person you can find with the lowest self-esteem who was raised under an oppressive Protestant work ethic. Then bluntly tell this person that you doubt they can build a great piece of software.

My mom's favourite cinematic moment is the scene from An Officer and a Gentlemen when Richard Gere, with newly minted officer’s commission and freshly-pressed white uniform, marches across the factory floor to sweep Debra Winger off her feet. The film ends with the promise of a life of love and romantic adventure for Winger's character as she follows her man through a series of exotic overseas airforce postings. My mother's fondness for this film is likely because it mirrors her own revisionist take on events leading to her marriage to my father. The only problem is that my dad - a budding Sapper in the Canadian Military Engineers - sabotaged his commission when he told his commanding officer to "fuck off" during his officer training in Wainwright, Alberta. As a result, love lifted mom and dad right up where they belonged, which was apparently the same place they had started from: Chilliwack.

So it was that I came to be born in a small but picturesque army town, comprised of three distinct socioeconomic groups: the fundamentally religious, the native Indians, and the white trash. None of these labels particularly fit me like Chilliwack, B.C.a glove, but through a process of elimination, I gravitated to the latter. My parents listened to Elvis (Aloha from Hawaii on 8-track as we drove around in our AMC Gremlin). My mom made hamburger stew every Saturday night, which we ate on TV trays while watching Hockey Night in Canada. None of my immediate nor extended family ever went to university, including my grandparents, whose opportunities for social mobility were yanked out from under them when their parents decided they wanted to be farmers in Western Canada instead of continue being solicitors in London or enjoy a leisurely retirement in Glasgow. The romantic call of the Canadian frontier (circa 1920) altered my family’s trajectory for at least two generations.

I grew up a latchkey kid in a working class subdivision. The homes were new, but nobody could really afford them without both parents working. As a result, the children of my neighbourhood enjoyed a significant amount of unsupervised play, especially during summer vacation when we often played road hockey as long as there was daylight. Without anyone around to cook for us, we ate a lot of Alphaghetti for lunch. Recently it has become popular to bemoan the lack of unstructured and unmediated time that children get to explore nature and to be themselves. But we all know what happens when you leave a bunch of white kids unsupervised for very long: Piggy gets killed. Lord of the Flies was re-enacted in some bastardized fashion on a daily basis in our suburban utopia. Now that I am an adult, ever day is Animal Farm on some level. As an aside, why is it that reading those two novels as children merely endows one with the ability to use their titles as allegorical shorthand, but does not teach one the lessons needed to avoid having opportunity to require their use?

At this point, dear reader, you are probably wondering how a rambling personal memoir is going to help you learn anything about software engineering. To be honest, so am I, except for this nagging feeling I have that a working class background coupled with low self-esteem is the recipe for the ideal software engineer. So bear with me for at least one more paragraph.

I was raised under the cloud of a Protestant work ethic and just enough physical abuse to ensure that I never wanted to disappoint any authority figure, especially my father. The male role models that surrounded me were from a generation that came of age during a time of unprecedented economic growth and opportunity in North America, which meant they could be and do pretty much whatever they wanted. Unfortunately it seems that many aspired to become a raging, violent, racist homophobe incapable of sensitivity or showing any sign of weakness. But they sure had high standards for us boys, and rarely hesitated to let us know when we were out of line or not measuring up. This was especially true of the vein-bulging hot heads that coached our hockey and baseball teams; the principal that picked children up by the ears; the father who chased his son around the yard slowly insisting that he "come here... come here..." so he could beat him with his belt in front of all of his friends; the uncles that molested our cousins; and so on. Ladies and gentlemen, the Silent Generation. Silent, but deadly.

Ultimately these stoic men made me feel vulnerable and weak. (Either that, or it was the abundance of Kraft Dinner I ate combined with my undiagnosed lactose intolerance.) Subsequently, girls made me feel inadequate (because I saw myself as vulnerable and weak). But man, did I ever refuse to lose, channeling all that anger and misunderstanding and disappointment into a competitive, guilty, fearful, responsible ball of anxiety and intellectual fury. As a result, I was not always a nice kid. And when you're not always a nice kid... well, in my day that meant you would often end up in a fight on the playground.

I was a skinny kid, and so I was beaten up. Usually in retaliation for some intellectual bullying I was engaged in, but sometimes for merely being in the wrong place at the wrong time. Perhaps it is due to this experience that I grew into an adult about whom people have often remarked "he has an edge to him." But I think (hope?) it is a good edge; trust me, you want people on your programming team that got beat up as children, for they will go the extra mile to please those in charge and to prove them wrong. You need someone like me: a white trash software engineer that can turn his hidden injuries of class into an advantage that will ensure your team ships on time and on budget. By making sure your team has at least one white trash software engineer, you greatly increase the odds of success.

Why? Consider the alternative: a team of well-heeled, secure individuals who were raised with unconditional love, whose only punishment they knew growing up was nurture and compassion. These are great citizens, and make for ideal friends and neighbours, but would you want to go to war with them? I think not. Given the often Herculean effort required to tackle the complexity of writing software that works, it can be difficult to get these people to devote the necessary mental energy and commitment simply because they don't really care if things don't work out. "There's a bug in my latest check-in...? Oh well. I've got to leave early as I have a friend from Zurich in town and she wants to join our run group this evening." Awesome.

Writing software is not easy. I have found that it helps if you have low expectations and a fear of failure if you want to persevere long enough to be successful at it.

Tuesday, October 14, 2008

Your favourite methodology is eXtremely gay

A good software process will not save you from bad programmers: the best methodology is to just hire the smartest people you can find and get the hell out of their way.

When programming, you can never have too many comments. No, wait… when programming, you can never have too many jelly donuts. I always get those confused. The former is an example of a coding standard, which can be part of a programming methodology. The latter is an example of a software development process. In this entry we'll discuss the difference between a software development process and a methodology, and why neither really matters.

Software Process and Methodology
As mentioned previously, software engineering is the name given to the branch of computer science whose research focus is the application of engineering rigour to the problem of building large, complex software systems on time and on budget. A significant amount of this research effort has involved attempting to develop a formal process (the engineering rigour part of the definition) that can ensure repeated success in developing software.

A software process is the set of steps taken in the development of a software product, where each step has a set of deliverables (usually in the form of documents) that are signed-off by an authority. This is known as visibility, and is considered an important aspect of being more rigorous. There are a lot of benefits to being visible, including automatic ass-covering (“but you agreed to this specification…”) and having something to give the new co-op student to read for the first few days when he or she arrives for work and you have nothing prepared for them to do yet. The waterfall model of the software development life cycle (covered in an earlier post) is an example of a software process.

A software development methodology fills in the details of how each step in the software process should be done: what are the specific deliverables - and their format - of each step in the process; how should the system be decomposed into modules; do we program in pairs or alone; do we utilize code reviews or unit test, etc.

A methodology is about taking a disciplined approach, whether it is to making coffee, investing in the stock market, or writing software. In the case of an investment methodology (following a particular method because you believe it is philosophically superior to other practices), the resulting discipline ensures that you will likely make money even if the majority of your stock picks don’t pan out. How? By cutting your losses early and letting your winners ride. Likewise, a good software methodology will serve to help minimize your losses by, for example, catching bugs as early in the process as possible, or by achieving a working system that is easy to change.

To further elaborate on these concepts, let us revisit the stages of the software development life-cycle.

Requirements
Stroustrup suggests that knowing what it is you are building is the crucial first step. However, if you have a target or pre-existing user community that will provide feedback (quite vocally, I may add), is that even possible? I can’t recall any significant software project that I have worked on where it was known ahead of time exactly what the artifact would ultimately become. Unlike many programmers, I don’t see this as a curse. Rather, I see this as one of the most compelling reasons to build software rather than less malleable consumer products. I think it is tremendously exciting to work within a medium that is so fluid. While I can see that you wouldn’t want to start construction on a condominium tower without first knowing what it was you are trying to build, there is no good reason to assume that such certainty is possible or even necessary with software.

The grouchy programmer’s insistence on a clear, complete requirements specification before design and coding begins is annoying. It is a fact of life that the requirements will change, so why insist that the spec exists beforehand? Embrace the chaos: come on in, the turbulence is fine! And if you can’t, might I suggest zoloft or effexor as an excellent means of combating that anxiety.

Design
If you don’t have the design right you might as well go home. Mistakes in any other phase can be corrected after the fact with relative ease compared to the pain of redesign. How do you learn good design? Same as how we humans learn anything: Through imitation of others who are good at it, and by doing it and learning from your mistakes.

Implementation
Alec Baldwin in Glengarry Glen Ross said it best: "A-B-C. Always Be Coding." At least I think that's what he says. I wasn't really paying attention.

Testing
In any online business, first to the post is a crucial advantage. If you take the time to follow a traditional software process, you risk failure for the entire enterprise. At abebooks.com, we developed our own software process (speed programming (tm)) to cut back on the time it took to ship new versions of the website. Its defining feature: user testing meant “the users tested the software for us.” Everything was in beta, all the time. Eliminating an entire phase of the process was certainly a massive time-saver.

With Alibris nipping at our heels (with their bags of American venture capital and glossy ad campaign), we didn’t have time for code reviews and regression testing. But fuck it if we were going to let those assholes beat us. We were there first and worked feverishly to maintain the lead.

Don't try this at home: skimping on test is always a bad idea. What saved our ass was that we were leaner, meaner, and smarter than the competition. Our agility (the ability to quickly recover from boneheaded mistakes) and complete control over the deployment process enabled us to survive. We were good, fast, and cheap. Over at Alibris, it was apparently good, fast, or cheap: pick any two. Or maybe it was gas, grass, or ass: nobody rides for free.

Maintenance
Alas, the relentless pressure to constantly be shipping will take its toll. The term Internet years is really about their effects on your body: one year of development in this environment will age you by five years compared to the six month release cycle employed by shrinkwrap and custom retailers. Inevitably bugs in the web app will arise due to unforeseen spaghetti effects and the compressed development cycle. At this point, it is not uncommon for management to ask for a post-mortem analysis of why that innocuous change resulted in 18 hours of downtime. At my previous employer this became unnecessary once they adopted what became known as Russ's Rule: it is always the DBA's fault. It became such a time saver in the whole software life-cycle that it was officially codified in their project workflow template.

The Problem
Software engineering is unique amongst engineering disciplines in that adherence to accepted methods and a rigorous, visible process does not guarantee success. We have yet to discover any software process and methodology that ensures repeatable results when it comes to developing large complex systems in an efficient, cost effective manner. But in my experience, it's not the methodology or process that is to blame if a software system fails to meet requirements. Rather, the blame lies squarely at the feet of the software engineers who were unable to tackle the complexity of the system under development.

Conversely, the best software engineers are able to repeat their previous success, regardless of the task laid before him or her, and regardless of the software development process or methodology utilized.

For example, building a scalable, highly available web application in a modern programming language like Java is subtly complex. The programmers need to have a strong grasp of programming concepts such as: multi-threaded programming and thread synchronization; caching; database connection pooling; database transactions; and, data structure and algorithmic analysis, to name but a few of the things that immediately spring to mind. The complexity of the underlying database management system alone can lead to project failure if someone on the team lacks the requisite knowledge to manage the system as the size of the database grows. These complexities have nothing to do with software process or methodology and everything to do with the intelligence, experience, and judgement of the people on the team.

This may be our industry's dirty little secret: programmers are not interchangeable commodities.

What to do?

First, understand that the weakest link metaphor does not apply to software development project teams. The team is much stronger than its weakest member. In fact, it is quite the opposite: a software team is only as good as its strongest member. Experience, judgement, and talent do not magically coalesce from the interaction of the individuals. Good software is not an emergent property of decentralized collaboration between average programmers. The project will succeed if and only if the complexity of the underlying system is within the grasp of at least one member of the team. The software process and methodology is irrelevant. But they remain the opiate of the management. Your favourite methodology won’t save you from bad programming, but it will give you something comforting to cling to while the ship is sinking.

Sunday, October 5, 2008

All we really need to know about software engineering is in the film Office Space

One of the difficulties with teaching software engineering to a class of undergraduates with very little industrial programming experience is trying to convey why some of the material covered is relevant and important. For example, the importance that designing for change plays in software engineering is difficult to explain to a group of students who have never experienced the joy and frustration of maintaining a program written by someone else. After emphatically waving my arms in front of the chalk board for twelve weeks, yet feeling unable to teach my students everything they would need to know about software engineering, I would screen the film Office Space on the last day of class.

The 1999 Mike Judge film Office Space chronicles the efforts of software engineer Peter Gibbons to find peace with the role of work in his life. One of the few artifacts of popular culture (Douglas Coupland’s Microserfs being another notable) to feature a software engineer as the protagonist, Office Space – despite its enduring mass appeal and broad comedic base – embodies many subtle gems of wisdom regarding the craft of software engineering.

The following rules are all found to varying degree in the plot and characters of the film.
  • design for change
  • software engineering is a social activity
  • an untested program does not work
  • not everyone gets to write video games
  • Brooks’ Law
  • you will be expected to work long hours
Design for Change
We are all taught as undergraduates that maintenance is the dominant phase of the software development life-cycle. Our responsibility as software engineers is thus to design systems that are easy to modify without introducing new bugs; i.e., to design for change.

Office Space is set in 1999 at fictitious Dallas computer firm Initech where Peter, like most software engineers, is employed maintaining an existing software system. In particular, Peter is busy fixing the Y2K bug in bank software. More than eight years on, many of us in the industry have mostly forgotten about the incredible person-effort expended on fixing all that code, which obviously was not properly designed for change. The film thus serves as an object lesson in how not to design systems. It is also a historical reminder of what the Y2K bug was all about, even if Peter’s attempt to explain it to his love-interest Joanna is aborted when he intuits her waning interest. Even without reminders such as these, the Y2K crisis is likely to remain a footnote in the history of computer science for one of two very different reasons:
  1. it was a such a monumentally anti-climactic non-event, suggesting the scope of the problem was severely overestimated by industry observers (and practitioners); or
  2. it was our finest hour – a triumph of modern software engineering that averted a global catastrophe.

Figure 1. Samir, Peter, Michael, and Tom in cubicle hell at Initech. Whoever decided that cubicles were an ideal way to layout offices should be forced to live out an eternity stuck in a cubicle in the middle of an expansive floor pinned beneath the incessant glare of fluorescent lighting. The demoralizing effects of these wall dividers, which offer little in the way of privacy and noise reduction, are legendary. Given that software engineering requires teamwork, and cubicles are an impediment to effective communication, they should be rejected outright.

Software Engineering is a Social Activity
Software engineers write modules, classes, and libraries that are used by other software engineers in the construction of larger systems. Teamwork is thus an essential component of the process. Moreover, these systems are constructed to meet the requirements of end-users who belong to a cultural group that is usually decidedly outside that of software engineering. Interaction with this group (the customer) is an important part of the development process.
Two significant problems with this reality are reflected in the film:
  1. software engineers as a whole – a point vociferously made by Business Analyst Tom Smykowski while explaining what it is he does at Initech – are not good with people; and,
  2. these people with the poor social skills (like Milton) are our teammates.
It is difficult not to sympathize with Smykowski, the first to be downsized during the ongoing business process re-engineering of Initech. Like Dan’s father in Microserfs, middle-aged Smykowski has woken up to an industry that sees him as irrelevant. When asked during his interview with the efficiency consultants (the Bobs) “what would you say you do here,” Smykowski defensively explains that he takes the specs from the customers to the software people, “because engineers are not good at dealing with customers.” This stereotype of the socially challenged software engineer seems the raison d’etre of the modern day Business Analyst, whose sole job requirement it often seems is simply – as Smykowski bellows ironically – to “have people skills!” Smykowski’s plight should serve as a reminder of the need to maintain relevancy: people skills alone are not enough.

Unlike Smykowski, Milton appears so completely socially retarded as to be unbelievable. To those of us in the industry, however, people like Milton are not an aberration. Indeed, some of the best programmers I’ve ever known share many of Milton’s personality traits, which are likely related to Asperger’s syndrome and a healthy dose of obsessive-compulsive disorder (OCD). These psychological afflictions are considered by some to be an advantage when it comes to the kind of thinking required to excel at software engineering.

Figure 2. Tom Smykowski, Business Analyst. Learn to love your Business Analyst, for they have the unenviable task of pinning down and documenting the customer specifications (i.e., specs). You can always blame the Business Analyst when the system you built works according to spec, but does something stupid.

While Peter’s patience is merely strained by Milton, others who do not share his neural processing style openly mock Milton and his OCD. In particular, Initech vice-president Bill Lumbergh delights in asking Milton to move his desk simply because of the unease it causes him. Likewise, in a scene reminiscent of the alpha-male who steals the geek’s ball and claims it was his in the first place, Lumbergh’s commandeering of Milton’s red Swingline stapler is a gesture that seems predicated only by his distaste for Milton as a person. While not endemic, this kind of manipulation and bullying of the more socially challenged amongst our peers is an unfortunate practice witnessed far too frequently.

Untested Programs Do Not Work
Peter’s colleague Michael Bolton – the classic passive-aggressive nerd – feels both exploited and disrespected by his employer despite his contributions to the firm’s intellectual property. Michael’s revenge is enacted when he unleashes a virus on the credit union mainframe designed to slowly steal fractions of pennies during interest calculations. However, the program is put into production without the opportunity for testing.

As Stroustrup succinctly puts it, “a program that has not been tested does not work.” To Michael’s dismay, his virus has a bug that results in a conspicuous amount of money to go missing, resulting in the likelihood of its detection. He afterward explains to his co-conspirators Peter and Samir that he must have had the decimal point in the wrong place, and that he always makes some stupid little mistake like that. In truth, all software engineers make stupid little mistakes like that, all the time. In fact, it’s often a complete shock to me that software works as well as it does given how:
  • massively complex systems have become;
  • escalated the corresponding attention to detail required to maintain these systems has become; and,
  • how easy it is to introduce run-time defects through simple typos and editing errors that escape compile-time detection.
This is why we test, and especially why we regression test after making even the most benign changes.

Not Everyone Gets to Write Video Games
Part of the apparent lack of job satisfaction indicated by the employees of Initech likely stems from the boring application area: banking software. We all want to work on the cool stuff, like games, artificial intelligence, virtual reality, porn websites… but let’s face it: the majority of software engineering jobs are in mundane application areas (like finance and corporate IT) that involve straightforward information storage and retrieval.

It begs the question, is Peter simply bored? Is Peter’s career crisis merely caused by underutilization and a coincident lack of motivation? The Bobs certainly feel that Peter’s problem is that he hasn’t been challenged enough, believing him when he explains “it’s not that I’m lazy, it’s that I just don’t care.” Software engineers should strive to find work environments and application areas that they find at least mildly interesting. If they don’t, they risk the predicament Peter explains to the Bobs: if your only motivation is the fear of losing your job, you will only work just hard enough to not get fired. As discussed in a previous post, software engineers instead should learn to assess their own work interest and its effect on their productivity.

Brooks’ Law
In his classic monograph on software engineering, Fred Brooks describes the phenomenon whereby adding manpower to a late software project only serves to make it later. This counterintuitive result, which has become known as Brooks’ Law, is due to many factors, including the increased overhead of communication, training, coordination, meetings, etc.

A possible corollary to Brooks’ Law is that within a software firm, individual programmer productivity is inversely correlated to company size. The larger the company, the less empowered programmers feel, or need to be; ineffectiveness can be hidden within the bureaucracy. Conversely, small companies engender ownership: employees see the direct consequences of their actions. I refer to this (purely anecdotal) observation as the Brooks Effect. Initech is a case study in the Brooks Effect. While interviewing for his own job, Peter admits to the Bobs that “in a given week I probably only do about 15 minutes of real, actual work.” The rest of the time is spent on such mundane activities as meetings, documentation, going for coffee with teammates, zoning out, filling out time-sheets (the infamous TPS Reports that serve as a running gag throughout the film), asking yourself “is this good for the company?”, playing Tetris, and eating birthday cake with coworkers.

The parody of the ubiquitous birthday celebration is recreated with particular precision in Office Space. From the uninspired singing to the disingenuous show of appreciation, no detail is left unskewered. In reality, once a company reaches the size of 50 employees, there is a statistical likelihood of one birthday cake event per week. For some inexplicable reason, this celebration is usually scheduled right before an afternoon code review. The comatose state that invariably follows can only partly be blamed on the resulting insulin shock.

Case in point, in my one year of work for Bell-Northern Research, as one of over 4,000 employees I managed to check-in to the code repository an underwhelming 16 updates to the DMS-100 digital telephone switch software. During that time span, however, I did participate in over 30 birthday celebrations and three farewell luncheons (not including my own).

You Will Be Expected to Work Long Hours
Peter’s boss Bill Lumbergh is a slightly exaggerated caricature of the prototypical alpha-male, and the film’s most memorable character. Either a programmer who rode his social skills through the management track to the top of his game, or an exemplar of the Dilbert Principle, the always coffee-sipping Lumbergh seems primarily concerned that his employees accurately complete their TPS Reports.

Lumbergh, who Peter believes represents “all that is soulless and wrong” doesn’t hesitate to ask Peter to “go ahead and come in [Saturday].” The key phrase is “go ahead”… as if to suggest that Lumbergh is doing Peter a favor by letting him work over the weekend. Software engineering – like many other professional positions such as law and medicine – is an occupation that can demand Herculean effort and extended work hours, particularly as deadlines approach. The overtime is rarely ever paid, despite efforts by jurisdictions such as California to legislate against this practise. But the legality doesn’t seem to matter, anyway, as many software engineers crave the intellectual calisthenics that programming offers, and will work overtime (or after-hours on open source projects) irrespective of compensation. In this regard, Peter is an anomaly.

When Peter fails to show up to work on Saturday, Lumbergh launches a never-ending stream of telephone calls to his home in an effort to locate and chastise his AWOL employee. This is my favorite scene in the film, not only for the comedic element of Lumbergh’s distinctive greeting that begins each of his 17 recorded messages, but because it echoes my own personal experience with two different employers. While employees everywhere have likely dealt with the inequity of the boss who parks the Porsche in the only reserved spot, the frantic cell-phone calls, pages, voice messages, instant messages, and urgent email seems particularly endemic to the life of the software engineer on the mission critical path.

Figure 3. Lumbergh asks Peter to work over the weekend. There is obviously a dress code at Initech (stated or implied) that forces all the men to wear a tie, even though they spend their day at a desk writing code. What on earth does wearing a tie have to do with programming? Why not specify that all software engineers have to wear a funny hat or wig? The tie is as necessary as a hat or wig, especially given (as Smykowski claimed) the software engineers are not allowed to meet with the customers. They made us wear a tie when I worked at IBM’s software development lab in Toronto (and made us start work at 8:42 AM because of the unionized manufacturing plant next door), except for one dude who was such a compiler genius that he got away with wearing leather from head to toe. As an aside, you know your career is progressing nicely if you get to exert more control over your wardrobe with each new position or employer. At Backstage we adopted a “pants optional” dress code because ass sweat is greatly reduced by the absence of pants. But be warned: people will make assumptions about your socio-economic status based solely on how you are dressed. The doorman Theo at my apartment building in Cape Town thought I was just a couch potato who passed his time surfing. He had judged me based on the way that I dressed, and had no idea I had a doctorate and was a president of a company. While the official uniform of software engineers is the unbuttoned dress shirt and khakis, I prefer sweat pants: comfortable, but apparently guaranteed chick repellant. “It’s a good thing you are married, because you’ll never get laid dressing like that” suggested my queer eye, Canterbury Dave from Smuggler’s Cove Pub, where I spent too much of my time as my marriage disintegrated riding the spiraling crashing waves of impotent despair from my usual seat at the bar. Of course, married or not, I wasn’t getting laid, anyway. At least the bartender used to put on the alternative 80s station on the satellite whenever I was there. But I digress.

Is It All That Bad?
Towards the end of the film, Joanna counsels Peter that it is okay to be uninspired by his work, because “most people don’t like their jobs.” This is one moment where the film rings hollow, for in my experience, software engineers love their work. We may not like our boss, and the meetings, and the politics, but at the end of the day, there is nothing else we would rather be doing.

As I ease into my 40s, Peter’s rhetorical question posed to Michael and Samir is particularly poignant: “what if we’re still doing this when we’re 50?” Personally, I sincerely hope I’m still doing this when I’m 50. As Lumbergh would say… yeah, that would be great.

END NOTES

I am amazed at how accurate the depiction of the software development process and software engineers is in Douglas Coupland’s Microserfs (Harper Collins: 1995) and his sort-of sequel JPod (Harper Collins: 2006).

Any textbook on software engineering will claim that maintenance is the dominant (longest) stage of the software life cycle. The seminal paper on how to design programs so they are easier to change is Parnas, David L. (1972): “On the criteria to be used in decomposing systems into modules,” Communications of the ACM, 15 (12), pp. 1053-58. In my experience, the adoption of early daylight savings in March of 2007 caused more software outages than the Y2K rollover.

Silberman, Steve (2001): “The geek syndrome,” Wired, 9 (12) discusses the tendency of computer scientists and engineers to exhibit autistic traits (in particular, Asperger's syndrome), and the strange cluster of autistic children born from couplings of techies in the Silicon Valley.

One of the best books on programming ever written is Stroustrup, Bjarne (1991): The C++ Programming Language (Second Edition), Addison-Wesley, from which I quoted page 380.

Brooks’ Law is from Fred Brooks Jr., The Mythical Man-Month: Essays on Software Engineering, Addison-Wesley, 1975.

The Dilbert Principle suggests that in many companies it is the worst employees who are promoted to management, because the company cannot afford to move the productive employees out of their current positions. Scott Adams wrote The Dilbert Principle: A Cubicle's-Eye View of Bosses, Meetings, Management Fads, and Other Workplace Afflictions (Harper Business, 1996).

The requirement to pay overtime to all software engineers was later overturned by California law S.B. 88 which exempts jobs that require skill and proficiency in the theoretical and practical application of highly specialized information to computer systems analysis, programming and software engineering.

An earlier version of this essay appeared in Software Engineering Notes, volume 30, number 4, 2005.

Saturday, October 4, 2008

The programmer life cycle

Legendary Canadian businessman Jimmy Pattison was famous for his ruthless policy of firing the poorest performing monthly salesperson at each of his many car dealerships. I could never do that at my own company – Backstage Technologies, a casual game studio and sometimes contract software engineering firm – because excellent programmers like the guys who work for me are extremely difficult to replace. But that doesn’t mean I don’t rule with an iron fist. In fact, I think there should be a Japanese-style TV reality show set at Backstage entitled Iron Fist. Each weekly episode of Iron Fist would conclude with me, adorned in iron gloves, whaling on the nuts of the employee with the fewest accumulated lines of code in production. Unfortunately, most weeks that would be me.

Why is my productivity waning, and why isn’t the amount of work that my employees do constant? Most textbooks on software engineering make at least passing reference to the notion of programmer productivity. The conventional wisdom is that an average programmer completes a paltry 500 debugged lines of code (LOC) per month. To students and professionals alike, this number seems absurdly low. The standard set of explanations for this phenomenon includes, among other things, the communication overhead of working in teams (i.e., Brooks’ Law), and the time allotted to non-programming activities like requirements gathering, documentation, and design.

These explanations tell only part of the story. While it has long been accepted that productivity rates can differ by an order of magnitude between the best and worst programmers, little has been said regarding the reality that work rates of the same programmer can likewise differ over time. Not unlike the software systems they construct, programmers follow a predictable life cycle. However, the programmer life cycle is not comprised of distinct activities (like the software development life cycle, which we discuss below) but rather by emotional stages that directly affect and predict productivity. The sequence of stages is characterized by an initial six month period of intense interest, at which time productivity rates are indeed much higher than the oft-quoted 500 LOC/month average. After a short period of volatility, the programmer then enters a prolonged phase of steadily dwindling interest, resulting in productivity rates that mimic the average. Each time a programmer switches employers or begins a significantly new project, the life cycle starts anew.

This conjecture is based purely on my experiences and observations while working as the senior software engineer for two successful Internet startups (abebooks.com in Canada, then Prophet.Net in the United States). The basic premise of the proposed relationship between stages of the programmer life cycle and productivity is the simple observation that employees are most productive when interest and satisfaction in their jobs is at its highest.

The Software Development Life Cycle
According to Webster, a life cycle is the series of stages through which something (as an individual, culture, or manufactured product) passes during its lifetime. In the case of a software product, the usual stages of its life are: requirements, design, implementation, testing, and maintenance.

Briefly, the requirements stage is when the product is conceptualized as a set of requirements, which are documented by a business analyst. This is what the product must do. The design stage is when a software architect imagines how the software will be built by decomposing the system into a set of interacting subsystems (database server, application server, web server, etc.), which in turn are decomposed into their constituent modules (database tables, transactions, stored procedures, classes, processes, threads, etc.). The architecture of the system is usually described by a set of directed graphs, where the nodes are the subsystems and the arcs denote relationships (e.g., “calls”, “stores”, “reads”). This is how the product will work. The implementation stage consists of database administrators and programmers realizing the design as a working program. The testing stage is meant to ensure that the program meets the requirements. Once this has been determined, a version of the program is released to its user community. Finally, the maintenance stage is where the program is changed into new releases that fix bugs that were not discovered during the testing stage, and new versions that add additional features.


Figure 1. The waterfall model of the software development life cycle.

Part of the goal of software engineering is to apply what is known as engineering rigor to the process of programming, the belief being that if it’s good for building bridges and toaster ovens, it should work just as well when writing software. Since engineers love linear project plans that involve milestones in the form of diagrams and documents, the first attempts at applying this concept to software development led to the infamous waterfall model of the software development life cycle. The waterfall model (see Figure 1) imagines each of the five stages of the life cycle as a self-contained step, where the output of each step falls only into the next. The model does not allow for backtracking or for information to flow in any direction but down. The obvious analogy to shit rolling downhill is left as an exercise to the reader.

The Stages of the Programmer Life Cycle
Everything we do as software engineers falls under one of these five categories that comprise the software development life cycle. The effectiveness with which we undertake these tasks is governed by where we are in the programmer life cycle (see Figure 2).

The programmer life cycle is comprised of six stages:
  1. Euphoric
  2. Productive
  3. Irreplaceable
  4. Resentful
  5. Bored, and
  6. Unproductive
While this particular life cycle model is perhaps most likely to apply to highly productive individuals (so-called star programmers) working in the 24/7 world of web development and e-commerce, it is my belief that there are fundamental truths in its structure that render it applicable to many software development situations and domains.

Euphoric. The first stage involves the start of a new job or significantly different project. The programmer is stimulated by both the newness of the environment and the challenges ahead. In many cases the programmer’s euphoria is fueled not only by these changes, but also by the coincident escape from a previous work situation that had become routine or was underutilizing his/her talents. This introductory phase is quite short in duration, lasting only as long as necessary to acclimatize to the new surroundings, learn a new development environment, and become familiar with the domain.

Productive. Once acclimatized, the programmer’s work interest reaches a peak. Because of this acute interest, productivity is at its highest. This stage – which lasts about six months – is when the programmer develops or takes ownership of some mission-critical software system. This coincides with steadily rising value to the organization.


Figure 2. Productivity as a function of stages of the programmer life cycle.

These first two stages constitute the “honeymoon period.” The next two are best described as the “volatility phase.”

Irreplaceable. Management soon recognizes that the programmer has become a valuable commodity. The programmer’s prestige within the organization is at an all-time high. As such, significant increases in compensation and fringe benefits are doled out in an effort to keep the programmer from ever possibly consider leaving. The golden handcuffs are sealed with the ubiquitous stock option grant, or the promise of one. The programmer feels on top of the world. Unfortunately this will not last.

Resentful. Management – sensing a sudden asymmetry in the employer-employee relationship – begins to bear resentment that a single individual (the programmer) is now responsible for the ongoing success or failure of the venture. Fearing a loss of control, management begins to act in a manner suggesting ownership of the programmer’s time and space. Symptoms of this malaise include requiring the programmer to carry a pager, work weekends, install broadband connectivity at home, and never under any circumstances take holidays. The programmer starts to receive email at all hours of the day from trigger-happy monitoring software reporting false alarms, and from managers requesting new features and emergency bug fixes.

In return, the programmer develops feelings of resentment towards management. This is exacerbated by management’s policy of rewarding the programmer’s competency not with bonuses and time off, but with additional workload and responsibilities. The first signs of complacency begin to appear in the programmer’s workplace attitude.

This unstable time of mutual resentment is necessarily short-lived, as emotions run too high for the process to carry on for more than a month or two. The working relationship can implode during the resentful stage, particularly if volatile personalities are involved. In the worst case, the programmer quits; the additional workload coupled with the stress of being irreplaceable yet resented becomes too much to take. However, in most cases the resentful stage merely settles into an equilibrium of mutual need: management’s need for the star to carry on keeping the software running, and the programmer’s need to be a star.

Bored. The post-resentment equilibrium sees the programmer’s activities shift more towards ongoing maintenance, consultative meetings with management, and internal knowledge transfer to other programmers and customer support staff. Because the initial challenges of the new project, environment, and technologies have all been met, the intellectual stimulation has dropped. This leads to boredom. Coupled with the excessive mental context switching demanded of the new activities, the programmer’s productivity (as measured by LOC/month) experiences a significant drop. Despite the tedium, however, this stage can last indefinitely provided the productivity remains above the minimum expectation level given the programmer’s current remuneration.

Unproductive. Like a manic depressive who goes off his medicine because he misses the occasional euphoric episode, or a love junkie addicted to the adrenaline rush of the first six months of a new relationship, the programmer is unlikely to remain in a state of boredom forever: something has to give. The change is triggered by a slide into the unproductive stage, characterized by the programmer working on his/her resume and visiting job sites on the ‘Net, while management views the programmer as “coasting,” overpriced, and expendable. One of two outcomes is inevitable: the programmer finds a new employer, or management moves the programmer to a significantly new role or project. Either way, the life cycle starts again.

Conclusion
This life cycle model should serve as a cautionary tale to both programmers and managers. The lesson for the programmer is to be aware of each stage and its effect on productivity levels, for ultimately one’s success as a software engineer depends on one’s perceived productivity. By recognizing the symptoms of boredom leading to unproductiveness, the programmer can proactively search for remedies, usually in the form of a frank discussion with management, or seeking out new projects and technological challenges.

Conversely, managers must understand the causes and effects of this life cycle in order to combat high levels of attrition and declining productivity. To get the most out of the organization’s stars, managers must avoid the resentment trap by resisting the temptation to over-burden the irreplaceable programmers with additional responsibilities lest they kill the golden goose. Instead, managers should look for challenges that will keep the stars at their peak performance level.

END NOTES

The waterfall model is due to Winston Royce, “Managing the Development of Large Software Systems,” Proceedings of IEEE WESCON 26, 1970, pp. 1-9. Ironically, Royce proposed the waterfall as an example of how not to build software systems.

Brooks’ Law, which states that adding manpower to a late software project only serves to make it later, is from Fred Brooks Jr., The Mythical Man-Month: Essays on Software Engineering, Addison-Wesley, 1975.

The first study to find significant differences in programmer productivity is Sackman, H. H., W. J. Erikson, and E. E. Grant, “Exploratory experimental studies comparing online and offline programming performance,” Communications of the ACM, 11 (1), 1968, pp. 3-11. Brooks himself estimates that good programmers are 5 – 10 times more productive than ordinary ones.

The 500 lines of code per month average is cited in Van Vliet, H., Software Engineering Principles and Practice (Second Edition), John Wiley & Sons, 2000, p. 175.

An earlier version of this essay appeared in Software Engineering Notes, volume 29, number 3, 2004.

I, Programmer: the Three Laws of Software Engineering

When teaching the third year undergraduate course on software engineering at the University of Victoria, I begin by asking students if they can define the topic of study, software engineering. Many of my students are in a formal engineering degree program; others are computer science majors within the Faculty of Engineering. Surprisingly, they struggle to offer a satisfactory definition of what it is they are studying.

So we start with engineering, which I take to be the building of useful tools for humanity. From this it must follow that software engineering is, by means of computer programming, the building of useful software tools for humanity. But isn’t that just programming? And how is this different than computer science? As we shall see, it is all about the assumptions we make about how the programming is done.

A common public misconception is that computer scientists design and build computers, when in fact that is the job of computer engineers. Computer science is about software: the mathematics of computation, and the science of abstraction. Eminent software engineer Fred Brooks suggests that the scientist builds things in order to study, whereas the engineer studies in order to build things. He further postulates that by any reasonable criterion, computer science is engineering. However, the folk wisdom is that computer science is to software engineering as physics is to electrical engineering. This is only half-true, for if it was completely true then most practicing electrical engineers would have a degree in physics and not engineering.

Who are the software engineers that write all those software tools we enjoy using on a daily basis? Typically, it is not formally trained engineers. The vast majority of computer science graduates are the ones that end up in careers as software engineers, writing practical software applications for other people to use. This anomaly led to a minor controversy in North America regarding the use of the title Engineer to describe what it is exactly that computer scientists do for a living. In fact, I can probably expect an injunction from one or more of the governing bodies of Professional Engineers for my improper and publicly misleading use of said protected noun in the title of this blog.

Whatever. The truth is, you wouldn’t want to hire any kind of Professional Engineer to create a software tool for you, your neighbor, or even your neighbor’s dog. They simply don’t have the requisite background knowledge and thus would invariably screw it up, blame you, then tell you how much harder their degree was to complete because they had to take physics.

More about this in later blog entries when we discuss the evils of self-taught programmers. For now, let’s summarize.

1. Software engineering is
  • the multi-person construction of multi-version software; and,
  • the name given to the branch of computer science whose research focus is the application of engineering rigor to the problem of building large, complex software systems on time and on budget.

2. Software engineers work in project teams, building programs for other people (the end-user, client, or customer) to use. Software engineering is therefore a social activity: programmers regularly interact with each other, and sometimes (if permitted) with the customer. Good communication and social skills are required.

3. Programming is the solitary activity of writing a computer program. Rather than complete programs, software engineers write small units of code that are combined with the code of other software engineers to create large, complex programs.

4. Computer science provides the knowledge and tools used by software engineers to effectively do their job. To be a good software engineer you need to know more about computer science than is taught as part of any of the traditional engineering specialties (e.g., electrical, mechanical, even computer).

My definition of software engineering (the multi-person construction of multi-version software) is generally attributed to David Parnas, who was actually my professor back when I was an undergraduate at the University of Victoria. Dr. David Lorge Parnas, the man who in 1972 invented the concepts of information hiding, encapsulation, and abstract interfaces, and thus drew the roadmap of modern best programming practices, was certainly the most famous professor I ever took a course with at UVic. I thought it was pretty cool that twenty years later I was teaching the same course at the same institution, using a textbook with an entire chapter devoted to his now famous case study in software design (the KWIK index).

My girlfriend at the time, Lynda, succinctly distilled the entire curriculum of Parnas’ class down to one law: design for change. She remarked that he could have saved us a lot of time if he simply had showed up for the first class and apologized for his lack of teaching skill, but explained that it did not matter because all he really had to say was “design for change.” He then could have handed over the actual work of instructing us on how to put that idea into practice to his competent graduate student and teaching assistant Gord Stuart. (I would later teach with Dr. Stuart at Camosun College. He pretended to remember me.)

David Parnas is undoubtedly a man of great intellect, and one of the most influential professors I have ever met. I still recall the exact moment when, sitting in the dark programming lab in the basement of the Clearihue Building completing a lab assignment for his course, I grasped the elegance of using another programmer's code only through the abstract public interface. I know that must sound silly or, like most great ideas, obvious in hindsight. But what my current crop of undergrads liked hearing about wasn’t the brilliance of his ideas so much as stories about what kind of professor he was. They wanted to know, did Parnas suck?

I first heard of the legend that is David Parnas while on a co-op work term. One of my managers – a jaded, humorous old mainframe programmer at the Ministry of Forests – referred to him as “feisty” and a sufferer of “little many syndrome”. He also insisted that when I returned to school that I take Parnas’ course on software engineering. I was only 20 years old at the time, and didn’t yet know anything about height and its effects on the male psyche; indeed, I can’t really say that I noticed people’s height at all, as everyone tended to just be shorter than me. Short or not, my take is that Parnas simply believed that he was right about everything and didn’t leave room in his mental universe for other opinions, or for very much else other than solving the world’s software engineering problems. He didn’t even know who Bobby Orr was, which should have immediately made him ineligible to work in Canada. This lack of cultural awareness meant that Dr. Parnas lacked street credibility with us, despite his repeated use of the punch line “after he regained consciousness” to describe how he dealt with public dissenters of his one true way to write software.

The great irony is that David Parnas wasn’t a good team player. He was a star programmer and an egoist; an ego-surfer before there was anything to surf. He was also a man of principle, exemplified by his refusal to work on the US Strategic Defense Initiative project. He carried a hanky and liked to blow his seemingly always congested nose during his lectures. I think he was slightly autistic. I liked him.

Superficially, the lower middle-class, suburban, provincial twenty year old students in his classroom (me included) couldn’t relate to him – we didn’t know many people who went to high school in the Bronx, or people that worked in Holland with Edsger Dijkstra, or programmers who wrote documentation in first-order logic, or who wrote cockpit software for the US Navy A6 fighter plane. Nor did Professor Parnas seem to worry about relating to us. He never drew on the board or used any visual cues to convey his ideas. Our textbook consisted of a photocopied spiral-bound collection of everything he (and only he) had ever published, much of which was impenetrable given our lack of intellectual maturity and professional experience.

Despite our immaturity and unbeknownst to us at the time, David Parnas taught us much of what we would eventually need to know about software engineering. In particular, he laid the foundation for a painless transition from procedural to object-oriented programming. After all, classes in an object-oriented language are simply a means of enforcing encapsulation, and object-oriented design looks to decompose a system into modules that each hide a secret (its implementation) that is likely to change. As well, I find myself time and again returning to his concept of the uses hierarchy as a means of defining project milestones as subsets of the complete system.

Parnas’ definition of software engineering tells us both what we should do, and how we should it. The science of software engineering is all contained in the two multis that make up his definition.

Multi-person: the software is written by a team of programmers. Each programmer writes modules (independent compilation units, or generally what amount to classes in an object-oriented language like Java) that are combined with modules written by other programmers to form complete systems. Before any programming begins, the system is designed by decomposition into these modules so that the work can be done in parallel by members of the team.

Multi-version: the software will change over time. Each module should hide one design decision behind an abstract interface. The modules interact (call each other) only through these interfaces since they are less likely (than the implementations they hide) to require modification as the software undergoes transformation due to evolving requirements, new features, and bug fixes. This will reduce the complexity of making changes to the software.

In other words, design for change. Just like Lynda used to say. It is such an important concept that it should form the foundation of our Three Laws of Software Engineering.
  1. A software engineer must design for change.
  2. A software engineer must obey orders given by his or her manager except where such orders would conflict with the First Law.
  3. A software engineer must protect his or her own continued employment, even if this conflicts with the First or Second Law.
As anyone employed for more than five minutes in our industry can attest, managers are always giving orders that lead to conflict with the First Law. The unending pressure to ship the next release of the software means that the Third Law dominates. At this point the software engineer can do one of two things: stick to principles and risk getting fired; or, cut corners and meet the deadline. The art of software engineering is to know when to violate the First Law, and to do it in such a way as to minimize its impact on those who will come to look upon the source code after you have long since moved on to another project.

END NOTES

The difference between science and engineering in the context of software is explained by Fred Brooks in his ACM Allen Newell Award acceptance lecture, “The Computer Scientist as Toolsmith II,” Communications of the ACM, 39 (3), 1996, pp. 61-68.

The definition of software engineering is found in the title of the David Parnas paper “Software Engineering or Methods for the Multi-Person Construction of Multi-Version Programs,” in Programming Methodology (eds. G. Goos, J. Hartmanis), Lecture Notes in Computer Science 23, Springer Verlag, 1975, pp. 1-11.

Dan Hoffman (who followed David Parnas to the University of Victoria) and David Weiss have compiled a greatest hits collection of papers that Dr. Parnas has published over the years as Software Fundamentals: Collected Papers by David L. Parnas, Addison-Wesley, 2001.

The three laws and the title of this entry are of course intended to pay homage to the short stories of Isaac Asimov.

The Lynda I mention was Lynda Wong – a sweet, funny, smart young woman who effortlessly made people smile. It was the spring semester of 1986 when we took CSC365 with David Parnas. We fell out of touch the following year after we broke up, but I was shocked and deeply saddened when I heard of her death in 2006. I wish I had treated you better, Lynda.

In the beginning, there was disappointment

There are books such as Winners Never Cheat and All I Really Need to Know I Learned in Kindergarten that romanticize the lessons of childhood and attempt to apply them to the problems of adulthood. Since software engineering is a social activity that is rife with politics, intrigue, conflict, and all the usual trappings that occur when you get two or more people with low self-esteem working together, one might write a book that suggests the key to better management of software projects can be found on the playground.

Do not expect me to write that book.

Maybe if my childhood had been different I would write that book, but the hand that I was dealt suggests that reality is more complex, and that most of us carry the scars of some pretty harsh lessons with us to work every day. Fortunately, this isn’t necessarily a bad thing, as we shall discover. However, if you don’t appreciate the simple fact that we are all imperfect humans that wear our injuries on our sleeves, your career goals - not to mention your personal goals - will likely suffer.

Fred Brooks once said that “adjusting to the requirement for perfection is, I think, the most difficult part of learning to program.” However, some of us found that adjustment quite painless, especially once we learned how to ask the debugger where that segmentation fault caused the core to dump. Adjusting to the imperfections of others is, I think, the most difficult part of learning to be a software engineer.

Given this belief, let me start by claiming something that is rarely examined in your typical undergraduate computer science class: The most important lessons you will learn during your career as a software engineer are the personal ones that make you a better human, not the technical lessons that make you a better programmer. In both the personal and the technical, experience is the key ingredient. The end result – personal and technical maturity – is where the journey takes you. Along the way, try to remember that we are all just doing the best we can.

To that end, try to realize that each of us brings with us to work each day a set of lessons crafted through a unique set of childhood experiences. Not all of these lessons are useful; the ones that aren’t are known professionally as baggage or issues. For example, let us consider some of the important things that I learned personally from my own childhood and see if we can derive their unintended effects within my workplace.

Lessons You Learned as a Child (But May Have Forgotten)

Your Dad will never buy you that mini-bike because he spends too much money on booze and cigarettes. As a result, I feel that colleagues can only act in their own interests, especially when addictions (to power, money, success) are in play. The modern day manifestation of this is the start-up where the owners can never seem to find time to put that promised stock option plan into practice.

Your Mom wishes you were more attractive. Even though I know I shouldn't have taken this personally since her fixation on my appearance was really just her way of dealing with the pain of being married to my father, I can't help but feel that I'm being judged by how I look, and that if I was only a little better looking I'd be more successful.

Poo goes in the toilet. Some kids learn this lesson really, really well. Too well. They grow up to become anal-retentive. You can spot these people by how their speech is peppered with scatological references; e.g., “crap” or “shit” as derogatory nouns. Conversely, some kids don’t see this as a rule so much as a guiding principle. They grow up to become anal-expulsive. You can recognize these people at work by their desk: their stuff spills out around them as a strange extension of their self, with no apparent order or structure. But they can’t help being this way any more than you can help being fixated on having a place for everything. It is a spectrum of behaviour. Placing yourself on this spectrum is easy: just ask yourself, “where are my keys?” If you are like most people, you know where they might be, and have a mental checklist of places you will look in order to locate them. If you don’t have a clue, you’re anal expulsive. If you know exactly where they are at all times, you are anal-retentive.

Don't talk back to the bully, or you'll get the living shit beat out of you in front of all your friends. Instead, I find it much safer to be passive-aggressive and bad-mouth the difficult people I work with to others. I believe this is what the inventors of instant messaging had in mind as the primary use of the new medium.

Your first girlfriend will have car sex with your best friend one night after they share a ride home from your house. Girls cheat, just as much as boys, but some find the cultural pressure to be a "good girl" so strong that they simply refuse to admit it. Or they adjust their moral compass to places that enable them to rationalize their behaviour and live with themselves. Either way, I struggle with trust issues.

(The careful reader will note that these last two lessons taken together leave me fearful of men, and suspicious of women. My tragic failure as a manager has been my inability to effectively teach dogs how to program.)

You are your own worst enemy. Think back to all those times and places where if you'd just acted differently how much better things would have turned out. Now I simpy find it difficult to trust my own instincts.

The new kid at school is always treated like shit. There should be an amendment to the UN Convention on the Rights of the Child that prohibits parents from moving a child to a new school, such is the pain and torture that awaits the new kid. Today, I never pass up an opportunity to piss all over the new guy.

Love is conditional. Do not confuse conditional love with tough love. Tough love says to you “I am here for you, but not if you continue to act in a self-destructive manner – I won’t enable you to do that.” Tough love is an actual act of love. Conditional love says to you “I am acting in a self-destructive manner, and it is your fault.” Your grandparents love you unconditionally, but they get sick and die before you need them most. Today, I always feel that my employer loves me conditionally. In particular, they love me when I'm writing awesome software, and they hate me when someone finds a bug in it. If you have people who were raised with conditional love on your programming team, they will go the extra mile. But only if they perceive the manager as the person (usually the father or mother… whichever one was alcoholic) in their life that withheld love. If it is safe to disappoint your manager, then you will. If it is not, you won’t... and you'll always ship on time and on budget, regardless of whether it kills you or not.

At least one of your friends was sexually abused, and it was probably the guy that kept suggesting you play “naked games” with him. If out of curiosity and a need for acceptance you decided to join in, that doesn’t make you gay. Oh, and the person doing the abuse is likely one of your uncles. I'm not sure exactly how this effects me on a daily basis, but I know it's not good.

Those are just some of the important lessons of my childhood that I bring to work with me every day. What are yours? What are the lessons of your colleagues and how are they different from your own? Figure out what they are and you are well on your way to understanding his/her fears and motivators.

END NOTES

The Brooks quote is from Frederick P. Brooks, The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition (2nd Edition), p. 8.

Software engineering is a social activity; forget that and your career is lost

Computer science is difficult. To create a non-trivial piece of software that functions correctly and reliably requires an above-average amount of logical thinking, perseverance, and intelligence. The basic level of mental competency necessary to solve problems in information storage and retrieval using the abstractions and models endemic to the software profession makes computer science a challenging academic degree program. A significant amount of mathematics, systems theory, and problem solving practice is crammed into the computer science curriculum in order that our graduates are ready for the obvious next step: a career in software engineering.

Thus, the goal of the undergraduate computer science education is to endow the graduate with the requisite level of mathematical maturity and problem-solving ability to be a good software engineer. However, nothing is typically done to instill the student with the requisite amount of plain old maturity that is required to succeed on the job. Apart from the usual trial-by-fire of working in random groups of four to complete team projects, very little is done to prepare the young person for the emotional minefield that is the modern office.

Stroustrup has said that programming is a human activity, and all is lost if this is forgotten. While I don’t know that everything is lost if we forget this, I will say that software engineering is a social activity; ignoring this fact is a recipe for career failure. But what exactly does that mean? In short, it recognizes that software development is done by teams of people working towards a common goal. Project success is effected by the team's ability to communicate and cooperate, and to understand each other and to be compassionate and supportive. While not exactly a marriage, many of the same concepts apply. Interpersonal issues - while not likely to cause a project to fail - will introduce delays and inefficiencies.

If you are reading this blog you are likely either preparing to become a software engineer, are already employed as one, or manage a group of them. This blog is about the social aspects of software engineering; or, what I call the human factors of programmers. While not meant as a self-help guide for computer scientist, you may or may not learn anything about your self or experience an introspective breakthrough that results in a happier career as a software engineer. But you might. In particular, if you are a student or new to the field, some of the things that I hope you can learn from reading this blog include:
  1. what the truly important stuff is that you learn in class;
  2. what you might expect from the real world in terms of career paths, work environments, and politics; and,
  3. that while your talent as a programmer will dictate how far your career takes you, how well you know yourself and your particular personality strengths and weaknesses will dictate how smoothly your career progresses.
There is often a cultural disconnect or divide between those who teach software engineering and those who practice it professionally. I know this first hand as someone who tried to balance parallel careers in academia and industry. One of the problems with this divide is that students often feel detached from their professors (to whom they can't relate), practitioners are out of touch with the results disseminating from academia, and professors are all whack-jobs who rely on institutional protection to thrive. While I don’t pretend that the motivations between educator, student, and professional are aligned, I do hope to build some bridges.

My intent is not to preach; it is to inform and entertain with a series of short essays that emphasize the qualitative, personal, and emotional issues you may experience as a software engineer. It is my hope that this leaves you better prepared to be a team player.

I don’t have any answers to the problems and complexities inherent in building large-scale software systems. I am not selling a methodology or a technology, but if I had to distill the essence of what I am selling down to one concept, it is this:

Overcome your insecurities and be nice to your coworkers.

While each of the entries is designed to be self-contained, there is a logical flow to the blog that I hope coalesces into this general theme of compassion through self-awareness and acceptance of oneself and others.