musings on managing software product development
I just finished reading Lean Software Development: An Agile Toolkit for Software Development Managers by Mary and Tom Poppendieck. It’s a delightful book; a good read, thought provoking.
I hold a bi-weekly staff meeting. At a recent meeting I thought I’d try one of the exercises from the Poppendieck’s book–I asked the team to identify sources of waste in their current development processes. One of the engineers shot back:
“this meeting”
Having just written about learning to stand on one leg, I’ve been reflecting on the art world vs. gainful employment in software.
art pays in negative money
The most money I earned as a modern dancer: $800 a month for six months of the year.
art words < programmer words
Dancing is a highly intuitive, visual, sensation laden experience. Working dancers spend much of their time learning choreography. You watch somebody else do the steps. You do them. Not that way. This way. You feel what it’s like to do the whole dance. Eventually you’ve learned the dance. We never wrote anything down. Most of the discourse between dancers was only barely verbal–we showed each other what we meant.
As a programmer, I spent all my time in text. Reading documentation, reading code. As a manager my time is split between text and words and white boards. The whole thing is performed slumped in a chair. I never get sweaty, and my feet don’t hurt at the end of the day.
Software is my second career. My first was modern dance.
Dance is a consuming life experience. Being a dancer is like being an opera singer; you are your instrument. The craft of dancing demands tremendous training. The art demands that you transcend that training.
I started dancing toward the end of my basketball career. I was attending UC Berkeley on a basketball scholarship and struggling to transition from center to forward. I needed to improve my mobility and speed. I thought dance classes might help. I never made the transition in my basketball game and my career fizzled, but dance stuck.
The learning challenges for a young dancer are immense. Consider the pirouette, in which the dancer spins on one leg. Here’s an example from the ballet world, Mikhail Baryshnikov performing in Giselle.
He does all kinds of hard stuff. The first pirouette comes about 30 seconds into the clip. Go ahead, try one. Spin around on one leg. You pick the leg. Try for 3 spins. See what I mean?
Just accomplishing the movement, hard as that is, isn’t enough though. Art demands transcendence. When you spin, make it so that the audience doesn’t see the effort, make it so they experience something profound. Maybe, the feeling of flying, or forgetting that they’ve just been laid off and leaving them with a sense of hope and beauty.
There is a way to learn to do the spinning, and there are ways to learn to approach the art. Those learning paths are what grabbed me and made me a modern dancer for 10 years.
The beginning of learning to spin is to learn to stand on one leg. First
you stand with your whole foot on the ground. Then you do it on tip toe, or releve. It took me about 3 years of daily classes to be able to stand on one leg in releve and not tip over. There wasn’t anything mechanical or procedural about learning how. It wasn’t like weight lifting, where if you just show up in the gym your physique changes. Learning to stand on one leg was mystical. Feeling what was going on when I fell over, listening to my teachers try to get through to me, feeling what it was like when it worked, experiencing the connections between skeletal alignment and the foot.
And then the art of it. A sudden pause in the midst of the dance, the dancer suddenly still, as if in mid air, standing on one leg.
At first, moments like that were terrifying. What if I fell? Eventually I arrived at the real learning. The dance itself would hold me there. If I didn’t hold back in any way, but gave myself over completely to the performance, it would happen.
It took me about eight years to arrive there–learning to give myself over to the dance. The breakthrough happened on stage, in a performance of a piece that was just beyond my comfort zone. That night I realized that the dance was much bigger than me, it was like a series of waves and currents. I had to let them take me over. That was 25 years ago. I remember it like it was yesterday.
The end game is the part of the development cycle between functional completion and shipment. It’s the easiest part of the cycle to manage. QA identifies must fix defects. Development performs surgical repairs without disturbing the rest of the system. Eventually an equilibrium between defect discovery and repair is obtained. One more regression test and we ship.
The end game is the easiest part of the development cycle to manage because it’s algorithmic. Only edit code to address something that prevents shipment. Team review every change. Manager review every check in. Team and manager review each change with QA: what’s the functional change, what’s the regression risk. All you managers out there, speak with an authoritative voice tone, enjoy the feeling of competent change management!
The end game is the easy part. What about the rest of the development cycle? How do you get the changes you want:
This is the change control that makes a difference over the life of a product. This change control is very different from the grumpy no more scope voice in the end game.
Getting the Valuable Features
I’m assuming you’ve a high-level list of features for a given release. The feature challenge lies inin picking the sub-features that haven’t been thought through at requirements time. Developers run into these non-spec’d areas all the time and have to decide what to do. Beware sub-features chosen in a vacuum! Foster a team culture where non-spec’d areas are a topic of discussion, ideally cross functionally. Frequent demo’s, walk throughs by developers of their code either in review or not, and a norm of discussion avoids surprises. The idea here isn’t that you, the manager, make all the choices. The idea is that the sub-feature decisions are known. Then the team can make intelligent decisions.
Re-factoring that Works
Before the Agile guys convinced us otherwise, re-factoring code was bad. Diffs no longer show functional change, making it hard to integrate branched back into a main tree. QA , who probably doesn’t trust you anyway, gets snooty and starts talking about all the testing that’s now been invalidated and will have to be re-run. And those are legitimate beefs about large amounts of code change.
But, in order to scale your development efforts, you have to be able to build parts of the system in parallel. You’ll need well defined and defended module and component boundaries or your developers will spend all their time getting in each other’s way. Your team will build better and better component boundaries over time if you create a climate conducive to smart re-factoring. Take a tip from Feature Driven Development and define work breakdowns at the object level, with interfaces between objects as the primary developer to developer contract. Hold regular reviews focusing on interfaces between the top level objects in the system design, and focus your code reviews on the implementation of the interfaces themselves. You might even want to have your team reviews focus on the unit tests for the interfaces, not the interface implementations. These are all formalisms to focus engineering attention on component boundaries. Over time, you’ll end up with a system where components can be significantly re-factored without throwing the whole product in the ditch.
The moral of the story is that the end game is the only time managers have much of a chance to directly manage change. By then, it’s too late to have an impact on the important stuff. The important stuff, spending time on valuable features, growing re-factorable code, happens when you’re in a meeting. But by creating a set of team cadences and practices that focus on communication and team interaction you’ll help your team do it’s own change control.
I read David Anderson’s post on how he manages his sustaining engineering function at Corbis at a time when my team was up to its eyeballs in support requests. It offered a glimmer of hope.
I started chasing down the links (what’s a “kanban system”?) and found myself in the conceptual world of lean software development; the Toyota way, thinking about software development like manufacturing, queue management, reducing work in progress, reducing inventory. Several years ago I read The Goal, Goldratt’s novel about continuous process improvement. I’d been inspired–I took notes, set up a meeting with others on the management team to discuss how the we could apply the concepts to our service engagements–but nothing actionable came of it and my inspiration faded. But now, discovering that there was an agile discipline built around those notions I was re-energized.
I re-read Anderson’s post, squinted and tried to figure out the rows and columns on the photograph of a whiteboard included in the article. I was trying to understand the technique, so I could copy it and get some relief from the bewildering frenzy of requests coming in for my engineer’s time. I couldn’t figure it out.
Then I had the ah-ha. I didn’t need to replicate Anderson’s system. I just needed to start with the basic underlying concepts. To my understanding those concepts are:
Here’s how I implemented phase one of the system. I started visiting Karl, the customer support manager everyday. I didn’t schedule meetings, I just stopped by. I’d show him the list of things we were working on and those things’ priority order. After a week or two of this, he had internalized my support capacity (one engineer out of six), and he internallized the concept that once we started working an issue we wouldn’t give up until the issue was resolved. Our conversations focused more and more on priority, and specifically about what things I shouldn’t work on. The anxiety level dropped–his and mine. Here’s a quote from Karl’s weekly account update:
I would like to offer a special thanks to the Engineering Team for their superb attention to the matters we have faced recently.
That’s the result of some modest agility, inspired by readings in lean software development.
Fifteen years ago, I was writing Smalltalk code as part of a small development team. I had learned about the Model View Controller pattern and had become a zealot. Tasked with building a key portion of the user interface for a looming demo to upper management, I held anxiety at bay and built an elegant object model for the entire project. Then I went home for the weekend, the user interface code incomplete. Kevin, the technical lead and the one under the most pressure to complete the demo, came in over the weekend and rewrote the user interface code I’d been assigned. And he discarded my object model in the process.
First thing Monday morning, Kevin showed the demo to our boss. Kevin explained to the boss that he’d had to rewrite my bit, but now the demo worked great. The boss was ecstatic. I was livid. Hell hath no fury like a programmer scorned.
Why did that experience bug me so much? I was embarrassed that I hadn’t completed my part, Kevin one upped me in front of the boss. But that wasn’t the core issue.
I wanted to be right. I wanted the rest of my peers and my boss to give weight to my opinions–every time. As a technical guy, I wanted that more than anything else–notably it mattered more to me that my technical judgements were valued than that other people could collaborate with me. I wanted to be the guru. When my object model was discarded, to the applause of the boss I might add, I didn’t feel like a guru, I felt like an idiot.
I wasn’t very graceful about it at the time.
Fast forward to the present, I’m no longer writing Smalltalk code (where is Smalltalk anyway?), I’m managing development teams. If I had been my own boss fifteen years ago, I’d have tried to convince me that being right isn’t an event, each technical discussion isn’t a show down on my fundamental worth as an engineer. In fact, the phrase “being right” is broken; rightness isn’t something a person is, it’s something a person, better yet a team, experiences. Rightness is something we look for, not something we own. To which the younger me might have responded, “then how do I compete for promotions and raises?” And that is another story, for a another time.
