Agile Software Development: The Cooperative Game (2nd Edition)

2017-07-07 02:10:07

Crystal Orange Web is a methodology we created for eBucks.com, a company delivering code to the Web in a continual stream. It differs from Crystal Orange in that this methodology does not deal with a single project but with a continuous stream of initiatives that require programming and with each initiative's results being merged with the growing code base being used by the public.

This methodology is still in its trial run. I include it here because

An increasing number of companies are finding themselves in this sort of situation

It represents the most recent application of the ideas in this book

It has a different shape than Crystal Orange

The eBucks.com situation was interesting for a second reason (the first being the continuous and criss-crossing web of demands from different customer groups). The company had already established a Web presence. It was no longer driven by time-to-market pressures but was beginning to feel pressures imposed by the cost of defects. Customer calls, arriving in exponentially increasing volumes, could easily negate profit margins. Thus, the company was shifting from productivity to defect freedom as its top priority.

There were about 50 people to coordinate in this situation: executives, business-people, managers, analysts, programmers, and testers. I classified this as an E50 situation.

The group was relatively new, so some process definition was needed to make clear who made which decisions and who handed what information to whom. Otherwise, people generally knew who they had to talk to in order to get their jobs done.

I performed interviews, as called for in the methodology-tuning technique. I interviewed people in each job role, from marketing through testing and system operations. The interviews revealed the following facts:

Convection currents of information were quite good. Everyone was on one floor. They had movable glass and whiteboard partitions as walls, so they could see and signal to each other while still having some privacy.

Ongoing distractions were keeping people from having the quiet time they needed to make progress on their assignments (in all job roles). Each person was working on multiple initiatives, with frequent interruptions.

Attitude, amicability, and morale were still quite good but were sinking because of the frequent interruptions and lack of progress. Also, the programmers sat on one side of the building while the business specialists sat on the other. This meant that the chit-chat in each group drove negative commentary about the other group.

The company was less than a year old, meaning that old habits had not yet set in and that people were open to inventing new work habits and conventions.

Brief Description of Crystal Orange Web

In keeping with the idea that a methodology is the set of conventions the group agrees to, we wrote the methodology as a set of conventions, in five categories. Here they are:

1. Regular Heartbeat, with Learning

The purpose of this category is to establish a core procedure for getting feedback on "how we work around here" and taking the time to reflect and improve on that. Every convention except the reflection workshop at the end of each cycle can be altered as an outcome of the reflection workshops.

1.1

Two-week development cycles. Overall production runs in fixed-length development cycles of two weeks. After each delivery, each team may opt for the next delivery to be either two or four weeks, depending on what the team can deliver of use to the public. Each team must deliver something useful to the public every four weeks.

1.2

Post-cycle reflection workshop, suggestions visibly posted. At the end of every cycle, the company meets to discuss what worked well, what didn't work so well, and which ideas to try out on the next cycle. The outcome of the meeting is a posted list of things to keep.

2. Basic Process

The purpose of this category is to organize who creates which pieces of work and who makes which decisions, in order to avoid duplication or gaps in the effort and to look far enough ahead to spot potential problems early. The process aims for delivery of business initiatives live to the Web.

2.1	A business owner writes a business use case and a system use case brief (Cockburn 2001c). The business use case illustrates the proposed new system features in operation, paying particularly close attention to the manual business processes that are invoked when things go wrong.
2.2	The brief is used by the technology group to estimate the work involved in creating the features. The business executives review the business use case, technology estimate, and value of the initiative before agreeing to further work. The user interface designers work with marketing and the detail business analysts to incorporate the features into the overall site flow and then produce screen designs and the XML for the screens.
2.3	The detail business analysts produce detailed use cases and data descriptions, which go to the user interface designers, the server programmers, and the servlet programmers. The servlet programmers work from the XML for the user interface, the use cases, the data descriptions, and the server interfaces to produce the servlets. The server and servlet programmers produce regression tests for their code and peer-review the test cases. When the test cases are deemed good and the code passes the tests, the code is passed to the integration testers, who pester the developers to fix whatever remaining errors they find before the major deployment.
2.4	The integration testers post the changes going out in the new release to the internal group and also to the call center.
2.5	For live code, the call center returns bug reports to a special SWAT team whose sole purpose is to fix problems in production. The SWAT team is selected from the development group on a rotating basis every two cycles.

3. Maximum Progress, Minimum Distractions

The purpose of this category is to ensure that people are working on what is of greatest value to the company and have time to focus and make progress on that work.

3.1	The top corporate key initiatives are prioritized and visibly posted for each two-week production cycle. They are allocated to individual people so that each person knows his top two or three personal priority items for the cycle.
3.2	Work is broken into what can be completed and tested in the two-week cycles and is further broken down into things that can be accomplished in one to three workdays. Each person who is working on more than one initiative is guaranteed at least two consecutive days to work on any one initiative without being pulled onto another assignment.
3.3	The developers post on the whiteboards outside their office the current status of the work they plan to complete during a given week. Every morning, the developers meet with the business owner of the current work initiative. They conduct a short meeting to determine the current state of the work and the top work priorities and to discuss any questions. The business owner is not permitted to ask for status again the rest of the day. The period 10:00 to 12:00 each day is declared "focus time," in which no meetings take place, and everyone in the company is encouraged to turn off the phone.

4. Maximally Defect Free

The purpose of this category is to construct a culture of "Kill bugs here!"

4.1	Every server and servlet class will have a set of automated regression unit tests, written by the programmer for his own code, using JUnit and HttpUnit or the equivalent. Programmers release code to integration test only when the tests have passed the scrutiny of a peer developer. The integration tester therefore gets the code, the test cases, and a note from another programmer saying that he will vouch for the quality of the tests.
4.2	The server contains a loopback mechanism so that the integration testers can maintain their own controlled test database (which other people can use).
4.3	A small, pidgin language will be used by businesspeople to construct sample business transactions and name an expected response. This little language allows integration testers, business owners, and the servlet writers to construct a test scenario and add it to the test database.
4.4	Screen-click statistics from the call center are posted in a visible place so that everyone can see where the public is having difficulties (either navigation difficulties or problems due to programming defects).

5. A Community, Aligned in Conversation

The purpose of this category is to indicate the long-term target toward which the company is aiming.

5.1	Eventually, the programmers, user interface designers, testers, business owners, marketers, and so on should sit in cross-functional teams to maximize the effect of conversation around delivering initiatives across specialty boundaries and to minimize the effect of rumoring about others' specialties. This will have to be balanced with staffing levels and growing space needs.

Reflection about Crystal Orange Web

Two things strike me about this methodology.

The first is the reduced role of process and work products in expressing the methodology. They are present but occupy only a fragment of the space usually devoted to them.

The second is the general absence of concurrent development, which is one of my favorite development speed-up techniques. Concurrent development is missing because of the bottlenecks in the system.

The programmers had an enormous work backlog, no spare capacity, and were being interrupted constantly. The people were quite inexperienced both in developing software and in the business domain. These two points together meant that the programmers were not able to do overlapped development and hold the requirements in an oral culture. They needed stickiness in the information, which meant having specs written down for them.

With time, this should change; as it does, I hope they will reduce the paperwork and increase the conversation. In the meantime, they need the paper.

Six Months Later

I present this methodology as it was constructed as the starter methodology. We would expect to see some drift over time, both as people thought up new ways of working and as they drifted away from the high-discipline practices.

Michael Jordaan, CEO of eBucks.com, made these comments about the group's work habits six months later:

"Obviously, when you left, some disciplines survived while others did not stand the test.

"The survivors include the fortnightly heartbeat with carefully planned cutoff times, which allows for developers and business owners to plan, testers to test rigorously, and customers to be informed upfront of scheduled upgrades.

"We discussed a three-week heartbeat, but this was considered too long. More complex issues than can be solved in two weeks are run at twice the heartbeat (four weeks), but we still encourage incremental rollouts.

"The post-heartbeat meeting is strictly enforced, and it has become one of the few times that I get to speak to the entire team. I have made quite a point of paying tribute to those involved in successful upgrades. Hopefully this public recognition is motivating. Mistakes are discussed and suggestions for improvements have been made at every meeting, supporting the learning culture we are creating.

"The quality of code going live has improved greatly as the testing team has a veto power, to prevent bad code from going live (and this can be embarrassing).

"The SWAT team, dedicated to eliminating live bugs, has also made great strides in responding to customer and call centre queries.

"Focus time is still adhered to (and we still ring a bell every morning at 10:00). If I go a single day without these two hours I now start panicking, so useful has it proven to be.

"Some things that did not survive were the habit of posting current priorities/work progress on a board. Maybe interruptions are less of an issue now, as people work from home, or maybe relationships between business owners and developers have stabilised. Maybe people are just lazy.

"Most developers have a maximum of three tasks at any given time, except for the two key people working on the back end, who may easily have a list of 15 each. Moreover, they are still interrupted by live issues, which interfere with their completion of tasks and lead to much frustration by other developers and business specialists.

"The issue here is lack of skilled resources. It is the age-old problem that training employees to assist, while undoubtedly the right medium-term solution, takes longer than simply doing it yourself."

In those comments, what I notice with some satisfaction is that the team still uses the core elements of the process: heartbeat with learning, and finding ways to modify even that heartbeat to fit their needs.

I notice the discussion of talent and skills as being critical to the project, and I notice them drifting away from what probably were embellishments in the methodology.