Component vs. Story Driven Estimating – Scylla and Charybdis

Facts and figures

Using component driven estimating, I under-estimate by a times 4 factor; a proof of concept before contracting doesn’t help.
Using story driven estimating, I over-estimated two iterations by 30%.

Pragma

1. I estimated two iterations using story driven development.
2. I am experimenting with a new framework
3. I am targetting a business domain I know well.
4. I made sure to estimate very small stories on a short event horizon.

Premisses

1. Component models afford and encourage glossing over interactions.
2. Functional models afford glossing over unused components. Logical dependencies constrain functional models to minimally correct component models.

Hypothesis

Story/Case/Interaction/Feature driven estimating is overall more acccurate than component driven estimating.

This post didn’t start off with facts, figures, premisses or a hypothesis. It would appear that stories, broken down into small (<0.5 day) tasks, afford better estimating than component analysis – if you’re in no reading mood, that’s all you need to know. I’m off rambling.

OK. I used to estimate projects using component analysis. This is how it works:

• Estimate each component as derived from requirements; do not include any correction factor.
• Sum and multiply by 4.

Recently, I moved to story based estimating – this is what happened:

• Broke down each story into 3 or 4 tasks. Each story was minimally small, representing an atomic value increment.
• Estimated each task separately.
• Summed and applied no correction factor (no prior iteration allowing to evaluate velocity).

It may be worth noting that story breakdowns are highly dependant on prior stories (if you shuffle your stories on the timeline, your story estimates are void, although your task estimates may still be accurate).

What happened?

Plato

My first encounter with object programming was an epiphany. Object modeling promised clarity, simplicity and re-use – so much so that I still find it difficult at times to come to terms with the conceptual overhead involved in teaching object programming versus scripting.

As essential models, objects promise clear, reusable encapsulations. Component driven estimating relies on high level separation and breaks into perspectives. From there, two attractions:

• Object modeling must be comprehensive. Modelling objects, we are thinking about potential. Uses and interactions far beyond business logic. We are modeling ahead of business goals.
• Object modeling must be pure. Modelling objects, we can’t afford letting arbitrary considerations taint our models.

I believed in objects long enough not to feel nostalgic. Don’t be fooled:

• You’re not gonna need it (however many times you will pay for it). However you believe your objects will pass the test of time, comprehensive objects afford more than is required.
  A friend of mine will always order the same dish at every Asian restaurant we go (Thai, Chinese, Indian, … ). Choice means little more than overhead in finding what you really need once your objects enter the maintenance cycle.
• No services. Platonic models don’t willingly endorse your business logic – If you try to reach the essence of your model, you will neglect your components (contr)actual uses and, like me, will reach meta-stasis, balancing ‘complete‘ objects (just need to fill in the blanks…) against a self-sustainable tasklist (an endless list of use cases)(1)

80/20?

80% of functionality (yes… the 80% your customers won’t need) is provided by objects – Pure, comprehensive models that do not contribute to your business logic. Thinking, modeling and estimating objects, we’re not even covering the 20% functionality that we contracted for. Worse, that 20% will account for 75% of actual development and maintenance time given complex dependencies woven into ‘pure’ representations

Aristotle

Down to Elysian fields, story driven estimating doesn’t suggest we entirely relinquish either objects or component analysis. Accurately, stories require modeling shadows; stories cut across layered architectures, grudgingly yield pragmatic models and hold no romantic promises to posterity. However:

• Stories require minimal object models. While essential (existentialist?) models do not require stories, stories require actors and props. For this reason, stories outline necessities driven by your business logic.
• Stories require percolation. Integration is all to stories, while component analysis comes second.

F* places story modeling at the heart of software architecture, but I won’t ascribe a short term incidental success in estimating to experimenting with a new pattern (however superficially relevant).

I’ll write off over-estimating to a long standing habit of pessimism – a stigma of pre-romantic beliefs in hidden essences, quickly glimpsed while designing pure abstractions, quickly lost as contractual requirements take over, driving the patching, deconstruction, scarification and disintegration of domain objects into business logic.

I’ll be linking Scylla and Charybdis when I have a moment (finally, not an obvious choice but could do for a nice read and the imagery is a random treat).

(1)From a neo-platonician perspective, I even find it hard to shake off unease while writing notifications for model objects – but why do object languages like Java not provide modifiers for making states and behaviours observable? Observer patterns are cumbersome combined with strong typing; shouldn’t this stuff be implicit to implementations?.

Feature Programming & Feature Annotations

In agile development, the emphasis is on delivering business value incrementally. While this explicits features within software cycle management, it may also reduce the developers’ visibility over software architecture; this may part in explain why agile methods rely heavily on refactoring.
In contrast, classic development approaches emphasize planning and upfront design. While this may promote architectural integrity, it has been largely demonstrated that  this reduces both customer satisfaction and software profitability, typically resulting in a large amount of unused functionality.
When debugging and processing change requests, development teams naturally adopt a feature oriented approach; this is because bugs and change requests are normally raised by end users.

Overall, software features, whether implicit or explicit to the development cycle as agile stories or bugs, remain largely implicit to code-bases because the high level structure of software applications follows a technical orientation – a paramount example in front end development being the unavoidable Model View Controller meta-pattern. Features typically cut across layered architectures. For agile programmers and maintenance teams, completing stories, fixing bugs and resolving change requests requires juggling between application layers. For traditional teams, reduced visibility over application features will result in a higher number of bugs and decreased customer satisfaction.

I define Feature Programming as a set of methods designed to reduce the cost of software development by revealing features within source code. This is mainly achieved in two ways:

• Feature Annotations using domain semantics allow software architects to retain a layered architecture while increasing feature visibility.
• Feature Architectures emphasize feature separation over layering when deciding the high level structure of an application.

Both approaches are expected to reduce costs in front end development. While feature annotations provides non invasive, lightweight techniques for teams that want to reduce liabilities in the software cycle, Feature architectures may be very suitable for decentralized and open source application projects and further afford the creation of reusable value increments.

In this article, I cover feature annotations; later I will cover feature architectures in another article and will also demonstrate a small framework guiding the development of feature centric front ends while affording elements of layering using a suitable interpretation of the MVC meta-pattern – stay posted!

Feature Annotations

Annotating features simply requires a fairly disciplined approach to tracing code. Feature annotations can be applied either during development or retrospectively, as a scaffolding technique used to reduce the cost of maintaining legacy code.

Feature annotations requires the following:

1. One or several feature classes define static functions expressing application features. Such feature traces must be easy to spot and suitable for automated discovery (in Java or ActionScript, you may use $ as a prefix).
2. Each function in a feature class encapsulates a trace statement. Traces may be logged, output to a console or even displayed in an application specific debug window.
3. Semantics for feature functions are typically provided either as story names or itemized requirements.
4. All code that relates to a specific feature must be annotated using trace function invocations (‘trace annotations’). Within an agile process, a developer writing a story will tag their code, and otherwise contributing code, using trace annotations (for each class involved in the developed story, at least one function or constructor should be annotated).
5. It is important that feature traces can be filtered easily either at runtime or at compile time (commenting out unwanted traces may be sufficient in simple cases).

Feature annotations constitute a non invasive technique. Adding feature annotations cannot break existing code and doesn’t require re-thinking your approach to software architecture. Developers working on agile stories need not be concerned with selecting trace annotations as the same annotation semantics are used over and over during story development.

In spite of this simplicity, feature modeling provides several benefits that will help in drastically increasing a development team’s reactivity to change requests while increasing software maintainability:

1. If you are doing code reviews, feature annotations reveal and delimit the code related to a given feature and afford validating the code from the point of effectiveness. Classical code reviews validate structural integrity, glossing over functionality scattered across application layers.
2. New starters observe feature traces at runtime and isolate features in the source; this complements a developer’s understanding of packaging and structure; developers also appreciate the software’s actual uses by reviewing feature classes. This results in decreased learning curves and reinforced consistency across the code-base given multiplied reuse opportunities.
3. Maintenance teams benefit orderly, channeled traces mapping business logic using domain language. Observing channeled traces and isolating defective features can save hours spent learning control flow and discovering all scattered elements potentially involved in a defect.
4. Liabilities associated with reuse are reduced given that concurrent, co-located traces provide tell-tale signs of reuse. A developer targeting a given code fragment will be able to determine which elements of functionality are affected, thereby correctly assessing the risk involved in modifying shared resources and identifying functionality that may require regression testing.

Free online project management

OK, so I reviewed XML editors and I didn’t find anything suitable for me. I drafted technical solutions, scavenged some useful components. Next question: where do I host my project.

I was looking for no-cost, online SVN repositories and quickly realised that I could get an integrated solution for project management.

• XP-Dev.com
• sliksvn.com
• unfuddle.com
• cvsdude.com (SVN is supported)

There is a short article on the topic: Free Subversion Hosting for Small Projects (David Bolton @ About.com)

I’m testing XP-dev. XP-dev has unlimited users and plenty storage (~500mb); other providers charge for projects with 2 users or more.
The interface is straightforward; advertisement it displayed, still OK on my netbook (1024 x 600) so won’t be a problem on anything else.

I scheduled a couple of iterations, 16 stories and 24 tasks. Agile rocks.