The F* meta-pattern

F***, I can’t find the code that froggles the wizzles.

If you have ever worked on a delivery team, you know how hard it can be to map the user perspective leading to bugs being raised to actual application code.

Feature architectures emphasize feature separation over layering, and may be especially suitable for front-end development and integration software.

Classic architectures emphasize layering over component separation. A paramount example is the MVC meta-pattern, where applications are divided into model, view and control layers, and a component hierarchy is designed across application layers. MVC has imposed itself in application development – so much so that a colleague recently suggested to me that MVC is no more than a simple warning – don’t shoot yourself in the head; in the meantime, MVC relies on an over-arching principle: separation. In software architecture, separation determines what you can, and cannot do. If MVC is good for writing applications, surely MVC will promote uses that are common to an application product’s life cycle. Granted, let’s review the situation:

• MVC makes it easy to entirely re-implement a complete layer of your application: you could be re-writing your Swing UI in OpenGL, or re-use your data model using a web front-end. Or even reuse your view and model and create a new product following a different business logic – as encapsulated into your controller.
• MVC lets you hire specialists to write the separate layers of your application – a GUI wizard may be writing the UI while an XML wizard develops the model.
• Given an existing model, MVC could help you design new ways of interacting with this model.

Now, here’s a couple of things that an MVC architecture won’t help you with:

• MVC won’t help you introduce new features in your application – since features typically cut across model, view and control, the more features you add, the more time it takes to integrate scattered elements of functionality within existing application layers.
• MVC won’t help you fix bugs – while users and testers associate defects to application features – elements of perceived functionality – MVC separation promotes feature obfuscation (feature semantics differ across layers) and require iterating investigations across layers, typically starting from the UI.

I have never met a developer involved in replacing an application layer wholesale; further, while assigning specialists to writing separate application layers may ensure that each layer is well designed, this also increases the chances that layers do not interact correctly and will often lead to over-design and creeping featurism within each application layer – leading to unused potential. Separation? Yes. Layering?

Agile resolves some of the problems associated with layered architectures – in Agile, developers focus on delivering value by completing the user stories that express a customer’s actual needs. Pair programming and pair swapping also ensure that Agile developers are generalizing specialists. While developers still juggle between application layers on a daily basis, communication failures – both socially, within the team, and technically, within the product – are much reduced and bug counts dwindle. Agile teams may even be more able to process change requests reactively and effectively: a generalizing specialist will be able to investigate a solution across layers, where a team of specialists may be involved in investigating a bug or change request, with all the communication overheads that this may entail.

Features, then…

Where MVC stands for model, view and controller, the F* meta-pattern primarily resolves software architecture as a collection of features. In other words, instead of packaging your application using, view model and controller root packages, you create a new package for each feature, and commit to primarily developing each feature using a self contained source bundle. MVC does not forbid identifying features as an after-thought (see [ref]), it just makes it harder; similarly, F* does not specify that you should not separate view, model and controller – it does make it harder, however. Like Test Driven Development, F* represents a significant paradigm shift in the way we model and develop software. In short, F* determines increasingly strong requirements with regard to feature separation:

1. (The source code for) any given feature should be physically separate from any other feature.
2. Erasing source for a feature should not result in a broken build or a broken application.
3. Erasing or disabling a feature should not result in other features (dependents) being disabled unless the dependencies are conceptually integer.
4. A developer needs not access the source code for a feature in order to integrate with that feature.

Given increasingly stringent requirements, F* boasts increasingly attractive benefits

1. Developing a new feature requires little knowledge of existing features; further, existing features provide self-contained, integrated examples to any developer learning the code-base; further, developers can work in parallel with little risk of causing merge conflicts or stalling on another developer’s critical path. Surprisingly, we have a methodology suggestively capable of satisfying software managers brooding over Brooke’s Mythical Man Month.
2. Features are developed better and faster. This will be especially true in an agile environment where developers complete Stories. Often, a single feature will match a unique story and keeping all the code for a story in the same place means that developers are more able to focus on completing the story and less likely to break code implementing another story.
3. Maintaining existing features requires little analysis. Where all the code for a given feature is gathered within one or a few classes sitting in the same package, debugging and analyzing such code can be achieved in finite time, without the risk of meandering within a large code-base – in contrast, fixing bugs across complex, layered architectures evaluates to systematic investigations that only discipline, tenacity and perseverance can resolve.
4. Given an appropriate infrastructure, decentralized projects can be conducted with reduced liabilities.
5. Remote developers can contribute either commercial or non commercial value increments on an ad-hoc basis.

F* prerequisites

F* requires a paradigm shift in the way we develop our code; I am currently experimenting with the development of ee-xml, an open source XML editor. First, let’s cover a couple of prerequisites to developing using F* in front-end development:

• F* requires a widget library. If this seems contradictory, consider the fact that MVC applications also require GUI libraries. Right or wrong, I tend to perceive the emergence of reusable GUI library components as an avatar of the MVC meta-pattern. I also believe that MVC fails to support application developers in inverse proportion of the size of a software company – the smaller a company, the least likely that company will be to benefit wholesale application layer re-writings. F* does not replace a GUI library, it requires (and benefits from) retrofitting view layer elements within features.
• F* requires a simple data model encompassing your domain logic. If this seems contradictory, take the case of PureMVC. Value objects typically implement domain logic, while Proxies realise an application’s business logic. F* requires (and benefits from) retrofitting your business logic within features.

For ee-xml, I use the Swing library and an object oriented XML library.

Developing using F*

In a classic, layered architecture, code is designed and written from each component’s point of view. In F*, code is written from the feature’s point of view. To this extent, F*, like procedural programming, is easier to understand than patterns like MVC. Even though, F* is no less object oriented than MVC.

F* is a separation driven architecture. Features are written as self contained bundles of resources; notifications are used to integrate features by enforcing their conceptual dependencies. Here is a recipe for designing a feature:

1. Consider your feature in isolation – granted that a feature may access shared resources, assume that you have already collected references to such resources. Resources are typically represented as fields within a feature class.
2. Resolve the interaction entailed by the given feature.
3. Register to receive notifications. in F*, notifications typically convey references to view and model objects. You receive and generate notifications because you wish to share resources with other features.
4. Provide code instantiating all resources that your feature needs create.

In a simple example, we have a first feature dedicating to opening an application’s main window; a second feature is tasked with allowing the user to open an auxiliary window from an existing frame. In this case, we will analyze the second feature:

1. We determine that our feature requires an existing frame and a menu, and will allow the user to select a menu item.
2. Upon activation of the menu item, we create a new window and issue a notification.
3. We register for receiving a notification when a new window is created. Discovering which notifications are available, and what these notifications mean, is best done at runtime assuming notification traces, but could also be done statically by referring to the list of available notifications.
4. Upon receiving a reference to a newly created window, we need to ensure that such window will be equipped with a menu bar, menu and suitable menu item. We then register to receive an event when the menu item is selected.

I have chosen a simple example to clarify our recipe for writing a feature. In a real world context, a feature will typically map a user story and may be more complex. Within the development of ee-xml, opening a file represents a feature. This entails accessing I/O resources as well as creating fairly complex view elements.

Given the above example the typical life cycle of a typical application feature is as follows:

1. The feature is instantiated by a registry. The registry maintains a list to all currently available features and will, in simple cases, instantiates all features on start-up. At this stage, the feature is neither initialized nor available.
2. Upon instantiation, a feature registers to receive notifications. In front-end development notifications will often be issued to signal that a UI resource has been made available.
3. Upon receiving a notification, a feature will initialize. The same feature may initialize several times (in the above example, the feature initializes every time a new window is created). Initialization consists in adding widgets or other UI components to the application and listening to events issued by such components.
4. Upon processing the event(s) the feature is listening for, the interaction is resolved and further notifications may be issued.

Ready to rock?

Back in 2000, I embarked on a long journey aiming at reducing the loss of momentum associated with layered architectures; I wrote several IDEs, eventually, lifting my limit from managing and maintaining just 50 to 3000 classes or more as a solo developer.

Any growing application entails an increase in physical and logical complexity; the Antegram IDE helped me reduce the perceived complexity of my applications. What Antegram achieves with the user interface, F* promises to achieve using adequate separation, and once again, I’m ready to rock, putting aside natural skepticism.

F* is a meta-pattern; it can be instantiated using several languages and implemented in various ways, following increasingly strict requirements, and providing increasingly strong benefits. I am currently finalising a compile safe F* framework in Java and writing the first open source F* application.

Don’t shoot yourself in the head.

Feature Architectures

How far can we go into the writing of an application without losing momentum?

As an application becomes richer, it typically becomes more complex. There are two aspects to this:

• Logical complexity increases as a result of new functionality binding to existing functionality.
• Physical complexity increases as a result of increased application size.

The result of increasing complexity is that developers find it increasingly challenging to add new features to an application. This complexity manifests itself in two ways:

1. Your workspace is increasingly cluttered with irrelevant entities – entities that do not contribute to the feature you are developing
2. Your code is increasingly dependant on existing elements of functionality.

Very large applications are typically broken down into modules. Modules reduce logical complexity by providing narrowing interfaces; physical complexity is reduced by keeping all the code for a given module separate from the code for other modules.

Feature architectures enforce micro-modularity:

• Each feature typically maps a single value increment or story.
• Features are abstracted from each other using channeled notifications.

Why design feature architectures?

Feature architectures are derived from story driven development. In agile, story driven development, developers collaborate towards the development of a given story cutting across the layers of a selected framework. Agile ensures that developers deliver value faster; in suitable contexts (assuming that value can be delivered incrementally), this maximises profitability while reducing the risk to produce large amounts of unused functionality.

Feature architectures take advantage of the separation between source spaces and runtime spaces to allow developing well formed (layered) runtime architectures while limiting linearly growing physical and logical complexity. This allows developers to focus on delivering atomic value increments without incurring mounting familiarization costs and splitting development across framework layers. Also, this may allow fairly large developer teams to collaborate without facing large communication costs.

In a nutshell, feature architectures and micro-modularity target value driven separation. In contrast, existing separation strategies and meta-patterns provide abstract roles that can be re-instantiated in a variety of contexts (the MVC meta-pattern is an example). This affords industry endorsement for high level design solutions. Beyond, classic separation models emphasize a component drive approach. While this means that component cores can be developed in parallel, this may result in glossing over communication interfaces between components.

Layered, component oriented architectures emphasize potential and structural integrity and are more suitable for long term investments and library development. Feature architectures emphasize value increments and may be ideal for writing application front ends:

• Reusable data and view components are already available
• Applications are feature rich, yet do not involve much complex code.
• Value increments typically involve cutting across layered architectures.

Working using a feature architecture

Feature architectures require developers to package features separately. Each feature is developed as an integrated micro-module – if you are writing an application, a feature will probably bundle view, model and controller functionality.

1. Each developer or pair are responsible for designing and implementing a feature derived from a user story.
2. The core functionality involved in developing a feature is developed independently from other features.
3. The source code related to a given feature is bundled within a separate package.
4. Developers integrate their feature by implementing handlers for channeled notifications.
5. Developers expose required resources by providing external notifications.
6. All source code for a given feature may be removed without causing the application to break at compile time.

Mmh… this may not be the best introduction to feature architectures I could write. I’ll try to improve on that and will also provide an application template very soon.

Feature Programming & Feature Annotations

In agile development, the emphasis is on delivering business value incrementally. While this explicits features within software cycle management, it may also reduce the developers’ visibility over software architecture; this may part in explain why agile methods rely heavily on refactoring.
In contrast, classic development approaches emphasize planning and upfront design. While this may promote architectural integrity, it has been largely demonstrated that  this reduces both customer satisfaction and software profitability, typically resulting in a large amount of unused functionality.
When debugging and processing change requests, development teams naturally adopt a feature oriented approach; this is because bugs and change requests are normally raised by end users.

Overall, software features, whether implicit or explicit to the development cycle as agile stories or bugs, remain largely implicit to code-bases because the high level structure of software applications follows a technical orientation – a paramount example in front end development being the unavoidable Model View Controller meta-pattern. Features typically cut across layered architectures. For agile programmers and maintenance teams, completing stories, fixing bugs and resolving change requests requires juggling between application layers. For traditional teams, reduced visibility over application features will result in a higher number of bugs and decreased customer satisfaction.

I define Feature Programming as a set of methods designed to reduce the cost of software development by revealing features within source code. This is mainly achieved in two ways:

• Feature Annotations using domain semantics allow software architects to retain a layered architecture while increasing feature visibility.
• Feature Architectures emphasize feature separation over layering when deciding the high level structure of an application.

Both approaches are expected to reduce costs in front end development. While feature annotations provides non invasive, lightweight techniques for teams that want to reduce liabilities in the software cycle, Feature architectures may be very suitable for decentralized and open source application projects and further afford the creation of reusable value increments.

In this article, I cover feature annotations; later I will cover feature architectures in another article and will also demonstrate a small framework guiding the development of feature centric front ends while affording elements of layering using a suitable interpretation of the MVC meta-pattern – stay posted!

Feature Annotations

Annotating features simply requires a fairly disciplined approach to tracing code. Feature annotations can be applied either during development or retrospectively, as a scaffolding technique used to reduce the cost of maintaining legacy code.

Feature annotations requires the following:

1. One or several feature classes define static functions expressing application features. Such feature traces must be easy to spot and suitable for automated discovery (in Java or ActionScript, you may use $ as a prefix).
2. Each function in a feature class encapsulates a trace statement. Traces may be logged, output to a console or even displayed in an application specific debug window.
3. Semantics for feature functions are typically provided either as story names or itemized requirements.
4. All code that relates to a specific feature must be annotated using trace function invocations (‘trace annotations’). Within an agile process, a developer writing a story will tag their code, and otherwise contributing code, using trace annotations (for each class involved in the developed story, at least one function or constructor should be annotated).
5. It is important that feature traces can be filtered easily either at runtime or at compile time (commenting out unwanted traces may be sufficient in simple cases).

Feature annotations constitute a non invasive technique. Adding feature annotations cannot break existing code and doesn’t require re-thinking your approach to software architecture. Developers working on agile stories need not be concerned with selecting trace annotations as the same annotation semantics are used over and over during story development.

In spite of this simplicity, feature modeling provides several benefits that will help in drastically increasing a development team’s reactivity to change requests while increasing software maintainability:

1. If you are doing code reviews, feature annotations reveal and delimit the code related to a given feature and afford validating the code from the point of effectiveness. Classical code reviews validate structural integrity, glossing over functionality scattered across application layers.
2. New starters observe feature traces at runtime and isolate features in the source; this complements a developer’s understanding of packaging and structure; developers also appreciate the software’s actual uses by reviewing feature classes. This results in decreased learning curves and reinforced consistency across the code-base given multiplied reuse opportunities.
3. Maintenance teams benefit orderly, channeled traces mapping business logic using domain language. Observing channeled traces and isolating defective features can save hours spent learning control flow and discovering all scattered elements potentially involved in a defect.
4. Liabilities associated with reuse are reduced given that concurrent, co-located traces provide tell-tale signs of reuse. A developer targeting a given code fragment will be able to determine which elements of functionality are affected, thereby correctly assessing the risk involved in modifying shared resources and identifying functionality that may require regression testing.