Translation and Extreme Programming

Extreme programming is an approach that brings together a variety of techniques with the aim of producing software more productively, with higher quality, and with speedy delivery of functionality to the user. The techniques it employs include the use of:
1. User stories - the idea that the programmers will select and develop one of the stories that the user uses to describe required functionality. These stories lead to rapid iteration since the objective of a story is to be met within one iteration (typically a fortnight).
2. Unit testing is paramount - to the extent that ideally, all tests will be written before any software is constructed.
3. Programming in pairs - to generate and utilise the effects of creative feedback on the software development process.
4. Constant and immediate access to the user.
Read More...

Requirements Specifications and Designs

Elaboration vs Translation

I described in my note comparing "mainstream" software development methods with "alternative" methods, I suggested that mainstream methods develop by elaboration and refinement, while alternative ones develop by translation. The comparison between the two was summarised by Sally Shlaer and Stephen J Mellor, in their paper "Recursive Design of an Application-Independant Architecture", and I cannot descibe the difference any better than by quoting from that paper:

Current View
Although practitioners within our industry hold diverse views, we believe that the following five widely held assumptions constitute the core perspectives on object-oriented methods, patterns, and architectures.

1. Analysis treats only the application—the subject matter of interest to the system’s end user, such as air traffic control, a telephone switch, an automated warehousing application, and the like.
2. The analysis must be couched in terms of the conceptual entities of the design. This assumption appears in much of the work on domain-specific software architectures, as well as in many object-oriented analysis and design methods. Hence a developer employing an object-oriented analysis method is necessarily constrained or biased toward an object-oriented design.
3. Despite the lack of a universally accepted definition for architecture, most people would agree that an architecture provides a view of the entire system. Because this view emphasizes the major elements of the system, many details are necessarily omitted.
4. Patterns are generally thought of as small units, consisting of a few objects with wellworked- out interactions.
5. Programmers expect to select, adapt, and modify patterns according to taste and experience. Patterns are seen as useful but advisory in nature: Use of patterns in general or in particular is not required.

An Alternative View
Our Recursive Design method is founded on a fundamentally different approach to this topic that makes five very different assumptions:
1. An analysis can be performed on any domain, not just the subject matter of interest to the system’s end user.
2. Use of an OOA method does not imply anything about the fundamental design of a system.
3. Because we treat the architecture domain just like any other domain, it can be modeled in complete detail using an object-oriented analysis method.
4. The OOA models of the architecture provide a comprehensive set of large-scale, interlocking patterns.
5. Because all the code other than purchased packages is automatically generated, use of the patterns is absolutely required. Read More...

Method Wars - NOT!

In the late eighties and early nineties when there was an explosion of different software development methods, there was a tendency for advocates of a particular method to develop arguments in favour of their particular method which went something like:
1. My method is undoubtedly the best (for all problems - an issue that I will come back to).
2. Because my method is the best, all others must be rubbish.
3. Method X (i.e. not mine) is rubbish because........(fill in your personal reasons here).
4. Just to reinforce my argument here is a detailed comparison between my (superior) method and method X.

This led to spurious method wars, with the advocates of a particular method slagging off the competition.

I am not interested in resurrecting any of the above method wars, but to describe my approach in building a system, I need to compare tendencies, rationales and approaches of the two streams. They are not absolutist - both strands of method development have learnt from each other - and neither strand is the definitive source of any one technique. My comparisons are of tendencies.

I am generally going to use Catalysis as the archtype of mainstream methods - because it is probably the most comprehensively documented and consistent method from the mainstream. Equally, the alternative stream will be described by reference to the Shlaer-Mellor method which is the most complete example of the alternative strand. Read More...

Software Development Methods Less Used -Bibliography and References

Borland www.borland.com
Dijkstra, Edsger: GO TO Statement Considered Harmful. Communications of the ACM. March 1968. Fowler, Martin: UML Distilled Second Edition A Brief Guide to the Standard Object Modeling Language 0-201-65783-X Addison-Wesley 1999
Hoare, C.A.R.: Communicating Sequential Processe ISBN 0-13-153271-5 Prentice Hall International 1985
Humphrey, Watts S.; Introduction to the Personal Software Process. 0-201-54809-7 Addison Wesley 1997
ISO/IEC: Information Technology Z Formal Specification Notation Syntax, Type System and Semantics ISO/IEC 13568:2002 2002
Jackson, Michael: Problem Frames Analyzing and structuring software development problems. 0-201-59627-X Addison-Wesley 2001
Jones, Cliff: Systematic Software Development using VDM, ISBN 0-13-880733-7 Prentice Hall 1990
Kennedy-Carter www.kc.com
Kristen, Gerald: Object Orientation The KISS Method From Information Architecture to Information System. ISBN 0-201-42299-9 Addison-Wesley Publishers Ltd. 1994
Pawson, Richard; Matthews, Robert: Naked Objects Wiley 2002
Shlaer, Sally; Mellor, Stephen J: Object Lifecycles Modeling the World in States. ISBN 0-13-629940-9 Prentice Hall 1992
D'Souza, Desmond F.; Wills, Alan Cameron: Objects, Components and Frameworks with UML. The Catalysis Approach 0-201-31012-0 Addison-Wesley 1998
Stevens, Richard: Perspectives Writing Better Programs: The Story So Far The Delphi Magazine. Issue 89 January 2003 pp28-32

Software Development Methods Less Used

Software and software system development is an inherently complex business. This complexity has grown as the scale of systems and the capability of existing infrastructure has increased.

There is a continual history of approaches to improve software and system development, and to help developers manage the complexity of system development. Many of these approaches - in recognition that software and system development are complicated, changeable, malleable, and error prone activities designed to develop complicated systems - have focused on improving the writing of software, and the processes associated with managing the outputs of analysis, design and programming.

While there are many well publicised and highly adopted methods, processes and techniques, there has been a parallel development of such activities that is less publicised and much less utilised This alternative approach has a different rationale and background, but there are many lessons to be learnt from these alternative approaches. This note gives a brief description of the various strands of development methods and forms the basis for subsequent comparison of the different strands, to identify the lessons each has to offer.

An article in Delphi Magazine by Richard Stevens gave a brief history of software development methods and processes. The article skipped through from Dijkstra's "GO TO Statement Considered Harmful", via Parnas' ideas on information hiding and modularity, into a description of the various structured methods, based on data flow diagrams and process modelling, such as Yourdon, Gane-Sarson etc. He mentioned that these methods adopted (to a greater or lesser degree) refinements such as entity life histories and entity relationship diagrams. He also mentioned techniques, from around the same time (the 70s) that gave more emphasis to the data modelling side of design, such as JSP (Jackson Structured Programming).

He describes methods at this point as based on the waterfall method of systems analysis, inflexible, generating huge amounts of paper, and not good at dealing with changes to requirements.

He goes on to describe the evolution of software development formalised processes such as the SEI Maturity Model and then onto the development of object orientated methods. These latter methods, typified by Gooch and Rumbaugh, "seemed to make it easier to build conceptual and coding models that more accurately their corresponding real-world entities." He also suggests that OO "seemed to afford much better opportunities for code reuse". He then describes people orientated factors in software development, arising from the work of Weinburg and Brookes.

Finally he describes the various agile methods, as incorporating the best elements of the past, discarding the worst, and building change of requirements into the software development process.

This history is a fair summary of the development of methods and certainly covers the main elements that have led to the modern mainstream view of software development methods. The outcome of these historic developments in exemplified by such things as:
1. Extreme programming and test driven development techniques designed to add flexibility to software development by incorporating change in requirements as a natural part of software development cycle; and to develop the software correctly.
2. Unified Modelling Language (UML) which has become the nearest thing in software development to a universal representation.
3. Borland's Development Studio, which incorporates (or can incorporate) UML modelling tools (Together), software configuration management tools (StarTeam), requirements management tools (Caliber), object persistence generation (ECO), as well as code editors and compilers with other elements to manage all the outputs from software development.
4. Detailed large scale software development methods such as Catalysis which is designed to build systems of made up of reusable components. It is perhaps the ultimate refinement of UML OO based development methods.

A comparable 80,000 feet altitude survey of what I am suggesting is an alternative historical strand of system development methods follows.

If Dijkstra's admonition to avoid goto (I understand that in the original paper by Dikjstra, his prohibition was on the uncontrolled, or unconstrained use of the goto statement - not an absolute prohibition) is taken as the inspirational spirit of what I have described as mainstream software development methods; then C.A.P. Hoares' work on communicating sequential processes was the inspirational spirit for what I am describing as the alternative stream of software development methods. This spirit fed into the work of Michael Jackson with JSP, and later into Jackson's System Development Method (JSD).

Developments were also being made with formal (provable) methods such as Z and VDM. These methods were intended to assist with the development of safety critical systems, which had extreme reliability requirements, such as the design of control software for nuclear reactors. The demands of the requirements are such that (portions of) the developed system have to be mathematically provable as meeting the requirements.

Subsequent method developments lead to KISS (Kristen Information & Software Services) Method, whose intent was to standardise the predictability of the software development process. The author was used to "evaluating, sizing and implementing projects based on energy analyses....The level of reliability and ease of implementation of these project was good....original expectation of an advanced computing environment was that this high level would be achieved. The reality was quite different." Further work was published by Michael Jackson on problem statements, and the need for system developers to have a toolbox of methods available.

The other major development on this strand was the Shlaer-Mellor method which was described as an object orientated method, but has a rather different feel and rationale than other object orientated software development methods. This has been extended and refined both by Mellor's own firm and by Kennedy-Carter in the UK. Also there were a number of independant developments that contirbuted to this alternative strand - most notably Naked Objects which advocated the direct manipulation of objects as a user interface, and to provide systems that were more suited to problem solving users, than users following a predetermined process.

References arising from the above will follow in a later blog.

Technical Notes - Rationale

I have spent about 50% of my working life (employed by the same, large, financial institution) involved in information technology. I started out as system analyst, but fairly quickly developed an interest in software development methods, and spent the bulk of my employment there becoming knowledgable about such methods,and their implementation of them in CASE (computer aided software engineering) tools. I became an instructor in two different software development methods; was involved in method and CASE tool selection; the specification of such tools, and amendments to existing tools; and consulting on methods within the organisation. Later I was involved in developing technology strategy and research, with particular reference to the software development environment.

Over a period of about 16 years I observed software development methods change and develop, and had exposure to virtually every development in the field.

One of my interests is Advanced Squad Leader (ASL) and I am trying to develop some computerised play aids for this game. As a software development this has some characteristics that make for complexity in the development process. The characteristics that give rise to this complexity are:
1 There are a very large number of rules within the game - the rule book is about 3" thick.
2 The rules have a very high level of interrelatedness - leading to sometimes arcane debates as to what the rules actually mean.
3 The rules are very situational dependant - some rules apply in some circumstances, and not in others. The way I believe is appropriate to handle this within a software development is to use reflexion - allow objects/classes/types to learn, from the system, about their own characteristics - to change the behaviour of objects in different circumstances.
4 The rules describe an inherently complex game - there are very large numbers of object types, relationship types and process types.

This interest has led to me try a personal system development project. This project has reignited my interest in ways of developing systems, and this in turn has forced me to clarify and codify my ideas about software development. As a result I am writing myself a series of notes about software development methods, which I am going to publish as a series of technical notes as I learn lessons from my development and codify my ideas on methods.

As a minor aside, there is a modern trend to use the word methodology rather than method in discussions of the software development lifecycle. Methodolgy to me is the study, understanding and comparison of different methods; a method is the way one goes about constructing a software system. I object to misusage of the word methodology, and believe that the software development is governed by a method (whether explicit, implicit, formal or informal).

De-coupling inheritance dependencies in Delphi

I had been having problems with testing a Delphi unit that fired events off and had many subclasses, when I realised that a different approach would solve the problem. The new approach solved a long-term problem that I had with my design – too much coupling between classes because of the way Delphi implements inheritance.

The long-term problem is one that applies in many places in my system design. It could be regarded merely as a theoretical issue, but a more elegant design sorts it out in many places.

In the requirements for the system, the following structure comes up repeatedly:

Several object types need to share a common interface, but each has its own behaviour. My standard way of defining these requirements is to define a category (see note below) which defines the interface. In addition I define the individual object types that have the different behaviours. The category is defined as the intersection of the individual object types. My original method of implementing this structure was to follow the suggestions in the definition of a category, to define a superclass corresponding to the category, and descendant classes corresponding to the individual behaviours. This enables common behaviour to be isolated out into the superclass.

Example
The requirements for part of my system involve the definition of structure nodes and part of the requirements for structure node is:

Thus structure node is existence dependent on the three categorised node types.

This was originally implemented in Delphi as five units, one for each specialisation, one for the abstract parent, and one for the interface of the datatype, as follows:
**********************************************************************
unit Node;
interface
uses Classes, Constants, Operations;
type
IJSPNode = Interface
function AddChild(aJSPTYpe: TNodeType = jspSEQ): IJSPNode;
....
procedure AddOperation(const NewOperation: string);
procedure Assign(Source: IJSPNode);
procedure SetChild(i: integer; aNode: IJSPNode);
property ChildCount: integer read GetChildCount;
property Condition: string read GetCondition write SetCondition;
....
end;
implementation
end.
*****************************************************************

unit NodeImpl;
interface
uses
Classes, Constants, Node, Operations;
function NewJSPNode(ATNodeType: TNodeType): IJSPNode;
type
TJSPNode = class (TInterfacedPersistent, IJSPNode, Iinterface)
strict private
....
strict protected
....
public
function AddChild(aJSPTYpe: TNodeType = jspLEA): IJSPNode; virtual; abstract;
procedure AddOperation(const aOperation: string); virtual; final;
procedure Assign(Source: IJSPNode); reintroduce;
....
end;
implementation
uses
Math, NodeAdmImpl, .... NodeItrImpl, .... NodeSeqImpl,StrUtils, Sysutils;
function NewJSPNode(ATNodeType: TNodeType): IJSPNode;
begin
case ATNodeType of
jspADM: Result := NewPositAdmitNode;
jspITR: Result := NewIterationNode;
jspSEQ: Result := NewSequenceNode;
....
end;
end;
....
end.
***************************************************************************

unit NodeItrImpl;
interface
uses
Classes, Constants, Node, NodeImpl;
function NewIterationNode: IJSPNode;
implementation
type
TIterationNode = class(TJSPNode)
public
class function SetJSPType: TNodeType; override; final;
function AddChild(aJSPTYpe: TNodeType = jspLEA): IJSPNode; override; final;
function AddChildBefore(AChild: IJSPNode; aJSPType: TNodeType = jspLEA): IJSPNode; override; final;
end;
function NewIterationNode: IJSPNode;
begin
Result := TIterationNode.Create;
end;
.....
end.
.....
*************************************************************************
There are a number of characteristics of this code, some related to my style of coding, but some forced on me by the constraints of Delphi.

The declaration of the interface is fine, but, in order to provide a class from which I can inherit I need to expose the interface of the implementing class, so that other units can see it.

The implementation of the class implementing the interface needs to know about all its descendants, which will provide maintenance problems as the system evolves.

There is a massive case statement within the factory method which needs to know about all the subclasses. This is again fragile under change.

The visibility of the elements needed purely by the demands of the implementing language presents an opportunity for developer's of the system subsequently to misuse the implementation parts from misunderstanding or a desire to shortcut some of the “clutter”.

The solution to these problems was to realise that the inversion of control did not need to be implemented by inheritance, but rather could be implemented by delegation. The categorised items do not become specialisations, but rather classes in their own right, with an internal variable of type category interface. Any methods that are implemented in the category class are delegated from the “specialisation” class to the internal variable. Any methods that would be overriden in the first solution are implemented directly in the categorised item class. This leads to a unit for the interface which is unchanged from that above, but the class definitions become:
*****************************************************************************
unit NodeImpl;
interface
uses
Classes, Node;
function NewStructuredNode: IJSPNode;
implementation
type
TJSPNode = class (TInterfacedPersistent, IJSPNode, Iinterface)
strict private
....
strict protected
....
public
....
function AddChild(aJSPTYpe: TNodeType = jspLEA): IJSPNode; /* Do nothing */
procedure AddOperation(const aOperation: string);
procedure Assign(Source: IJSPNode); reintroduce;
....
end;
uses
Math, StrUtils, Sysutils, Constants, Operations;
function NewStructureNode: IJSPNode;
begin
Result := TJSPNode.Create;
end;
....
end.
*************************************************************************

unit NodeItrImpl;
interface
uses
Classes, Node;
function NewIterationNode: IJSPNode;
implementation
uses
Constants, NodeImpl;
type
TIterationNode = class(TInterfacedPersistent, IJSPNode, Iinterface)
strict private
aStructuredNode: IJSPNode;
public
class function SetJSPType: TNodeType;
function AddChild(aJSPTYpe: TNodeType = jspLEA): IJSPNode;
function AddChildBefore(AChild: IJSPNode; aJSPType: TNodeType = jspLEA): IJSPNode;
end;
function NewIterationNode: IJSPNode;
begin
Result := TIterationNode.Create;
end;
constructor Create(....);
begin
aStructuredNode := NewStructureType;
....
end;
function AddChildBefor(....)
begin
Result := aStructuredNode.AddChildBefore(....);
....
end;
.....
end.
.....
*************************************************************************
This approach does not quite get rid of all the problems of coupling. It is still necessary for something external to this part of the system to know what type of structured node is being demanded, but this is natural in this system, so not eliminating this lingering remnant of coupling is not significant.

Naming Rationale
I prefer this name to generalisation as the latter suggests that the system is to be implemented in a particular way. This construct arises in statements of requirements whereas generalisation arises in specifications. This term was used in the KISS method by Kristen[1] and was defined there as:
Category
A category takes care of the reuse of action types by different object types, weak object types or gerunds. For a category we always identify two or more parents. For transformation to the implementation environment, the category can incorporate the generic attribute types of its parents, so the attribute types have to be specified only once. In an object-orientated programming language the category can be implemented as an abstract superclass. The category allows us to model polymorphism.
[1] Object Orientation The KISS Method from Information Architecture to Information System.
ISBN 0-201-42299-9
Gerald Kristen 1994
Addison-Wesley Publishers Ltd.