Monday, August 3, 2015

Enterprise Modernization - The Next Big Thing!

“We tend to think the strong will survive, but a virus is a very small thing that kills big things.” Horace Dediu, Clayton Christensen Institute, speaking about the fall of Nokia.

All enterprises, be they large or small, national or multinational, commercial or government agency, American or Chinese, Japanese or European, are carrying the dead weight of their history and almost certainly continuing to add unnecessary complexity and excessive cost that will progressively reduce effectiveness, with the potential to trigger existential crises. Newer enterprises including Internet platform and Cloud based companies are not immune from this effect. As Horace Dediu has commented, Nokia is a classic case of a large enterprise brought from market leadership to irrelevance and zero value in an extremely short space of time.

In the past, modernization has become synonymous with technically related "application modernization" The term "modernization" is commonly used to refer to information systems, applications and technologies. And it is very common for systems or applications to be "modernized" because a technology has come to the end of its life, or an application is so antiquated that it cannot support newer business process patterns such as multi-channel and mobile. More generally organizations clearly implement modernizing programs without necessarily using the modernizing name. For example transformation programs or projects will often involve considerable elements of modernization. Similarly adoption of Agile methods may be considered a form of process modernization. Or the introduction of DevOps practices, enterprise architecture, master data management, life cycle management etc. But it is notable that each of these forms of modernization are discipline centric. It is very rare for an enterprise to undertake concerted effort to understand what modernization really means for a specific enterprise and to plan and execute modernization that delivers inter-discipline collaboration in support of specific modernization objectives and goals.

Each enterprise has unique modernization needs. A bank may see a primary modernization goal to establish core banking systems that reduce the risk of negative customer impact, such as delays or errors in transaction processing. Also to be able to introduce price and product change in days or weeks. A healthcare company may similarly see the requirement to change prices rapidly but equally the need to rapidly implement new legislation. A retail enterprise may see the ability to interact with customers using processes that span multiple technology, shopping and delivery channels, and to be able to influence the customer decision making process to achieve maximum customer satisfaction.  A key element apparent in all these modernizing goals is that modernization is not about achieving a new plateau of capability and functionality. Rather it is about enabling continuous, short cycle time response to change, targeted at the very specific areas that are mission critical for the individual enterprise.

The problem with modernization is that it is widely perceived as slow, very expensive and high risk because the core business legacy systems are hugely complex as a result of decades of tactical change projects that inevitably compromise any original architecture. But modernization activity must not be limited to the old, core systems; I observe all enterprises old and new, traditional and internet based delivering what I call “instant legacy” [Note 1] generally as outcomes of Agile projects that prioritize speed of delivery over compliance with a well-defined reference architecture that enables ongoing agility and continuous modernization.

What’s required is a modernization approach and strategy that progressively breaks out business and technology components that establish highly independent units of business capability that comply with a reference model and architecture that ensures architecture agility and implements clear fire breaks between the components.
But as discussed, enterprises are extremely reluctant to undertake modernization as they see it as all cost and risk. Transformation projects are widely viewed in the same way. And anyway immediate business priorities always trump housekeeping!

There are various aspects to a successful modernization approach. But the most important are:
1. Define Agility Vectors. Identify the top priorities for business agility and integrate relevant actions into all business projects. Here are some examples:
a. Radical improvement (time and cost) in response to legislative change
b. Generalization of existing capabilities to support new products and services
c. Dramatic reduction in new product time to market
d. Integration of existing capabilities to leverage disparate channels
e. Separation of common and customer/channel specific capabilities
I refer to these as “vectors”. Mission critical goals that will require actions and change right across the organization. Single projects are generally not going to cut it. Therefore the vectors provide a mechanism to exert influence (demand shaping) over the incoming application demand management pipeline and to select and coordinate multiple project actions.
2. Mandate Reference Architecture for Agility. Establish a reference architecture for business agility that defines the minimum necessary architecture compliance to ensure continuous business evolution. Mandate that all core business capabilities are implemented as software services and components.
3. Integrate Agile Architecture AND methods. Implement Agile practices that give equal weight to agile architecture and agile project delivery. Demonstrate small, incremental delivery steps, business capability by capability, service by service.
4. Govern business agility. Automate governance by establishing model driven architecture and development tooling that ensures compliance with the reference architecture.
5. Integrate Business and IT. Practice conceptual business modelling to establish common business and IT vocabulary independent of existing applications, align business and software services and reengineer the organization around business capabilities and services.
6. Get top level sponsorship for the Enterprise Modernization Roadmap. Recognize enterprise modernization as a major business priority and elevate to the highest levels of the enterprise to make it happen.

The enormous, disruptive creativity of Silicon Valley is transforming how companies make decisions, store data, reach potential customers, outsource activities, processes and capabilities, and how people communicate, make friends, protest and shop. No enterprise is immune from this effect.

Accordingly, modernization today means reinventing the way enterprises work, the business model and systems, and being genuinely agile. You only do this by ripping up today’s world and turning it into a genuinely flexible structure of independent components that can flex and morph continuously.

Application Modernization should be long dead and buried. Enterprise Modernization is an existential priority for all enterprises, not just those with mainframes and COBOL!

Note 1: Understanding Agile Adoption Failure

Friday, March 20, 2015

The Microservices Pattern

For those of us that have been practicing SOA for over a decade, it's surprising that there's so much interest in microservices. In fairness microservices don't look like the vendor play that was early SOA in the early noughties. But experienced SOA practitioners everywhere will be wondering if microservices is actually a good thing. You see microservices is basically an SOA pattern that inherits all the well-known SOA principles and adds characteristics that address the use of SOA for distributed, finer grained software services. And like all patterns, microservices are not applicable to all situations, and it's very important to understand the cost/benefit equation. Those folks that are heralding microservices as the "new SOA" or "the right way to do SOA" are flat wrong. Microservices is a specialization of SOA that has applicability to a relatively narrow problem space.

Componentized Services and Decentralized Data Management.

Capability based services that have reduced dependencies and single point responsibility for data has been a widely used SOA pattern. What's different with microservices is the decentralization of data management and encapsulation of the data in the service component. This is a useful pattern for services that can really own the data AND have high probability of churn in the service information model. The reduced horizon of change will be highly effective in responding to continuous business evolution.  But there's a serious cost to this, and Fowler et al surround the distributed data management advice with caveats.  They say, "Decentralizing responsibility for data across microservices has implications for managing updates. The common approach to dealing with updates has been to use transactions to guarantee consistency when updating multiple resources. . . . Using transactions like this helps with consistency, but imposes significant temporal coupling, which is problematic across multiple services. Distributed transactions are notoriously difficult to implement and . . . as a consequence microservice architectures emphasize transactionless coordination between services, with explicit recognition that consistency may only be eventual consistency and problems are dealt with by compensating operations."

Anyone who has dipped their toe in to this complex and murky pool knows that you do distributed data only when there is a real business requirement that demands this level of sophistication. In the typical enterprise environment there's a requirement for a range of SOA relevant data patterns. Distributed data is just one of many.  

Decentralized Governance

The vision of microservices is very reasonably to facilitate heterogeneous technology platforms. But the Fowler guidance goes much further. "Perhaps the apogee of decentralised governance is the build it / run it ethos popularised by Amazon. Teams are responsible for all aspects of the software they build including operating the software 24/7. Devolution of this level of responsibility is definitely not the norm but we do see more and more companies pushing responsibility to the development teams."
But you don't have to be a Luddite to think that there are elements of governance that need to be centralized; and I'll bet are standardized in Amazon. For example, at a basic level security, logging, reference data and return codes, contract and API meta data and data management (see above), life cycle management etc. But in our practice we also see massive advantages in standardization of much of the (platform independent) architecture runway, SOA patterns etc. The advantages are not just about productivity and quality, which are considerable. They also deliver consistency of approach that has many less tangible benefits in terms of skills and resources, knowledge sharing and integrity of the delivered solutions, while still supporting heterogeneous target platforms. Perhaps this demonstrates the divide between the architect and developer viewpoints. And I have seen far too many enterprises getting into deep water because they have allowed the pendulum to swing too far towards developer independence.  

The concept of microservices is clearly creating much interest and confusion. I commented on the Newman book in an earlier blog that the author didn't seem to understand the difference between a pattern and a process. And perhaps this is the issue with microservices. The concept is essentially about decentralization and distribution, creating freedom of action for developer led teams that deliver something useful for (probably) a narrowly defined set of users. It all looks good in terms of ticks in the "project delivered" box. But it also looks remarkably tactical. And as I commented in my recent blog, lack of consistency leads to inability to govern and increased time to market. And my guess is that microservices, in the way they have been defined, will lead down a very similar path.

Nonetheless, there are some useful aspects to microservices. DevOps specialists are seeing real benefits from smaller units of deployment. Promotion of some of the basic elements such as componentization, automation and evolutionary design are a good thing. But adopting the microservices characteristics as a unified pattern as described is really only applicable to a narrow problem space. And as soon as you adopt the characteristics piecemeal, you have to reassess the cost/benefit. I expect that the widespread interest in microservices will lead to it morphing into a simpler, less complex pattern that does address a wider problem space. But surprise, surprise, we solved that a long time ago, and all we are doing is renaming highly independent Capability based services as microservices!

References: Microservices, Lewis, Fowler, March 2014

Wednesday, March 4, 2015

Microservices Unplugged

Given my (well-known and enduring) interest in all aspects of services, I have followed Martin Fowler's writing on microservices. But I will admit I always found the original paper more confusing than insightful. And in my client work I have resisted the temptation to use a microservices pattern, for precisely the reason that it would more than likely confuse. So I was interested to see the book Building Microservices by Sam Newman published last month, particularly as Newman is part of the Thoughtworks stable, which presumably means it is authoritative.  

Right off the bat, Newman advises that we should "think of microservices as a specific approach for SOA in the same way that XP or Scrum are specific approaches for Agile Software development". These analogies are very interesting because my expectation was that microservices is a pattern. So I might infer that microservices is a set of process techniques as opposed to an architectural approach. Yet in the book, Newman clearly includes some elements of concept model and architecture as well as process and organization.

Probably the biggest weakness of Newman's book is the absence of a coherent reference model. He discusses a series of heuristics such as:
                - Microservices are small, autonomous services that work together.
                - With high cohesion.
                - Gather those things that change for the same reason, and separate those things that change for different reasons.
                - Using business boundaries.
                - How small is small? Could be written in 2 weeks; small enough and no smaller; can be managed by a small team
                - Separate entity. All communications between services are network calls. Decoupled. Minimum dependency.
                - We need to think about what our services expose, and what they should allow to be hidden. If there is too much sharing our consuming services become coupled to internal representations.

And these are mostly useful principles, but are most unlikely to guide a consistent approach. Given Martin Fowler's involvement, I would have expected at least some good patterns. But mature SOA needs a defined concept model, like CBDI-SAE, SOA-RM etc.

This lack of clarity is fundamental. Many organizations treat operations as services, and they typically end up with thousands of services. They will be confused by the very term microservice because they already have what they perceive as small services. Perhaps the term microservice has been chosen because of the ubiquitous use of the ITIL service, which undoubtedly means monolith. Most mature SOA organizations define services and operations as a primary technique to establish cohesion guided by reference models such as SOA-RM and CBDI-SAE. Just as important is the clarity and precision around specifications or contracts and what is exposed to consumers (APIs) and providers. Yet the book contains nothing on the subject of rich service specification or contract as the primary means of achieving implementation independence and business alignment. Organizations adopting microservices will need a defined reference model. Without that there will be architecture governance by personal opinion. I wonder how Thoughtworks advise their clients in this area? 

The book contains some quite general advice on business capability based decomposition - which is a tried and tested method of defining services. However there is little or no advice on what a well-formed business capability is, it's characteristics and governance considerations. In my experience this is an area for considerable confusion; the business capability concept is so well known, but little understood, it's an accident waiting to happen. And as for the relationship between capability and service, there's no model provided, and more importantly, the granularity of a business capability service is likely to be non-trivial.

Similarly there's no guidance on types of service. Do all services contain a mix of channel, process, business and data behaviors? Which you might expect given the explicit advice to gather those things that change for the same reason, and separate those things that change for different reasons.

The one area that might be helpful is the concept that microservices are highly autonomous, and a capability cluster of services would be a unit of deployment. But sadly for Newman, this is an idea that I and my colleagues worked on and popularized nearly 20 years ago. We called it Component Based Development, and the concept was closely allied to Rich Service Specifications which enforced the component boundaries, a concept that seems to have escaped him.

Once upon a time I had expected there to be a microservice pattern. But having read the book, all I see is a set of established ideas being rehashed in a very superficial manner, under a new name; with many key concepts missing or misunderstood. Apparently there is a lack of consensus on how to do SOA well. That failure of SOA is caused by vendors’ products, issues with granularity or picking wrong places to split a system. These assessments of the problem illustrate the lack of understanding of the author and his colleagues. Yes, there have been SOA failures, but mostly they have arisen through a) a focus on integration technology not business, or b) lack of reference model, reference architecture and governance. As an aside, in mature SOA we don’t consider “places to split a system”; we define the scope of a service using well defined heuristics and techniques that vary by type of service. To claim therefore that microservices is the answer to problems with SOA is downright disingenuous. And given the amount of hype I see about the topic generated by supposedly responsible organizations such as Thoughtworks and Gartner, that’s downright irresponsible. Disappointing. 

Public Domain information on Mature SOA architecture and delivery practices


Monday, January 5, 2015

Understanding Agile Adoption Failure

The most common concern our customers voiced in 2014 was the unexpected outcomes of Agile projects. They don’t talk about failure as such. But they do talk about loss of consistency; inability to govern; lack of coordination AND THE INCREASING TIME TO MARKET caused by these precise issues.

I was struck by the results of the Agile Adoption Experiences Survey 2014 published by Scott Ambler. The really significant result to me is that 40% of respondents rates their organizations adoption of Agile as neither a success or a failure. Add to this the categories of Failure, Great Failure and Too early to tell and you have 58% that are not successful! This synchs with my customer feedback referred to above.

The advice my colleagues and I give when customers approach us looking for answers to these questions, is to look at how architecture is integrated into Agile projects. And there are some key areas that we look for in our assessments:
1. Is there a good reference architecture and associated contextual patterns?
2. Are there clear policies attached to work products together with the rationale?
3. Are developers and architects working as a community of interest to evolve the reference architecture, patterns and policies?
4. Are the reference architecture, patterns and policies integrated into the tooling and the architecture runway?
5. Is the architecture runway model based – allowing it to provide a reusable design time platform to be evolved by projects?

Agile projects can be successful in an enterprise situation. But architecture and governance need to be coordinated for consistency and mechanisms (automation) enforced to ensure consistency.

I wonder why the Agile Adoption survey didn’t ask any questions along these lines?

Thursday, September 25, 2014

Agile Self Governance

I guess most readers of this blog know that Ireland went through a massive bust in 2008/9. The primary cause was a massive building bubble. And because the economy was dependent upon building related taxes, the crash was brutal. One of the side effects of the bubble was that building standards suffered. In one extreme case a block of apartments was constructed with no fire safety protection! There were many, many more prosaic examples. In my own case I had building works completed that had to be completely reworked within a couple of years! The reasons for the low quality were complete lack of governance. In the boom times many people set-up without adequate skills and many developers over-committed to deliver more work than they could manage competently. In Ireland, unlike say the UK, there was no independent building inspection regime. Architects merely had to give approval for work to proceed at the major stages. There was no independent oversight.

Today in the Agile project world the idea of self-governance is pervasive. But the parallels with the Irish governance regime in the noughties is too close for comfort. The Agile principles guide that projects should be built around motivated individuals, given the environment and support needed and trust them to get the job done. Further valuing working software over comprehensive documentation is effectively encouraging teams to dispense with transparency and traceability. While this may work in small scale environments, in a large enterprise the idea that all teams will be highly skilled, properly resourced and motivated contradicts general experience. So in many large organizations conventional governance is still a requirement that can be highly detrimental to the efficient Agile process.

In my last blog post I described a new Agile governance model that addresses the empowerment question with:
a) A defined Agile governance model
b) Defined principles and reference architecture that establish ways of working together with articulation of business value
c) Automation systems that progressively incorporate the principles and reference architecture into frameworks, tooling, design time platforms, deliverable profiles and knowledge management systems.
d) A Community of Interest (CoI) responsible for the governance system and communications
e) A communication system that ensures Agile projects are fully implementing the governance system, and providing at least retrospective feedback to the CoI that contributes to a common asset base as well as practice maturity.

And in this post I want to explore further the organizational issues. In the image I show there are three primary stakeholders in the Agile process. A Design Authority (DA), the Community of Interest (CoI) and everyone else; let’s call them the Crowd. And what’s really important is this is not a conventional hierarchy. The Crowd, developers, architects, product owners and business analysts are the drivers of the process, engaged on business improvement delivery projects. The CoI are also part of the Crowd insofar as they are also engaged on delivery projects, but they are individuals that have a further role of taking responsibility for coordinating consistency of the approach taken by delivery teams. As shown the CoI will be responsible for reference and enterprise/domain architecture and have an approving role for solution architecture and design, platform components and factory configuration. And the key point here is that all of these key deliverables start by being crowdsourced. The delivery teams are in the best position to establish solution architecture and design. The delivery team member with a CoI role is responsible for identifying and promoting candidate components of reference architecture, the design platform etc to the collective CoI.

The DA is different. It’s still not hierarchic, but it does approve demand shaping decisions, reference and domain architecture. This generally happens in response to delivery project concerns, for example when there are affordability or timescale issues with “doing the right thing”. You know the deal, if we don’t worry about technical debt, we can deliver on time and budget. But if we address the bigger architectural questions, and deliver solutions that will reduce complexity, reduce operating and maintenance costs etc, then the delivery date will be pushed out and additional resources required. Frankly this is the right time to address the value of architecture, because the questions are real.

So what I have described here, and in the last post, is an evolving governance model in which the definition of “self” is stretched a little to encompass the role. The Crowd has a major role in informing the requirements for policy and architecture as required for specific projects.  The CoI are responsible for synthesizing these requirements to meet some broader, longer running remit and the DA guides on bigger questions of architectural compliance from a strictly business value perspective. The Crowd and CoI are also responsible for automating policy wherever possible; selecting exemplar solution components for platform and factory implementation, so that the very best solution components become reusable as patterns, configuration items, coding standards, services etc.

Whether it’s in a building or software development context, simplistic self-governance doesn’t work in complex enterprise situations. However understand the roles and responsibilities, work bottom up with Crowd sourcing and you may just motivate a very large team by enabling them all to contribute to the greater good.

See Also: Agile Governance

Wednesday, September 3, 2014

Agile Governance

Last year the decision was finally made to mandate Agile across our enterprise. The decision was taken, even though there were many unanswered questions. The assumption was that forcing the migration, along with adoption of popular “enterprise Agile methods” would ensure resolution of the outstanding questions. In practice, Agile methods have been very effective in delivering specific digital business initiatives. But almost inevitably the distribution and delegation of architecture has resulted in duplication, inconsistency and increased complexity, across all project types including legacy and new projects. We are now concerned that we no longer have an effective governance capability. The question is how do we fix this without losing the undoubted benefits of Agile methods?“ Enterprise Architect, F2000 company.

Over the past few months I have heard this message over and over again. While Agile is being successful, it is increasingly in conflict with broader goals. And this is clearly becoming a major issue, manifest in increased complexity, horizon of change and coordination issues as well as inconsistent customer experience.  I am now regularly advising a practical approach to resolution by addressing from the governance perspective.

In many organizations governance is still practiced by phase or stage gate peer review, and Agile projects are forced to accommodate, which leads to WaterScrumFall or worse. But governance criteria and policies are often very weak anyway, out of date or non-existent. Consequently governance is frequently a matter of opinion and experience, highly dependent upon the experience of individual reviewers. As we all know, a basic principle of Agile methods is delegation of responsibility, and ideally we need to delegate governance to the Agile practitioners and teams. So the question is how to implement self-governance and ensure quality and consistency of governance?

I think it was an old John Cleese training film in which Cleese himself plays the part of a manager telling a subordinate that he is now empowered, and he scatters magic dust over him and shouts some magic words saying, “You are now empowered!” Clearly this isn’t any more useful than telling an Agile team that they are now self-governing! Rather we need to go back to basics and define and communicate what governance is required and provide Agile teams with guidance on what is expected.

That sounds good in theory, except that in practice no one is going to be able to accurately define all the governance requirements; certainly not in a fast changing, Agile business and technology environment; nor will Agile development teams be able to keep up with a bureaucratic regime that continually issues edicts that everyone is expected to adopt.

What’s required is a governance system that works in an Agile environment. The parameters of the Agile governance system comprise:
a) A defined Agile governance model
b) Defined principles and reference architecture that establish ways of working together with articulation of business value
c) Automation systems that progressively incorporate the principles and reference architecture into frameworks, tooling, design time platforms, deliverable profiles and knowledge management systems.
d) A Community of Interest (CoI) responsible for the governance system and communications
e) A communication system that ensures Agile projects are fully implementing the governance system, and providing at least retrospective feedback to the CoI that contributes to a common asset base as well as practice maturity.

Agile Governance Model

This is a much simplified Agile governance model. Key points to note are:
1. the centricity of the architecture runway, and the tight relationships between policy, reference architecture, reusable assets and the automation platform. 
2. variants guided by various dimensions of scope, which may include applicability, business or technology domain, or even the maturity of the business or technology domain to achieve compliance.

Principles  are a great place to start. Self-governance is going to be a key part of Agile governance, and if we can’t articulate and communicate what’s important then we are dead in the water. Some examples shown here:

Reference Architecture is a critical capability, defining the architecture styles and patterns and applicability. But reference architecture shouldn’t stop at models, it must be realized in code in the Design Platform – which progressively realizes the reference architecture as application level infrastructure reusable across multiple projects. The design platform is typically managed by the CoI, a collaboration of senior developers and architects that decide what should be in the platform and develop the models and code as exemplars that succeeding projects will be happy to use, customize and or extend. The design platform is therefore a critical governance tool that actively evolves, managed by the CoI and constantly challenged by project developers  to provide optimal solutions to delivering on the principles and reference architecture. As a by-product the mature design platform will also be a major productivity tool; for example in the Everware-CBDI Agile Service Factory, over 80% of the code for typical large projects will be automatically inherited from the platform.

As shown Principles are generic patterns or techniques that guide strong solutions to business problems. The application of Principles is achieved by Policies. But not your father’s policies! In many (most?) organizations Policies are outdated lists of standards. In Agile governance, policies should be a context based record of how the principles and reference architecture have been realized. Like principles, policies are not mandates from senior management, they are transparent  communications of pragmatic decisions made by the CoI on the best way of delivering an optimal result in a particular context, reusing tried and tested methods supported by existing architecture and design assets. This is therefore a continuously evolving body of knowledge, specifically tailored for one enterprise’s needs. Examples below. Note in particular the Policy Context that highlights applicability and exemption.

Many Agile teams are now using the Scrum of Scrums approach to coordination of multiple projects. This is a highly effective mechanism to manage the pan-Scrum backlog. However this coordination must not be confused with architecture realization. The Community of Interest is not a Scrum of Scrums, it is a group of the most respected architects and developers who will be active practitioners in architecture and development projects, who coordinate the realization of the architecture, the models and implementing code, typically in direct response to project demand, but involving CoI members as appropriate to review, refine and contribute to improve the solution, to be optimal, generic, principle and policy compliant. The Architecture Scrum may therefore on occasions be a series of architecture specific sprints, perhaps at commencement of new programs, or in response to significant new areas of reference architecture or design platform requirement. But in momentum situations the Architecture Scrum is more likely to be integral to multiple development Scrums.  

In a generic sense, governance is concerned with ensuring the integrity of the delivered product. This requires a strong focus on the architecture and how it is realized. As many organizations are now realizing, delegation of architecture in an uncontrolled manner is high risk. The approach outlined actually encourages delegation of architecture but to a coordinating body, the CoI, which itself is charged with supporting project demands and broader organizational objectives. But the approach outlined also recognizes that there needs to be explicit documentation of architecture principles and policies and their application in order to allow communication and review, and justification of business value. This is a necessary level of documentation needed to communicate to the many stakeholders involved. 


Reliance on opinions expressed on a case by case basis, or architect resource involvement in projects without the backing of strong, defined reference architecture, gives programs or projects far too much discretion. Whilst we may laugh at John Cleese’s magic dust, in practice the embedding of key architectural code into the platform layer actually does make governance considerably more effective. But even if it’s incredibly effective, it’s not magic. An effective CoI comprising the very best architect and developer skills available means all projects have access to optimal solutions as well as automatic compliance. Agile governance as described in this post is therefore not an extension of Agile methods per se, rather it is a bridge between Agile methods and agile architecture that defines and ensures desired outcomes, without compromising the integrity of the Agile process. 

Monday, July 21, 2014

Agile and the Fairy Godmother

Once upon a time, a long, long time ago, well some 20 years actually, some clever folk figured out a way of structuring work that was quite revolutionary. So revolutionary in fact that most people didn’t understand it. Now a few folk in a parallel world worked hard over the years to make this method work, and they invented more ideas, frameworks and tools. And every day they are constantly improving their productivity and quality; and many of them also use Agile methods, as they are entirely complementary to the revolutionary method. The method – Design by Contract.

However the mainstream of the market seems to be fixated on Agile methods, as being the metaphorical silver bullet. Yet we all can’t help noticing that Agile Land is not such a happy place. There is continuing debate about the difficulties of making the methods work in the enterprise; the fragmentation of the original Agile principles and the outbreak of religious wars. And I am minded to comment, again, that the root problem with Agile methods is that they are one-dimensional - solely focused on people and process to the exclusion of all the other opportunities to create agile businesses.

At the heart of the conundrum is the need to focus on some different questions. Such as:
- How can we reduce the amount of work that has to be done?
- How can we structure the outcomes (deliverables) so they are inherently agile?
- How can we structure the work so that there is real traceability between the intrinsic business model and the delivered systems and services?
- How can we ensure that overall technical debt is always reducing faster than new functionality is being delivered?
- . . .
You get the idea.

So back to my parallel world. In Design by Contract the business problem, typically the Use Case, Services and Operations are attributed with Pre-conditions, Post-conditions and Invariants (rules that must remain constant). These artifacts provide us with functional and design level specifications that can be produced in an Agile, iterative manner, that
a) Deliver implementation independent service specifications (descriptions if you prefer)
b) And therefore also for publishing API specifications
c) Form the basis for structuring the code that is fully traceable to the business model
d) Create inherently agile systems and service structures
e) Create the structure (stubs) for Unit, Integration, Functional and Regression testing
(e.g all conditions and rules within a given test scope must be tested)
f) Depending on the technology employed to define the conditions and invariants, (rules engines, pseudo languages etc) both code and test cases can be produced automatically.

I was prompted to write this blog because I happened to read Rob Marvin’s useful blog on testing, and his very interesting ideas for improving test productivity to keep up with Agile projects. In his piece he talks about automation, but actually seems to miss the opportunity to auto-generate the test cases. Even more important, he seems to believe that the current state of Agile projects is his benchmark that he has to keep up with. I would comment that state of the art Design by Contract projects together with model driven frameworks are delivering order of magnitude greater productivity with exceptional quality, and this ought to be the where testers should set the bar.

Sometimes it seems to me that the Agile methods community is rather in the situation of hoping a Fairy Godmother will appear, wave a magic wand and all will be well. Everyone will use Scrum like they ought to; enterprises will waive their awkward little requirements for inter-project coordination and all will be well. But while the Agile community continue to ignore the broader scope of agile enablers this day won’t come.

If you are interested in an example of Design by Contract at work take a look at the Agile Service Factory.