Application Modernization

Over the past few months I have been exploring how SOA is morphing into BAU (business as usual). The parallels with Climate Change are uncanny. There is a highly vocal lobby that would tell you SOA is not happening. Yet all the evidence from both personal experience and industry surveys tells us that SOA is happening for real. How often have we observed that after the hype has died down, real learning and rollout just happens quietly and in private?

I have commented previously that SOA will morph and converge with CEP, EDA, Web 2.0 etc. Also that ecosystems (intra and inter company) will be the primary route to a more strategic version of SOA, rather than enterprise SOA. I refer to this as the Smart Ecosystem.

But smart ecosystems need a basic platform of technologies and business services to get beyond first base. What I observe is considerable activity in what is being termed application modernization. As we start to emerge from the recession there is real business pressure to keep costs and complexity down, and to be able to support the inevitable business demands for new ways of doing business.

A Forrester report commissioned by BluePhoenix shows a majority of IT leaders placing IT modernization as the top software issue. A very high number of respondents indicate their intent to consolidate or rationalize enterprise applications. A very high proportion also indicate they will be using SOA to sort out their legacy problems.

It’s a no brainer really. We know how to architect and deploy SOA; but efforts to deliver “enterprise” SOA have foundered for lack of relevance to business programs and priorities. In contrast, handled correctly modernization can provide a sensible platform for sorting complexity and agility issues while delivering business programs.

Most application modernization in process today is strongly technology focused, with objectives relating to platform and language replacement and reengineering. Critically much modernization is application specific, just replacing one arbitrary application scope with the same implemented in a modern language.

But if this activity is business and architecture driven, the opportunities to deliver business value in a series of coordinated increments have the potential to radically reduce complexity, increase agility while delivering urgent business programs.

At CBDI we are working on enhancing our already popular SAE tools and practices framework to create an integrated application modernization approach. We will be publishing the first cut of this work in the December CBDI Journal. Knowledgebase detail will follow. Needless to say, there’s no industry agreement on definitions. Our first cut on Application Modernization is as follows:

Application Modernization: Rationalize one or more applications or a portfolio to improve business support, technology usage and life cycle and run-time delivery process. Objectives include:

• Rationalize - eliminate duplicate applications; make multiple overlapping applications consistent.
• Modernize – upgrade delivery and operational technology and processes including managed service, offshore, outsourced delivery.
• Componentize – reorganize arbitrary boundaries to align with business morphology and enable business flexibility.
• Service Enable – move to service architecture that aligns with business capabilities, services and events.

Can incorporate: Integration, Migration, Reengineering, Rewrite, Replacement, Acquire, Buy not build, Elimination, Functional Improvement, Outsourcing, Offshoring

Be very pleased to hear what others think.

Reference: Application Modernization And Migration Trends In 2009/2010, Forrester,

Ecosystem 2

Last month I explored the concept of the ecosystem, and how breaking down enterprise boundaries will be a major focus in future. I discussed how the enterprise perspective needs to change in order to optimize business processes in a manner that spans conventional boundaries. Whilst I may have been a little controversial by suggesting the “death of enterprise architecture” the careful reader will know that my comments were directed at the issues of how to scope architecture and delivery projects.

I return to the theme this month in an article in the November CBDI Journal in which I explore how SOA will evolve. I examine a number of key influences which inform the evolution in which scoping forms an important part. I also look at key technologies:

1. How the confluence of SOA and maturing EDA, CEP, smart systems and Cloud Computing will encourage a revised view of solution scope that is simultaneously more narrowly drawn around specific business goals, yet spread more broadly to involve all the stakeholders that naturally participate in the ecosystem.

2. How SOA and Cloud Computing make the virtual world practical.

3. That well architected SOA should deliver highly independent, stable capabilities that implement core business logic for major resources which can support agile event processing.

4. A well architected EDA layer increases loose coupling and transforms orchestration overhead into business relevant event rules that can rapidly respond to business change.

5. CEP enables a system in which multiple stakeholders can participate in an ecosystem and concurrently derive stakeholder relevant events in support of a broader ecosystem purpose.

6. Maturing technologies including Web 2.0, Sensors and Analytics, which have naturally evolved in relative isolation, will increasingly interact with core systems architectures.

Putting this all together I suggest we have a number of really interesting new patterns that in various permutations and dimensions have a disruptive effect on our view of enterprise.

So what is an ecosystem? Try this . . . a set of business capabilities that collaborate to support a common purpose and exhibit high levels of interaction based on event relationships, shared information and data concepts. An ecosystem may comprise part or whole of one or many business processes; part or whole of one or more enterprises. It is most likely to be a series of subsets of conventional scoping mechanisms.

An ecosystem doesn’t have to be drawn on a huge scale as we measure scope conventionally. It may be sub enterprise or cross enterprise, focused on key goals of better results, in say a particular field of medicine; or key areas of high cost and low profitability. The key point is to start with a purpose, and then to examine the entire scope of business types, information needs, events, business capabilities, business services and software services. Equally important we should explore the entire set of stakeholders and actors in order to surface a rich model of complex events, capabilities and services that we can then consider various scenarios for radical change in business processes, business intelligence and management information availability in the light of new technical capabilities such as sensors, event processors and mashups.

Ecosystem might not the right word. Maybe a better definition is NOT ENTERPRISE!

Comments on The IT Complexity Crisis: by Roger Sessions

Roger Sessions has written a paper on IT Complexity. When I met with Roger in London earlier this year (at the EAC) we had a short conversation, and it was clear we agreed on numerous things. This paper is useful because it provides a basis for more detailed discussion.

Roger’s paper:

Comments:

1. Cost of complexity is a little suspect to me. Primary issue is basing numbers on government data – government projects have high probability of failure, at least in the UK. More important I think complexity has more impact on full life cycle costs. Direct IT cost is one issue, but lost business opportunity is probably the major cost. Hard to estimate. However we can all agree, it’s a it’s a big number.
   
   
2. I am always wary of “functional decomposition”. It’s notoriously imprecise. At CBDI we recommend a combination of capabilities and business types (data). The outcome of Roger’s example SIP analysis looks similar to what would emerge from structured business type analysis, but in my experience the latter method is more reliable. More importantly I am vitally interested in developing an architecture that is a) demonstrably stable where it needs to be (managing business types such as customer, product and so on) and agile by design where it needs to be (managing process behaviour, transient data and events). I accept this may appear to be a form of religious debate, and we may have to agree to disagree, but I am interested to have the discussion.
   
   
3. But regardless of how you arrive there, Roger and I are completely in agreement on the need for components. The component must be completely encapsulated, own it’s own data store(s) and be the sole provider of owned services. Roger’s components appear to be capabilities. No problem there, because in the CBDI reference architecture there are options. But I would be looking to separate out layered behaviour so that business type components (stable) are separate from process and event components (unstable, subject to change).
   
   
4. The computation of SCUs (standard complexity units) is interesting. Roger’s methodology will lead you towards larger components with more internal dependencies. What’s the internal architecture look like Roger? The dependencies are still there. I am not concerned if the number of dependencies is very high, PROVIDING the dependencies are all well formed, reusable services and operations. Then it’s simply a question of ensuring you have good management (life cycle and run time) systems, which we do know how to do. I am more concerned by poor reference architecture (bad patterns, convergence of behaviors that should really be separate, bad or non existent contracts . . . )
   
   
5. Finally I put more faith in principles, patterns and reference architecture than in numbers. Numbers can lie, and they often do. Whereas patterns and reference architecture are the distillation of good (and bad) experience and provide intelligent people good guidance to become even more intelligent.
   
   
6. At CBDI we have developed methodology and process for modernization. We see real demand for rationalization and modernization of existing systems (full life cycle costs). The essence of the approach is to a) create the SOA façade and b) componentize. You might like to take a look at our method for CA Gen, it’s just one method for one environment, but it illustrates we [practice what we preach. There’s a slideshare there that describes.

Good stuff Roger, fully support the component approach. Happy to dialog in more detail.

SOA Manifesto Marketing Twaddle

Last week a small group of mostly vendor representatives plus a couple of analysts and authors came together last week in a seemingly self important manner to develop an SOA manifesto. Of course the primary reason for the event was marketing, and we shouldn’t be surprised that the result is superficial in the extreme. The primary worry I have is that they came up with a manifesto that actually confuses rather then assists in wider, deeper understanding.

CBDI members will know from our long standing research that the key to architecture, and SOA specifically, is to define principles that a) uniquely differentiate the architectural style and b) enable governance. I wonder how this group managed to omit crucial matters such as separation of provider and consumer, standards based and abstraction?

Meaningless marketing twaddle, but because the vendors sponsored this, political correctness will prevail. Yuck.

The Death of Enterprise Architecture?

- Introducing Smart Ecosystem Architecture (SEA)

In recent months there has been considerable debate about Enterprise Architecture or EA. Practitioners who have embraced EA are now hotly discussing whether EA is a business or IT architecture and the balance between the two domains. Without any doubt most EA today is IT focused even though practitioners know it shouldn't be. And the new EA standard TOGAF 9 merely reinforces this. Although there is a vocal lobby who would like to gain more power and influence in the enterprise by taking the lead in business architecture and design, typically they are unable to achieve this because they can't articulate or demonstrate why the business should let them.

Yet my own observations are that EA has not been an overwhelming success. The tensions between EA and delivery teams remain undiminished. EAs are frequently out of touch with delivery issues and do not command the respect needed to exert strong governance over the delivery work streams. From an SOA perspective, there are few enterprises that have delivered a comprehensive Service Portfolio Plan, let alone the implemented enterprise service portfolio. You may wish to ponder on whether this is a failure of EA or SOA. I couldn’t possibly comment!

Many years ago when we published the first CBDI Maturity Model we recognized that SOA maturity would be defined by the ecosystem. This maturity stage followed the enterprise stage. I will readily admit that in the early days the ecosystem stage was ill defined; in fact no one was interested. Apart from a small minority of enterprises that have always operated an ecosystem business model, the focus of attention was always heavily on the enterprise. Today we can see things have moved on apace. Various influences particularly Complex Event Driven Architecture and Smart Business and IT are strongly predicated on optimizing business design and processes involving all the ecosystem stakeholders. Examples:

- Airlines, airports, airport concessions, airport services, hotels and rental car companies form an ecosystem that optimize their resources and prices on a dynamic basis dependent upon external events, raw demand and actual traffic. Actually we wrote a research report around this model years ago – see reference.

- Smart power grids with tighter linkage between customers, suppliers, generators, assets and operations manage power supply to optimize use of green energy, price to consumers and profitability.

- Also see IBM's Smart Planet site. Reference below.

The emphasis on business optimization is very interesting. This means that conventional architecture domains of operational systems, business intelligence (BI) and management information (MI) become much more inter dependent and dynamic. New types of information, perhaps conventionally classed as operational, now become critical MI. Real time feedback loops require derived information to be available in the operational timeframe. Conventional (sic) SOA layered architecture policy requires extension to manage new patterns.

In these examples of the new enterprise each participant collaborates to optimize the overall ecosystem and in the process optimize their own position vs their collaborators. In this world the architecture is absolutely not enterprise wide. It is goal driven, focusing on a scope that is business results driven.

Does this mean that EA is dead? As discussed, in its current form EA has actually been less than successful. The current level of debate is merely confirmation of the patently obvious. Personally I recommend a more practical approach to architecture that is more grounded with stronger relationships and shorter feedback loops to business value and delivery projects. Given the current issues with EA it would be expedient to have unambiguous naming without the baggage of EA. I am temped to say that Business Architecture works because most grownups understand that Business and IT are indistinguishable at this stage of the game. But I see a lot of immature debate from EAs that want to elevate EA away from IT, which is plainly wrong. Also I don’t have a strong opinion on the "role" of EA. Rather I worry about the nature of the architecture. I want to get everyone thinking about a new perspective and scope.

On balance therefore I prefer a naming that emphasizes the architecture scope and deliverables, the differences from what we have been doing and the need for change. I suggest Smart Ecosystem Architecture (SEA).

References:

IBM Smart Planet

CBDI Report: Modeling for SOA - Worked Example - The passenger departure process - from arrival at the airport to boarding the plane.

CBDI Report: A Web Services Maturity Model

The Failure of SOA Standards!

You will have heard the joke before: “The nice thing about standards is that there are so many to choose from.” Sadly in SOA land this is reality.

The need for standards in SOA are multivarious including standard concepts so we all speak a common language, standard meta models that underpin profiles, repositories, deliverables, specifications etc. standard protocols that allow implementation and platform neutrality for service publication and consumption. In this note I am interested in concept standards that support architecture and the full life cycle.

In this area we all know we have three “horizontal standards” organizations (OASIS, The Open Group and OMG) that have each followed their own path, ignoring the blindingly obvious need to have consistency across their work. The situation is so ridiculous that the three organizations resorted recently to publishing a document to explain how to navigate around the SOA architecture standards. Curiously although the document is branded by the three organizations and acknowledges participation and contribution from all three organizations, the document is formally a joint OASIS and Open Group document. One might be forgiven for assuming they couldn’t agree the legalities of the collaboration. Also the document doesn’t address inconsistencies between standards and guidance produced by the same organization.

The document declares two goals – 1) to help users to understand the strengths of each body of work and select products appropriate for their needs. 2) to encourage consistency. But the latter is declared only as a secondary goal!

But inconsistency is only part of the problem. We must judge any standard in its support for the user community and crucially user adoption. The primary purpose of these standards under discussion is achieving common definitions of important concepts and simply put they have failed to achieve widespread adoption because of the competing efforts have confused the user community. Further individual efforts have in my opinion failed to meet user needs.

• While TOGAF is getting considerable traction our experience is that the parts of TOGAF that get used are the ADM and the contracts. Not the concepts and meta model, because they are patently not based on real world experience. They are both too verbose for some purposes and insufficiently detailed for others. And the guidance for use is non existent.
• Similarly the OASIS model is widely regarded as academic – not useful in practical application.
• I have more respect for the OMG’s work in SoaML, because it is usable in support of code generation, but it suffers from over generalization, which makes it very hard to use by mere mortals.

I observe that in the referenced “navigating” document the nearest the authors come to agreements on core concepts is at the most superficial level. I see no evidence that there is any real intent to deliver real convergence at a meaningful (i.e detailed, defined) level of abstraction.

Over the years CBDI has encouraged standards efforts; we have consistently made the case that common concept standards are essential at all levels of abstraction. We didn’t just sit by; we developed our own meta model which actually predates most of the work referred to above, and made it available to standards bodies. Frankly we fully expected that our work would be superseded by the standards work and that we would progressively align our SAE models to a converged standard.

Well you won’t be surprised to hear that the complete opposite has happened. This month we are publishing V3 of the CBDI-SAE Meta Model for SOA. We have actively participated and contributed to the OMG SoaML work and we are pleased to announce a high level of alignment. This is possible because of the highly generic nature of the SoaML work.

The CBDI-SAE meta model has always been very different to the various standards bodies work. We have focused on what may be termed a detailed Business Type Model (DBTM). The goals of the DBTM are to provide consistency of concept for use across the life cycle at a level that can be implemented in tools and inform practical architecture and full life cycle deliverables. And of course users of the CBDI-SAE meta model tell us that is exactly what they use it for – populating architecture and service repository schemas, underpinning UML profiles and defining detailed of deliverables that can provide full life cycle traceability and governance.

What we didn’t anticipate at the outset was that our meta model would actually form the basis for a further purpose – to provide a mapping between the other concept standards such that users forced to work with multiple conceptual schemas (few of us live in a homogeneous world) would have a basis for coordination and possibly transformation.

So as a user what do you do? Currently we see minimal use of these concept standards because they are inadequate. We recommend that you continue (or commence if you are a late adopter) using the CBDI-SAE meta model. We will commit to map to the other standards and, if and when they ever get to a credible position of convergence, we will certainly plan model convergence also. But we will need more than alignment of the concepts, we will need to see practical abstraction and detail required for real work.

We will be announcing a user review of the V3 model very shortly and welcome all input.

---

Navigating the SOA Standards Landscape Around Architecture, Copyright The Open Group 2009, OASIS 2009,

V3 SAE Meta Model note this is currently for CBDI subscribers only. Public domain version follows shortly for the review.

Don't Believe the Hype!

I note Gartner has published its 2009 Hype Cycle Report^[1]. I must admit I find it a very strange beast. It claims to be an evaluation of technologies. But it is plainly a mix of technology maturity AND technology adoption maturity. For example SOA is classified as being half way up the “slope of enlightenment” and 2 to 5 years to mainstream adoption. Yet it is very clear that SOA technology is reasonably mature at this stage, but it has equally obviously run way ahead of users’ ability to deploy it, because it is an architectural issue facilitated by technology.

The report also includes a technology referred to as Context Delivery Architecture (CoDA) as being in the Technology Trigger stage and less than 2 years to mainstream adoption. Two issues here. First so called CoDA is simply part of SOA. By Gartner’s own definition it’s about architectural response to the end user’s context such as location, preferences, identity, etc. and delivering the information that is most suited for it. Second the Differentiated Service pattern^[2] defined by CBDI in 2000 has high levels of support in many tools and platforms that are fully mainstream in terms of technology maturity. Yet Gartner believe CoDA will become mainstream in less than 2 years, even though they advise SOA will only become mainstream in the 2 to 5 year timeframe. Doh!

And I wonder how Location Aware Applications (slope of enlightenment) differs to Context Delivery Architecture?

And then there’s Cloud Computing. Currently shown by Gartner as being at the top of the “peak of inflated expectations” with 2 to 5 years to mainstream. Let’s examine this more closely. SOA is central to Cloud Computing. I note various commentators saying that Cloud simply enforces the encapsulation and virtualization principles of SOA. Of course there are new technologies in Cloud. Infrastructure as a Service (IaaS) in particular is key; yet this is still just SOA for infrastructure, which many vendors have already done a great job of cracking, and have many user deployments.

Web 2.0 is also on the list. Whilst Web 2.0 is a very ill-defined trend spanning many disparate technologies, SOA is clearly central to most of the use cases. Yet it’s shown in the Gartner report as towards the end of the “trough of disillusionment” and less than 2 years to mainstream adoption, ahead of mainstream SOA!

And then there’s the heterogeneous nature of the Hype Cycle Report. Because it includes such a wide and disparate range of technologies, it’s really more confusing than informing. Further it ignores one of the most important influences on maturity – standardization. Reading the report this year I was tempted to simply ignore it. But the inconsistencies highlighted above persuaded me that this report is actually quite dangerous guiding decision makers in highly undesirable ways. Listen to Gartner and you are encouraged to think that SOA is a technology. But the A is not just important, it's the architectural foundation to many of the evolving trends. Yet Gartner seem to place them all on the same level as individual technologies. I suppose the real reason why is because it plays to vendors marketing programs, of which Gartner is an intrinsic part. However I see it as distinctly unhelpful to enterprise customers.

My recommendations on hype cycle analysis are as follows:

- Decide which classes of technology you need to track in terms of technology maturity. Create a grid for this. Use this to inform R&D and Discovery projects.

- Decide which classes of trend you need to track in terms of adoption maturity (include architecture, practice, technology, products). Create a separate grid for this. Use this to inform project chartering decisions and governance review criteria.

- Change both grids to include an assessment of when important and relevant standards will be stabilized.

- Create dependency models of the trend areas, so you can understand the relationships between such clusters as SOA comprising Cloud, IaaS, EDA etc).

Clustering and dependency techniques are important. Not only do they allow you to plan sensibly, they also facilitate completeness reviews. And strangely I observe Gartner omit the area of semantic integration, a technology domain that I see as starting to become very important for many of our customers. Of course it's an integral part of the SOA cluster, and it's strongly driven by SOA maturity and the requirement to deliver on consistent information services. By chance I am publishing a report on the Progress Data Exchange Semantic Integrator this month, and I also note that last month Oracle acquired a provider of a similar product - Golden Gate. But the Gartner methodology apparently doesn't seem to surface such an interesting area.

The basic idea of a hype curve has been around for decades. I recall doing something very similar back in the mid ‘80s with an oil major client. But like all models, the deliverable is highly dependent upon some clear definitions and rigour that make the opinion based report credible and comprehensive. Sadly Gartner seems to have been caught up in it’s own hype.

---

^[1] Gartner's 2009 Hype Cycle Special Report Evaluates Maturity of 1,650 Technologies
http://www.gartner.com/it/page.jsp?id=1124212

^[2] Design Pattern: Differentiated Service, Fewer Interfaces than Components

http://www.cbdiforum.com/secure/interact/2000-12/design_pattern.php3