David Sprott's Blog

Understanding Agility the Hard Way!

2012-01-12T03:23:00.000-08:00

Agility is one of those words beloved by software industry marketing people. Over time it has become almost embarrassing and meaningless. Yet if you are in doubt I suggest asking Eastman Kodak, RIM, Palm, Yahoo or Nortel what they think the term means. When you don't have agility you understand it all too well.

In today's FT John Gapper says: "Kodak’s experience has a lesson for companies in the grip of rapid technological change. As Kenny Rogers sang, “You’ve got to know when to hold ‘em, know when to fold ’em”. Unfortunately, most public companies are run by people who hate folding ’em, and instead keep returning to the shareholders and bondholders for more chips. . . . Few senior executives, when debating options for a technology company in decline, admit defeat and run it modestly. Instead, they cast around for businesses to buy, or try to hurdle the chasm with what they have got. Sometimes they succeed but often they don’t, wasting a lot of money along the way."

It brings to mind Fred Brooks essay on software engineering. "No scene from prehistory is quite so vivid as that of the mortal struggles of great beasts in the tar pits. In the mind's eye one sees dinosaurs, mammoths, and sabretoothed tigers struggling against the grip of the tar. The fiercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that he ultimately sinks."

When all is going well investing for agility may seem a luxury. Why make capabilities genuinely independent so they can be switched in or out, or truly generic so they can be used in many different contexts if there's no obvious or short term ROI? By the time you can accurately compute the ROI it will probably be too late.

Ref: FT - The smart technology loser folds

Ref: Fred Brooks: the mythical man-month, Addison Wesley, 1975

Let's Kill the Confusion About Capabilities for Once and All!

2011-10-26T02:09:00.000-07:00

I note in the Business Process Trends Advisor Paul Harmon is confused about what is a capability. What Harmon is missing is that capabilities are not just another modelling device that further helps to understand the business. Rather they are core structural concepts that allow us to establish independent units of capability. Capabilities are coarse grained business components that are inherently reusable. We define the characteristics as follows:

Service Oriented: Offers a software service.
Composite: May encapsulate all manner of behaviors including process, utility, core business (data) services.
Enduring: Outlives changes to how it is realized or the business processes that use it
Process Independent: May be used within several business processes
Implementation Independent: independent of how, where or by whom the capability is realized. Does not pre-empt or expose how each action is executed internally. Internal processes could therefore be changed and not impact the user of the capability service providing the contract remains unchanged.
Minimum Dependency: May depend upon another where a) it needs some other capability to have been exercised first or b) where there are internal dependencies. Some capabilities can be independent or self-contained. See below.
Measurable: Performance must be measurable

CBDI recommends:

whilst “not standalone” capabilities may be tactical necessities, standalone, fully independent capabilities are essential for maximum business agility, enabling replication, offshoring, ecosystem partner provisioning etc and reducing the horizon of change
also enabling maximum business agility, capabilities should offer a software service which is externalized, exposing the business capability to a wider world.

Harmon asks for examples. Look no further than Amazon – they offer well-formed capabilities that externalize the Amazon business such as Cloud Service; Storefront etc.

For more on this topic see the October CBDI Journal. It is free with simple registration.

Business Process Trends Advisor - Capabilities Again

The Service Oriented Enterprise

2011-10-13T05:43:00.000-07:00

It’s now fourteen years since CBDI was formed. In the early days our mission was, as a think tank, to evangelize new ideas of modularity, separation, contract based and service orientation.

We had a vision then; it was the enterprise, every enterprise, as a set of services deployed in a cloud.

Of course over the years we have become immersed in the details of best practice, education, and assisting larger enterprises to leverage and realize the SOA concepts. But the vision is still there.

It was around 2005 that we first started thinking about capability as a formal concept. The first report on the subject was in June 2006, and at that time the CBDIers started a long running debate (well argument actually) about this concept. There were two primary issues. First, should a capability be standalone or could it be an integral part of a broader application and technology infrastructure. In the end we couldn’t agree and we compromised in identifying capabilities as standalone or not standalone. I will admit I was in the first camp, and remain there today.

The second issue was should organizations be planning for the entire enterprise to be comprised of capabilities. To be honest this issue received less attention and in the end we never published advice on this. Perhaps it was just too early. Also in those early days there was confusion over capabilities. Was a capability part of a business process, or did the capability subsume the business process? Well of course these questions have long been resolved with experience.

But whilst the capability concept is now widely understood, I wonder how many enterprises have actually taken it to the logical conclusion and organized (their SOA and indeed the business) around them? I see many pretty capability maps, but fewer capability services.

The reason I am returning to this subject is because many organizations are now rebooting their SOA initiative and they should be looking at capability services as the foundation of their architecture. Why?

1. Business capabilities represent a common perspective shared by business design and service implementation.

2. Properly structured capabilities are completely independent (see above) and facilitate change with the minimum horizon of impact. They also support practical analysis and classification of services as core or context which facilitates the sourcing decisions.

3. Looking at how the world works today, the primary strategy must be around ecosystem collaboration. Choose what capabilities are core to your enterprise and specialize in them. Organize around them. Conversely identify the context capabilities and figure out what collaborations will be most complementary and go make partnerships, or offshore or outsource or rent from the Cloud.

4. Plan to implement the entire core of the enterprise as a set of strategic capabilities that are smart, autonomic, and talk directly to the real “end user”. Eliminate all other process interventions and optimize the core capabilities using the very latest technology. Architect and engineer the capabilities to be capable of continuous evolution.

5. Stop talking about SOA, start talking about the Service Oriented Enterprise. It’s a business issue.

We have dedicated this October 2011 edition of the CBDI Journal to this topic. We advise that business design should be synonymous with capability design. In my report on Business Design for the Service Oriented Enterprise, I explore a series of patterns for the smart, continuously evolving, virtual enterprise. I discuss the convergence of ecosystem automation and autonomics, architecture for continuously evolving business, together with the merger of consumer and business IT and how these will have a profound impact on conventional business models, which will in turn affect business modeling techniques and enterprise architecture. In addition we have in depth reports on Capability, Planning and Analysis.

CBDI Journal October 2011

Business Design for the Service Oriented Enterprise

Capability Planning and Analysis

Mission Impossible? Or how to achieve the SOA vision.

2011-10-04T08:31:00.000-07:00

When I am asked about the state of SOA, I sometimes comment that anything involving architectural change is bound to take a little time. But my more considered response would be that whilst the impression of SOA is now widespread, true implementation of the SOA vision, for most enterprises remains a distant vision, if indeed they still remember what that was.

For me the vision was encapsulated in the report by one of our customers on their SOA progress in 2009. They reported their systems were exploding in size and complexity. They had scant standardization, and there was no single truth. If a core process broke they would change it to fit the application, rather than the other way round. This was crazily expensive to maintain. After four years of transformation they report a 20% reduction in IT staff, 1500 systems closed down, the ability to turn services on automatically for customers virtually as they place their orders and a massive reduction in complexity demonstrated by a rental price change that previously required changes to 42 systems – followed by three months of testing, now requires just one platform adjustment that automates the change process. THAT’S STRATGIC!

In contrast I read a Forrester survey[1] from last year that reported while 47.4% of respondents work in organizations where SOA projects are underway, the original reasons for SOA, reuse and cost reduction, have morphed into data integration, legacy integration, flexibility of application development, and department-level application integration. Perhaps this is why we at Everware-CBDI are observing numerous inquiries about “SOA Reboot”, which is variously explained as interest in doing SOA properly, realizing the vision and or delivering real business benefit.

For many enterprises the root cause of this lack of achievement is very straightforward - SOA requires a strategic initiative that looks longer term than most enterprises are able to do. But for most enterprises this is mission impossible, they are bound by short term goals and budgets.

The solution is not rocket science. What’s needed is a governance system that manages a progression from tactical to strategic. Many SOA efforts today are business process project focused, because simply put that’s where the business priority is today. What’s needed is a governance system that ensures project service solutions can be evolved to become enterprise services, where it makes sense. The overhead in making this leap is that a few new policies are needed that spell out better working practices. Consider some candidate policies.

All new components and services MUST comply with a defined minimum level of reference architecture.
Implications and strategy for future service reuse is a REQUIRED element of all Plan or Feasibility phase end reports.
All projects MUST reuse and evolve existing (loose coupled) services and components before acquiring or building new components

There’s more; to make this work needs good governance plus a product (sic) management system in place, because it will get complex. But it works.

I am writing this practice up for the Quarter 4 CBDI Journal. Make sure you are a registered subscriber so you get a copy on publication.

[1] TechTarget/Forrester Research State of SOA Survey for 2010

http://media.techtarget.com/searchSOA/downloads/TTAG-State-of-SOA-2010-execSummary-working-523%5B1%5D.pdf

… so you should do a SOA Certification! But which one? And how?

2011-08-11T06:56:00.002-07:00

After a near death experience SOA is alive and well. After mixed early learning experiences, most enterprises are seeing SOA in a fresh light. Just the other day one company was discussing with me their "SOA Reboot" project! They are in good company.

I observe widespread activity that is based on doing SOA properly a second time around, including portfolio planning, shared services, good governance and not least education and certification of in-house practitioners and service providers’ personnel.

There are several sources of SOA education and certification including ZapThink, SOA School and Everware-CBDI plus several of the larger vendors. At Everware-CBDI we have approached the certification process as a result of our extensive model based practitioner experience and the creation of a comprehensive methodology and approach documented in the CBDI-SAE Knowledgebase. Our customers who have subscribed to the CBDI Journal over 14 years acknowledge us as an authoritative source of in-depth practice guidance for business driven architecture, specification, design, management and governance.

Our approach has been rather different to the other vendors.

1. We realize that while certification is important, it is critical to minimize the cost and particularly the time involved in obtaining a quality certification.

2. We have prioritized making the entire process online from start to finish.

3. Time and cost optimization is particularly important for service providers who are driven by high utilization targets. Equally end user enterprises and government departments are also under pressure and appreciate sharp focus.

4. We don’t re-label videos of Face to Face classes as eLearning; we create short, narrow focused, fully scripted elearning modules that are purpose designed.

5. We organize the modules into syllabuses for different roles. Of course SOA is about architecture, but we package the learning for architects, designers and project managers to make it as efficient as possible.

6. We believe education and certification is part of continuous learning and skills development. That’s why our certification is backed up with the CBDI-SAE Knowledgebase containing detailed meta models, profiles, process descriptions, patterns, task and technique descriptions, together with deliverable templates and other useful tools, plus a vast inventory of guidance. All cross referenced and consistent with the eLearning. A practitioner’s toolkit!

The table below attempts to compare the options available for education and certification. E&OE.

	CBDI	ZapThink	SOA "Schools"	Vendor Training
Online Learning and Certification	☑	☑	☒	☒
Online Price	$1200 or $399 per syllabus	$1995	☒	☒
Integrated Best Practices, Guidance and Resources	☑	☒	☒	☒
Syllabuses for Architecture PLUS other roles	☑	☒	☑	☒
4 day Course Price	$1800 Inhouse only $450/ day for 8 delegates	$1995 $499/ day	$2796 $699 /day	Up to $5000
Self Study Resources	Bundled	☒	$1596 or 50% discount with F2F course	☒
Certification and Exam Cost	Bundled	Bundled	Bundled	$200 - 300
In house Corporate or individual use. Volume discounts	☑	?	?	?
SCORM compliant for enterprise LMS integration	☑	☒	☒	☒
Vendor Independent	☑	☑	☑	☒
Deployed online by world leading service providers and enterprises	☑	?	?	?

Try the sample CBDI eLearning and Certification Modules (no charge registration required)

Cloudburst - Coordinating Vectors for Business Transformation

2011-07-08T03:34:00.000-07:00

I note Tom Graves, Robert Phipps and Richard Veryard discussing vectors as a metaphor for enterprise change. They argue As-Is and To-Be models are way too static. The organization is not a series of snapshots, rather it is a complex, multi-dimensional, interconnected series of continuous changes. I agree.

For some time now we (Everware-CBDI) have been advising and practicing continuous evolution. We realize this concept in modernization projects by baselining the current state and modernizing that; actually rejecting the inevitable change request list. The outcome of the project is therefore compliance with improved architecture, (SOA, WOA, BPM, events, cloud etc) and organized as services, capabilities and components that are a) designed to change continuously and b) managed in an agile evolutionary process and c) with managed, current organizational knowledge.

More recently my thinking has evolved further to consider modernization as a multi-dimensional activity.

The de facto modernization approach today is still a linear development from legacy transformation - in which applications are transformed from a legacy platform to a modern, standards based platform, OK with some optimization of portfolio alignment and EA compliance.

But I suggest we need to go further; to view modernization as a more holistic and multidimensional activity which plans and delivers a progressive transformation of the As-Is business through a series of maturity states on a number of (yes OK) vectors. The vectors may include:

- Architecture: Business, Service, Information, Social Network, Implementation, Technology, Deployment

- Portfolio: progressive componentization and transition engineering

- Service architecture: progressive delivery of the service based business

- Cloud: progressive unbundling

- Standardization and Differentiation: adopting commodity services where it makes sense and delivering strategic differentiation where it is a business imperative. Naturally using cloud as a key enabler of both

- Organization: transformation of IT organization ownership to a core business activity

- Process: transformation to continuous evolution

- and there are many more. Note CBDI-SAE practitioners, we might label these Streams.

My trigger for thinking about the evolution of modernization was the impact of Cloud on modernization practices. I realized that we needed a fundamental rethink based on a pattern that would be better described as Cloudburst rather than Transformation. Cloudburst suggests a dramatic unbundling; the breaking down of conventional boundaries and forceful redistribution. Of course Cloud is not the only trigger, but it is evident that it is going to have profound impacts on everything we do.

So back to the vectors. These are important and provide context for the unbundling of the current state - distributing and componentizing capabilities across architecture layers, suppliers, technologies, processes and ownerships. They provide context for change and ensure we don't remain isolated looking at the world solely from our own specialism and perspective.

Relevant Blog and Research Note: Integrating Strategic Planning for Cloud and SOA

Service Oriented Cloud (SOC)

2011-06-23T10:08:00.000-07:00

I am almost shocked by the vast volume of tweets hitting #Cloud. Of course it’s a reflection of the frenetic level of interest in the subject. But it’s also because Cloud Computing is such a huge, complex domain.

Following in the well-trodden path of many new information technology concepts we might expect morphing to occur. Much of the Cloud focus has been about infrastructure and technology, plus commodity applications, productivity tools and multi-tenant Web applications. Whilst all the really good Cloud environments are Service Oriented, it’s very much the minority of consumer SaaS that is today.

Yet it’s very obvious the next stage of Cloud will be about enterprise services. And as private and virtual private Clouds become respectable and trusted, we should expect a huge push by enterprises to demand modernization and rationalization of application landscapes into the Cloud with cost and agility objectives in mind.

But while everyone calls everything a service there’s potential for huge confusion. Further everyone needs to know that right now few existing applications, regardless of how recently they have been “modernized” are Cloud ready. Regardless of private or public Cloud deployments, they need to be genuinely secure, componentized and service enabled and many of them need to be multi-tenant architecture if they are to deliver the expected cost benefits.

To differentiate between the morass of stuff that’s happening and what’s needed in a genuinely Cloud ready SOA environment, I propose we start right now referring to the SERVICE ORIENTED CLOUD or SOC for short. It’s small step, but it will make life easier for everyone, and indeed allow those of us focused on the SOA enabled SaaS layer to have a nomenclature that works, as opposed to continuously committing unintended double entendre.

I recommend we start by hitting #serviceorientedcloud

Integrating Strategic Planning for Cloud and SOA

2011-06-15T08:13:00.000-07:00

In October 2010 Gartner Group assessed that Cloud is at the top of the Hype Curve, the peak point of inflated expectations. But no one really needs this authoritative assessment to confirm an opinion that the industry in general is still in the early stages of Cloud Computing. An even more recent survey reveals that just 7% of organizations have approved cloud strategies and most organizations are now planning to establish strategic plans. So notwithstanding the cautionary reports, most government and commercial enterprises are planning to move rapidly to embrace the profound change inherent in this computing model.

As many organizations embark on strategic planning for Cloud Computing they should be strongly advised to think hard about the scope of their planning activity. What’s becoming very obvious is that we are in the midst of a significant paradigm shift. Cloud Computing is a major change in its own right, but in fact it is essentially only one phase in a much longer running cycle that commenced back around the year 2000 and continues to transition the entire industry from monolithic IT at all levels to service oriented everything.

Whilst Cloud is the “trend de jour”, a study of the leading platform offerings, particularly Amazon, Microsoft and Oracle, shows that Cloud is synonymous with SOA. These leading platforms are all completely service oriented and demonstrate sophisticated, next generation SOA implementations. All the platform capabilities are delivered and consumed as services and whilst the end user may choose to deploy conventionally architected applications, they would be highly suboptimal in terms of cost, portability and business agility.

At this juncture we also need to revisit the question of what is the Cloud. Not surprisingly, like most important trends, the concept is morphing as it matures. In the early stages of Cloud, the emphasis has been heavily focused on cost restructuring particularly in the areas of automation and standardization of technology infrastructure and rationalization and optimization of those resources. The PaaS and SaaS layers have been primarily used for conventional Web and commodity applications such as office productivity and email.

But the next stage of Cloud will be focused on business services and the operation of an ideal service architecture which rationalizes the morass that is the typical enterprise application portfolio. In this process the Cloud based services will align with customer facing business services to form the service oriented enterprise.

One might even wonder if the term Cloud will be relevant in just a few years time? The term is already coming under pressure because Cloud covers such a multitude of architecture concepts. Even the early standards work undertaken by NIST is rapidly dating as for example multi-tenant is no longer a unique identifying characteristic, and as the base models of private, hybrid and public are changing with the evolving with the development of the Virtual Private Cloud.

It seems probable therefore that the industry will go full circle and Cloud will be a primary enabler of the Service Oriented Enterprise, or, heaven forbid SOA 2.0.

This discourse is not wild speculation, IMO it is high probability. As a consequence I recommend that strategic planning for Cloud should be fully integrated with SOA planning. This is particularly the case when strategic ambitions are broader than purely technology infrastructure rationalization.

Further details see my Research Note:
Integrating Strategic Planning for Cloud and SOA

Cloud-SOA Meta Model L1

2011-06-06T06:53:00.000-07:00

There's a lot of loose talk about Cloud and SOA. From what I see most SaaS today is not SOA. In fact it's mostly multi-tenanted Web applications with a browser interface. This "may" be satisfactory for SMEs but for larger enterprises this likely to be unacceptable. The primary inhibitor delaying enterprises migrating to the Cloud is less likely to be security than portability. And here enterprises need an architecture driven approach that forms the basis for good governance at all levels.

In retrospect, when we required effective governance for SOA, as an industry we developed rigorous models and profiles that allowed us to establish repeatable structure for deliverable and governance tooling. For Cloud we need to develop those vanilla SOA models to incorporate new classes and relationships. Interestingly, whilst there are some very important modifications to the base SOA model, for the Consumer perspective the differences are quite limited.

I have posted a discussion draft of an L1 Cloud Meta Model. Note this is a Consumer View and is a conceptual level model, based on the CBDI-SAE V3 Meta Model, intended to explore the key concepts and relationships for scoping purposes. Of course each package needs to be extended and or developed. I will be documenting this and providing explanatory materials in an upcoming CBDI Journal article. Meantime you can download the base CBDI-SAE V3 model and specification for definitions.

Be very interested to get feedback.

NB: See earlier posts on this topic.

Industry Cloud models are all very hazy!

2011-06-02T06:44:00.000-07:00

I am currently working on extending the CBDI-SAE meta model for Cloud. I started this exercise by making some specific assumptions:

1. I am interested in the larger enterprise perspective; enterprises probably need more knowledge of the underlying capabilities than SMEs if for no other reason than good governance

2. For similar reasons I am exploring primarily the SaaS and PaaS layers because most enterprises will usually outsource the infrastructure, even if its private.

3. And in defining the meta classes for these layers I am interested in supporting an application modernization process and life cycle because, again, that's where most enterprises are at in their Cloud efforts.

In looking at some of the excellent work that has been published already on Cloud, particularly by NIST, I was struck by the Provider centric focus of all the deliverables. I suppose I shouldn't be so surprised because most of the work is being supported by service provider based people. Yet, in context with my assumptions above, I am finding the models not as helpful as they might be. The NIST Reference Architecture is mostly Actor centric, yet the Provider capabilities are expanded, whereas the Consumer, Broker and Auditor are either not expanded at all or at a very superficial level. Well it's all superficial, but nevertheless it presents a very unbalanced picture.

So pretty soon I decided that (at least) my initial model would be a Consumer View - because that's the primary requirement of enterprises. And of course I find a high level of overlap with the SAE model because the Cloud from the Consumer perspective is all about services. But there are some interesting alterations and extensions that I will be detailing in due course.

If anyone is working on meta models in this area I would be very interested to hear from them.

SOA was yesterday’s issue?

2011-05-31T02:57:00.000-07:00

Don’t you believe it! For many larger organizations SOA is only now becoming central to key business strategies. Whilst SOA has been around for almost a decade, few companies have achieved real maturity. Governance issues in particular have allowed fragmented service oriented solutions to predominate. Even where consistent service facades have been established, the underlying legacy application estate represents a colossal inhibitor to significant or rapid change.

Today SOA is a major priority for:

- delivering smaller application components for effective Cloud computing deployment
- and thereby modernizing the application estate, reducing costs and enabling . . .
- flexible service support for strategic BPM projects and programs

To achieve SOA maturity, organizations need much improved SOA skills. Not just for architects but for all roles, planners, business analysts, architects, project managers, procurement specialists, designers, testers and so on. Recognizing this, at Everware-CBDI we have been focused on creating skills development products that address this need. We have created a comprehensive eLearning portfolio plus certification modules. These have been developed to minimize time and cost while focusing and maximizing learning. They are not warmed up videos of face to face classes requiring many hours to view.

The products are now being used by very large service providers and enterprises. Last week we launched an on-line store, and these can now be procured by individuals in a cost effective manner.

SEE DETAILS, TRY and BUY at the new online store.

Beware gurus promoting their new book on SOA Governance

2011-05-19T04:15:00.000-07:00

SearchSOA.com interviewed co-authors Thomas Erl and Anne Thomas Manes to discuss their recently published book, SOA Governance: Governing Shared Services On-Premise and in the Cloud (Prentice Hall, April 2011). The Q&A didn’t encourage me to buy their book.

Question: How you define SOA governance?
Answer: Manes “The quick definition is 'governance makes the rules.' . . . the best governance system is one that people appreciate, that helps people get work done, with the highest quality, and that is beneficial to business.

Commentary: Governance doesn’t just make the rules. SOA Governance is a process that guides policy setting and compliance to ensure service and solution delivery and operations are delivered and remain in compliance. SOA policies are architectural, practice and organizational decisions designed to deliver business value through application of key principles.

Question: You say the first step in an SOA governance effort is to establish a SOA governance program office. What would that consist of?
Answer: Manes “Having a recognized office with the authority to establish rules is a prerequisite for any program. “Erl “The SOA governance program encompasses the governance system, but also all the other logistical aspects of that system, so project plans, roadmaps, tools, and the steps to make it part of the overall means by which projects are regulated.”

Commentary: The first step in SOA governance is to publish policy for delivery programs and the framework of deliverables and responsibilities for publishing and compliance reviews. This activity is best done under the auspices of an existing governance practice. Of course it requires SOA specialist skills, but the objective should be to integrate SOA into business as usual as soon as possible, not to create separate organizational structures that create and perpetuate divergence, or even worse, to consolidate all project practices under governance.

Question: Is there such a thing as agile governance?
Answer: Manes “When defining your precepts you need to constantly be willing to reassess if you’re achieving your goals, by reevaluating and understanding if they’re helping you deliver better solutions more quickly and then going back [and addressing them if they’re not]. That’s the definition of agile.” Erl “The governance system for an SOA initiative needs to be inherently responsive to business changes. To me, a governance system improves the responsiveness of those that function within it because so many decisions have already been made.”

Commentary: This sounds like a recipe for encouraging “policy waivers”. We’re doing Agile, so we can’t comply with the policy! If there was something called Agile governance in my opinion it would apply review/gate criteria that ensured that delivery projects are using agile approaches in a prescribed manner that delivers good business outcomes, specifically – systems, services and ongoing support processes that can evolve at minimum cost and cycle time. This generally requires that SOA delivery programs implement twin track (service and solution) delivery, strong componentization of implementations, iterative delivery of baselined, well architected functionality that is designed to evolve on a continuous basis.

Question: What are the governance considerations specific to cloud-based services?
Answer: Erl “When your resources are hosted by a third-party cloud provider, there is a limitation to the extent of control as to how they can be governed. That’s something to factor into the governance system. If your governance says it has to comply with industry standards and the supplier doesn’t support that, you have to say ‘How can we [govern] within the cloud given these constraints and still be in support of our business requirements?’” Manes “You have more inherent risk in deploying in the cloud and need to take appropriate risk mitigation by deploying [governance] precepts specifically for the cloud.”

Commentary: For many if not most large organizations “business as usual” is delivered by outsourcing and service provider suppliers. The governance of outsourcing will normally be focused on outcomes and risk management. The governance of Cloud based SOA is an extension of the same process – focusing on encapsulated service contracts and bullet proof service level agreements. This is not to trivialize Cloud specific issues, rather to push back against the trend that says we have to reinvent everything we do for Cloud. That’s patent nonsense, we should use proven best practices and architectural patterns to get Cloud deployments up the maturity levels as soon as possible.

Disclaimer: Please note I haven't read the book. I see there are numerous authors involved in this work and my comments are based entirely on the Q & A which, as I said, didn’t encourage me. Which is a shame for the other authors.

Version 3.0 of the CBDI-SAE UML Profile for SOA

2011-05-11T03:02:00.000-07:00

Everware-CBDI has announced the immediate availability of the CBDI-SAE UML Profile for SOA V3 (SAE Profile). The SAE Profile makes Model Driven Architecture (MDA) for SOA practical for everyone. Whilst introducing a sensible level of compliance with industry standards such as SoaML, the SAE Profile provides a more extensive and detailed coverage of the complete lifecycle, from business models to deployment. As well as capturing requirements in a precise manner, the ability to model service architectures and service specifications facilitates SOA governance with the production of more formal models and other key deliverables that conform to a detailed meta model.

Modeling Service Architectures and Service Specifications
Making MDA for SOA practical for everyone
Enabling sensible compliance with industry standards such as SoaML
Facilitating SOA governance, with production of formal SOA models and specifications

The CBDI-SAE UML Profile for SOA V3 is now available for download from the Everware-CBDI website. The profile is available by simple, no cost registration with the CBDI Forum, or login by existing members.

The SAE Profile allows architects and designers to use UML tools such as Sparx Systems Enterprise Architect and No Magic Inc's MagicDraw and to create purpose designed, consistent deliverables for SOA.

The SAE Profile is based on the CBDI-SAE methodology which is defined in the CBDI-SAE meta model for SOA. This is a detailed set of meta class models that define SOA concepts at a level of precision suitable for project deliverables. The models are broadly scoped to integrate with architecture, design and solution delivery practices and span the entire life cycle.

Making the CBDI-SAE Meta Model for SOA available as a UML Profile enables users to model SOA design and architecture using diagrams that are UML compliant and to progressively define detailed meta data that can be used directly in key project and governance deliverables including all architecture views, service specifications, implementation, technology and deployment specifications.

A new report “An Introduction to Service Architecture Modeling with the CBDI-SAE UML Profile for SOA V3” provides guidance on how to use the SAE Profile and walks through the process of modeling a service specification architecture.

Microsoft buys Skype!

2011-05-10T07:44:00.000-07:00

So Microsoft is to become the new owner of Skype! This is big news for us that run our business and personal communications on the platform. I suspect we were all relieved when eBay finally recognized the inevitable and sold a majority stake to investors who have largely trodden water with the product while they figured out how to realize on their investment. So is the Microsoft acquisition good or bad news?

Good News? Microsoft will probably be a good custodian because they are primarily concerned with leveraging the capability to strengthen their social networking products, and less interested in direct revenue return. So hopefully impact on our free communications network will be minimized. Similarly Microsoft will be keen to integrate Skype with social network products, telecom and video networks and gaming. These will probably be more important than integration with legacy integration with Windows.

Bad News? Where I would like to see Microsoft go with Skype is integration of various communications channels. The (industry analyst) party line is that communications futures are likely to center around Facebook – and I will freely admit I see this as a nightmare. What I would like to see is much more integration between Outlook, Gmail, Hotmail, Skype, Twitter etc. This could be an amazing way for Microsoft to move beyond it’s Office legacy environment, and to establish a world class, multi channel communications system. But sadly Microsoft has two major limitations that I expect will come into play:

1) Microsoft does proprietary and competitive. Expect them to strengthen Hotmail and other Microsoft specific links in order to compete more aggressively with Gmail. Skype may become more proprietary and less open. Will Skype remain on the iPhone?

2) Even within their own world, Microsoft doesn't do cross product integration. This is because of the way that Microsoft is organized. I recall years ago asking Bill Gates the question "why didn't MS formally componentize its portfolio?" His answer told me all I needed to know - he doesn't believe in portfolio level (enterprise) architecture, and optimizes at the product group. And I am not sure MS has changed. So IMHO the likely outcome is that Microsoft will move Skype into video and gaming and in that way make some revenue. But they will miss the strategic opportunity to go head to head with Google and Facebook and at the same time rejuvenate it's legacy portfolio.

Still it will be interesting to watch.

Parallel Universe Syndrome?

2011-04-19T07:46:00.000-07:00

I have become increasingly concerned that in the typical enterprise the major business improvement disciplines operate to some extent as silos. EA is often disconnected from Business Architecture. BPM is frequently not well connected with EA and Application Modernization. Governance is commonly applied at discipline level rather than being coordinated across discipline. I have blogged on these topics recently in Beware the New Silos! and Silos What Silos!

The term Parallel Universe comes to mind in which a hypothetical self-contained separate reality coexists with one's own. Although strictly we should refer to this as a Multiverse, let’s not get too technical. Think about it for a moment; disciplines are inward looking, they have their own language, industry standards, specialists, champions, culture and sponsors. In fact they have a significant critical mass of their own. Furthermore we can observe standards bodies for the various disciplines expanding their scope to encroach on the natural space of other disciplines, instead of establishing agreed boundaries and coordination mechanisms.

In the BPM world there appears to be a profusion of frameworks, all of which cover similar ground, but no real convergence. Other disciplines have equivalent frameworks, and they are similarly discipline centric. No surprise here. They all use different perspectives to manage dimensions of a common business objective. The issue is that all these frameworks need to work in a coordinated, networked manner. But in addition to the culture, language and other areas of difference mentioned above, they operate on different life cycles and timescales in a concurrent but often conflicting manner frequently with very fragmentary coordination.

I don’t believe we need yet another framework to address this question. There are a huge number of possible dependencies and interfaces, and given the heterogeneous nature of the overall business improvement environment, it seems preferable to encourage outcomes not techniques. What I propose is that because governance is typically exerted for each discipline, what’s required is a set of governance review criteria for each discipline that raises the key questions that allows the organization to monitor and govern across discipline.

In this month’s CBDI Journal I have introduced such a cross discipline governance system. I have suggested a simple approach which extends the widely used COBIT governance framework. There’s no attempt to say “how” the disciplines should work together; simply to focus attention on the outcome for the collaboration to ensure that appropriate coordination is considered.

So Nobody is Doing EA Eh!

2011-04-08T02:30:00.000-07:00

I get rather tired of the “Enterprise Architecture doesn’t work” debate. The latest comment from ZapThink is predicated solely on the fact that “architecture must precede build”, therefore because an enterprise, by definition already exists, architecture is redundant. In the line of facile arguments this one must rank very highly.

Simplistic analogies never work. Architecture for bridges or buildings doesn’t scan. In the physical world there are any number of examples of evolutionary enterprise architecture in which the product is evolved on a continuous basis. Think airports, highways, networks etc.

For years we at CBDI have been advising (and indeed practising) how to modernize portfolios of applications and technology in order to establish continuously evolving capabilities. Of course we practice enterprise architecture, but it’s about clear understanding of the levels of abstraction that provide minimum necessary detail to plan and execute the next phase of activity in an agile, iterative process that is common to initial build and ongoing evolution. And even more important, it’s about triage – understanding what elements must have an enterprise context and what elements are entirely local.

All this bottom up and top down, inside out, emergent and so forth are so much rubbish bandied about by those that need to spend some time breaking rocks. Think continuous evolution of business architecture, enterprise architecture and solutions. Rant over!

Silos, what silos?

2011-03-24T05:55:00.000-07:00

Following on from my blog on "Beware the new silos!" today I was speaking with a business process analyst who briefed me on how his company, a medium sized financial services organization, is using a leading BPMS together with a collaboration based tool. He described how they are doing amazing, utterly brilliant work in the business process layer, but appear to be oblivious to everything else. They have no databases let alone information architecture, no services or service architecture, no applications and no governance and no interest from business or IT management in anything other than delivering business processes. Everything is managed in the one platform and organized in business process silos. He commented, “this is a major train wreck waiting to happen!”

Beware the new silos!

2011-03-18T10:06:00.000-07:00

Coordinating EA, SOA, Business Design, BPM, Business Capability Management and SOAM.

It’s a complicated world, and one of the primary ways we all cope with that is by specialization. For years I have characterized myself as a specialist in SOA; others will specialize in EA, BPM, PM, etc. But over the last year or two as SOA has moved into the mainstream we at CBDI have broadened the scope of our consulting, research and frameworks in both life cycle and discipline dimensions. The new scope includes practice support for standards such as ITIL, TOGAF, COBIT, eSCM and disciplines such as project management, transition management, BPM and testing, to name just a few. But it’s increasingly clear that the industry generally, as well as individual enterprises, is at risk of creating silos of the primary disciplines.

Many enterprises are now embracing the BPM discipline. After many years in the doldrums, the BPM market is unmistakably maturing rapidly. Acquisition of smaller, best of breed product suppliers by the major vendors must have been a primary contributor to increased activity, coupled with strong business pressures to rationalize and modernize business processes.

Similarly there is evident maturing of SOA. Again after many years of strategic commitment without effective governance, it is clear that SOA is now mainstream for many enterprises, a mandatory architecture pattern and technology for all projects and programs. We might discern two interrelated demand signals to:

1. deliver services from disparate back end systems to support BPM projects.
2. modernize application portfolios (rationalize, standardize, move to cloud) to reduce cost.

In practice these initiatives are frequently operating as silos with different sponsors, scopes, timescales and different objectives. To compound this there are other critical activities in EA and Business Design, which should be exerting governance and coordination but without due care and attention these may also be operating independently.

It’s not surprising these various silos can emerge almost by accident. The genesis of the disciplines is inward looking. Standards and frameworks have emerged that are primarily about each discipline. Of course, standards bodies typically compete not collaborate, and conflicting standards for the “same” discipline merely complicate the issue. Collaboration over inter-discipline standards is immature, and it’s common for standards some bodies to simply expand their efforts to cover new spaces without regard for clear separation of concerns.

A similar picture may be observed in many enterprises. In most organizations champions of a discipline emerge and there is healthy competition for mindshare, primarily of course with the status quo. The relative maturity of emerging disciplines will also be highly variable. It would not be uncommon for a single enterprise to have excellent experiences of mature and successful BPM projects with relatively immature SOA capability for common services and governance. And vice versa!

What’s required is a governance driven approach that provides just enough coordination to ensure the disciplines collaborate in an appropriate manner. In fact we recommend you should view the disciplines as components connected by services. Yes it’s an SOA! See image.

As the diagram illustrates there are three major tracks in play here – business architecture, business modernization and application modernization. Underlying all of these is SOA. In recent years there have been some attempts to recast EA as business architecture; but rightly these have been rejected because of the deeply entrenched IT perspective of archetypal EA. Rather what’s required is outline architectures for business and IT that can coordinate portfolio development and program management across the BPM and application domains.

Business modernization as the name implies is the transformation of business processes. Naturally these initiatives will span logical business components and domains. More worryingly they will look at slices of larger problem spaces, and so the need for coordination is paramount. BPM creates the demand for application and capability services and here each BPM project will frequently represent demand for a particular slice of a shared service. We recommend there should be a separate discipline for Business Capability Management which is equally as important as BPM. Consider a twin track (supply – demand pattern) operating between BPM and BCM in which BCM supplies cross cutting business capabilities that enforce enterprise standard behaviors and information.

The second important twin track is between Business Modernization (BCM plus BPM) and Service Oriented Application Modernization (SOAM). The SOAM discipline is tasked with meeting the BPM/BCM led demand, within the context of the EA, Business Architecture and SOA.

The SOAM needs to be agile and iterative implementing the service architecture on an evolving basis while progressively modernizing (rationalizing, componentizing) the application portfolio. But even more important, the SOAM needs to establish a capability to continuously evolve modernized application and service capabilities that can meet Business Design, BCM and BPM demand with the minimum cost and cycle time.

So it’s more complex than simply staging service delivery to meet BPM project demand. But I am not proposing an umbrella framework; rather I am recommending that disciplines are managed as components and offer and consume services from the other components. This approach will constrain scope creep in each discipline and allow specialists to work with defined dependencies with others.

In addition to services offered and consumed we should also place policy responsibility and authority upon each component. For example (at the aggregate component level):

Business Architecture: standard business capabilities; standard business rules; standard business process behaviors; standard business semantics; shared service portfolio . . .

Business Modernization: business process ownership and design; business capability ownership and design; context specific service contracts; delivery priority . . .

SOAM: service contract design and ownership; component boundaries; supply contracts . . .

I will expand on these concepts, plus detail the services and policies for each of the disciplines in detail in an upcoming report to be published in the April CBDI Journal. You may subscribe at no charge here.

Bridging the gap between abstract SOA models and reality

2011-02-17T01:33:00.001-08:00

Many organizations tell us they are classifying SOA as mainstream – that is the architectural pattern and associated technologies are mandated for all projects and programs and strategic applications are progressively being modernized with service interfaces. Further, as organizations’ use of SOA matures, we observe increasing commitment to common, cross-cutting capability and core business services which naturally lead to standardization of the service portfolio.

This progressive maturing of SOA capability requires consistency of approach across disparate programs that facilitates collaboration and effective governance and naturally creates the requirement for standardization of reference models and reference architecture.

Whilst there is a plethora of reference materials from the standards organizations such as OASIS, The Open Group, The OMG and ITIL(OGC) it’s clear that in addition to inconsistency of definitions and expression, most of the standards are abstract and narrowly focused on core concepts and ontology. Whilst these standards are often very useful in guiding conceptual understanding, they may not provide the detailed models on thebroader scope required by practitioners to establish best practices for architecture and solution delivery teams.

What’s required from an SOA reference model:

- a basis for common agreement on concepts across disparate groups that allows sharing of models.

- at a level of detail that is unambiguous and supports tooling. A fully detailed UML model, defined, enumerated and attributed.

- across a scope that supports a typical end to end business project covering: business models, service specification, service implementation, solutions, deployment and runtime, technology, testing, policy and organization.

- an attempt at some level of compliance with the various standards groups, recognizing that there is no single model or agreed ontology, nor standardization of definitions or expression.

In order to support use cases:

- integrate SOA into wider business practices:

- - - - enterprise architecture

- - - - solution architecture

- - - - business modeling

- - - - BPM

- - - - application delivery projects

- support common service definition across ecosystems such as industry groups, supply chains, business partners

- define policy relating to reference model compliance (and thereby support governance of same)

- define required traceability

- support seamless interaction between teams (and parties) carrying out business modelling, service architecture and design, provisioning/procurement, implementation, test, service management and operation

- eliminate ambiguity in service agreements with outsourcing and offshore parties

- support common schema definition for what may be a disparate set of tools being used in modeling, asset management, cataloguing, version and configuration management

- support definition of a set of common and or project specific deliverables across the architecture and delivery life cycle

- support creation of UML Profile or domain specific language (DSL)

The Everware-CBDI SOA meta model attempts to meet the above requirements and support these use cases. You can download the V3 model here.

Beyond IaaS - the Business Service Network

2011-02-15T04:26:00.000-08:00

I note Joe McKendrick at ZDnet asks "Will telcos become the Cloud brokers for integration as a service". It's a good question. In fact it's a really good question that I posed back in 2005 in a CBDI report titled Enabling the Business Service Network. In the report I said:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Over the next decade there is high probability that large and small enterprises will transition away from providing their own service collaboration and management infrastructure.

A critical element of all these infrastructure process improvement initiatives is to enable consistent information on the infrastructure capability and status that supports policy driven deployment. Although it might seem too simplistic, the overall ambition of these process and technology improvement initiativesis to turn the IT capability into a black box that largely self managing.

IT resources are shared, not isolated or application/organization specific
Resources and services are managed according to predetermined policy
Unpredictable demand is managed to deliver predictability and consistency
Operational costs are reduced through automation and reduced human intervention
More complex architectures and scalability are delivered without proportionate dependency on people and skills
Error and failure rates are reduced
Standardization increases operational efficiencies
Infrastructure knowledge is collected in a single information model
IT services are enabled as self service to users
Service levels are improved by dynamic change of services
Increased agility is achieved by rapid provisioning of new services or resources and reconfiguration of existing services

The Business Service Network (BSN) acts as an intermediary between service providers and consumers providing many value adding services that make service execution more reliable, controllable and adaptable. The BSN architecture is independent of location and organization and by design capable of supporting continuous change in the usage, behaviours and deployed resources of the business service and supporting technical infrastructure.

The basic BSN architecture is an SOA at both business and infrastructure level. All capabilities are implemented as services in a layered architecture in which each layer encapsulates lower layers, simplifying utilization and management. Individual services form components which provide atomic capabilities that can be reused in many situations and create an inherently adaptable architecture. Provisioning is an assembly based process in which services are contracted to provide resources to specific assemblies on a dynamic basis.

But the BSN architecture is more than an SOA. The BSN must automate the entire life cycle processes of business and infrastructure service delivery and should do this with minimal human intervention. To do this the SOA needs to be complemented with an Operational Support System, a comprehensive management environment that controls the supply/demand of resources to services.

The BSN architecture represents a significant advance on discrete SOA and OSS architectures because the consolidated view of business service and infrastructure layers creates opportunities for advanced management and control capabilities, which are much more difficult to deliver from separate component parts. These capabilities are enabled by the shared information model that maps the physical resource description and usage model to the logical assembly level model, that in turn is mapped to the business service viewpoint. The shared information model allows static and dynamic metadata from all the technical domains in the aggregated stack to be pooled and used for end-to-end management control purposes including:

· Comprehensive dashboard and reporting structure

· Business policy driven infrastructure – a clear linkage between the business service perspective and the technical infrastructure and resources allowing management of resources for specific services. For example policies (rules) may be defined that call for levels of resource to be made available dependent on business service traffic.

· Infrastructure resource status influences business decisions (e.g. allocation of call centre resources)

· End to end Service Level Agreement (SLA) – a common SLA structure internal to the BSN in which all collaborating components integrate with a common SLA protocol that allows aggregate SLA status reporting on a common basis

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

However from what I can see, the really important point that I made in 2005 hasn't been picked up - this isn't just about a broker pattern for integration as a service - it's the provision of an END TO END SERVICE that delivers an SLA including the network and security. One of the biggest worries about Cloud is the probability of becoming locked in to single Cloud providers because it's the only way to deliver security and end to end SLA. However in my BSN model the telcos provide that single integrated SLA covering the totality of the service provision. Back in 2005 I said it would take a decade to realize this, and it seems I wasn't far wrong.

David

See the January 2011 CBDI Journal for other reports on Cloud

Head in the Clouds?

2011-01-28T02:43:00.000-08:00

Cloud is confusing, period!

· Is SaaS Cloud? Or is it multi-tenancy ASP?

· What’s the difference between Server Virtualization and Private Cloud?

· Where are the standards in the Cloud?

· Is SOA a prerequisite for success in the Cloud?

· Is an ESB required in the Cloud?

· Is the Cloud secure and low risk?

· Will Cloud mean the demise of the IT Department?

Many enterprises are exploring Cloud computing. A select few vendors, such as Amazon and Google, are already delivering serious revenues. Others such as Microsoft, IBM, HP et al view Cloud as central to their long term strategy but are less far along the road. Many smaller enterprises are embracing SaaS, often as business driven initiatives, but may find they are up “dead end street” when they try to integrate with core business applications. Most large enterprises are going slowly down the early learning route with Private and Hybrid Clouds.

It’s clear Cloud is still at the top of the hype curve. There’s huge debate and uncertainty which is inevitable before a new concept stabilizes. One of the primary issues is that Cloud is all inclusive of many existing technologies, including grid, virtualization, automated infrastructure management, ASP etc. So many organizations find it hard to understand what’s on offer.

For large enterprises particularly, the question is highly relevant because they are under pressure to reduce costs and increase time to market, but the security issues surrounding Cloud are self evident. If an enterprise implements VMware-tuned servers with automated, self service provisioning for multiple internal user usage, surely that’s the same as a private Cloud? And the answer is a qualified yes – because even if end users are charged on a pay per use basis, someone has had to make the investment to establish the capability. And the investment costs, as well as the additional capacity to allow for elasticity, have to be shared within the enterprise.

Cloud is clearly a major trend that will eventually have profound implications for IT architecture, economics, organization and markets but its still in a very early stage of evolution. The level of uncertainty is very high, perhaps because it’s becoming clear that Cloud will not be a simple market similar to other utility industries such as electricity. Rather it will be a complex market with many layers and points of entry that mirrors the complexity of the IT product stack.

In this edition of the CBDI Journal (free with registration) we commence with guidance on definitions of the components of the Cloud. Clearly large enterprises will have complex Cloud strategies that comprise a mix of solutions, and a need to sort out the nomenclature, not just of the Cloud, but also of the momentum business which will include many of the Cloud technologies deployed as point solutions.

Then we explore a Business Driven Cloud Strategy - how large enterprises should make explicit links between their Cloud computing strategy and their business goals in order to move beyond the tactical and technology centric activity that is inevitable in early markets. Of course nearly everyone (well in large enterprises anyway) is focused on Cloud as a technology for infrastructure because this represents low hanging fruit. And that’s fair enough because there’s a lot of learning to be done. But we recommend strongly that this needs to be undertaken with an eye to the future roadmap. Just consider at what point did Amazon realize it had a major new business in prospect?

Finally in the third CBDI Journal report this month we push the envelope further on information architecture in context with responsive business processes. Over the years we have published several reports on the intersection of information architecture and business intelligence, and tooling is now available to realize the concepts we have been exploring. We illustrate how business process behaviors can expand beyond the transactional to incorporate rules and automated, real time responses to a wider range of events, rather than after the fact from data warehouse or similar systems. Clearly in order to do this we need to rewrite the book on how we architect data requirements for the business process.

Defining Cloud

2010-12-20T13:02:00.000-08:00

I observe considerable confusion over "what is the Cloud?" The absence of industry standards in this space is clearly a concern. In the early stages of any new trend there is a confusion effect as the industry (formally or de facto) decides what’s what. Of course it’s also interesting to note how new trends merge into the stack, often without discernable separation in a relatively short time.

In context with Cloud, the NIST criteria seem like a good start.

- On demand self service, Broad network access, Resource pooling, Rapid elasticity, Measured service

These components are actually useful as a starter set of principles. In the same way we defined SOA in terms of principles I would suggest we do the same here. I suggest Cloud Principles might include:

The provision of virtualized, encapsulated services
Self service, on demand consumption
Multi-tenant, dynamically scaled resource pool
Managed and monitored
Pay as you go usage
Using industry standards

I accept the industry standards is, as yet, a bit of a stretch. . .

I wonder if the perceived confusion de jour arises because we are focused on the current maturity model state – infrastructure provisioning. My contention is that the next stage will add commodity business services (of all sorts not exclusively Web Services and SOA) and the provisioning will morph to include self service, assembly and orchestration. In fact we are seeing this already in BPaaS. In this model many SaaS providers including Salesforce would qualify.

Cloud Maturity Model

2010-11-29T02:46:00.000-08:00

Over 7 years ago CBDI published its SOA maturity model and roadmap methodology. Over the years it has been widely used by larger enterprises as well as cloned and copied.

Those familiar with the model will recall that the maturity states use service usage patterns to characterize relative maturity - Early Learning, Integration, Enterprise and the mature state is referred to as the Ecosystem, which “follows” the Enterprise state.

Over the years I have worked with many organizations, assisting them to develop SOA maturity models and roadmaps and in most cases the focus has been on achieving Enterprise capabilities. The Ecosystem, while interesting, has been way beyond most enterprises' vision. That situation is changing, and it's interest in Cloud that is the catalyst.

Today the cloud is in a relatively immature state. But at some point the cloud and SOA maturity models will converge and at that point we should fully expect ecosystem like behaviors including commoditization and the concomitant price and cost reduction, standardization of higher order levels of functionality and massive reduction in cost of integration and deployment. At this point the changes to the IT landscape will be profound. The massive swing to specialized service providers that has happened may well be significantly reversed as mobile enterprise, functional standardization and improved economics become normal.

As usual profound market shifts of this nature will cause rationalization of market players – and in the case of cloud it’s by no means certain that big will be beautiful – because the nature of the cloud can actually favour small players.

It’s not too far fetched to imagine cloud based innovation driving macro economics, in the same way that in the past players such as Microsoft, IBM and Apple to name just a few have had deep impact way beyond their conventional market space.

We are still in the stage of maximum hype. Think SOA - it took 12 years to get here, and we are far from done. Cloud has similar architectural impacts but it's the economic and organizational impacts that may actually take SOA on to the next stage. And that was why all those years ago, I and my colleagues envisioned Ecosystem as the mature stage of the SOA maturity model.

I am working on ideas for updating the CBDI Maturity Model and Roadmap to incorporate and detail Cloud based capabilities, and of course to explore the wider capabilities. I hope to publish some initial work in the December/January CBDI Journal. I would be very pleased to hear from anyone who has interest in this area. I have some questions and would be pleased to dialog on same:

1. Is Cloud part of the SOA maturity model and Roadmap? Or is it a separate, parallel model?

2. What are the major stages in Cloud capability maturity?

And a related question - 3. How and when will the security issue be resolved?

4. How and when, if at all, will standards play a part in the Cloud? Looks a bit like the wild west today!
5. What does the Cloud cost benefit model look like?

Interested in Cloud? You may like to read the CBDI Report on Service Portfolio Planning and Architecture for Cloud Services

Minimum Set of SOA Standards?

2010-11-19T02:23:00.000-08:00

This following is my reply to one of our customers that asked me to review an SOA standards document. For obvious reasons the client must remain confidential, but I am publishing this as my replies may be of interest to a wider audience.

Dear Xxxxx, The SOA Compliance Standards and Guidelines document you have provided me with for review is a good start. It addresses many of the technical level matters that are needed to create a consistent SOA. However, in my opinion, the document does not go far enough to establish a bare minimum set of standards.

The Reference Architecture illustrates a layered service architecture separating service behaviors. Beyond the diagram and superficial text, it is not clear whether there is additional policy information defining the behaviors at each layer. This impression is reinforced by the section on service granularity which talks in abstract terms about optimum granularity. If each service layer is well defined then service granularity will be appropriate to the service type and their behaviors.
The requirement for a service contract is good. However WSDL and Schema definition are insufficient. The contract should define a set of properties and behaviors that define the obligations of the parties to the contract in an unambiguous manner that a) effectively hides the implementation details, b) facilitates governance and c) test case generation. Pre Post conditions are strongly recommended.
The MDM strategy of entity ownership and (I assume) an event based subscription architecture to replicate changes is a good start, however this seems somewhat immature.
- There will be entity occurrences that must be synchronized in real time and therefore be part of the core transaction.
- It is usually preferable to classify individual some COTS services as underlying services and to expose consistent (canonical) core business type data using common services, and drive the pub sub from those common services.
The data strategy underlying the SOA is critical to service classification and design, and I would be concerned that this has not been sufficiently thought through.

As we discussed, it’s good to have a strong focus on policy setting – getting consistency of approach in key areas of practice. What is the absolute minimum set of policies I would expect to see in place (at what appears to be an early stage in the SOA maturity model)?

Here I recommend you use the policy structure in the Knowledgebase as a start point. I have summarized below, but the detailed tables will provide you with a much richer list of candidate policy types.

Planning Governance	Organization and control of Services.
	- establish a Service Portfolio Plan – this is an essential step to avoid proliferation, and to decide how you will manage the COTS provided services, and indeed shape the RFPs
Architectural Governance	Ensure the architectural quality integrity
	- get the reference architecture in place asap – again this is crucial to have this in place before you decide on the COTS acquisitions – so you place the COTS services in your architecture, as opposed to simply acquire a disparate set of services and then have to resort to conventional integration.
Sourcing Governance	Determine and control Service Sourcing
	- recommend you adjust RFPs as appropriate to align with architecture - recommend you require suppliers to provide comprehensive service behavioural contracts, not just WSDL
Usage Governance	Determine and control Service usage
	- decide service ownership and as discussed above, whether COTS services are accessed directly or indirectly through core business services
Operational Governance	Control run-time services, including QoS
	- you have a good start in this area – but see the Service Specification template – as this includes the opportunity to review the need for variations from defaults
Best Practice Governance	Determine appropriate development strategies and approaches; including modeling and documentation
	- recommend policies on data modelling and service specification
Service Asset Management Governance	Policies that manage Services as assets including control of state changes
	- bare minimum is change management and specification coordination
Organization Governance	Determine the organizational factors such as roles & responsibilities; funding strategies
	- service ownership needed

Regards, David

Searching for Goldilocks

2010-11-17T09:22:00.000-08:00

Found on a ZapThink email:

"Not too technical, not too high-level

Unlike courses offered by SOASchool, CBDI, Web Age, SOA Institute, SOA Certified Professional (SOACP), Architecting the Enterprise (AtE), IBM, Oracle, Software AG, and others, we cover the technology without getting lost in the details. We discuss the big picture but connect it to the day-to-day reality of the IT shop. "

They say all publicity is good publicity, and I am minded to agree. But ZapThink should think carefully before suggesting CBDI courses are too technical! CBDI is renowned for providing a balance of business and technology practice at an actionable level for practitioners – independent of specific technologies.

The proof of the pudding - not too hot, nor too cold . . . . we are happy for you to test run our eLearning classes FREE of CHARGE!

Application Overhaul

2010-11-17T06:28:00.000-08:00

Get ready for a stream of well worn ideas given a makeover as Gartner delivers its annual conference. Notably: Application Overhaul. The new name for Application Modernization.

According to Gartner “ . . . Application overhaul is a new take on legacy modernization methods that deal with the backlog of already-built applications – this overhaul is a task that must be successfully executed before IT can move out from the tremendous maintenance burden that saps budget from innovative projects.”

For the last couple of years I and my colleagues have been advising that Application Modernization needs to be more than simply technology driven legacy reengineering. Rather Modernization needs to be just that - application renewal that embraces modern practices particularly business driven SO architecture, but equally component based implementation, portfolio integration and best practices in the areas of contract specification and change management.

So what do we get from Gartner? The equivalent of taking your automobile down the road to the repair shop for a service! Change the oil, fit new brake pads and a good wash!

Concise OED: Overhaul - thorough examination (with repairs if necessary)

Lazarus or Thomas Effect

2010-11-03T07:06:00.000-07:00

I note the Burton Group has published a paper titled the Lazarus Effect: SOA Returns. Key message is after a troubled time SOA is being reintroduced and recast in a business context. Don't bother reading it. The Burton Group (remember Ann Thomas Manes?) has a checkered track record in spinning SOA is dead, SOA is being resurrected with a new manifesto etc. It's simply marketing spin from an analyst group that need to keep telling us they still exist, even though they were acquired by Gartner. Or perhaps they feel they need to be active in self promotion to impress their new owners.

While Burton, Gartner and others make an industry by worrying about whether SOA works the world has moved on. Most serious enterprises ARE taking SOA seriously and just doing it. It's only the industry analysts that continue to play Doubting Thomas because they never actually get their hands dirty.

Practitioners in the real world realized years ago that SOA is not an end in itself; it's an architectural style that facilitates better solution delivery and business agility. Rant over!

Dealing with search complexity

2010-09-28T03:53:00.000-07:00

We live in a complex world. One way to deal with that is through Views. A search engine is actually a View generator, but I suspect we all feel that search engines, however good, are still blunt instruments and frequently return unsatisfactory results.

I turned my mind to this because in our CBDI-SAE Knowledgebase we have huge repository of SOA knowledge, and even as a contributor I don't always easily find what I am looking for. So what hope for a customer!

The KB has a search engine, menu hierarchies and lots of topic based portals. But it definitely takes work to unlock the value, and we are all busy people and we need efficient mechanisms to find what we need now!

It occurred to me that the concept of a View might be useful. The KB does have stakeholder and work package Views. But it doesn't have Views that relate directly to goals which perhaps more directly reflect what KB users are tasked with doing. As a consequence I developed the idea of a series of Use Cases. For each use case I modeled Actors, Processes and Capability. I modeled the Use Cases in Powerpoint and added links to the Capabilities and published as a PDF. You can see the result here.

If you are already a KB user you will (I hope) recognize some of the Use Cases as being relevant and be able to immediately find resources that may accelerate your work package delivery. If you are not a KB user yet, you may still find value in this set of models a) it provides a goal based picture of capabilities you may require regardless of source or b) it may interest you to look further at the KB and to utilize it further.

Finally therefore I wonder why search engines don't utilize a model along these lines by providing focused search arguments. Take a look at the Google advanced search. It merely asks you to provide more "words". Yes it provides some conditionality, but surely it would be more effective to ask context questions in a straightforward manner that everyone would be able to answer such as:

"what are you doing?"

"what is your role?"

"what is your objective"

and thereby deliver narrowly targeted results.

Happenings at CBDI

2010-09-27T06:51:00.000-07:00

This month we are making some big changes to the way we work at CBDI. We are announcing that, commencing with the September edition, the CBDI Journal is now available on a quarterly basis and freely available on registration.

We have published the CBDI Journal 11 times a year continuously since 1997 and this is one of the biggest changes in that time. Regular readers will know that over the past five years we have undergone a planned transformation from SOA industry analysts to become practice developers for SOA based application modernization. The CBDI-SAE practices and resources are now made available in the CBDI-SAE Knowledgebase (KB) – a framework toolkit for best practice using modern, modular architecture and agile practices.

The CBDI Journal will continue to provide detailed, independent practice guidance that helps architects, project managers, business analysts and other IT professionals but it will be quarterly and available to potentially a much wider audience.

The September Journal is available now.

SEPTEMBER 2010 CBDI JOURNAL CONTENTS:

- Editorial: Ten Tips for Outsourcing/Offshoring Success

- Update: The CBDI-SAE Reference Framework in 2010

- In Depth: Integrating BPMN into the SAE Architecture and Delivery processes

- Knowledgebase Update and Usage Guide

You can get your edition now at: September CBDI Journal

When Policy Should be Mandatory

2010-08-12T07:46:00.000-07:00

I was asked the question today, "when is point to point architecture justified?" Or conversely, "when is it not necessary to use the ESB?"

My answer was along the following lines:

In the end, most policies are discretionary - there are always good reasons why a waiver may be granted. But the use of the ESB and broker pattern is one area where the policy should be mandatory, and no waivers granted. The broker pattern is fundamental to good architecture because:

1. It separates concerns, eliminates hard dependencies

2. It creates an inherently flexible architecture and can easily cater for unforeseen requirements

3. It standardizes onto a common infrastructure, removes considerable work from projects and programs, achieves economies of scale.

4. Enables common services such as audit, logging, security . . .

5. . . . . and all the usual stuff . . .

But there's a further reason that's even more compelling, perhaps particularly so to business managers that might be pressurizing for a DIY initiative. The SOA architecture is actually just a foundation backbone. Enforcing all traffic onto the common bus allows us to scan ALL message traffic without exception for reasons that may or may not be in the purview of the business process or delivery team. The opportunity to use business rules based monitoring opens up a whole raft of possibilities:

- real time business intelligence (BI)

- real time management information (MI)

- complex event processing

- . . . .

And for this reason there should be no exceptions for any reason whatsoever. The success of real time BI and MI will be the ability to bring together disparate events and alerts to apply new rules in new ways. Leave out one domain, application area, business division etc and potentially the value of the BI is destroyed. Methinks business people will understand this.

Cyberwarfare – a real threat or an over reaction?

2010-07-20T13:05:00.000-07:00

There’s been a flurry of resurgent interest in cyberwarfare in the last couple of months. It’s clearly a topic that we as citizens should take seriously, but there’s a strong temptation to dismiss it as science fiction. But as architects it is a topic we should perhaps take very seriously indeed.

Like most people of my generation war has passed me by. It’s a matter for professional soldiers, not for the man in the street. But Cyberwarfare introduces a fundamental shift insofar as the war is more likely to be waged against a nation’s infrastructure, its banks, power and water utilities, logistics infrastructure and so on; quite possibly waged by anonymous perpetrators whose aims are to seriously destabilize another society.

My interest was sparked by the Economist article of May 24^th (Cyberwarfare, Newly nasty. Defences against cyberwarfare are still rudimentary. That's scary). By sheer coincidence a couple of weeks ago I picked by Michael Dobbs book The Edge of Madness. I like Dobbs’ highly realistic political thrillers, and this one is no exception.

The Daily Telegraph said “Dobbs, best known for his political trilogy House of Cards, and for being a former adviser to both Margaret Thatcher and John Major, is a master at seizing straws in the wind and building a bonfire with them. He has found enough straws lately to ignite civilisation's funeral pyre. An unexpected entry on his wife's credit card statement, suggesting she'd been gambling on the internet, propelled him towards The Edge of Madness, his 13th novel. . . . You don't need the wiles of a computer hacker to see that Michael Dobbs may be on to something with his fictional prediction of a cyber-war in which one nation - in his scenario, China - reduces every other to economic rubble by tampering with the global electronic life-support systems.”

This led me to reflect that our de facto approach to security architecture is obsessed with access – identity, authentication, permissions, rights. In the SOA world many have moved beyond the fortress model, but the primary advance made is to apply the same old checks at a finer grained level. So what’s to stop an intruder able to get past the screens wiping bank balances en masse; scrambling demand and supply data for power grid supply, publishing personal details of prison officers, corrupting aircraft maintenance records and so on. The scary thing is that the motive is not profit, it’s simply to destabilize and cause terror. For how many weeks will your civilization persist without electricity, gas/diesel/petrol, food supplies and money?

If we are to take this threat seriously we have to rewrite the book on security. We have to work on the basis that threat assessment is not based on criminal intent, rather on calculated attempts to cause chaos.

Of course the service architecture is inherently suited to this purpose. We tend to promote separation as a means to enable agility, but it “could” also be a key defensive strategy.

Over the years CBDI has identified a number of patterns that perhaps show us the way.

- I have advised numerous banks over the years on real time, independently provisioned pre and post conditional audit operations.

- we also developed the idea of a disruption tolerant service network, where meta data is collected independently to track the status of say a logistics system, in order to advise end user on the real system status

- a couple of weeks ago we came across the concept of forensic operations. The idea of, again independent operations that collect meta data that can be used to investigate abnormal events.

This class of pattern, and once we put our mind to it we will surely discover many more, is clearly not used systematically by many organizations. Perhaps the time has come for one of more independent businesses to provide such services. But without a clear and present danger will anyone spend money on prevention. The cynic may say, "the banks have already done a good job of screwing up our society, why would other countries bother?"

Yet, the requirement for this form of defensive architecture is actually increasing. We are all aware of the event driven, smart systems architecture, many applications of which are focused on critical infrastructure such as power and water.

I am reminded of the devastating floods that occurred last November in Cork City. Faced with unprecedented rainfall upstream, the Electricity Supply Board were forced to open the Inniscarra and Carrigadrohid dams to release large volumes of water which, with little warning, inundated the city. In this case the dam was operated under manual control, but there is strong interest in automating systems such as these in order to better manage the flows. Just imagine if hidden code were placed by a developer, perhaps to be executed years in the future when particular water conditions arise, in which dam managers are provided with erroneous information that caused them to repeat the disaster? In Dobbs fascinating novel he uses this pattern extensively – not complete, immediate systems failure, rather a divergence of reported data and reality.

I would be very interested to hear others experiences and views. Tell me you already scan inhouse, outsourced, offshore developed code for hidden bugs; that you have self checking operations that prevent malicious operations . . . .

Cyberwarfare, Newly nasty. http://www.economist.com/node/9228757 [note this is premium material.]

The Edge of Madness', by Michael Dobbs (Simon & Schuster 2008)

Outsourcing Madness

2010-07-05T06:09:00.000-07:00

Recently I wanted to alter the regular premiums of a pension policy I have with a very well known company; a simple transaction you might think. After 4 months of delay and much correspondence, I received a letter from the said company saying “their systems were not working because of their offshore partner, and they were unable to do even the most simple transactions.” They apologized profusely and promised to fix as soon as they could. Three months later they have now solved the problem as far as I am personally concerned, however my guess is that as a company they are still completely screwed up. Certainly to admit internal problems in this manner suggests at the very least a failure of policy and management.

It just so happened that by sheer coincidence I consulted to this very same company a couple of years ago. At that time the engagement, to assist in establishing SOA reference architecture and process, was curtailed because the organization was in complete chaos. They were offshoring huge chunks of their core business in what looked like a mad panic in order to radically reduce costs. My observation was that uninformed business managers, over promoted because of their aggressive management style were riding roughshod over highly experienced professionals and managers and demanding results in impossible timescales; handing their problems over to naïve overseas companies who were happy “to bite off more than they could chew” because they didn’t understand the awesome complexity of the core business systems.

In this process the company destroyed years of organizational intelligence and knowledge.

Is this situation unusual? Not in my experience. Over the past couple of years I have observed numerous companies taking extraordinary decisions of this nature. Perhaps the most incredible was the one that eliminated an entire EA function because it delivered no short term ROI and the offshore partner would be able to undertake project architecture as required!

Yet outsourcing has become for perhaps a majority of larger enterprises, the de facto business operating model. A couple of years ago I wrote up the insightful experience presented by Denis Hageman[i], in which he described how his then company ABN AMRO grew their outsourcing capability over many years. In that process they discovered that outsourcing is a complex management task and required dedicated attention to deliver on what often looks like easy cost savings. He described a journey, not a quick fix.

For all these reasons I was interested to discover that Carnegie Mellon and its spin-off company ITSqc have created a capability model for what they refer to as eSourcing. CBDI members will know we have long been advocates of capability management as a means to managing complex change and the eSCM[ii] (eSourcing Capability Model) is a valuable contribution to this discipline.

This month I have published an article on eSCM. In the report I draw the conclusion that the eSCM is a useful model, but that it is probably too generic and too broad for most organizations because it attempts to provide a generalized model that can cover all forms of IT enabled outsourcing. I wonder how the challenges of call center outsourcing can be similar to IT program and service delivery? They can be at a very superficial level of abstraction, but a useful model needs more granularity, and therefore tighter domain focus.

In my report I explore how eSCM might be extended and detailed by integration with core aspects of SAE and ITIL which I believe can create useful practice guidance. This is an exploratory work and I will be pleased to hear members’ views and particularly any experience with eSCM for IT services delivery.

Outsourcing is not an inherently flawed strategy. Quite the reverse. But used as a tactical strategy, in the wrong hands; without due care and attention, outsourcing may be a highly dangerous strategy. We badly need better practices to make it work reliably and to provide a benchmark for providers to certify themselves and client organizations to make assessments and manage the relationship.

[i] How to Stay in Control of Outsourcing, Denis Hageman, ABN AMRO

http://www.cbdiforum.com/secure/interact/2008-06/enterprise_architecture_europe_conference_notebook.php#session_2

[ii] eSCM http://www.itsqc.org/

Videos from Microsoft Architect Council UK April 2010

2010-05-27T10:10:00.000-07:00

Matt Deacon interviewing me - discussion on SOA past and present

http://channel9.msdn.com/posts/mattdeacon/Talking-Architects-with-David-Sprott/

My keynote: Convergent Architecture

http://www.microsoft.com/showcase/en/gb/details/a31051dc-7705-4612-849b-8fa6fd06e7de

SOA is business as usual?

2010-05-27T09:04:00.000-07:00

I know I have used the term before – SOA IS business as usual; and in this context I have been emphasizing that SOA is simply the way we must do things today. But there are many data points that suggest we have a way to go – that SOA requires considerable change in practices and that it’s by no means easy to figure out all the changes required, let alone implement them.

This week I was talking to a customer who said that they have considerable SOA experience. They have an ESB installed and lots of services. However he said that the services are all point to point, and that there’s almost no standardization or reuse. Now that they have some of the basics sorted they want to gear up and realize their longer term ambitions. And that’s when we started the conversation about reference architecture, policy and so on and so forth.

Another data point occurred this week – Lawrence I were working on ideas for portfolio management for the report in this month’s CBDI Journal. As you do, Lawrence had trawled the web and found various “authorities” on portfolio management and it caused us to have some serious debate because it’s clear the portfolio management world is still locked into “application management”. Shuffling around arbitrary lumps of capability that they happened to build or buy.

Our guidance is that once you accept that “application” is an unfortunate unit of Portfolio Management you are recognizing that the portfolio comprises of a mix of assets that will have higher levels of reuse and inter-dependence. This has a very interesting knock-on effect. Because we have a mix of asset granularity, many portfolio decisions will be taken across the life cycle, not just in the Planning Phase. Of course we will continue to scope out Solutions and Services in the Planning discipline and phases, but we will continue to discover opportunities to create and reuse assets throughout the Analyze, Design and Delivery stages, and our processes need to positively encourage it.

If you have been following our (riveting) series on application modernization over the past few months you will have read about on iterative life cycles for modernization (aka SOA) which leads directly to the conclusion that of course iterative portfolio management is a natural in the SOA world. The idea of a planner or architect making binding decisions on portfolios is over. Today’s portfolios are assembled out of reusable, moving parts which the projects and programs are actively contributing to. They can be broadly scoped out in terms of services, frameworks, platforms and components, but the detailed portfolio management work will get done in the delivery cycle.

For most of us I suspect this is not yet business as usual. Delivery cycles are too focused on . . . delivery! But we need to temper that enthusiasm (business pressure) and place some portfolio responsibility on delivery teams. For SOA to be really effective, that has got to become business as usual.
Lawrence's report on Portfolio Management

In-memory madness.

2010-05-19T07:34:00.000-07:00

I note SAP is set to make a major announcement this week.

“SAP’s software dispenses with the separate “relational databases” where the data behind such transactions are typically stored, and instead retains the data within the server’s memory. This, says Vishal Sikka, the firm’s chief technologist, not only speeds up existing programs—it also makes them cheaper to run and easier to upgrade, and makes possible real-time monitoring of a firm’s performance.” The Economist May 15th

If you have shares in SAP and haven’t yet sold them, now’s the time.

SAP’s rise to market leadership was because it emphasized business not IT. This is a total reversal.
One time pioneer of SOA, separation of concerns, transparency of technology, SAP has just reversed everything it stood for over the years. In-memory data is putting implementation constraints on business processes and likely to wreck any chance of a consistent information architecture.

Pursuing technology wizardry is simply business madness.

Back to the Future – What IS a Service!!

2010-04-28T11:37:00.000-07:00

Last year I set a goal of integrating SAE with other frameworks and standards. Inevitably this has been a considerable task, but to date I am pleased to say we have mapped to the TOGAF 9 framework, and of course aligned the SAE meta model and profile closely with the SoaML standard.

I started to look at the ITIL framework last year, but to be honest found it a slippery beast. But I have now created the time to have another go and I am pleased to report I have this month published guidance on how to use the SAE and ITIL frameworks collaboratively. The issue that I struggled with last year is that ITIL is a highly generic framework. It provides process guidance for service management activity, where the service is almost anything you want it to be. Whilst it is perfectly understandable that the service providers like HP, IBM, TCS et al envisage the scope of their services to be ever expanding, it doesn’t make it easy to get to grips with processes that are reduced to highly generic descriptions.

Further the nomenclature is very hard to work with. Whilst there is an ITIL glossary, there is no meta model, and similar to the Event Architecture space where, last year I also struggled with the Event Processing Glossary[i] for the same reasons, I found lots of inconsistency and no attempt on the part of the ITIL community to address core issues of nomenclature.

The root problem, (there are many more but I will focus on the core issue) is that ITIL describes the service as, “A means of delivering value to Customers by facilitating Outcomes Customers want to achieve without the ownership of specific Costs and Risks”. The term is in direct conflict with the SOA community that has established formal standards backed up by automation around the term Service. However the ITIL community choose to ignore this, leaving their customers to resolve the inevitable confusion.

The IT infrastructure professional makes word associations with Service that go along the lines of Network Service, ERP Service, Desktop Service and so on. Whereas the Application Architect thinks of a Customers Core Business Service, or an Events Service.

The ITIL documentation is really no help. In fact there is a section in the ITIL books headed WHAT ARE SERVICES? Which admits “.. over the years organizations have debated the definition of what is a service”, yet completely fails to provide answers, just generalizations. I have to admit some frustration with this, and it probably caused me to drop the research topic last year. No less than 10 years ago we were grappling with nomenclature in the service domain, and it seems incredible that we should have to repeat the experience.

In consequence, in my work this month on the collaborative use of SAE and ITIL I felt I couldn’t write sensible guidance unless I had some semantic consistency. I have therefore proposed a nomenclature and a simple classification system which I hope will at least avoid senseless time wasting over semantics.

I suggest in context with SAE and ITIL in collaboration, the ITIL service is called an IT Service and the first class SOA concept is called a Service. I have then provided outline meta models for the (ITIL SAE) intersection which is aimed at creating delivery and life cycle consistency.

In my report I do say that “we recommend SAE users adopt this approach as . . . it is essential to have clarity and consistency of core concepts. However we are not wedded to the specific term IT Service, and we will encourage standards organizations to resolve this. So in the meantime local solutions may be preferable. Consequently, if a major service provider chose to substitute the IT prefix with their company or division name, that might actually be a great solution that also representing very effective marketing!”

A wider issue is how and whether we should incorporate the concepts into the SAE meta model without any rigorous work from the ITIL community.

I do believe this is an area where the standards bodies need to undertake some work on an urgent basis, and maybe this needs to happen outside of the ITIL community in order to achieve a sensible result. Meantime, if you follow our advice, you will at least have internal consistency.

CBDI Report: SOA and ITIL Framework Collaboration

http://www.cbdiforum.com/secure/interact/2010-04/s_i_f_c.php
ITIL (The IT Infrastructure Library) is becoming widely adopted as the standard for IT Service Management. Described as a service management framework, the service concept at the heart of the ITIL process is about the general capability delivered by a service provider, which might coincidentally include SOA based services amongst the range of components delivered and managed. In this report we provide guidance on how to use the ITIL framework in collaboration with the Service Architecture and Engineering Framework (CBDI-SAE) for the Service Strategy and Design stages, and how to deliver better service management for SOA and modernization programs. By David Sprott

[i] Event Processing Technical Society Glossary
http://www.ep-ts.com/component/option,com_docman/task,cat_view/gid,16/Itemid,84/

Thriving on Chaos?

2010-02-22T08:30:00.000-08:00

I live in a very old property and I am reliably advised that once every 100 years or so, a thorough renovation is always going to be necessary. Well that is good news! The bad news is that the house is currently a building site and my wife and I have been required to move out – into a cottage that we fortunately have on the property. One of the traumatic experiences was to empty the house of all our effects. But there was a silver lining as it required me to sort out my books.

I have happily been discarding classics (sic) such as COBOL/CICS Programmers Guide, Understanding COM+ and the MS-DOS Users Guide. However I have spent happy hours reacquainting myself with gems from authors such as Yourdon, Martin and DeMarco which actually never really date in terms of the essential truths they espoused. Similarly I also came across a copy of Tom Peters book – Thriving on Chaos. This book should be required reading today.

I turned to the last chapter titled Building Systems for a World Turned Upside Down. You may remember Tom Peters – he was one of these breathless characters that emitted ideas at a speed that would apply better to a machine gun. Under the heading KEEP IT SIMPLE (AND VISIBLE) I read, An accounting firm executive recalls his surprise at the systems used by a GM-Toyota venture. He had expected state of the art . . .instead he says laughing, “it was flip charts, red and green lights depicting a systems status . . . absurdly simple, but to the point. . . . “. Maybe over the years Toyota allowed the attention to detail to drop a little and forgot the other Peters strictures such as MEASURE WHAT’S IMPORTANT, SET CONSERVATIVE GOALS and DEMAND TOTAL INTEGRITY.

Peters was talking responsiveness and flexibility 22 years ago! But at the same time he was demanding a balanced approach that “while turning the world upside down” also insisted on metrics that provide really appropriate instrumentation.

He provides a sample list of unconventional measures that match Prescription and Measure. Here’s a few of examples:

Prescription	Measure
Service	Ten attributes of customer satisfaction
Responsiveness	Speed of response to customer needs; percentage of customers covered by electronic linkages; new links added to product every 90 days
Listening	Number of customers called per week
Pilots	Number of small starts; percentage of R&D budget devoted to small starts
Support Fast Failure	Number of awards for interesting failures, constructive defiance of rules

Which makes me think we should be able to apply similar thinking to today’s business and IT governance. I am frequently reminded that the level of governance commonly exerted is inadequate. Often it’s the absence of clear policy, sometimes it’s politically appointed governance boards that operate by opinion rather than bringing deep experience to bear on key issues and contributing actively to organizational learning.

I always advise effective governance requires a) clarity of principle and policy, b) formal deliverables produced at the right time (that is synchronized with key decisions and allows alternative actions to be taken where necessary) c) published governance criteria against which governance boards can act and d) publication of policy AND governance decisions that avoids reinvention of wheels.

Reading Peters again, I am minded to suggest we also need measures such as:

Prescription	Measure
Business Responsiveness	Actual change performance vs predicted/contracted
Consistent Information	Number of data services vs portfolio target, Number of duplicate data sources; cost of data (correction effort, customer queries)
Process Improvement	Number of proposals for improved pattern/policy approved
Supplier performance	Number of governance waivers requested;

It’s all too easy to think that in today’s fast moving, technology laden world that controls aren’t needed. If you read the Accenture report on Millennials' Use of Technology: Jumping the Boundaries of Corporate IT you might be persuaded also. However I suspect this is typical fluff that eventually ends up in tears – just like Toyota’s braking fiasco. Back in 1988 Peters clearly understood that you need the combination of fast moving, agile practices but backed up by tough controls that ensure your business doesn’t go off the rails. It’s called governance.

References

Thriving on Chaos, Tom Peters, MacMillan, London 1988

Millennials' Use of Technology: Jumping the Boundaries of Corporate IT

Agile Application Modernization?

2010-01-26T06:48:00.000-08:00

Is Agile Application Modernization an oxymoron? I have started with the principles as defined by Beck, Beedle, Bennekum, Cockburn, Cunningham et al and suggested some key alterations. Be very interested in all feedback.

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Replace

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

With

Manage change to absolute minimum to deliver high quality foundation that can then support continuous delivery of change as necessary.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Business people and developers must work together daily throughout the project.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development.

The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Replace

Continuous attention to technical excellence and good design enhances agility.

With

Continuous attention to best practice architecture delivers an inherently agile product.

Simplicity--the art of maximizing the amount of work not done--is essential.

The best architectures, requirements, and designs emerge from self-organizing teams which can then form the basis for reference architecture and policy.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Getting to grips with Agility

2010-01-24T09:08:00.000-08:00

For the past few years the concept of business agility has been much in evidence in vendors’ SOA marketing materials. It is a good idea to promote agility – clearly SOA can deliver inherently agile applications because it increases loose coupling, and in many ways the marketing of the benefits of service architecture has closer relationship with reality than many forms of IT marketing that frequently rely on hype and superficial nonsense.

As SOA marketing spend diminishes to be overtaken by new kids on the block such as Cloud, Virtualization, Green IT, Social Software, Application Modernization and so on, the business agility message is being used ever more widely. No surprise because there is intense post recession interest in managing change as enterprises drive out costs and reprioritize to focus on new business opportunities.

Yet in many cases the use of the term agility is a stretch. At least SOA marketers could reasonably claim that service based products genuinely would reduce coupling. But the core issue is that business agility is an elusive concept and no one gets held to account because the agility offering is unquantifiable. We might wonder how many buyers of SOA products and services feel in retrospect, and whether with 20:20 hindsight they might consider the claims for agility were over stated!

What’s needed is ways to talk sensibly about the topic of agility. I observe efforts to measure agility by measuring complexity. I don’t agree with this – I think it’s fine to measure relative complexity, but this doesn’t help in the understanding of agility. As an aside, the agile methods community have developed ideas around an agility quotient[i] which assesses 11 factors to assess and rate skills and attitudes relevant to agile delivery. But this is quite different to measuring the agility of delivered application capabilities.

We need a technique to facilitate a rational dialog between business and IT managers on the subject of agility. The dialog needs to be an integral part of business requirements planning – that encourages exploration of where business change may occur, where articulation points need to be, and the type of change that may be required, or not as the case may be.

In this month’s CBDI Journal I propose a technique that develops a relative measure of potential agility. I refer to this as Potential Application Agility (PA²). The measure is developed for business processes and or business capabilities and can be used to analyze change to assess the monetary value of agility in different areas of the business and to create a heat map that can inform architecture work. The PA²measure then provides a useful input into architecture development effort.

I recommend initially developing an “agility architecture” highlighting Policies, Patterns, Standards and Practices for each of the architecture Views (Business, Specification, Implementation, Deployment and Technology). The agility architecture will form part of the reference architecture and with suitable annotation provide guidance and governance to achieving appropriate levels of agility as indicated by the business analysis.

I anticipate a response that says, “we can’t afford the time to do that sort of analysis; it’s inimical to the agile (project) best practices.” My response is – how can you afford not to undertake this form of analysis. In today’s fast moving world, it’s impossible to bottom out comprehensive business requirements. Rather it’s much more important to figure out the morphology of the application and or service in terms of agility and stability and to deliver a stable foundation that can support appropriate levels of adaptability.

The PA²analysis can be undertaken in an iterative manner and on progressive levels of detail. Initially it would make sense to do it in outline, and then in time honoured fashion to drill down in areas that merit attention. A few hours of facilitated workshop style discussion with the right stakeholders should be more than adequate to achieve an outline agility analysis, and direct more detailed work into areas of serious business value.

[i] The Agility Quotient http://www.ddj.com/architect/184415255;jsessionid=DXLSRHZYJJ44JQE1GHPCKHWATMY32JVN?_requestid=58498

The meaning of life and application modernization

2010-01-07T07:41:00.000-08:00

I observe lots of modernization going on that addresses narrow technology concerns – particularly moving to new technology platforms and languages. But there are so many other dimensions that we should be considering.

I have written a lot over the years about how good (fit for purpose) architecture is based on principles, and we might do worse than consider what architectural principles are relevant to modernization.

First the SOA principles of loose coupling, abstraction, contract based and suitably event driven are very appropriate. It would seem strange (a bad investment) if a modernized system was not service oriented. But in addition consider:

Model driven architecture and design – enables governance and ongoing management of inter and intra application architecture.
Component based implementation – ensures containment of complexity, defined responsibilities, boundaries and dependencies.
Virtualized – cloud ready.
Application knowledge is a byproduct of delivery platforms and tools – ensuring precise and current documentation of the (business logic and rule) details of the application.
Defined change management capability with SLA.

Modernized applications should be model driven (ideally, but not necessarily using MDA/MDD per se) and service and component based in order to achieve the same level of agility and asset management that can be achieved in the service architecture, at the implementation layer. These two principles can easily be measured by the existence of a) models and b) well formed components and are viewed as important characteristics of modernization and key criteria for assessment of existing systems.

A defining feature of legacy applications is the absence of understanding of what an application does. In contrast a modernized application should not be defined solely by being coded in a modern language, but should also have detailed documentation of structure, business rules and logic compliant with a comprehensive meta model as an integral part of the technology architecture and delivery process. In this way currency and accuracy should be guaranteed.

Finally there has been widespread debate about what agility means, but few answers. The obvious way to tell whether an application is agile is to have explicit agility architecture together with a Change Service Level Agreement that provides estimating guidelines for relevant types of change. Then it would at least be measurable.

Everware-CBDI is evolving it's service architecture and engineering framework(SAE) to incorporate application modernization. Resources on SAE2:

– Application Modernization Framework (free white paper)

– Application Modernization Process (free white paper)

– December CBDI Journal – On Application Modernization (on subscription)

Dear Mr President, You have ordered a review of Air Travel Security Systems.

2009-12-28T03:22:00.000-08:00

Today I have read that you have ordered an air security review; you want to know how a man carrying explosives managed to board a flight from Amsterdam to Detroit.

The answer is very simple. Today catching terrorists is evidently not a priority for the security services.

Since 9/11 the world’s governments, airports and airlines have put in place a huge security operation. It has revolutionized flying – making airport processes slow and very uncomfortable, for millions of ordinary passengers at huge cost to travellers, because we pay for this through airport taxes. Yet every regular traveller will tell you the security system is completely useless. It is not designed to catch terrorists; it is designed to demonstrate that governments are doing something, anything. When did the airport security systems catch a terrorist – never! I feel I can safely say that because had they done so, you can be sure we would have heard about it.

For years we have been telling governments, airlines, anyone that will listen that the answer to terrorism is differentiated security. In 2002 Richard Veryard writing in the CBDI Journal identified three patterns - Ghetto Security, Herd Security and Differentiated security. Today the security systems use Herd Security.

In contrast, differentiated security means that each individual belongs to a different class defined by profiling. And I note the Christmas Day bomber was on a “watch list” but was not considered sufficiently dangerous to be on the “no fly” list.

What’s needed is a system that uses profiling intelligently as part of a sophisticated range of checks that include before the event profiling as well as pre travel checks. The current system is binary – everyone gets the same security check unless they are on the “no fly” list - in which case they don’t get to travel.

But in 2003 we said that in differentiated services, service outcomes depend on context. This means we need to have better ways of selecting who gets closer scrutiny. In addition to profiles we must also have a range of scanning techniques, not to find the devices – because frankly these will always keep changing and security systems have a hard time keeping up. We need to scan the individual first as input to a triaged process:

- Electronic kiosks could support a range of biometric checks logically equivalent to those a human agent might make - is the passenger sweating, agitated, looking frightened, etc.

- Software can also measure the response time of the passenger, and may make useful inferences about the passenger and his context from any lengthy hesitation - just as a human agent does. Uncharacteristic hesitation may be a sign of impersonation, or it may signal a need for help.

- But actually these high tech solutions could easily be done today for an intermediate (not low or high risk) class of traveller by interview. The Israeli government does this today for all passengers which doesn’t scale – hence the need for profiling.

Profiles plus human and automated scanning can drive a triaged process that allows regular travellers, aged grandparents and indeed the 99.999% of travellers to be treated with human dignity, while the .001% are given an appropriate level of scrutiny.

The systems and technology are available to make this happen and could be brought to bear very rapidly. Clearly from recent public statements the various levels of profile information are available, they simply are not being used properly. The primary impediment to action is bureaucracy operating without executive direction. They need to be told what to do! This needs someone like you, Mr President to tell them go catch the terrorists and stop harassing ordinary, honest citizens.

Differentiation and Security - Three patterns of security

Application Modernization

2009-12-11T06:20:00.000-08:00

Over the past few months I have been exploring how SOA is morphing into BAU (business as usual). The parallels with Climate Change are uncanny. There is a highly vocal lobby that would tell you SOA is not happening. Yet all the evidence from both personal experience and industry surveys tells us that SOA is happening for real. How often have we observed that after the hype has died down, real learning and rollout just happens quietly and in private?

I have commented previously that SOA will morph and converge with CEP, EDA, Web 2.0 etc. Also that ecosystems (intra and inter company) will be the primary route to a more strategic version of SOA, rather than enterprise SOA. I refer to this as the Smart Ecosystem.

But smart ecosystems need a basic platform of technologies and business services to get beyond first base. What I observe is considerable activity in what is being termed application modernization. As we start to emerge from the recession there is real business pressure to keep costs and complexity down, and to be able to support the inevitable business demands for new ways of doing business.

A Forrester report commissioned by BluePhoenix shows a majority of IT leaders placing IT modernization as the top software issue. A very high number of respondents indicate their intent to consolidate or rationalize enterprise applications. A very high proportion also indicate they will be using SOA to sort out their legacy problems.

It’s a no brainer really. We know how to architect and deploy SOA; but efforts to deliver “enterprise” SOA have foundered for lack of relevance to business programs and priorities. In contrast, handled correctly modernization can provide a sensible platform for sorting complexity and agility issues while delivering business programs.

Most application modernization in process today is strongly technology focused, with objectives relating to platform and language replacement and reengineering. Critically much modernization is application specific, just replacing one arbitrary application scope with the same implemented in a modern language.

But if this activity is business and architecture driven, the opportunities to deliver business value in a series of coordinated increments have the potential to radically reduce complexity, increase agility while delivering urgent business programs.

At CBDI we are working on enhancing our already popular SAE tools and practices framework to create an integrated application modernization approach. We will be publishing the first cut of this work in the December CBDI Journal. Knowledgebase detail will follow. Needless to say, there’s no industry agreement on definitions. Our first cut on Application Modernization is as follows:

Application Modernization: Rationalize one or more applications or a portfolio to improve business support, technology usage and life cycle and run-time delivery process. Objectives include:

Rationalize - eliminate duplicate applications; make multiple overlapping applications consistent.
Modernize – upgrade delivery and operational technology and processes including managed service, offshore, outsourced delivery.
Componentize – reorganize arbitrary boundaries to align with business morphology and enable business flexibility.
Service Enable – move to service architecture that aligns with business capabilities, services and events.

Can incorporate: Integration, Migration, Reengineering, Rewrite, Replacement, Acquire, Buy not build, Elimination, Functional Improvement, Outsourcing, Offshoring

Be very pleased to hear what others think.

Reference: Application Modernization And Migration Trends In 2009/2010, Forrester,

Reference: CBDI Application Modernization Resources

Ecosystem 2

2009-11-21T08:46:00.000-08:00

Last month I explored the concept of the ecosystem, and how breaking down enterprise boundaries will be a major focus in future. I discussed how the enterprise perspective needs to change in order to optimize business processes in a manner that spans conventional boundaries. Whilst I may have been a little controversial by suggesting the “death of enterprise architecture” the careful reader will know that my comments were directed at the issues of how to scope architecture and delivery projects.

I return to the theme this month in an article in the November CBDI Journal in which I explore how SOA will evolve. I examine a number of key influences which inform the evolution in which scoping forms an important part. I also look at key technologies:

1. How the confluence of SOA and maturing EDA, CEP, smart systems and Cloud Computing will encourage a revised view of solution scope that is simultaneously more narrowly drawn around specific business goals, yet spread more broadly to involve all the stakeholders that naturally participate in the ecosystem.

2. How SOA and Cloud Computing make the virtual world practical.

3. That well architected SOA should deliver highly independent, stable capabilities that implement core business logic for major resources which can support agile event processing.

4. A well architected EDA layer increases loose coupling and transforms orchestration overhead into business relevant event rules that can rapidly respond to business change.

5. CEP enables a system in which multiple stakeholders can participate in an ecosystem and concurrently derive stakeholder relevant events in support of a broader ecosystem purpose.

6. Maturing technologies including Web 2.0, Sensors and Analytics, which have naturally evolved in relative isolation, will increasingly interact with core systems architectures.

Putting this all together I suggest we have a number of really interesting new patterns that in various permutations and dimensions have a disruptive effect on our view of enterprise.

So what is an ecosystem? Try this . . . a set of business capabilities that collaborate to support a common purpose and exhibit high levels of interaction based on event relationships, shared information and data concepts. An ecosystem may comprise part or whole of one or many business processes; part or whole of one or more enterprises. It is most likely to be a series of subsets of conventional scoping mechanisms.

An ecosystem doesn’t have to be drawn on a huge scale as we measure scope conventionally. It may be sub enterprise or cross enterprise, focused on key goals of better results, in say a particular field of medicine; or key areas of high cost and low profitability. The key point is to start with a purpose, and then to examine the entire scope of business types, information needs, events, business capabilities, business services and software services. Equally important we should explore the entire set of stakeholders and actors in order to surface a rich model of complex events, capabilities and services that we can then consider various scenarios for radical change in business processes, business intelligence and management information availability in the light of new technical capabilities such as sensors, event processors and mashups.

Ecosystem might not the right word. Maybe a better definition is NOT ENTERPRISE!

Comments on The IT Complexity Crisis: by Roger Sessions

2009-10-26T13:37:00.000-07:00

Roger Sessions has written a paper on IT Complexity. When I met with Roger in London earlier this year (at the EAC) we had a short conversation, and it was clear we agreed on numerous things. This paper is useful because it provides a basis for more detailed discussion.

Roger’s paper:

Comments:

Cost of complexity is a little suspect to me. Primary issue is basing numbers on government data – government projects have high probability of failure, at least in the UK. More important I think complexity has more impact on full life cycle costs. Direct IT cost is one issue, but lost business opportunity is probably the major cost. Hard to estimate. However we can all agree, it’s a it’s a big number.
I am always wary of “functional decomposition”. It’s notoriously imprecise. At CBDI we recommend a combination of capabilities and business types (data). The outcome of Roger’s example SIP analysis looks similar to what would emerge from structured business type analysis, but in my experience the latter method is more reliable. More importantly I am vitally interested in developing an architecture that is a) demonstrably stable where it needs to be (managing business types such as customer, product and so on) and agile by design where it needs to be (managing process behaviour, transient data and events). I accept this may appear to be a form of religious debate, and we may have to agree to disagree, but I am interested to have the discussion.
But regardless of how you arrive there, Roger and I are completely in agreement on the need for components. The component must be completely encapsulated, own it’s own data store(s) and be the sole provider of owned services. Roger’s components appear to be capabilities. No problem there, because in the CBDI reference architecture there are options. But I would be looking to separate out layered behaviour so that business type components (stable) are separate from process and event components (unstable, subject to change).
The computation of SCUs (standard complexity units) is interesting. Roger’s methodology will lead you towards larger components with more internal dependencies. What’s the internal architecture look like Roger? The dependencies are still there. I am not concerned if the number of dependencies is very high, PROVIDING the dependencies are all well formed, reusable services and operations. Then it’s simply a question of ensuring you have good management (life cycle and run time) systems, which we do know how to do. I am more concerned by poor reference architecture (bad patterns, convergence of behaviors that should really be separate, bad or non existent contracts . . . )
Finally I put more faith in principles, patterns and reference architecture than in numbers. Numbers can lie, and they often do. Whereas patterns and reference architecture are the distillation of good (and bad) experience and provide intelligent people good guidance to become even more intelligent.
At CBDI we have developed methodology and process for modernization. We see real demand for rationalization and modernization of existing systems (full life cycle costs). The essence of the approach is to a) create the SOA façade and b) componentize. You might like to take a look at our method for CA Gen, it’s just one method for one environment, but it illustrates we [practice what we preach. There’s a slideshare there that describes.

Good stuff Roger, fully support the component approach. Happy to dialog in more detail.

SOA Manifesto Marketing Twaddle

2009-10-26T10:59:00.000-07:00

Last week a small group of mostly vendor representatives plus a couple of analysts and authors came together last week in a seemingly self important manner to develop an SOA manifesto. Of course the primary reason for the event was marketing, and we shouldn’t be surprised that the result is superficial in the extreme. The primary worry I have is that they came up with a manifesto that actually confuses rather then assists in wider, deeper understanding.

CBDI members will know from our long standing research that the key to architecture, and SOA specifically, is to define principles that a) uniquely differentiate the architectural style and b) enable governance. I wonder how this group managed to omit crucial matters such as separation of provider and consumer, standards based and abstraction?

Meaningless marketing twaddle, but because the vendors sponsored this, political correctness will prevail. Yuck.

The Death of Enterprise Architecture?

2009-10-26T09:56:00.000-07:00

- Introducing Smart Ecosystem Architecture (SEA)

In recent months there has been considerable debate about Enterprise Architecture or EA. Practitioners who have embraced EA are now hotly discussing whether EA is a business or IT architecture and the balance between the two domains. Without any doubt most EA today is IT focused even though practitioners know it shouldn't be. And the new EA standard TOGAF 9 merely reinforces this. Although there is a vocal lobby who would like to gain more power and influence in the enterprise by taking the lead in business architecture and design, typically they are unable to achieve this because they can't articulate or demonstrate why the business should let them.

Yet my own observations are that EA has not been an overwhelming success. The tensions between EA and delivery teams remain undiminished. EAs are frequently out of touch with delivery issues and do not command the respect needed to exert strong governance over the delivery work streams. From an SOA perspective, there are few enterprises that have delivered a comprehensive Service Portfolio Plan, let alone the implemented enterprise service portfolio. You may wish to ponder on whether this is a failure of EA or SOA. I couldn’t possibly comment!

Many years ago when we published the first CBDI Maturity Model we recognized that SOA maturity would be defined by the ecosystem. This maturity stage followed the enterprise stage. I will readily admit that in the early days the ecosystem stage was ill defined; in fact no one was interested. Apart from a small minority of enterprises that have always operated an ecosystem business model, the focus of attention was always heavily on the enterprise. Today we can see things have moved on apace. Various influences particularly Complex Event Driven Architecture and Smart Business and IT are strongly predicated on optimizing business design and processes involving all the ecosystem stakeholders. Examples:

- Airlines, airports, airport concessions, airport services, hotels and rental car companies form an ecosystem that optimize their resources and prices on a dynamic basis dependent upon external events, raw demand and actual traffic. Actually we wrote a research report around this model years ago – see reference.

- Smart power grids with tighter linkage between customers, suppliers, generators, assets and operations manage power supply to optimize use of green energy, price to consumers and profitability.

- Also see IBM's Smart Planet site. Reference below.

The emphasis on business optimization is very interesting. This means that conventional architecture domains of operational systems, business intelligence (BI) and management information (MI) become much more inter dependent and dynamic. New types of information, perhaps conventionally classed as operational, now become critical MI. Real time feedback loops require derived information to be available in the operational timeframe. Conventional (sic) SOA layered architecture policy requires extension to manage new patterns.

In these examples of the new enterprise each participant collaborates to optimize the overall ecosystem and in the process optimize their own position vs their collaborators. In this world the architecture is absolutely not enterprise wide. It is goal driven, focusing on a scope that is business results driven.

Does this mean that EA is dead? As discussed, in its current form EA has actually been less than successful. The current level of debate is merely confirmation of the patently obvious. Personally I recommend a more practical approach to architecture that is more grounded with stronger relationships and shorter feedback loops to business value and delivery projects. Given the current issues with EA it would be expedient to have unambiguous naming without the baggage of EA. I am temped to say that Business Architecture works because most grownups understand that Business and IT are indistinguishable at this stage of the game. But I see a lot of immature debate from EAs that want to elevate EA away from IT, which is plainly wrong. Also I don’t have a strong opinion on the "role" of EA. Rather I worry about the nature of the architecture. I want to get everyone thinking about a new perspective and scope.

On balance therefore I prefer a naming that emphasizes the architecture scope and deliverables, the differences from what we have been doing and the need for change. I suggest Smart Ecosystem Architecture (SEA).

References:

IBM Smart Planet

CBDI Report: Modeling for SOA - Worked Example - The passenger departure process - from arrival at the airport to boarding the plane.

CBDI Report: A Web Services Maturity Model

The Failure of SOA Standards!

2009-10-01T02:40:00.000-07:00

You will have heard the joke before: “The nice thing about standards is that there are so many to choose from.” Sadly in SOA land this is reality.

The need for standards in SOA are multivarious including standard concepts so we all speak a common language, standard meta models that underpin profiles, repositories, deliverables, specifications etc. standard protocols that allow implementation and platform neutrality for service publication and consumption. In this note I am interested in concept standards that support architecture and the full life cycle.

In this area we all know we have three “horizontal standards” organizations (OASIS, The Open Group and OMG) that have each followed their own path, ignoring the blindingly obvious need to have consistency across their work. The situation is so ridiculous that the three organizations resorted recently to publishing a document to explain how to navigate around the SOA architecture standards. Curiously although the document is branded by the three organizations and acknowledges participation and contribution from all three organizations, the document is formally a joint OASIS and Open Group document. One might be forgiven for assuming they couldn’t agree the legalities of the collaboration. Also the document doesn’t address inconsistencies between standards and guidance produced by the same organization.

The document declares two goals – 1) to help users to understand the strengths of each body of work and select products appropriate for their needs. 2) to encourage consistency. But the latter is declared only as a secondary goal!

But inconsistency is only part of the problem. We must judge any standard in its support for the user community and crucially user adoption. The primary purpose of these standards under discussion is achieving common definitions of important concepts and simply put they have failed to achieve widespread adoption because of the competing efforts have confused the user community. Further individual efforts have in my opinion failed to meet user needs.

While TOGAF is getting considerable traction our experience is that the parts of TOGAF that get used are the ADM and the contracts. Not the concepts and meta model, because they are patently not based on real world experience. They are both too verbose for some purposes and insufficiently detailed for others. And the guidance for use is non existent.
Similarly the OASIS model is widely regarded as academic – not useful in practical application.
I have more respect for the OMG’s work in SoaML, because it is usable in support of code generation, but it suffers from over generalization, which makes it very hard to use by mere mortals.

I observe that in the referenced “navigating” document the nearest the authors come to agreements on core concepts is at the most superficial level. I see no evidence that there is any real intent to deliver real convergence at a meaningful (i.e detailed, defined) level of abstraction.

Over the years CBDI has encouraged standards efforts; we have consistently made the case that common concept standards are essential at all levels of abstraction. We didn’t just sit by; we developed our own meta model which actually predates most of the work referred to above, and made it available to standards bodies. Frankly we fully expected that our work would be superseded by the standards work and that we would progressively align our SAE models to a converged standard.

Well you won’t be surprised to hear that the complete opposite has happened. This month we are publishing V3 of the CBDI-SAE Meta Model for SOA. We have actively participated and contributed to the OMG SoaML work and we are pleased to announce a high level of alignment. This is possible because of the highly generic nature of the SoaML work.

The CBDI-SAE meta model has always been very different to the various standards bodies work. We have focused on what may be termed a detailed Business Type Model (DBTM). The goals of the DBTM are to provide consistency of concept for use across the life cycle at a level that can be implemented in tools and inform practical architecture and full life cycle deliverables. And of course users of the CBDI-SAE meta model tell us that is exactly what they use it for – populating architecture and service repository schemas, underpinning UML profiles and defining detailed of deliverables that can provide full life cycle traceability and governance.

What we didn’t anticipate at the outset was that our meta model would actually form the basis for a further purpose – to provide a mapping between the other concept standards such that users forced to work with multiple conceptual schemas (few of us live in a homogeneous world) would have a basis for coordination and possibly transformation.

So as a user what do you do? Currently we see minimal use of these concept standards because they are inadequate. We recommend that you continue (or commence if you are a late adopter) using the CBDI-SAE meta model. We will commit to map to the other standards and, if and when they ever get to a credible position of convergence, we will certainly plan model convergence also. But we will need more than alignment of the concepts, we will need to see practical abstraction and detail required for real work.

We will be announcing a user review of the V3 model very shortly and welcome all input.

Navigating the SOA Standards Landscape Around Architecture, Copyright The Open Group 2009, OASIS 2009,

V3 SAE Meta Model note this is currently for CBDI subscribers only. Public domain version follows shortly for the review.

Don't Believe the Hype!

2009-08-25T03:03:00.000-07:00

I note Gartner has published its 2009 Hype Cycle Report^{^[1]}. I must admit I find it a very strange beast. It claims to be an evaluation of technologies. But it is plainly a mix of technology maturity AND technology adoption maturity. For example SOA is classified as being half way up the “slope of enlightenment” and 2 to 5 years to mainstream adoption. Yet it is very clear that SOA technology is reasonably mature at this stage, but it has equally obviously run way ahead of users’ ability to deploy it, because it is an architectural issue facilitated by technology.

The report also includes a technology referred to as Context Delivery Architecture (CoDA) as being in the Technology Trigger stage and less than 2 years to mainstream adoption. Two issues here. First so called CoDA is simply part of SOA. By Gartner’s own definition it’s about architectural response to the end user’s context such as location, preferences, identity, etc. and delivering the information that is most suited for it. Second the Differentiated Service pattern^{^[2]} defined by CBDI in 2000 has high levels of support in many tools and platforms that are fully mainstream in terms of technology maturity. Yet Gartner believe CoDA will become mainstream in less than 2 years, even though they advise SOA will only become mainstream in the 2 to 5 year timeframe. Doh!

And I wonder how Location Aware Applications (slope of enlightenment) differs to Context Delivery Architecture?

And then there’s Cloud Computing. Currently shown by Gartner as being at the top of the “peak of inflated expectations” with 2 to 5 years to mainstream. Let’s examine this more closely. SOA is central to Cloud Computing. I note various commentators saying that Cloud simply enforces the encapsulation and virtualization principles of SOA. Of course there are new technologies in Cloud. Infrastructure as a Service (IaaS) in particular is key; yet this is still just SOA for infrastructure, which many vendors have already done a great job of cracking, and have many user deployments.

Web 2.0 is also on the list. Whilst Web 2.0 is a very ill-defined trend spanning many disparate technologies, SOA is clearly central to most of the use cases. Yet it’s shown in the Gartner report as towards the end of the “trough of disillusionment” and less than 2 years to mainstream adoption, ahead of mainstream SOA!

And then there’s the heterogeneous nature of the Hype Cycle Report. Because it includes such a wide and disparate range of technologies, it’s really more confusing than informing. Further it ignores one of the most important influences on maturity – standardization. Reading the report this year I was tempted to simply ignore it. But the inconsistencies highlighted above persuaded me that this report is actually quite dangerous guiding decision makers in highly undesirable ways. Listen to Gartner and you are encouraged to think that SOA is a technology. But the A is not just important, it's the architectural foundation to many of the evolving trends. Yet Gartner seem to place them all on the same level as individual technologies. I suppose the real reason why is because it plays to vendors marketing programs, of which Gartner is an intrinsic part. However I see it as distinctly unhelpful to enterprise customers.

My recommendations on hype cycle analysis are as follows:

- Decide which classes of technology you need to track in terms of technology maturity. Create a grid for this. Use this to inform R&D and Discovery projects.

- Decide which classes of trend you need to track in terms of adoption maturity (include architecture, practice, technology, products). Create a separate grid for this. Use this to inform project chartering decisions and governance review criteria.

- Change both grids to include an assessment of when important and relevant standards will be stabilized.

- Create dependency models of the trend areas, so you can understand the relationships between such clusters as SOA comprising Cloud, IaaS, EDA etc).

Clustering and dependency techniques are important. Not only do they allow you to plan sensibly, they also facilitate completeness reviews. And strangely I observe Gartner omit the area of semantic integration, a technology domain that I see as starting to become very important for many of our customers. Of course it's an integral part of the SOA cluster, and it's strongly driven by SOA maturity and the requirement to deliver on consistent information services. By chance I am publishing a report on the Progress Data Exchange Semantic Integrator this month, and I also note that last month Oracle acquired a provider of a similar product - Golden Gate. But the Gartner methodology apparently doesn't seem to surface such an interesting area.

The basic idea of a hype curve has been around for decades. I recall doing something very similar back in the mid ‘80s with an oil major client. But like all models, the deliverable is highly dependent upon some clear definitions and rigour that make the opinion based report credible and comprehensive. Sadly Gartner seems to have been caught up in it’s own hype.

^{^[1]} Gartner's 2009 Hype Cycle Special Report Evaluates Maturity of 1,650 Technologies
http://www.gartner.com/it/page.jsp?id=1124212

^{^[2]} Design Pattern: Differentiated Service, Fewer Interfaces than Components

http://www.cbdiforum.com/secure/interact/2000-12/design_pattern.php3

Repeatable, Reusable, Rapid?

2009-06-17T03:41:00.000-07:00

I am intrigued by the amount of attention and support being given to “In Praise of Slow”. Originally published in 2004, this book by Carl Honoré is an entertaining and thought provoking commentary on our “culture of speed”. Honoré grabs our attention in the opening pages by describing how his lifestyle has led him to optimize his time with his son, searching out the shortest books for the bed time story, and pondering on why Snow White couldn’t have made do with 3 dwarves! His thesis is that we should pay attention to detail and do important things right first time. Like the slow food movement, set up originally to combat fast food, it’s about preserving culture, heritage, localization and small scale.

I am instinctively supportive of this idea of “doing things right”. In our own industry I worry about unfettered offshoring, agile development adopted purely for speed, the compromise of architectural principles trading short term gain for life time cost.

I note Ron Tolido’s blog develops this theme and extends the idea to Slow IT. Ron suggests “It is about using the principles of Enterprise Architecture to create a platform for continuous business change. This is not a paradox: only on top of a simplified, secure and flexible foundation of building blocks we can orchestrate and change solutions on a daily basis.” He calls it “Slow IT, the art of careful technology”.

I am completely with Ron in rejecting the superficial - Web 2.0, panic package acquisitions and the like for use in serious enterprise business processes. Yes we need to transition enterprise systems to modern componentized architecture that permits continuous upgrade of smaller moving parts.

However for all that I do believe that Slow IT is not going to go anywhere fast! We already have Slow IT today and the opportunities for misunderstanding are legion.

Last week I attended a presentation from Nick Cheetham at the Department of Work and Pensions in the UK. He commented that his organization is one of the few that right now is experiencing growth – because the unemployment rate is set to treble. But for most of us the imperative is to do more with less. And it’s interesting to observe different responses to this pressure.

I see tangible evidence that companies are increasing the rate of offshoring, in order to cut costs. I see others slashing the number of projects and programs, and focusing on the core business. But the primary observable effect is redundancy – reduction in head count, which is driven simply by the numbers. Then it’s up to the retained staff to figure out how to do more with less.

Maybe “slow” is an unintended but inevitable consequence of the current situation the archetypal enterprise is in, but I suggest a more appropriate focus is along the lines of “repeatable, reusable, rapid”. And this applies to everything – processes, services, components, infrastructure, skills, etc.

In this month’s CBDI Journal we publish a report on Implementation Architecture and Automation Unit Specification. This is an area in which we have been teaching and advising for some time, but we realized we hadn’t documented the guidance. It’s a critically important area – you have the business designs, the service specifications, so how do you deliver an effective software design and implementation, and demonstrate to governance reviewers that you are complying with SOA and EA principles and policy? And it’s a classic opportunity area for practicing repeatable, reusable, rapid techniques.

Like many folk, CBDI has been talking and advising on matters relating to repeatable, reusable, rapid for years. I seem to recall the phrase “reuse before you buy before you build” was coined by a colleague in TI around 1994 when we were developing the ideas around Component Based Development. It seems what goes around comes around, which perhaps says “good things come to those that wait”. But that’s different to advocating “slow”, which seems a bit like turkeys voting for Christmas.

Modernizing Legacy – Thoughts on Analyzing and Classifying Existing Application Landscape

2009-06-03T01:17:00.000-07:00

The last few weeks I have been working on strategic direction for a government’s applications in a particular area of citizen support. It has prompted me to challenge some of our widely held assumptions about Legacy and how we manage it.

First what do we really mean by Legacy? The general understanding of legacy means something handed down from the past, an inheritance, gift or donation. Clearly in computing generally the term legacy has become synonymous with obsolete – still functional to some extent but does not work optimally with modern systems.

But this really isn’t adequate. Old doesn’t automatically mean obsolete – to be superseded. Old fashioned shouldn’t automatically mean redundant. What’s needed is a better taxonomy that allows us to communicate more meaning about the nature of the application portfolio.

I recall the presentation last year by Colin Smart of HBOS on their work in classifying their application landscape. Colin suggested an expansion of legacy to the following:

· Strategic – everyone should use this

· Retiring – no one should use this (but should use this instead)

· Contained – no one should use this (but we haven’t an alternative)

which I like a lot. It immediately provides an answer to my basic concern regarding supersession and or redundancy. Incidentally the Contained class relates to our Chernobyl pattern – which we describe as “wrap in concrete, don’t expect to replace any time soon”.

But I feel we could go further in the classification particularly to integrate with the emerging SOA.

A significant part of the application area we were analyzing had been developed using the Information Engineering methodology and presumably at that time the IEF (Information Engineering Facility), now renamed CA Gen. A particular feature of this delivery approach was to establish an integrated, business driven data model which transformed directly to an integrated database. Now regardless of the issues surrounding a tightly integrated database, it was interesting to see how easily the aging application portfolio could map to the Core Business Service layer because the integrated, business context database effectively inherited an implied Business Type model (BTM). Rather than publishing Underlying Services, the CA Gen application would be easily able to publish Core Business Services that would be directly called by the Business Process layer.

I just happen to have been looking at a CA Gen application, but of course back in the 80s and 90s Information Engineering was widely used, and I wouldn’t be surprised if many legacy applications exhibited similar characteristics. However it must be said that many older delivery technologies wouldn’t have the same internal integrity as the IEF, and so any original quality may have deteriorated with time.

But if you do discover this, the alignment of the (very large) application area with the BTM also provides more confidence that the application area could be componentized – to break up peer to peer calls (between modules) and to turn those into service calls, thereby reducing the unit of change, deployment and release. And believe me, the example I am advising on is a BIG application area, so there is real value in doing this. The same action opens up the possibility of replacing or reengineering selected modules if appropriate – simply giving the customer more options on how to manage the application area.

So to return to my taxonomy question, I recommend that there are more useful classifications that would really help to better communicate what’s really going on in the application estate – Usage (as per Colin’s suggestion), Service Layer and Separation.

Usage	Service Layer	Implementation
Strategic	Core Business	Components
Retiring	Underlying	Componentized modules
Contained	Underlying	Monolithic structure

In most cases I would imagine Strategic applications would be supporting Core Business Services directly. Sometimes there will be situations where the API architecture is inadequate, and where a Strategic application is shown as supporting the Underlying Service layer, it serves to highlight the potential conflict. Similarly if a Retiring application is supporting the Core Business layer, then some urgent action is needed.

It might be expected that Strategic applications would have some level of componentized implementation. Conversely if a Strategic application has monolithic implementation architecture it highlights the risk, likely cost overhead and reduced options.

I would be very interested in hearing about other legacy classification systems that members have used.

Principles - A prerequisite for effective design.

2009-05-07T09:06:00.000-07:00

There’s a long running broadcasting history radio program on Sunday mornings on Radio Telefis Eirann (the national Irish broadcaster RTE) edited and introduced by John Bowman. The program comprises interesting clips from RTE’s extensive broadcasting archives.

A couple of weeks ago the first item was a piece on Frank Lloyd Wright, the well known American architect, triggered by the 50th anniversary of his death. The piece recalled several young Irishmen who in 1950, sailed to America to work with Lloyd Wright for a few months.

One of the men, Sean Kenny, recalled that they were rather pleased with themselves having completed a rather adventurous crossing of the Atlantic, and were somewhat taken aback when they first met Lloyd Wright. Expecting to be put to work on architecture related tasks they were more than a little surprised to be given a pick and shovel and directed to work on the construction of one of Lloyd Wright’s designs. At the conclusion of their stay they recalled they didn’t come away with a deep insight into how to practice architecture. However they did learn a huge amount about the principles of using space in design to create solutions that worked for the user.

Kenny returned to Ireland and related how he set up as an architect. In the first two years he was very disappointed – prospective customers didn’t beat a path to his door, in fact during that period he obtained just one small engagement. Yet in later years Kenny became very well known as a stage and set designer for the Irish theatre.

Once again I return to a common theme. Deep understanding of architectural principles is a prerequisite for effective design. The primary challenge of SOA is interpreting the business goals into architecture that uses relevant principles to deliver an appropriate and affordable level of agility, and then exerting governance over the delivery and ongoing maintenance to ensure the integrity is not compromised.

But Kenny’s story also suggests that “architecture” shouldn’t always be a discrete activity. Rather it is a way of thinking that will be applicable to many roles and actors in a wider process.

This month, in not unconnected pieces, I write about Agile methods and their relationship to SOA. I am reminded that everything we do is a series of compromises. The issue with Agile methods is that the required compromises often appear unacceptable in context with architectural principles. Yet there are many aspects of Agile methods that can deliver real value to an agile architecture. The key is determining the important business goals and then using architecture principles and patterns that can deliver appropriately. Merely cloaking your activities in “agile methods” is likely to create a suboptimal result. Like Sean Kenny, we all need to understand what represents good architectural outcomes, and then be prepared and indeed capable of picking up our metaphorical picks and shovels and delivering effective solutions.

Finally, regular readers will know that while most of the Journal material is produced by CBDI people, occasionally we publish guest articles. In future I would like to increase this balance of guest material and I would like to invite CBDI members to consider making a contribution to the CBDI Journal. I am interested particularly in members’ experiences that develop and extend the body of Service Architecture and Engineering (SAE) practice guidance. For example practical experiences in service classification, policy setting, governance criteria, use of the SAE meta model and profile etc. If you would like to make a contribution, please contact me to discuss.

CBDI SAE Knowledgebase Roadmap

2009-03-25T09:57:00.000-07:00

I penned this note for one customer last night and figured it might be interesting to a wider audience:

Since Version 2, CBDI is updating the KB on a regular basis rather than issuing major releases at less frequent intervals. Also CBDI has separated out the development and production versions, and we now create migration packages as part of our normal process. Therefore this creates more flexibility for licensees.

Licensees may now choose to a) migrate Journal updates on a regular monthly basis, as this will be a relatively trivial task, and there will be demand from KB users for this to be made available when they see CBDI publishing and promoting new reports

b) to migrate other changes on a timeframe that is convenient to them.

In terms of the overall roadmap we published our intentions for 2009 in January. See: http://davidsprottsblog.blogspot.com/2009/01/cbdi-r-d-priorities-for-2009.html

We are tracking reasonably closely to this agenda:

Productivity, cost reduction and business value.
We have undertaken significant work on project management in the last few months and this will be scheduled for KB update in due course.
SOA as BAU
We have already published reports on CM, SLC management and tooling and are currently working with various vendors to produce tool specific guidance and repository schemas.

As part of current work on new eLearning modules (see below) we have moved our guidance forward on the Implementation, Deployment and Technology Architectures, plus the Governance discipline and we will incorporate this work into the task, technique structure shortly.
Standardization
We have recently published a major report on our work with UPMS and SoaML.
We are currently finalizing a detailed guidance note on TOGAF 9 and SAE and a journal report will publish very shortly; detailed mappings and guidance will follow in the KB. The comments re ITIL and other frameworks are very much in plan.
Better Business Modeling
Recent reports published in this area. We plan to continue this work. We are also giving a public workshop on the topic in June and expect the outcomes of that to be published in the KB.
Evolving SOA Architecture
On the plan

In addition to the above, we are currently delivering a major upgrade to our eLearning modules. These are designed to provide fundamentals level education beyond the architect community – which we perceive as a significant area of weakness in most organizations. Further modules will be planned to cover more advanced topics shortly. Our policy is that all eLearning modules are available to KB users.

In addition there is a constant and progressive process of refining our guidance based on customer feedback and our own consulting engagement activity. We have recently opened up the discussion forums in the KB and are requesting customers to provide feedback on development prioritization.

SOA in the Recession

2009-02-11T12:26:00.000-08:00

I don’t want to get political, but I am mildly puzzled by what’s going on in the real economy. Whether you look at the US, UK, Iceland (outstanding example) the primary cause of the current macro economic crisis is clearly a “credit bubble”; a huge expansion of debt way beyond borrowers’ means to repay. Yet the answer to the crisis in the US and UK at least seems to be to spend one’s way out of trouble! And whilst this may be perfectly fine for large countries that can sustain increased debt over the long term, for small countries, individuals and large and small companies there’s a compelling case to reduce risk – to reduce costs and to reduce or eliminate debt wherever possible to prepare for future uncertainty. So while politicians might like us all (businesses and individuals) to go out and spend, there’s no chance of this happening any time soon until future prospects become clearer.

Now let’s apply that same thinking to the IT space. In business systems terms we have created our own bubble. It’s happened without us realizing it over the past 20 or so years because most organizations (commercial enterprises and governments) have expanded application portfolios in an uncontrolled manner creating duplication, inconsistency, lower customer satisfaction, high cost and slow response to change. In the good times organizations were happy to spend, spend, spend with little or no governance over architectural integrity or consistency focusing only on short term needs. Effectively we have massively over inflated the assets of the enterprise, knowing all the while that under the surface there is a toxic mess that can’t be easily cleared up. Let’s call it the toxic IT assets.

Most projects today are still tactical in nature, responding to immediate needs but this shouldn’t rule out SOA. Service architecture is a multi-faceted beast which should create stronger linkage between business and IT architecture, standardize common services where appropriate and importantly facilitate rationalization of the existing mess. So let’s go back to my rhetorical question. Just because we are in a crisis do we continue to expand the bubble? Although we may be cancelling projects and reducing activity, unless we get smart on architecture we are increasing the toxic mess. Because that’s effectively what’s being advocated by suggesting that SOA activity will be slowed, or curtailed because the business cannot afford it.

The issue of “whether to SOA or not” will arise if the SOA budget line is treated exclusively as an IT responsibility. In most enterprises the incremental cost of delivering properly architected applications and infrastructure is actually a very minor cost in the overall scheme of things. Further presented with the business impacts of doing things right, my experience is that business management see the value very easily. Application rationalization benefits in particular can be achieved in the relatively near term, and have immediate benefits in terms of reduced complexity as well as reduction in real costs of ongoing managed operations, maintenance and licenses.

The right course of action is to involve business management in project initiation and governance decisions; to share with them the cost and benefits of architectural decisions, and to do this on the basis that service architecture is business as usual (BAU) and the key questions to be answered revolve around the level of standardization, componentization and rationalization in business and IT.

In these recessionary times it would seem very appropriate that we focus SOA efforts more strongly on IT rationalization - reducing risk, reducing cost and preparing for future uncertainty and create clear linkages between the IT and business rationalization efforts. Now we know how to avoid further inflating the bubble, it would seem reckless not to advise the key stakeholders how to better manage the organization’s assets and reduce business risk.

SOA Concept Standards

2009-01-21T03:58:00.001-08:00

In the first half of this decade there was big push on Web services standards, with a (fairly) good convergence around a core set of standards. However the state of SOA concept standards – that is standards underlying the architecture and engineering concepts - is a very different matter.

There are at least four international bodies currently developing standards around SOA - OASIS, OMG, Open Group and W3C, plus some important national bodies (US Federal Government, DoD, MOD).

The key standards bodies have several relevant artefacts.

- The OASIS SOA Reference Model TC has approved an SOA reference model – OASIS-RM.

- The Open Group is working on an SOA reference architecture and has published an SOA Ontology.

- The OMG has recently released a draft specification of SoaML, the SOA Modeling Language, a UML profile previously UPMS.

The DoD and MoD are looking to use UPDM (UML Profile for DoDAF and MoDAF) from OMG. This profile is relatively close to SoaML and shows that these three groups seem to be coming closer together. This is likely due to the fact that DoD/MoD needs to use standard tools to do their modelling and the OMG is the organization that develops the specs used by most of those tools. The Open Group have based their Ontology on the OASIS-RM. However beyond this it is hard to see much convergence between the bodies.

In addition to lack of convergence between the bodies, it is also obvious that there is widespread inconsistency between parallel groups within standards organizations. As ever multiple, incomplete standards is nothing new. As “users or advisors” we have to assess what value each candidate standard brings and whether it is fit for purpose.

In the context of SOA there are several considerations:

What value do (specific) standards bring? Common vocabulary, high quality, reusable concepts, standard artefacts that can be both reused and shared?
And conversely what is the cost of inconsistency?
And crucially is the standard providing the right level of detail in a manner that supports practical use?

CBDI has pioneered many aspects of SOA standards. The CBDI Frameworks, meta model and UML profile pre-date the work of the standards bodies working on defining SOA structural standards. The CBDI frameworks have been made widely available; the CBDI SAE Meta model and related UML Profile have thousands of downloads and are widely used. Some industry groups have based their work on it. (Notably HL7). Many CBDI Forum members have used the SAE meta model as a basis for their asset management schemas, and the UML Profile as a basis for SOA architecture and design deliverable formats.

For CBDI and our Forum member users the question is how do the CBDI models compare and should they move to adopt one or more emerging standards and when?

CBDI SAE is an SOA methodology which provides considerable depth in practice guidance based on the rigorous underlying meta model. The OASIS-RM is a conceptual model of SOA and there is significant alignment between the CBDI work and OASIS-RM. However CBDI has focused it’s efforts differently to OASIS – The CBDI Meta Model is not purely a conceptual model, rather it is a working level, detailed meta type model that provides the basis for life cycle meta data that will be managed in registries and repositories. In contrast the OASIS-RM and Open Group concept models provide either no or limited cardinality or optionality rules, and are therefore open to interpretation, leading to inconsistencies in implementation.

CBDI is monitoring the work of OASIS-RM and may a) contribute to certain areas and b) consider alignment of the CBDI meta model as appropriate.

CBDI is also monitoring the work of the Open Group and plans to assess potential for alignment when the current SOA work is published. The Open Group SOA Ontology is a derivation of the OASIS-RM, and is a similar concept level model. We have discussed with the Open Group the possibility of donating the CBDI Meta Model as a way to deliver further detail.

CBDI has supported and contributed to the UPMS work, now renamed SoaML. There is significant alignment between key areas of the CBDI meta model and UPMS. The SoaML is somewhat more comparable to the CBDI models insofar as it is fully detailed. Where it diverges is that the SoaML is a UML profile – and therefore its purpose is to support tool design; the CBDI models provide richer metadata whereas SoaML simply provides a means to represent the basic elements in a model.

You may well ask the question, so what’s the difference between the SAE UML Profile and the SoaML? First the SAE Profile is broader in coverage that the SoaML – it spans the entire service life cycle; we would be the first to say, the leaf node detail is not necessarily fully consistent and detailed, but that’s the intent, whereas SoaML is focused purely on the service modeling domain. Second the SAE Profile supports the SAE methodology.

For end users the question is how these various standards should be used. The conceptual models (OASIS-RM and the Open Group Ontology) are useful in providing conceptual consistency. But they do not provide the necessary detail that informs repository and deliverable design. In contrast the SAE meta model and UML Profile are being widely used to define asset schemas and deliverables, and the SoaML will of course be used by tool vendors to develop same.

We are currently engaged in a detailed assessment of the SoaML and SAE profile and anticipate we will report in February. Following this we will canvas our Forum members’ opinion on the level of convergence that is desirable.

To summarize, it seems to us that higher level conceptual standards have their place while the industry is learning. But as the industry and its customers demand production strength support, the requirement is for standards that guide deliverable creation and tooling. This has been the objective guiding CBDI work for some years, and while we see some merit in alignment with the higher level conceptual models, we believe alignment with SoaML is a priority because of its comparable rigour. The levels of inconsistency observed in the purely conceptual models will be extremely difficult to resolve at that level. For that reason we believe the standards will evolve from the CBDI SAE and OMG SoaML models because they are at the level users will require.

The CBDI models are organized into packages because the breadth of SOA clearly indicates multiple domains. It seems likely that standards will evolve at different rates in each of the domains. Alignment around SoaML is clearly going to happen in the Modeling domain. We envisage a plug and play approach where different standards bodies will develop strengths in different domains. In this process CBDI will continue to influence events and to provide a detailed Business Type Model “view” that helps real users make sense of it all.

Finally, we can all see the Web services standards were hugely successful because IBM and Microsoft drove the process. Similarly today we note IBM is a big supporter of the OMG, and specifically SoaML, and significantly Microsoft joined the OMG last September.

Notes:

- This discussion is focused on SOA Concept standards. I will return to the topic of framework standards in due course.

- The comparison report between SoaML and SAE is planned for the February CBDI Journal.

CBDI R & D Priorities for 2009

2009-01-20T08:20:00.000-08:00

Every year CBDI attempts to make an assessment of where we are at, and the key drivers for our next years R & D program. Last year (2008) we identified “make it real” as a theme, and in many respects we have delivered on this. Clearly the major achievement this year has been the huge advance in the CBDI Knowledgebase. With Release 2.0 now launched we now have a body of guidance that is now getting used in earnest by early adopters and ready for widespread usage. In addition the tooling and profile work we did earlier in the year, plus detailed modeling and a strong emphasis on practices. More recently we have been focused on addressing “real world” topics which has spawned M&A, Multi-channel, etc.

Looking forward to 2009 the macroeconomic situation is of great concern, but my reading is that SOA is moving out of the hype phase which has been characterized by tactical application to becoming business as usual (BAU). Larger enterprises, governments and consultancies are now acting to put in place high quality, repeatable practices. So while the credit crunch will have a temporary effect by reducing overall demand, there is evident and increasing demand for methods and guidance in the area of quality, repeatability and productivity.

For CBDI research I see five primary areas of demand for our R & D:

Productivity, cost reduction, business value. – Survival in recessionary times. Focus on key cost and headcount reduction strategies including Testing, Offshore and Outsourcing, Managed Services. Estimation. M&A.
SOA as Business as Usual. We have already started articulating this message, but in 2009 we need to make a major transition where we go beyond SOA to BAU. For example:- develop resources such as templates to the next level of detail (XML schema).- work with industry providers to create tool specific reference implementations- refine service specification and other deliverables by life cycle stage - export task lists to Project Management tools- reference architectures for technology platforms (e.g SCA)
Standardization. CBDI has blazed a trail where others are following. Our meta model and profile, SOA process and life cycle have been widely adopted, cloned and copied. We have donated our core concepts and models to various groups including the OMG, HL7 and TM Forum. We have also participated in the OMG’s UPMS. In 2009 we will look to influence more rigorous concept models and align our work with emerging standards. We will look to integrate or align with de facto Project Management approaches, ITIL, and other frameworks (possibly TOGAF as it matures).
Better Business Model. CBDI believe that de facto business modeling and requirements approaches are woefully inadequate for purpose. Lots of the individual components (techniques, languages etc) are fine for a narrow purpose, but they are all predicated on yesterday’s world in which we abstracted away from the real business problem in order to make realization with the limitations imposed by technology. Today many of these technology limitations are removed, yet we still draw line and box diagrams, and never give a thought about a richer conceptual model that underpins a loose coupled world that is in constant evolution.
Evolving SOA Architecture for Cloud, SaaS, IaaS and Virtualization. SOA architecture is an integral component of the emerging technology environment. There is a need to move our reference framework forward to provide initially, the conceptual Framework for SOA in these emerging domains.