Model-Driven SOA

UPDATE: I’ve posted a new article SOA is dead; long live Model Driven SOA, with more details on how to implement Model Driven SOA.

Enterprise software has become too complex to be effective! As a percentage of total investment, IT rose from 2.6% to 3.5% between 1970 and 1980. In 1990 this has increased to 9% and by 1999 to 22%! A huge amount of money which is spend for 70% to 80% to keep existing systems running. In many companies, it takes the IT department one to two years to implement a new strategic initiative [1]. How did this happen?

Proliferation of Complexity

According to Cynthia Rettig [1] this situation comes from a proliferation of complexity in software code. It is estimated that for every 25% increase in complexity in the tasks to be automated, the complexity of the software solution itself rises by 100%! Triggered by centralization of software through enterprises and automation of more and more tasks (because sometimes business managers seems to think that only buying more software will cut costs and improve operations) the complexity of tasks to automate has increased very fast. This has lead to an enormous complexity in the software trying to automate these tasks. No single person within an organization could possibly know how a change in one part of the software will affect its functioning elsewhere.

SOA, the solution?

The promised solution for this big problem is SOA (Service-Oriented Architecture). By using a modular approach SOA promises a separation of functionality on a more higher level than object-oriented programming does, thereby reducing the complexity or at least structuring it. By separating the implementation and orchestration of services, the resulting system can also easily be changed by orchestrating the services in another way or by just using other services in a certain orchestration. That’s the promise, but how is reality?

The transformation to SOA is a very long process for enterprises. Not only technological changes have to be made, organizational changes can be very deep and radical too! However, it is questionable whether enterprises can actually maintain a focused strategy long enough to align their core business processes with IT [1]. The current dynamic business environments doesn’t give enterprises that time. However, these environments call for the flexibility promised by SOA. It’s a bit the chicken or the egg! I think we can conclude that an enterprise at least needs a sound Enterprise Architecture (see Implementing Enterprise Architecture ) to overcome this problem, but that again is a whole paradigm shift for a lot of enterprises.

Beside these organizational problems also a couple of technical problems has to be solved before SOA can makes its promises come true. First, the identification of services in a SOA isn’t trivial (see SOA and Service Identification ).
Flexibility highly depends on the granularity of the services used. In principle we can say that if services become smaller, the flexibility will increase. However, complexity will also increase if services are getting smaller and smaller. This situation of choosing the granularity of services suggests a ‘green-field’ situation, which almost never exists in practice. When transforming an existing IT-landscape into a SOA it is even more difficult to have an end result with loosely-coupled services which are granular enough to offer some of the flexibility enterprises desperately need. In most situations enterprises are ending up with, what I tend to call, some SOA lipstick on a pig. In such cases SOA is just an additional layer on existing software, resulting in even more complexity. Software engineers will still have to deal with the layer of enterprise applications below the modular business processes.

I tend to think, that SOA isn’t the solution for the problems enterprises are dealing with nowadays.

Model-Driven SOA, we need it!

Cynthia Rettig concludes that enterprises need a closer communication and collaboration between IT and business. A recent study by Forrester Research Inc. of Cambridge, Massachusetts, found that only 28% of CEOs thought their CIOs were proactive or creative in terms of business process improvement. Forrester’s advice to CIOs is to get more deeply involved in the business issues and educate executives on what IT is and what it actually does. [1]. Cynthia Rettig adds that its maybe also time for business executives themselves to become more proactive. Executives could educate themselves more about technology.

I think Model-Driven SOA can solve this problem, while also solving the "SOA lipstick on a pig"-problem. By Model-Driven SOA I don’t refer to process modeling tools. Most of those tools can only orchestrate webservices, meaning that they only can create a layer on top of existing webservices. These webservice mostly consist of legacy applications, so we don’t reduce any complexity in this way. What we need are full range tools doing information/data modeling, presentation modeling and business-logic modeling (see From Software Engineering to Business Engineering ).

Model-Driven SOA has a lot of advantages, making it a more feasible solution for the problems outlined before than SOA ever can be:

• Business solutions created by Model-Driven SOA still comply with the principles of a Service-Oriented Architecture.
• The model is the code: no compiling or translation needed. This really is a big advantage compared to code generators, you will not end up with non-maintainable software code.
• Models are in "business language", thereby bridging the gap between business and IT, CEO and CIO.
• It’s not only possible to integrate existing enterprise applications (EAI), but you can replace the existing complexity part by part. Because of the Model-Driven approach development of new system parts can be done very fast.
• Once the core business processes of an enterprise are transformed it will finally have the flexibility so hardly needed to gain a competitive advantage. Changes in a business process can be changed in the model, the software automatically changes too. That’s what I call Business-IT alignment!

I tend to think, that Model-Driven SOA finally can bring us Business-IT alignment, making enterprises agile!

Model-Driven SOA: dream or reality?

Sounds good, but is it even possible? I think so, of course difficulties exist which have to be solved. Most complex is finding good models which are both business-understandable and executable. Another thing is connecting all models needed and checking the consistency between them. By example, in a process model you want to use (or refer to) elements from your information model and presentation model.

The "Ontologies for SOA" working group of "The Open Group" works on ontology models enabling Model-Driven SOA. If their work can become mature fast enough they can play a big role in vendor-independent modeling formats. Of course vendors have to create a visual layer above it making the ontologies business understandable. More pragmatic is our (Mendix’s) current work on Model-Driven SOA. We’re really far in modeling enterprise applications in a ‘business-way’. Also connecting models and checking consistency is done in a sound way, meaning that you can’t execute a non-consistent model.

I’m sure, that Model-Driven SOA is the way to go. Not in a far, far future, but now. It’s possible!

[1] Cynthia Rettig, The Trouble With Enterprise Software, MITSloan, August 8, 2007. http://sloanreview.mit.edu/smr/issue/2007/fall/01/

[2] The Open Group, Ontologies for SOA, http://www.opengroup.org/projects/soa-ontology/