Friday, September 24, 2010

IT Transformation Lifecycle. The Wrap Up.


Perhaps you’ve noticed but up till now my series of blogs have followed a generic IT Transformation life cycle that roughly has these steps:
  1. Identifying the business needs
  2. Defining the drivers for Transformation and developing the business case
  3. Evaluating the future
  4. Understanding the scope and requirements
  5. Defining the technology strategy
  6. Making the Business Case
  7. Applying SOA as a Solution Architecture
  8. Defining the Services Taxonomy and the SOA Framework
  9. Applying the right SOA approaches & techniques
  10. Engineering the solution
  11. Establishing the right Governance and team
  12. Executing on the project via appropriate Project Management techniques
  13. Migrating to the new system
The diagram below summarizes this life cycle in terms of the purpose for each step:
We have now reached the end of the cycle. Upon successful migration you and your team are entitled to celebrate and reward the key performers. There is a lot to be happy and grateful for. However, you’d be wise to keep your cell phone active even as you celebrate. Initially at least, there is the likelihood that you will be receiving a large number of calls regarding deployment problems. Truth is that it will take some time before the system becomes truly stable. Indeed, it is a well know fact that all systems follow the so-called “bathtub shape” when it comes to failure rates:


A new system starts with a high failure rate that will (hopefully) diminish in time until it eventually becomes stable. True, failures will never go away during the system’s productive years (the “Rubber Ducky years”, I call them), but the failure rate should remain reasonably low and under control. However, after cumulative changes and stresses of continued improvements are applied throughout the years, you will notice that, with the weight of age, natural entropy eventually takes its toll making the system more and more unstable. Problems will arise; changes will become more difficult, meaning that, in time, the system will become sufficiently stiffened and inflexible and be ripe for yet another IT transformation!

That’s right folks.  From the moment you cut that ribbon debuting your new system, the system becomes “legacy”.  So, what was really accomplished after all the effort and the millions in investment to create a new solution? Well, if you did things more in the right way than in the wrong way during this transformation, the “length” of the bathtub will hopefully extend for many more years than it would have otherwise. That’s right you’ll move from a bathtub shape to a swimming pool. Also, the functionality of the new system will have been much improved. All in all, a good IT Transformation effort should be something to delight in. Like a good bath!
A new cycle does not imply repetition or more of the same. IT Transformation is about progress. The true shape of progress resembles the picture below, an upward cyclic progress towards new solutions.



Mirroring this “spiral of progress”, my blog is also resetting somewhat.  It has now reached a plateau. My future blogs will continue to cover the general theme of IT Transformation.  Getting set to walk up another flight of stairs, I will cover general aspects related to emerging business needs.  After all, we are now facing the Social Media explosion and the heightened pervasiveness of mobile computing. Who can say what the true impact of the advent of iPad and soon to follow iPad-like devices will be?  I intend to continue discussing current drivers for transformation and technologies that will shape the future.
Rather than making a weekly appearance, I will endeavor to publish an article every other week, time and cycle of life permitting. 

Till then . . .


Friday, September 10, 2010

Evaluating the SOA System Migration Alternatives


Even with traditional systems designs you have an array of options as to the best way to approach your system’s migration. The ultimate migration strategy will be driven by the specific business requirements and characteristics of your system. For example, you can do a so-called Big-Bang approach or you can migrate on a functionality basis. SOA gives you an additional option of migrating on a layer-by-layer basis (Presentation, Process and Data layers)[1].
A system wide Big-Bang migration should be avoided at all costs, but if you are able to segment the target audience so that you can deploy the system with a series of coverage based “mini-big-bangs”, you’ll have a great option. For example, if you can “big bang” a particular region or country to the new system (the smaller, the better, as you are using them as guinea pigs), and then gradually add new regions as you grow in confidence, this can be a winning strategy.
Unfortunately, this type of coverage-based migration is not typically possible in transformed systems. After all, IT Transformation programs tend to be enterprise-wide and holistic in nature. In these cases, you need to check whether is possible to graduate the migration on a function by function basis.
In functional-based migrations you gradually introduce specific parts of the presentation, process, and data layers for each functional subset identified. An example of a functional based migration would be one in which you first introduce the new CRM system, then add additional functional blocks, gradually sun-setting the legacy environment.
In order to identify if functional-based migrations are feasible, I suggest preparing a dependency map listing the specific subsystems that support each autonomous function. This dependency map will identify the services and subsystems in each SOA layer that can be deployed as standalone deliverables.
As you may imagine, functional-based migrations will demand integration of the legacy to the new function—at least for the duration of the migration. The good news is that you can leverage SOA’s ability to integrate legacy systems to new systems via the service encapsulation (i.e. service wrappers) of old functions. The bad news is that this type of integration can quickly become so onerous or complex that functional based migration becomes unmanageable.
Another approach is taking advantage of the fact that well-designed SOA systems facilitate layer-by-layer migrations. This additional level of granularity is another tool in your arsenal whereby segments of the new functionality are introduced gradually via stepwise introduction of the new SOA layers. The question then is which layers to introduce first and how. In this case you will be faced with an array of choices. What you choose to do, again, will depend upon the specific characteristics of your project. Following are sample pros and cons of each approach:
Migrating the Interface first.
This implies mounting the new GUI and interfaces against the legacy environment.
Pros
o Technically this is a low risk approach, provided the new interfaces easily replicate the functionality of the legacy interfaces.
o If the new GUI is friendlier and provides some GUI-specific functionality enhancements, you will be able to benefit from this early on.
o It can be helpful in expediting the training of users on the new system interfaces.
Cons
o The business team could be disappointed and confused by this approach as they will naturally expect improved functionality that might not be yet available due to the continued use of the old backend systems during the migration.
o You will need to train the staff prior to completing migration. Depending on needed future refactoring for processing and data layers, this training could become outdated.
o You will need to develop an encapsulated SOA layer so that the new GUI can access the legacy. This is essentially throw-away work.

Combined GUI and Processing Layers First
While theoretically you could first introduce just the processing layer, setting aside the legacy and data layers till a later time, this approach is usually not feasible. You would need to decouple your legacy presentation logic from business logic, and chances are that this would not be trivial. Accessing legacy data from a new processing layer is perhaps easier, but the new functionality will then be severely constrained by the legacy data schemas. A more practical approach is to move the processing and the GUI layers as a unit. You will then have to take into account these considerations:
Pros
o You can introduce enhanced functionality that does not depend on the data layer. For example, new validations or user workflows.
o Since data migration is usually the trickiest thing attempted in a migration, by first migrating the combined GUI and processing layers, you will be able to replicate existing functionality with minimum risk.
o You will more easily be able to fallback the system, if needed.

Cons
o The business areas will expect more functionality than you can provide. e.g. “Where is the internationalization?” This might not be feasible as long as you continue using old DBMS schemas!
Processing and Data Layers First
This is equivalent to a city building infrastructure (water mains, electrical wiring, sewage) but doing so in a way that the population remains unaware of the changes below ground.
Pros
o You can do the heavy lifting without troubling the users.
o Easy to fallback migration as you are making the change in a transparent fashion to the user
Cons
o You will need to emulate the legacy interface processes and, depending on the legacy system, this might not be doable. Note that I am not recommending modifying the legacy presentation layer to talk to the new system! This would entail too much throw away work.
o Lots of work that nobody will notice at first. Management will be asking, “Where is the beef?”

Data Layer First
Frankly, the idea that your legacy processes will be modifiable to use new data schemas is not a practical one. At best, you can expect to do a pure data replication and transformation as an initial stage in what will become a Processing plus Data migration. Migrating only data is not a step I would recommend.
In any case, any data migration effort should endeavor to create a Y-split that will ensure that legacy and the new data can coexist at least for the duration of the migration. When migrating processes plus data, you should ideally create process switches that will allow Y processing (parallel processing) of transactions between legacy and the new system. It would be a trial run for the new system with the plan to eventually refresh the new system and start from scratch.
Under any scenario, make sure to use appropriate monitoring and control switches as well as planning for fallback options. Place the appropriate switches to turn the new elements on and off as needed, and include the necessary logging to ensure you know what’s going on.
The best migrations are those that get you to that brave new world in one piece, with satisfied business users, and with your sanity intact!


[1] See my previous blog on the SOA Distributed Processing Pattern at: http://www.soa-transform.com/2009/08/soa-distributed-processing-pattern_20.html)

Evaluating the SOA System Migration Alternatives


Even with traditional systems designs you have an array of options as to the best way to approach your system’s migration. The ultimate migration strategy will be driven by the specific business requirements and characteristics of your system.  For example, you can do a so-called Big-Bang approach or you can migrate on a functionality basis.  SOA gives you an additional option of migrating on  a layer-by-layer basis (Presentation, Process and Data layers)[1].
A system wide Big-Bang migration should be avoided at all costs, but if you are able to segment the target audience so that you can deploy the system with a series of coverage based “mini-big-bangs”, you’ll have a great option.  For example, if you can “big bang” a particular region or country to the new system (the smaller, the better, as you are using them as guinea pigs), and then gradually add new regions as you grow in confidence, this can be a winning strategy.
Unfortunately, this type of coverage-based migration is not typically possible in transformed systems. After all, IT Transformation programs tend to be enterprise-wide and holistic in nature. In these cases, you need to check whether is possible to graduate the migration on a function by function basis.
In functional-based migrations you gradually introduce specific parts of the presentation, process, and data layers for each functional subset identified. An example of a functional based migration would be one in which you first introduce the new CRM system, then add additional functional blocks, gradually sun-setting the legacy environment.
In order to identify if functional-based migrations are feasible, I suggest preparing a dependency map listing the specific subsystems that support each autonomous function. This dependency map will identify the services and subsystems in each SOA layer that can be deployed as standalone deliverables.
As you may imagine, functional-based migrations will demand integration of the legacy to the new function—at least for the duration of the migration. The good news is that you can leverage SOA’s ability to integrate legacy systems to new systems via the service encapsulation (i.e. service wrappers) of old functions. The bad news is that this type of integration can quickly become so onerous or complex that functional based migration becomes unmanageable.
Another approach is taking advantage of the fact that well-designed SOA systems facilitate layer-by-layer migrations.  This additional level of granularity is another tool in your arsenal whereby segments of the new functionality are introduced gradually via stepwise introduction of the new SOA layers.  The question then is which layers to introduce first and how. In this case you will be faced with an array of choices. What you choose to do, again, will depend upon the specific characteristics of your project.  Following are sample pros and cons of each approach:
Migrating the Interface first.
This implies mounting the new GUI and interfaces against the legacy environment.
Pros
o   Technically this is a low risk approach, provided the new interfaces easily replicate the functionality of the legacy interfaces.
o   If the new GUI is friendlier and provides some GUI-specific functionality enhancements, you will be able to benefit from this early on.
o   It can be helpful in expediting the training of users on the new system interfaces.
Cons
o   The business team could be disappointed and confused by this approach as they will naturally expect improved functionality that might not be yet available due to the continued use of the old backend systems during the migration.
o   You will need to train the staff prior to completing migration. Depending on needed future refactoring for processing and data layers, this training could become outdated.
o   You will need to develop an encapsulated SOA layer so that the new GUI can access the legacy. This is essentially throw-away work.

Combined GUI and Processing Layers First
While theoretically you could first introduce just the processing layer, setting aside the legacy and data layers till a later time, this approach is usually not feasible. You would need to decouple your legacy presentation logic from business logic, and chances are that this would not be trivial. Accessing legacy data from a new processing layer is perhaps easier, but the new functionality will then be severely constrained by the legacy data schemas. A more practical approach is to move the processing and the GUI layers as a unit. You will then have to take into account these considerations:
Pros
o   You can introduce enhanced functionality that does not depend on the data layer. For example, new validations or user workflows.
o   Since data migration is usually the trickiest thing attempted in a migration, by first migrating the combined GUI and processing layers, you will be able to replicate existing functionality with minimum risk.
o   You will more easily be able to fallback the system, if needed.

Cons
o   The business areas will expect more functionality than you can provide. e.g. “Where is the internationalization?” This might not be feasible as long as you continue using old DBMS schemas!
 
Processing and Data Layers First
This is equivalent to a city building infrastructure (water mains, electrical wiring, sewage) but doing so in a way that the population remains unaware of the changes below ground.
Pros
o   You can do the heavy lifting without troubling the users.
o   Easy to fallback migration as you are making the change in a transparent fashion to the user
Cons
o   You will need to emulate the legacy interface processes and, depending on the legacy system, this might not be doable. Note that I am not recommending modifying the legacy presentation layer to talk to the new system! This would entail too much throw away work.
o   Lots of work that nobody will notice at first. Management will be asking, “Where is the beef?”

Data Layer First
Frankly, the idea that your legacy processes will be modifiable to use new data schemas is not a practical one. At best, you can expect to do a pure data replication and transformation as an initial stage in what will become a Processing plus Data migration.  Migrating only data is not a step I would recommend.
In any case, any data migration effort should endeavor to create a Y-split that will ensure that legacy and the new data can coexist at least for the duration of the migration.  When migrating processes plus data, you should ideally create process switches that will allow Y processing (parallel processing) of transactions between legacy and the new system. It would be a trial run for the new system with the plan to eventually refresh the new system and start from scratch.
Under any scenario, make sure to use appropriate monitoring and control switches as well as planning for fallback options. Place the appropriate switches to turn the new elements on and off as needed, and include the necessary logging to ensure you know what’s going on.
The best migrations are those that get you to that brave new world in one piece, with satisfied business users, and with your sanity intact!


[1] See my previous blog on the SOA Distributed Processing Pattern at: http://www.soa-transform.com/2009/08/soa-distributed-processing-pattern_20.html)