Sunday, August 29, 2010

Migration Strategies

IT transformation is a perilous business and regardless of the emphasis you place on the quality of the product and the testing, failures will occur. It therefore makes sense to plan for failure. 
The key elements to ensure controlled migration are:
·         Doing the migration in phases
·         Ensuring migration is fully reversible, at least during the initial phases
·         Defining safe fall back points when doing a migration in the more advanced phases
·         Allowing accelerated end run migration
You must ensure that while attempting the migration you don’t leave any orphans behind—missing functionality, legacy processes and systems, etc. This validation is not trivial given that, in most instances, the new system will not be formally equivalent to the old. That is, the old system may have data bases, reports, process tools and other features that aren’t needed in the new system or that are being covered via different means.  Ensure you are leaving nothing fundamental behind by assessing, on a case-by-case basis, the way the old system functionality will be provided in the new system. This is,  map the functionality as opposed to the system components.
Additionally, you will need to keep a strong focus on data. If you are performing database conversions from the old system to the new, you should double-check that the new databases maintain all the needed index keys and metadata information. Perhaps the old system uses some data that’s a result of processes that will be sunset, but you must also do a referential integrity check to ensure you don’t end up neglecting important information such as historical files or exception logs.
In order to achieve a flexible migration, you will need to identify the various subsystems that can be moved without affecting each other. Also leverage the power provided by the new system’s SOA capabilities to wrap-and-encapsulate old business logic and infrastructure as a process-step for a controlled migration.
In general, the more you accomplish a migration plan that allows a module-by-module migration, the better the chances of ultimate success.
In addition to allowing staggered migration of components, you can reduce migration risks by investing in automation of key processes required during the migration.  By now, you should see a trend: migration planning may require a certain amount of extra development. Like the scaffolding work needed to create building structures, this work may support only the creation of the new system and may be disposed of once the new system is in place.  Most project plans I’ve seen, fail to account for the need to develop these migration support elements; thereby forcing delays or affecting the quality and viability of the process.  No one likes to spend time and effort on work that ultimately may be thrown away, but I have also been pleasantly surprised to find that automated processes, originally envisioned as assisting specific migration steps, becoming part of the reusable toolset for ongoing operations.
Having said this, the next question is how to best migrate a complex system? In particular, if we are following a 3-Tier architecture model, the logical question to ask is whether one should migrate everything at once or focus first on migrating the data? Some suggest it is best to first migrate to the new user interface first and so on. What’s best? 
More on this, next week. . .

Friday, August 20, 2010

Migration Tenets & Principles

The transition from the existing system to the target system is where the success or failure of the effort will ultimately be determined.  Too often, wrongly placed beliefs in the solidity of the target system encourage risky migration moves. Other times, people forget that the migration strategy must be adapted to the type of systems they are migrating from.  
For example, while most of the current enterprise-based legacy environments are based on Mainframe technology, it is also true that there are several legacy environments that are remnants of the “distributed” PC trends of the eighties and the “Mini-computer” architectures of the nineties.  The way you migrate from a Mainframe environment is likely to be very different from the way you migrate from a distributed Novell server farm. Already, many a company that has launched its technology base with first generation Web technologies is now facing the challenge of migration to more current technologies.  
Not all migrations are equal. In general, the migration strategy you must adopt will be a compromise of business requirements, technical capabilities, budgetary constraints, and cultural style. The outcome will define how aggressive or moderate the migration strategy must be as well as the detailed transition steps you must follow.  
Pursuing general “sanity-proven” principles should help you define the specific migration strategy. This strategy will define more narrowly defined tenets and standards.  At a minimum, the following migration principles must be considered:
·         Make the migration a planned & controlled exercise.  Ideally, make the migration a phased effort. Each migration step should have a clearly delineated objective. Avoid Big-Bang migrations as much as possible. You can minimize migration risks by making transition steps as granular and manageable as possible.
·         A corollary to the previous tenet is that you shouldn’t force an all-or-nothing migration scheme. The migration plan must be as flexible as possible, even when this flexibility adds cost or latency to the migration process.
·         Be pragmatic. You don’t have to throw away the baby with the bathwater. If you can, leverage technology capabilities from an existing system whenever possible.
·         Decouple the migration process from specific infrastructure dependencies as much as possible. Use middleware layers or service encapsulation to minimize risk.
·         While the end state will comprise novel technology elements and functions, from a functionality point of view, the process should follow a migrate-then-extend precept. In other words, don’t try to solve world-hunger in a single step. Oftentimes, a migration step is needed simply as a tactical scaffolding process for, say, data access and system management in preparation for future migration moves.
·         Avoid disruption to components not being migrated whenever possible. The two previous tenets should help you achieve this.
·         Without a doubt, there will be new subsystems created by the IT Transformation that can be introduced without the need to migrate something old.  These new subsystems can be introduced to the environment as soon as available and can be integrated to the legacy as needed via services, but their introduction does not preclude the need for appropriate regression testing.
·         Extract-Transform-Load (ETL) existing data as much as possible, but don’t forget that a new system requires clean, well structured taxonomies. A migration is a cleaning house exercise and this is a step where you should also focus on doing all necessary data cleansing and normalization work. Migrating of old content can sometimes be more painful and ultimately more unfeasible than simply creating or acquiring new content from scratch. Data that is unreliable or not comprehensive must be re-entered as part of the migration plan.
·         Ensure the migration process is complete and comprehensive in replacing all the appropriate legacy components and data elements. Remember that the end system will only be as strong as its weakest link. If you fail to migrate a key component that could become a bottleneck or a reason for some of the newly promised functionality to not be implemented, then the migration will have failed.
Finally, don’t make “Falling Forward” a strategy. Once, while planning for a migration, I suggested allocating some extra time as a cushion for migration fallback and other contingencies.  Committed to a fixed deadline, the project leaders looked at me with worried frowns that indicated they weren’t keen on the idea. “If we encounter a problem we’ll fall forward; that’s the strategy,” one stated reassuringly.  Everyone else nodded as if “falling forward” really was a valid strategy.
Let me say this in no uncertain terms, this is utter non-sense.  When it comes to these kinds of mission-critical projects there is no such thing as “falling forward.” Falling forward is a sure way to break your teeth. Perhaps when dealing with simple, non-mission critical projects we can “fall-forward” into forcing the deployment of a project even if this means falling flat on our face rather than accepting that the legacy system should be restored and the attempted migration temporarily abandoned. When dealing with IT transformation of mission critical systems there’s no way to “fall forward”.
Mind you. I understand that during the life of a project there will come the time when one has to cut the umbilical cord of safety and make the new project final. That moment, when we finally cross the Rubicon, will come with its natural nervous uneasiness. However, we should have presumably proven the new project will work at least as well as the legacy, that we are comfortable in the belief we are not risking the life of the business. Even if one is to acknowledge that it is not feasible to expect absolute perfection prior to turning the legacy system, it is paramount to cover the “derriere” of the migration by including well-planned contingencies and remediation actions. Then, and only then, can you abide by the Evel Knievel  style of “falling forward”.

Friday, August 13, 2010

Prototypical Migration Goals

The primary purpose of a migration plan is to successfully introduce the new system as seamlessly as possible. Simple migrations can easily accomplish this, but IT transformation migrations require a more structured approach that can accommodate the inherent complexity and extended timelines involved. Migration is sure to drive transformation tasks and deliverables that need to be planned from the start; so defining the migration process once you’re well into the transformation execution is a little too late. A transformation initiative should be planned with migration in mind as early as feasible. Depending upon the nature of your transformation initiative (is it an evolutionary change or a revolutionary scrap-and-replace effort?), you will need to approach migration with a clear set of goals. Typically, the migration goals you’ll need to weigh represent a cost and focus compromise between the following objectives:
Value Acceleration. The migration should provide benefit early on.  One of the main reasons for the failure of a transformation project is extended delivery timeframes that expose it to the loss of support from new executive teams or built-up financial pressures that lower its priority. When building a cruise liner, one doesn’t attempt to sail it before it’s been completed. This is unlike IT Transformation. IT Transformation is more like a public works program were new infrastructure can gradually enable renovated sections. Waiting for the entire transformation initiative to complete before starting migration is unwise. Instead, you should endeavor to put into production the transformation elements that can deliver value as soon as feasible. When you do this you are providing life support to the transformation initiative.
Some might argue that this approach carries the risk of the company being tempted to halt the remainder of the transformation once a few of its main benefits are enjoyed. But, if management is going to stop a project they will stop it regardless. At least, by following this approach, the transformation will be viewed as having produced value. If the program is stopped under these conditions, you will be in a good position to defend the outcome.  Also, if the initial value represents a significant percentage of the total value of the transformation, then it actually makes sense to give the company a chance to amortize its investment.
A more serious argument against the approach of accelerated migrations is that most transformation initiatives require heavy work on revamped infrastructure before they are able to deliver actual user-perceived value.  This is a true conundrum and one that is very difficult to explain to business customers eagerly awaiting actual functionality. Again, it all goes back to how the project was planned and how well you segmented the various infrastructure deliverables to accomplish a quick return in value. Ideally, you should have segmented the transformation into sub-systems. The horizontal layers provided by the infrastructure should have been segmented along levels of service and functional subsets that can be scoped independently. For example, perhaps you can deliver those data analysis results sooner by relying on a simple data repository that does not have the entire resiliency in place.
In general, it’s best to avoid designing the program along the lines of first building the entire infrastructure and then delivering the applications. The end user does not care how nice the infrastructure is; only that the functionality and expected services are in place.
Non-disruptiveness.  Regardless, your migration effort should never impact on day to day business. You should focus on reducing risks and minimizing impact to business every step of the way. Furthermore, ensure that the migration meets the committed SLAs or you will be exposed to earning a bad reputation from the users. There is nothing worse than having a newly introduced piece of technology fail, causing the credibility of your entire program to suffer. It’s best to avoid “Big-Bang” migrations that require switching off the old system in order to introduce the new.  Through millions of years of evolution, nature has given us the best example of a proper migration plan with the way a caterpillar transforms into a butterfly.
Efficiency and Cost Effectiveness. The migration effort must not force excessive use of resources, whether human or material. Also, a migration effort that’s too expensive is not going to be well received; so you should endeavor to reuse as much of the existing capabilities as possible. Remember, most of the tools and resources needed for migration will usually only be used once.  Migration effort is one of those cases where it makes sense to tap into external resources on an outsourcing basis. Finally, don’t forget to make the required training a core part of the migration plan.
Timeliness. Having expressed an opinion that, if possible, migration should take place sooner, let’s not speed migration for simple political expediency. A migration has to be done in accordance with an overall timeline that properly balances the risks, the benefits, and the application of resources. Migrate too soon, before the system is placed into operation, and you are going to have a recipe for failure. Migrate too late and you’re going to face a situation where any room for failure is effectively shut down
Avoid expedited migrations just to meet a deadline or for purely PR reasons. It’s not worth it.  Migration for migration’s sake is going to leave your executive sponsors and the business people scratching their heads. Then, there are times when migration must take place at a purely infrastructure level in a way that will not be apparent to the end users. When doing so I recommend keeping the PR at a minimum. The last thing you need is a disappointed sponsor wondering what all the brouhaha was about. Think about it, when was the last time your municipal government bragged about replacing those sewage lines?
The key thing to remember here is that migration timelines, the way you manage the expectations for each migration step, and the degree of exposure you apply (PR) must align with the business requirements in order to be effective milestones in accomplishing transformation goals. Accomplishing this requires defining a migration strategy that includes the tenets and principles to be observed.  More on this, next week. .  .

Friday, August 6, 2010

Notes on Training

Ongoing training of all levels of your transformation team is critical both to the success of the project and the company.  Ironically, training is often looked at as the ugly ducking in the budget cycle and as the line item that can serve as a “reserve cushion” when the inevitable need to make budget cuts occurs (you know the drill, around September, finance begins to panic and all budgets must be reduced, “non-essential” travel curtailed, all hiring suspended, and all reports must focus on budget control). Despite being the usual victim of end of year budget cuts, it is ironic that training budgets are often unused by the end of year. Training is a use or lose proposition, but who can spend time on training during the last three months of the year when there are pressing and holidays are approaching?
Some companies have a training department, formally tasked to dealing with training requirements for the organization. In my experience, these departments tend to focus on aspects related to the organization as a whole: company processes and values, inter-personal relations, and training in personal improvement areas commonly found in the catalogues offered by third parties.  Depending on the company governance, you might have to work with this training department but, when it comes to IT Transformation, it’s important that you take the lead in the areas and focus for the training related to the transformation.
When it comes to transformation, training should be given much more r-e-s-pect.  As with testing, training should be an inherent task in the overall transformation planning and not merely an after-thought.  For starters, since you are dealing with the introduction of new technologies, make it a practice to include vendor training as a part of any software or hardware acquisition.  Everyone involved with the new technology should be trained in it. However, focus on train-the-trainer for those skills you wish to definitely preserve in house. In fact, vendor training arrangements ought to be primarily focused on train-the-trainer levels as opposed to basic 101 courses.
The same 101-training avoidance should apply in all cases.  Invest in training that covers intermediate or advanced levels only. Don’t waste time and money sending your people to introductory courses. When possible, let them purchase and expense books and web-training programs on introductory topics (the “Just-for-Dummies” series comes to mind).  Allowing your team to read these books during company time is a much more efficient use of their time and the company’s money than having them attend 101 courses elsewhere.  If this doesn’t suffice, establish an internal training and mentoring program to enable transfer of knowledge from experienced staff to junior staff.
Also, don’t forget the challenge of retraining those who will need to re-qualify their skills due to the elimination of personal areas of expertise caused by the transformation.  Still, the same training principles should apply to re-training staff: no introductory courses, focus on applied mentoring, etc., However, given that you are dealing with the sensitive human resource aspects and morale issues created by the transformation, be aware that those individuals may  approach such training warily.  The will certainly (and probably rightfully so) feel that their professional future is at stake and that it all depends on their ability to get trained properly.  In this case you owe them a more structured training and knowledge acquisition program. In this circumstance you must work very closely with your HR department as you may need to face the fact that a number of the staff will simply not be trainable. Tough calls will then have to be made about career progressions (transfer to another area if appropriate, or departure from the company, if no other alternative exists).
An additional observation is that if you check the training statistics for any given timeframe, you will notice that typically eighty percent of the training is consumed by twenty percent of the people (“Pareto’s Principle” at work again!). Some employees shun all forms of training (“I don’t need it!”); whereas others become professional training participants, particularly those who like to make a career at collecting all types of certifications.  You’ll need to me more pro-active when it comes to developing training schedules for the team. It’s best to circulate a training questionnaire and have everyone list their own training preferences (subjects, levels, and available times) in order to get a sense of how best to balance the training efforts and to plan training accordingly.
Attending conferences and conventions may or may not be considered valid training efforts. Depending on the specifics, attendance may be more of a boondoggle than a true training event. You will need to make sure the training budget is truly applied to efforts intended to give expanded knowledge to the team.
Also, when I speak about training, it’s best to think holistically about training all affected constituencies. Don’t forget the need to also train the users of the new solution, from the business stakeholders to the front end employee using the applications.  Negotiate with the appropriate line managers so that you can tap their respective training budgets for this effort.   
Finally, it should go without saying, it is often difficult to differentiate between the training needed for the transition period from the training needed when the system has reached a steady state. Training for the transition period should be part of the migration plan and should be structured as a one-off basis. Planning for ongoing training will require further study and the appropriate development of recurring training tools, including web-based training, internal company training curriculum establishment, and even partnering with external training companies. Training programs and materials for steady-state are, in fact, part of the transformation deliverable.