Friday, July 31, 2009

The Laminated Tenets

The Category II Tenets

These are the types of tenets you can laminate in plastic and frame for the office wall next to the water cooler. They don’t often change, but change they do. These tenets usually define the current state of the industry preferences for specific technologies. While usually applicable to current environments, there is no guarantee they won’t eventually need to be revised. Take a trip down memory lane and imagine a return to the year 1969. Yes, Woodstock happened and you probably don’t remember it, either because you hadn’t been born yet or because you were in fact there. After listening to Janis Joplin, Santana, and Jimmy Hendrix, you show up to the office and are given the task of defining the tenets for a data processing application. You clear your mind and readjust to the fact you are back at work. You define a tenet that states customer records shouldn’t exceed 80 characters. Why? Because each record must fit on a punch card, and a punch card holds a maximum of 80 columns. Clearly, when there’s a need to code for the year field, you’ll opt for conciseness and drop the “superfluous” 19 from the year, etc. Another tenet might have been to make certain that all job submissions are hand-delivered to the “Batch Service” window with the user signing the log sheet.

Unlike Category I tenets that tend to be industry-agnostic, Category II tenets have a more focused applicability. Some of these tenets will be more appropriate for a particular industry segment, geography, or IT situation. If you are in a service industry, for example, you will have tenets that emphasize functionality over other attributes; if you are in the manufacturing industry, you will probably focus on tenets geared to reducing costs, and if you are in the videogame industry, your tenets will highlight usability and playability.

The list of Category II Tenets can therefore be extremely large. Below, I show some such tenets as examples. Again, when it comes to creating your own tenets, you’ll need to assemble your team and map those tenets that best apply to your specific conditions and that contribute to meeting the category I tenets previously defined:

· Move to a common development environment to maximize use of common support processes and tools within any of the deployed platforms.

· Object-oriented methodologies and approaches will be pursued whenever possible. Percentage of reused code will be a success factor.

· Use state-of-the-art programming tools and techniques to ensure maximum development productivity. Use Life Cycle and Service Studio components to facilitate service reuse. Optimize the development cycle by using design methodologies that permit analysis of the system under a business view and rapid prototyping for proof of concept. Use UML whenever possible.

· Ensure reliability through redundant clustering, either purchased or built. The strategy is to seek system level resilience via n-plus one redundancy with automatic fallback/recovery.

· Consider security requirements from the start. Establish security, logging and billing mechanisms based on service-oriented constructs.

· Message queuing is the strategic middleware standard. Mapping to other middleware will be supported as required, but the canonical service distribution protocol will be SOAP based.

· Unify connectivity protocols and communications interfaces end-to-end. Reduce the number of different access methods and protocols used inside the central complex. Use open, off-the-shelf protocols and systems. TCP/IP is the preferred connectivity protocol.

· Hide the complexity of the DBMS from the applications, users, and processes that access the data, by encapsulating these database accesses with service oriented APIs.

· Enable distributed data access with connectivity to heterogeneous DBMS within one unit of work.

· Avoid DBMS access across the wide area network. Propagate remote data queries via a service-oriented approach.

· Portals will deal with user sessions and presentation and will orchestrate the user interaction, including content presentation, but the business services will be provided by backend engines and will not reside in the front end portals.

· All systems to be adapted, built, or bought must provide the following features:

o Proactive and reactive monitoring capabilities

o Fault tolerant, load balancing, and fail-over capabilities

o Performance measurement, logging, and system health reporting

o Integration with system administration and management standards.

In addition, Category II Tenets are the natural realm for all SOA-related tenets. Those tenets deal with desired levels of service-granularity, SOA governance, SOA management approaches, etc. Because the grand theme of IT Transformation is the use of SOA, I will later cover these tenets in detail.

The Category III Tenets

These are the tenets that are defined at the departmental level. They describe naming conventions, documentation standards, reporting mechanisms, and pretty much anything dealing with the nitty-gritty of running a project and are therefore as important as the other categories. It is a good idea to keep a formalized track of how these tenets are defined, communicated, and enforced.

A typical challenge is that sometimes, tenets are created as Level III when in fact they ought to be vetted with a Level II scope. The opposite is also true. A sure sign of an organization trying to over-control a project is having Category II tenets that should be left to a subsidiary unit. Remember that allowing a degree of departmental freedom in choosing tenets at this level is actually a good way to ensure fluid dynamics and engagement of the base in the overall standardization process. In the end, you may have something akin to British Common-Law whereby Level III tenets become so widely adopted that in time they emerge as de-facto Level II tenets!

In my experience, developers tend to converge around novel tools that, due to their emerging nature, have not yet come onto the radar of the corporate bigwigs. Sometimes this results in conflicting viewpoints between the directives driven from the top and the preferred technologies driven from the grassroots. When thishappens you will need to open the floor to debate and be prepared to make some compromises.

An example of this situation is the manner in which TCP/IP was adopted as the preferred networking protocol. It is a fact that TCP/IP emerged from the grassroots, even as corporate directions coming from the top were trying to enforce the OSI or SNA protocol stacks. The ‘can-do-now’ and ‘can-do-cheaper’ pragmatism espoused by departmental gurus won over the ‘comprehensive’ and ‘structured’ OSI dogma espoused by the architects.

Because the OSI standard was a camel (i.e. a horse designed by committee), it was practically impossible to implement, and the grassroots’ preference for TCP/IP did turn out to be the right choice. To be fair, there are examples where grassroots-driven adoptions ended up being counter-productive. Take the wildfire adoption of the distributed Novel servers in the latter part of the eighties that ultimately led to the organic emergence of extremely complex environments; environments with a high degree of data heterogeneity and processes that companies had a hard time extricating themselves from. Now, I am not suggesting that the Novel corporation or its technology was at fault (after all, it was in fact a fairly leading technology for its time). Rather the issue was that much of the deployment of this technology happened pretty much ‘under-the-radar’. Most companies ended-up with duplicate data bases, data bases that lacked the necessary security, badly managed data, and in many instances, application code that was written for the very proprietary vendor environment and which could not be reused.

So what are examples of Category III tenets? The list can become fairly detailed and nuanced. I suggest breaking down these tenets into logical groups so that you have at least a means to compare and contrast tenets coming from different groups. The following groupings are recommended:

Data Tenets. Specific to the data, these tenets would include naming conventions, partitioning approaches, etc.

Foundational Tenets. Tenets dealing with system infrastructure and management would come here. Network IP standards, server naming standards, etc. would come here.

Presentation Tenets. These cover usability standards, use of presentation elements, navigation, etc.

SOA Tenets. XML tags, repository placement, service levels, etc.

General Tenets. Tenets dealing with general governance and project processes, including vender-selection strategies, evaluation and testing methods, etc..

Also, as a matter of fact, if you can also group the Category II Tenets along these lines, it would be a extremely useful way to ensure coherence between the Category III and Category II Tenet narratives.

That’s it. . . If you are “teneted-out” I don’t blame you. It’s time to dive into the Level II Architecture stage and, in particular, into SOA as the solution for that architecture level.

Friday, July 24, 2009

Chiseling the Stone: The Category I Tenets

Yes, these are the tenets that must be established to withstand the passage of time and to survive the challenging trials a complex project presents. Category I Tenets are not meant to vary over time, all the more reason for them to typically prescribe common-sense best practices. In many ways these tenets are so common-sense that their ‘motherhood and apple pie’ texture make them appear as though they are just a collection of distilled clichés. I suspect that when you read through the sample of Category I Tenets below, you’re going to say, “Duh! These are so obvious, why even bother to mention them?”

Well, we live in a world where many believe we didn’t really land on the Moon; where the so-called “birthers” refuse to acknowledge President Obama’s birth certificate, and where many think 9/11 was a self-inflicted government conspiracy. Irrationality loves the vacuum. Even if you discount all irrational arguments, the fact is there are a surprising number of cases where rational counter-points can be made to what seems to be a solid, “clearly-uncontestable” tenet. Even “obvious” tenets should be stated explicitly or you could risk someone, somewhere along the line, taking the counterview based on the validity of an isolated exception to the rule, or on the strength of an individual’s forceful opinion.

Take for instance the tenet, “Be Proactive; not Reactive”. This statement seems so obvious as to be unnecessary, but there are those who might argue that being reactive is a lower risk proposition, especially if agility to quickly catch-up with the competition’s first moves can be maintained. To them, the tenet should read, “Be reactive, but be quick to adapt to changing conditions”. In this light, the former tenet doesn’t sound so obvious anymore. Who knows, based on differing conditions it could well turn out that you transformation strategy should after all be reactive.

It’s important to state the Category I tenets openly and unabashedly. Following are a few sample Type I Tenets, representative of the kind of Category I Tenets you’ll need to define. Even though I am quite certain these tenets will hold water under a large number of conditions, they were selected under specific situations that might differ from yours. Clearly my list is not exhaustive, and yes, my lawyer also advices to remind you that you should be able assess these tenets in the light of your own transformation effort and use alternate or even opposite tenets as appropriate!

· Functionality requirements must be driven by the business requirements.

This statement simply reinforces the need to ensure that business is involved from the start in defining the specific features and priorities. However, getting the attention of business for what is essentially a strategic initiative can be a difficult undertaking. Business folks are typically immersed in addressing short-term issues. More significantly, typical business executives rarely focus on strategic technology issues (actually, that’s a good thing as this should be your responsibility; not theirs), so you will have to go the extra mile to extract and map the business requirements needed to create the technology plan.

· Be Architecture Driven; Not Vendor Driven.

This is a Zen-like “Be-the-Architecture”. Remember, we are talking about the transformation of IT elements that provide key competitive differentiators to your business. ERP is sufficiently commoditized so that most companies should be happy to follow the lead of vendors in this area. And yes, if you buy into the SAP or PeopleSoft views of the world for these particular IT domains, you will be just fine. Remember, however, that if you are to buy into the complete architecture view of a vendor for the development of your new core technology, you are making a generational commitment to that vendor—a commitment that ties your company to the vendor’s future for eons to come. I can’t tell you how many times I have come across companies that fully bought into a particular vendor’s architecture only to see that vendor go the way of the Dodo (an extinct bird—check it out in Wikipedia). What this tenet says is: design your architecture according to your company’s needs; make the architecture vendor-neutral, and then and only then approach vendors to ask them how they can help fulfill your vision.

· Use of off-the-shelf industry standards and products is preferred for all commodity-level elements

I’ve seen my share of companies who’ve spent an inordinate amount of time and effort developing their “own version of network protocols”, or their own “very cool, in-house version control system.” Unless there’s a clear cut reason for you develop a commodity component in-house, you’re better off having your team develop only those components that will create differentiating solutions. It’s wiser to focus on providing value-added features not readily available on the open market; the kind of features that leverage your company’s core competencies rather than divert resources to areas that can be easily met by a simple product purchase.

Now, this tenet might be seen as possibly contradicting the prior one (Become Architecture Driven; Not Vendor Driven); so let’s clarify: defining a “proprietary” architecture is in no ways wrong, but in fact is advisable. A bespoke architecture, however, does not presume the use of proprietary components. In fact, an architecture that’s well tailored to your needs will wisely identify the areas of focus for differentiated, value added development, and the areas and manners for interfacing off-the-shelf components.

· Architect from a Holistic Perspective

Don’t focus just on the needs of a particular business function or department. If the sales department requests a Prospect Tracking Tool, you could spend you energy defining the contact management tool requirements, designing the workflows and data bases involved, and defining a terrific prospect tracking tool. However, if you fail to think holistically by envisioning the synergies that can make the tool better you will be de-meriting the solution. What if you could tie in this solution to a more generic customer data base? What if you could gather customer purchase history and tie that information in with the tracking tool? Say, you were to find out that for the past two years you spent X amount of hours wooing a customer who generated only a minimal amount of business, while another customer, who had been contacted only a few times now represents 10% of your revenue? Wouldn’t you like to design a system capable of managing such a broad view?

Before you accuse me of over-scoping with a “what if all that’s needed is a tracking tool; nothing fancy or expensive!” I am not suggesting that you actually implement this type of holistic design. I’m suggesting that you architect the solution so that, should you decide to implement it, it will take into account the potential interactions with the rest of your system.

· Move to Service Oriented Architecture.

Enable an end-to-end, distributed service oriented framework that provides all necessary services independently of the underlying technical infrastructure. Arguably this is a Category II tenet, but given SOA’s establishment as the de-facto architecture pattern for modern systems, I think this is now a tenet that has attained a chisel-in-stone status.

· Minimize costs by driving reuse and convergence

This is like understanding the need for exercise, not smoking, and avoiding fats—obvious advice that’s rarely followed. Many choose time-to-market even with the added costs this heterogeneous solution demands. “You want a new campaign management solution? No problem! Let’s buy that super-expensive module from that company that invited us to the Ryder Cup last year! Later on, we’ll figure out some way to tie their customer data base in with ours!”

· Place automation of core business functionality in back-end systems

It is better to have the bulk of the capabilities served via back-end engines rather than designing a monolithic front-end that implements all functions. Many Web sites that have attained a large size organically are now suffering from limited scalability due to the failure of their designers to move the business logic to backend engines. The front end component should be limited to providing user interface capabilities such as interaction workflows and presentation level integration, and access the backend business processes via services.

· Keep the number of programming languages (and redundant technologies) to a minimum.

I am old enough to remember FORTRAN, a language widely used for scientific work. Then COBOL was invented by the great Grace Hopper to be the language of choice for “business applications”. Next came PL/I, an IBM specification that kind-of-combined COBOL with FORTRAN. As if PL/I were not bloated enough for the US Federal Government; the Department of Defense invented ADA, which is probably gathering dust in some of those old computers running the country’s nuclear silos (so it’s a good thing that ADA was a strongly typed language!). Meanwhile, a host of other languages sprouted-up like weeds in an unkempt backyard. We had Basic, LISP, Algol, Modula, Eiffel, and C, then C++, followed by Java which is “like C++ but with garbage collection and no pointers”; then came the wave of scripting languages that have come to look more and more like actual languages. Enter the Ruby vs Python debates.

I am certain that ten years from now, a similar passionate debate will be raging between followers of two as yet unknown languages (I don’t envision the day when computers will be programmed in plain English. Call me a nerd, but why we would we want to tell the computer to “Move money in the customer savings account to his checking account” when a simple Customer.TransferMoney(From, To, Amount) would do?)

Bottom line is that there are now literally thousands of computer languages (compiled and interpreted) out there.

You just don’t have to use them all!

The trouble with languages is that they tend to become the object of emotional preferences. It is natural that programmers familiar with a given language will gravitate toward that language. In large projects especially expect to be pressured to allow whatever programming languages your programmers’ desire. If there is an area where the proverbial stake needs to be sharpened, polished, and put into the ground, it is on this one! It is important that early on in the project you establish the tenet of using no more than two programming languages, and no more than one scripting language.

This tenet should also extend to the use of any other redundant technology: keep the number of J2EE Platforms to a minimum; keep the number of RDBMs to a minimum . . . You get the idea.

Now, I am not saying: “Have only one RDBMs solution”, or “Have only one Language”. Strive for simplicity, but not so much so that you become simplistic!

Monday, July 13, 2009

On the Creation of Tenets and Standards

You have defined the technology strategy and the architecture scope. On the table sits a clear architecture model and on the wall you have a nice poster with architecture reference framework that will serve as a coherent beacon throughout the effort. What’s next?

Four decades ago, men set foot on the Moon. The beginning of that incredible journey was promulgated in this mission statement by JFK: "This nation should dedicate itself to achieving the goal, before this decade is out, of landing a man on the moon and returning him safely to Earth."

This is one of the best examples of a mission statement that you will find anywhere. It is succinct, clear, and comprehensive (I especially like the bit “…and returning him safely to Earth”—very thoughtful.) What was not clear was what would follow next. It wasn’t obvious at first that NASA would be the agency responsible for the mission (the Air Force wanted the gig), and then there was the question of who would make a good astronaut (the idea to use circus performers, divers, and mountain climbers was first considered, but disposed of in favor of test pilots). Then there were the questions of the type of technology to be used and the project governance (using myriads of contractors working under a common NASA governance was the one chosen). The number of decisions and choices that had to be taken was staggering.

As with the Moon mission, to reach the sky you must first touch the ground. Even at Level I, the architecture must move into the “where-the-wheels-touch-the-road” area. Work may start as abstract and conceptual, but eventually it has to reach the point where it is necessary to explicitly articulate the specific rules, tenets and preferred standards that will constitute “the practical canon” of the architecture. This canon will help clarify the preferred approach and will serve as a lighthouse for dispute resolution purposes. As covered earlier, the US Constitution is a perfect example of Tenets and Standards defined to make possible America’s “mission” as represented by the Declaration of Independence (the part where we are all entitled to pursue happiness?)

When applied to systems architecture, you’ll need to start defining concrete approaches. For example, it’s all very well to pursue a distributed processing model, but will you use the suite provided by a specific vendor? And if so, which one? Or, will you prefer to play the integration game and select amongst best-of-breed solutions. If so, which ones and under what criteria?

You’ll need to define tenets.

The dictionary definition of tenet is this: a principle, belief, or doctrine generally held to be true; especially one held in common by members of an organization, movement, or profession. In our context, the tenets should be mapped within the end-to-end architecture model and should represent a summary of the consensus and direction of specific strategic choices.

Tenets[1] should be documented in such a way as to provide broad exposure to the technology principles without being buried in dense documents. Tenets can be categorized, and sometimes should be prioritized in the case (hopefully rare) where two tenets result in contradictory directions.

There are different types of tenets:

· Category I Tenets represent clear-cut best practices that apply to all projects. These are the tenets that are core to the project and therefore the ones that you need to worry about first.

· Category II Tenets tend to revolve around the “technology context”, or industry, and are usually a more detailed specification of process and technology choices.

· Category III Tenets represent choices made at a departmental level. These are options that have a narrower scope and therefore their selections can be more a matter of individual or department preference. That is to say, given a set of feasible options, you simply choose those that you believe are the most appropriate to your company.

Just to state the obvious, Category II Tenets should not violate Category I Tenets, and Category III Tenets should conform to Category II Tenets.

The formulation of tenets is as much a key to the entire transformation project as is the framing of a constitutional convention. After all, it is here that you define the rules that will serve as the de-facto guide for all future decisions regarding the project; so let me cover a bit of the “how” you can accomplish this step.

First, if you have not yet defined clear governance for an architecture group, you should do so now.

At this point in the process, the core architecture team should be relatively small; formed of a few senior individuals able to cover the span of knowledge needed for the transformation (systems, applications, data, SOA, etc.) Most importantly, this core team should be tasked to facilitate the elaboration of tenets and standards.

The next question is what specific role you and the architecture team will take vis-à-vis the tenet and standard decision-making. Here are some ways you can arrive at them:

1. Moses Coming Down from the Mountain. Here you and your architecture team go to a retreat, hash out the Category I Tenets and return wearing long white robes to announce your “commandments” to the rest of the technology areas. In my experience, this style only makes sense in two scenarios:

· A project has been so delayed that any more debate or analysis-paralysis will place its success in jeopardy. A strong “do now-ask questions later” stance is required.

· The great majority of the technology team is formed by external resources and therefore not emotionally committed to the project

Otherwise, I recommend not following this approach, unless you wish to disenfranchise the very people whose support you’ll need in the grueling months and years ahead. You would also risk losing the benefit of the insights and opinions of the rest of the technology team.

2. Edicts from the Benevolent Dictator. First let me state I believe the term “benevolent dictator” to be an oxymoron. The Benevolent Dictator approach is a softer form than the Moses style because it at least allows the voicing of opinions of all key technical members through the means of a Technology Steering Board. You will then be able to assess all the opinions and decide upon the chosen tenet. If this process was truly unbiased and resulted in a decision that properly accounted for the strength of an argument favoring one tenet over another then this method would actually be ideal. In truth, it is almost impossible to remove the natural biases of the decision-makers. You risk alienating the rest of the technology members if they perceive the entire exercise as just playing lip service—an exercise on futility.

3. The Democratic Caucus. Here you debate and ultimately decide the preferred tenet, based either on consensus or outright vote. Obviously, you will need to allocate extra time to allow for the very heated debates to follow; plus secure the assistance of HR and a good therapist to deal with the aftermath of bruised egos! In principle, this process may work under some circumstances but it also has its issues if you are facing an organizational structure that comes already fragmented into groups. Let’s face it, the chances that you will get a consensus between mainframe people and people who dabble with Java is not very high.

4. The Grass-Roots Convergence. Why even bother with a process? Let people decide themselves! After all the Internet emerged with approaches and standards that eventually replaced those of established vendors. I believe this approach is actually effective in small start-ups and in those situations where you have a highly involved and well-experienced base. I am afraid that it might not lead itself to being effective in the more cavernous world of large enterprises where the process can lead to anarchy if not well structured.

So what’s best? My own view is that Category I Tenets should generally be chosen via the Benevolent Dictator form, if only because of the wide span and strategic importance they represent. Also, Category I Tenets should closely adhere to the strategic objectives already defined for the project. You can’t expect the entire technology base to have all the information needed to make well-informed decisions (maybe you are privy to information regarding a secret acquisition of a company that uses a certain technology, but which can’t be shared). In any case, I advise against the Moses approach so as not to shut down the chance to learn valuable insights from the team.

I suggest that for Category II Tenets you try the Democratic Caucus approach, with the proviso that if no decision is forthcoming after a healthy degree of debate that you retain your Benevolent Dictator prerogative. This should ensure a decent amount of compromise as your team surely will not want you to make the final decision!

If at all possible, to the extent that Category I or II tenets leave room for flexibility (as they should), it is worthwhile to allow the base team to recommend specific departmental Category III tenets. Obviously, you will need to oversee that the Category III decisions do not deviate from higher level tenets.

A final point, during this phase it would also be wise to establish the process and principles under which tenets will be enforced and revised. Your entire team should be well aware that the establishment of tenets and standards is a serious matter and that its ultimate outcome will be the reduction of stresses resulting from byzantine arguments and the focus gained by the shared understanding of the environment.

So, what are these Category I, II, and III Tenets I have been talking about? Next I will be covering some examples.

[1] Tenets are also referred to as “Principles” by many. I prefer to reserve the term “Principles” for the meta-category of tenets about tenets.