Tuesday, June 30, 2009

The Architecture Scope

Because architectures should strive for flexibility, there is always a danger that, if they are not properly scoped, you may end up with a pile of “architecture” documents that no one will care to read, let alone implement. When defining the architecture it is essential for you and your team to define it with the right scope. If you are to keep it real, boiling the ocean is not an option. (“Let’s boil the ocean to get rid of those German U2s, the implementation is up to you.”)

For example, when defining the scope, ask yourself if you are trying to address only the customer-facing automation side of your business, or if you will also include the backend support systems [1]. If the latter, chances are you will need to consider integration to the ERP vendor. In any case, you’ll need to define whether to take an end-to-end customer lifecycle perspective or to focus on specific core business processes only.

Should you support international environments? If your company is a conglomerate, will you cover multiple business units or just one particular division? Are you even empowered to define the backend architecture?

Answering these questions requires a candid, non-delusional understanding of whether you have the kind of organizational governance to define far-reaching enterprise architectures or are limited to specific areas under a narrower span of control. That subsidiary headquartered in Paris might not appreciate your trying to dictate their technology!

Understanding your own limitations is essential. Clearly, it would be nice if the scope of the architecture encompassed as much of the “problem-space” as possible, but the fact is that real-life problem-spaces are highly irregular.

Were you to depict a typical problem space using a shape, chances are you would end up with a splattered blob like the one that follows:

Non-withstanding your governance limitations, you could define the architecture with a scope so broad that it covers all possible points of the problem space—like throwing a stick of dynamite into lake to do some fishing. This ‘solve-world-hunger’ approach will specify a great deal of things that just don’t belong to your specific transformation initiative. While all-encompassing, this ambitious approach will most likely be very expensive and non implementable.

The next diagram depicts a more realistic approach. The architecture scope here represents a compromise between specifying the largest possible conjunction of common elements in the problem set while ignoring the more “far-out” cases. It is this conjunction that forms the core of enterprise architecture—the main area of focus for the strategic architecture group.

Areas of the problem space not covered by the core architecture can always be handled as exceptions on a case-by-case basis. Who cares if content integration with the three person art-department happens with an 8GB USB stick? And yes, you have to accept the branch office in Namibia entering their one daily purchase order on a typewriter and then faxing the batched weekly orders on a Friday night. When the day comes that Internet connections become cost-effective there, you can then bring them into the fold of your architecture.

Such pragmatic exceptions avoid the need to over-architect data bases or data exchange formats and do help keep the architecture simpler. Of course, it is best to minimize these special cases as much as possible. Were you to set the scope in a too-narrow a fashion then most everything will becomes an exception! In addition to assuring yourself hours of the extra work needed by this fly-swatting approach to solving issues, you will also be costing the company more than necessary.

Even though the exceptions should be in the periphery of interest for your architecture team, you should at least be aware of their existence. Better yet, the architecture group should establish preferred guidelines on how best to admit exceptions so that the possibility of their future alignment with the core architecture is not compromised. For instance, establishing the use of a standardized Comma-Separated-Value (CSV) file for “non-traditional” data exchanges (including the art department’s USB stick), or defining the product codes to be typed in the manual purchase orders from Namibia, would at least ensure that exceptions can be better handled in the future.

It’s also worth remembering that the fluid nature of a business requires the ability to append new problem space areas to the original space (imagine when a business merger or company reorganization occurs). Typically, these “out-of-space” problems will have to be first handled on a reactive basis following the rule that new features are always needed “by yesterday”. However, you should have you team work on integrating these to the mainstream architecture as soon as practical. When this occurs, you will need to start the process of redefining the architecture scope all anew.

But what does the term “Scope” means in practical terms? What’ is it precisely that your architecture team should be defining as being either in or outside of scope?

The architecture scope should define the layers of technology to use (off-the-shelf or homegrown). Also, the scope should detail the applicability of the architecture as it concerns your organization’s structure, both horizontally (how many departmental functions and geographies it spans), and vertically (what level of detail will the architecture attain? Will you stop at the regional level or define the departmental elements?). In regard to timelines, your scope should go all the way to the long-term—there is no such a thing as a short-term architecture.

Perhaps you recall that back in the late eighties there was an Open Systems Interconnect standard (OSI) being pushed by the International Standards Organization. The specification reflected a design-by-committee approach that failed to take hold when faced with more effective de-facto standards such as TCP/IP, but one of its most enduring legacies was the definition of the seven-layer interoperability stack. This stack divided the various system elements in accordance to the type of service provided by each layer.

This framework can be useful in showing how the IT industry has been traversing the OSI layers toward commoditization. In the early days of IT, much blood was shed in battles centered on which technologies were to be chosen for the networking layers. Architecture scopes had to deal, for example, with whether to use SNA or DECnet, or even whether to implement something proprietary (it happened; one of my deliverables in a project long ago was for a proprietary communications protocol). Today it would be incomprehensible to debate against the use of TCP/IP as the protocol of choice, unless you are in a research organization testing the next big thing. The industry has converged to standards that moved the battleground upwards in the stack; toward OSI’s highest layers.

The diagram below shows the OSI layers. Shown in the horizontal axis are the trends toward standardization. In consequence, these days the focus is on actual application and service specifications; not on the underlying networking layers that, for all practical purposes are now completely commoditized.

For all the criticisms against it (the OSI specification truly became a boiling-the-ocean example[2]), it at least served as a unifying schema. The problem is that the OSI Reference Model is now outliving its usefulness. Despite the industry trend toward standardization and commoditization, an area of “infrastructure” that remains yet to be standardized is the Service Distribution Fabric[3]. SOA is today in a similar state of maturity as the networking world back in the eighties. With the so-called Enterprise Service Bus (ESB) we have vendors resisting standardization measures in order to sell their own proprietary solutions and get competitive advantage. Just as happened with TCP/IP, the world awaits a vendor independent SOA standards specification.

More work is needed to define the detailed structure standards for the Presentation and Application layers. This burden has been taken up by a variety of organizations such as the W3C, the OMG (Object Management Group), the Java Community Process, IEEE, many others, but we are not yet at a point where we can say that application and presentation layer standards have been universally adopted (Web browsers are still not guaranteed to display content the same way), but I have no doubt the industry will continue to converge towards more standards for SOA and the upper application development layers.

While that happens, your role will be to apply you know-how to best adapt from the range of available technologies and standards the solutions that best meet your specific needs. Defining the scope of the architecture, even when “only” focusing on the highest layers of the OSI stack, remains a challenging and fundamental proposition.

Life is still fun!

[1] As per my definition, a pure backend ERP project does not an IT Transformation project make, but there can be super-duper projects that require front-facing and backend integration.

[2] Ironically, the OSI standard is an example of an architecture scope run amok. Its broadness made it extremely difficult to implement. In fact, the standard became so bloated that various “implementation subsets” were defined. The irony of that compromise was that if you had Vendor A implementing subset X, there were no guarantees this vendor could interoperate with Vendor B! implementing subset Y. The whole point of having a standard in order to ensure vendor interoperability was lost!

[3] This is commonly referred to as Enterprise Service Bus, but since we are not yet at that point where vendors even agree on what the ESB term really encompasses, I prefer to use the more generic term: Service Distribution Fabric.