Friday, January 7, 2011

Considerations on Experience and Expertise

If you are going to build a star team capable of delivering SOA systems, you will need to ensure the right mix of skills and experience. Just as in a good gourmet recipe, you will need to gather the proper ingredients and bake them together just right to achieve the desired outcome.

In addition to the more karmic aspects of this team formation: style, culture, motivation and focus, the element I believe to most define the likelihood of success is experience. The question that then arises is this: What, after all, is experience?

My older brother told a joke about a man showing up for a job posting. “How long have you been working in this role?” the interviewer asks. “I have never worked in this role,” the applicant answers. “Then why are you applying for the job?” The applicant pulls out the want ad he had seen in the newspaper and points to it. “Well, at the bottom of the Help Wanted Ad you wrote, ‘Useless to show up without experience.’ It just so happens that I have no experience and, according to my wife, I am quite useless.”

This story aside, experience is not something easy to define. Just as the US Supreme Court refused to define pornography, experience is something usually identified with the, “I’ll know it when I see it” mantra. For starters, let’s be candid and agree that experience is not necessarily what resumes claim it to be. The quality of the experience matters. How many times have you met someone who claims “thirty-year” experience when, in fact, what they have is the experience of one year, repeated thirty times? Also watch out for resumes claiming experience-by-proxy. You know, the “I am not a doctor, but I play one on TV” kind. You can identify these pseudo-experiential claims under resume sentences that read like this: “Worked with the team that developed XYZ” or “I was there when NASA placed a man on the moon.”

Furthermore, given that our objective is to build a team that can build and operate a world-class SOA system, experience is actually pointless if it does not lead to expertise. Herbert Simon, Nobel Laureate in Economics, and one of the pioneers in the field of Artificial Intelligence, stated that expertise was the ability to do a task at least three different ways. When I lived in New York, I knew of only one route to get to La Guardia from my home. However, a good cab driver knows the detours to take when traffic is jammed. (I know, today’s Navigation systems can help me match the capabilities of this cab driver—technology can be a substitute for experience).

Assuming good-quality experience, then it can be measured in terms of years of practice. According to some popular books on the subject (“Blink” by M. Gladwell, for instance), anyone can become an expert at anything by simply practicing and practicing and practicing. Interestingly, the point is made that the magic number of practice hours needed is about 10,000. I may agree with that estimate, but those quoting the study also refute the existence of natural talent under the “anyone can be a Mozart” argument, “if only they practice as much as Mozart did”. As someone who has practiced guitar for eons and who listens with envy to the virtuoso guitar playing of an eight year child prodigy (http://www.youtube.com/watch?v=NMnw7CcWO7E), I couldn’t disagree more. Still, if we ignore those savants as outliers for the sake of discussion, a 10,000 hour period represents a minimum of five years, assuming forty hours a week. This means that, unless you are a natural prodigy, you should expect it to take five years of practice (and in my book practice equals experience) to become an expert at something.

Experience can be quantified, but expertise is somewhat more ethereal. Back in the eighties, a class of software known as ‘Expert Systems’ made its appearance. This software was intended to replicate the knowledge of experts. The idea was to “reverse engineer” this knowledge via a series of heuristic rules that could then be made accessible to the rest of us mortals. As one of the most committed practitioners of this idea, I was actually able to experience the difficulty of this endeavor. First, to have experts explain how they arrived at their conclusions was more than difficult, if not downright impossible. Oftentimes, experts couldn’t even explain how or why they reached the conclusions they did!

Fact is, deep expertise is all about moving knowledge from the realm of the conscious to the realm of the intuitive; not unlike when learning how to drive makes us no longer think about the mechanics of driving. True knowledge is embedded knowledge; knowledge that has become part of our core mental fabric. Yes, Expert Systems can encode and replicate some basic If-Then-Else rules, but true expertise is often expressed by flashes of insight that are not so easy to explain, let alone dissect.

If you are familiar with the TV show House M.D., you know about the show’s leading character uncanny ability to diagnose a variety of obscure diseases thanks to his insights. President Lula of Brazil , who just recently stepped down after eight years as president with an 87% acceptance rating in the polls (imagine that!) was asked about his secret to governing. His reply: “Good government is simply the art of doing the obvious.” Of course, that’s easy for him to say. What is obvious to him may not be obvious to the rest of us. This is what made him by all accounts a great president. Stating the obvious is something we all try to do. And therein lies one of the main issues in accessing the services of true experts: It is practically impossible, at first, to differentiate the insightful advice of a true expert from the opinionated shallowness of a fool. A know-it-all fool can at times sound like an expert. Witness those cult leaders who frequently lead their followers astray with the decisive and resolute way in which they express their convictions.

But I digress. . .

You can weed out the shaft from the gold by becoming an expert at recognizing expertise. If someone claims experience in something ask them about it. What logic, constrains, rationale did they use in solving problems in the past. Get specific examples of their past performance. Evaluate their decision-making logic. Assess these and other thought-process aspects; and do not just focus on the number of years of claimed experience.

You should be just as careful when selecting the type of experience you wish to acquire in your team. Forming a great SOA team is not unlike forming a championship-level football team. You’ll need it all: the agile quarter back able to scramble and throw the ball far, the quick receivers, the strong offensive line and even the flimsy-looking field-goal kicker. Like a good football team you will need a variety of talents. The one resource with the Doctor House-like knack to quickly resolve system outages is not likely to be the one person who introduces innovation. The project manager with the ability to sense drifting timelines is much needed, but do not expect her to figure out the types of SOA services to write. The programmer able to draw up complex algorithmic code is a key resource, but he won’t be your go-to person to define the best quality assurance processes. Think of the natural leader who can motivate the team into action as someone priceless, but don’t expect her to excel at keeping track of the written status reports.

Mix the team with a variety of experiences. Remember, there are people who have a great deal of knowledge in very narrow areas (do make sure that the degree of specialization is not so narrow that you end up with someone who knows everything about nothing!) and those who have knowledge covering a wide range of areas, although their knowledge in each might be shallower. Try to have a reasonable mix of these two types of knowledge holders, and dismiss the ones who claim deep knowledge about everything. Those are the fools I referred to earlier.

For some key areas of expertise, it is and recommended to have some redundancy (the best football teams have a great second-string quarterback). In these instances, you will benefit from getting differing viewpoints from more than one expert and will be in a better position to reach you own conclusions to help decide courses of action. Having this kind of depth of expertise is not really feasible in all areas, but you should try at least to having it in the group responsible for overall architecture decisions. Lastly, make sure you integrate some bright, inexperienced talent in your group. In addition to helping you build your team for the future, having this type of resource will give much needed fresh perspectives. Potential experience can be as valuable as actual one!

In the complex world of SOA, uniformity is not bliss and experience morphed into expertise does matter.