Friday, November 19, 2010

Information Distillers, Aggregators & Your Electronic “Mini-mes”

As the years ahead move us toward the enabling of understanding and wisdom, we should expect an increase in the commercially available services leveraging these new automation models. For example, consulting is already an embedded part of services provided by professionals, but in the future, consulting will evolve into a set of online services provided via moderated access to human experts or via the access to software-based expert systems. Whether they are made of flesh or metal, these will be bona-fide Information Distillers will always be ready to augment your thirst for information at the push of the button and the opening of your Pay-Pal wallet.

Emerging Information Distillers will successfully locate and turn the required information into “understandable bits” which can be digested by customers under several revenue models. While in principle these services are not fundamentally different from those provided by traditional consulting entities such as the Gartner Group or your corner H&R Block, the difference is that they will be democratized and available to all—individuals and companies alike. For example, in travel, distillers will not only publish travel magazines (electronically or via hard-copy), but will also package tours and offer special negotiated travel deals. ( can be seen as a first generation distiller leveraging the power of social networking.) However, information distillers in the future will be able to provide personalized advice either from paid human experts or from next generation expert mining tools.

As electronic commerce becomes more pervasive, and the speed and specialization of business increases, proxies or electronic avatars will become more prevalent. Functionally, such an avatar will not be much different from today’s travel agency role when booking travel for a client. However, whereas today’s agencies do not truly represent the interest of the traveler (agencies, in principle, represent the interest of the supplier), future avatars will act as your proxies—your electronic “mini-mes”—working automatically under business and engagement rules that you’ll define in order to be presented with the best deal.

As artificial intelligence becomes mainstream, and as technical standards facilitating electronic brokering are implemented, these avatars will become virtual software entities capable of representing you, the consumer. Eventually, avatars will completely broker and execute the best possible arrangements for you.

This type of avatar is already a reality in the hectic world of electronic trading, where complex software algorithms make nanosecond level decisions on whether to buy or sell stock assets. From this world, we should be forewarned that, as proxies become more commonplace, we must be prepared to face the consequences of relying too heavily on software avatars endowed with automated decision making permits. On September 7, 2008, in an already volatile and jittery financial setting, a Florida newspaper accidentally entered an old web article detailing United Airline’s 2002 bankruptcy. Google, all-obligingly, indexed the article and distributed it to e-mail subscribers who had requested alerts on any news regarding this airline. This is where automated software proxies took over. The stock trading software scanned the article and found the keywords “bankruptcy” and “United Airlines” and automatically ordered sales of UAL’s stock portfolio. Other software robots, responsible for monitoring unusually large trade volumes in the stock market, quickly took notice of the sudden sale of UAL stock and proceeded to sell their stock. The outcome was a selling frenzy that resulted in more than one billion dollar loss to UAL stockholders. The Securities and Exchange Commission began an investigation to determine responsibility. After all, who is at fault? The Florida newspaper? Google? The developers of the software? The companies that transact stock in such a perilous manner?

Clearly, we are entering a brave new world that requires added protocols to safeguard software agents going rogue and to answer the myriad concerns related to protection of privacy; not to mention the expected security issues related to fraud and software impersonators, with the logical progression to identity theft. In the meantime, if you are in the supplier’s side, you can start designing your systems to enable this future “Electronic Mini-Me” concept. Define and be prepared to have the appropriate services and architecture layers that can leverage the deployment automated selling brokers.

As you define this architecture, you will have to rely heavily on the implementation of publish/subscribe systems and asynchronous response patterns. You will also need to focus on implementing a sophisticated combination of Business Rules and Business Process Management based systems that can allow your business team to easily configure and define the automated way broker services will be made available to your customers. For example, these brokers could be configurable to making distressed inventories available electronically and able to dynamically price on-line offers with available inventory via dynamic revenue management rules as applicable. Think of how an electronic auction process in works but on steroids.

Just as the electronic avatars discussed here are a practical instantiation of the move towards cyber-understanding, future systems applying basic rules-of-wisdom will emerge. True, Wisdom will always be a purview of humans and not computers. However, following the precepts of “Wisdom of the Masses”, we are now experiencing the benefits of the wisdom provided by virtual communities; areas where we find reviews in a broad range of areas, “How-To” tips, and better deals. A case can be made that this wisdom is an emergent property, resulting from the aggregation of large catalogues and information, and the associated tie-in of user areas and access to content. These areas are best represented by “Virtual Malls” such as, and, but are also expected to rapidly merge with social networking sites in the so-called Web 2.0 world.

There is already a linkage between merchandiser sites and places such as,, and This integration will ultimately occur via business partnerships or mergers, but it will be initially accelerated by automation known as Collective Intelligence, the process that combines the behavior, preferences or ideas of a group of people or sites to gain new insights[1].

Analogously, it is to be expected that, as this vertical industry matures, we will continue to see the emergence of portals specialized according to industry. That is, we will see “electronic virtual malls” integrating offerings on the one hand, and acting as “aggregators” dealing with the specific industry groupings. The aggregators will be able to convert the volumes of data found on the Internet into useful information. This information will be presented in a form which will be customized for information seekers as a consolidated package of knowledge. The automated assembly of related knowledge designed to fulfill the "seeker's" goals can be related to the area of specialization of the site. This trend will be evident first in consumer-facing verticals such as travel sites,, and various other special-domain sites such as and The question you’ll need to answer is how to make your company part of this new world?

[1] Programming Collective Intelligence—Toby Segaran