The following are a selection of commentaries by C3BC published in Pund-IT

22.    IBM’s OPAL – Open Treasure or Hidden Riches? 

IT software customers live in an increasingly expensive world. CIOs worry and worry still more as the escalating cost of software licenses and consequent maintenance consumes a greater and greater portion of their budgets. Towering above even these concerns, in C3B Consulting’s experience, is the trauma and cost of integrating disparate pieces of software, particularly middleware, into operations management. 

IBM offers customers one approach to this in the form of an extensive library of over 1850 — often free — ‘integrations’ that can be obtained over the Web. This library rejoices in the name OPAL (standing for Open Process Automation Library), which can be found at www.ibm.com/software/tivoli/opal 

It can be argued that OPAL is a hidden jewel which should shine especially brightly in tough economic times. Yet, on another level, OPAL is classic IBM: the jewels are hidden. Try finding what you want, and if you are not already Tivoli-educated (for OPAL is thus far primarily Tivoli-oriented), you may have real problems — even though the solution you need may well be there. The oddity here is that so much of what OPAL offers remains so hidden.

So what are these ‘integrations’ found in IBM’s OPAL? From C3B’s discussions, it appears they are pieces of code or applications which enable two or more pieces of software to work with each with each other, one of these being some part of the Tivoli family of products. Take for example: TelAlert for Tivoli NetView, a tool IT can use to deliver and communicate urgent information and thereby improve the management of work flow processes. TelAlert integrates with Tivoli’s system management framework so that customers have global, secure guaranteed bi-directional notification to multiple devices — including PDAs, Blackberries, IVR and mobiles. Moving such information out to a variety of devices requires multiple protocols which usually means difficulties. Yet a solution is already available in TelAlert.

Indeed, it is this immediate availability that attracts. Yet, what is more significant to C3BC is that these integrations more than just are the work of IBMers; ISVs and partners already provide over 20% of the integrations available. The real and potential benefits of OPAL offerings are, therefore, far-reaching.

According to IBM, integrations are used by end-user customers, as well as by ISVs/SI/business partners and even by the IBM sales force (in the latter case to identify solutions in advance to integration issues that will likely occur once software products have been installed). IBM also indicates that such pre-canned integrations are a significant factor in winning business — and C3BC can believe this: how rare it is to find that an integration solution exists before you encounter it.

Analysis

Despite all the many positives, C3BC remains puzzled. The OPAL concept is broadly excellent but its public manifestation is narrower than one would expect. That ISVs (as well as IBM) have developed integrations clearly suggests recognition of the importance and value of such an integration library.

Yet, at least thus far, even though these integrations are ready and available and low cost (or free, as many are), C3BC believes IBM seems intent on missing an opportunity.  Indeed, it seems to C3BC that IBM should more clearly observe the literal meaning of OPAL — the Open Process Automation Library — by opening it to platforms beyond Tivoli. By doing so, OPAL could become a universal library of diverse solutions which are ready to enable integration.

In C3BC’s analysis, this could be a door opener for Tivoli as well as for other Software Group products — and fits well with today’s need to contain and reduce IT costs. With an already impressive number of integrations available, it should be the integrations that are the highlight, the focus of OPAL: after all, simplifying integration issues is what IT users look for. With the increasing importance, and arrival, of the need to integrate technology to support a smarter planet, a greener earth, OPAL could become still more significant.

In medieval times, “the opal was considered a stone that could provide great luck because it was believed to possess all the virtues of each gemstone whose color was represented in the color spectrum of the opal. It was also said to confer the power of invisibility if wrapped in a fresh bay leaf and held in the hand (Source: Wikipedia).” This strikes us as an excellent metaphor for the power and transparency that an integrations library should achieve.

The open question in our mind, remains, will IBM capitalize on what it already possesses and take OPAL further and faster for the benefit of an even wider customer audience — and itself? Or will it keep the jewel of its Open Process Automation Library hidden from all but a limited audience?  


21.    Is Software Maintenance and Pricing at the Tipping Point?

C3B Consulting continues to argue that Non-Linear Licensed Software (N-LLS) – the concept that businesses do not need to buy software on a core processor license basis – is something that large organizations increasingly want and need (q.v. September 2009, INSGHTSPECTRA, www.insight-spectra.com). The ‘by-core’ pricing model is simply too expensive when radically less costly options are available.

Fuel for this fire is added as large organizations face the fact that software maintenance is eating up an ever larger proportion of their IT budgets (as hardware costs fall and increasing automation reduces the people cost). Currently, software maintenance is some 20-25% of the product cost per instance installed. What makes this worse is that many software vendors have come to rely on maintenance and support fees to deliver their profitability. In addition, the software industry’s ‘habit’ (admittedly ‘connived at’ by buyers) of delivering incomplete or bug-ridden products imposes cost burdens that are increasingly unacceptable. The fact that this becomes circular – that incomplete products require more support – is hardly encouraging to customers.

As John Parkinson of Parkwood Advisors LLC describes it (about a major financial sector client where he has been working for the past 2+ years on re-architecting the production computing infrastructure), what most organizations want is pay for the compute power consumed. For example, Parkinson has been talking with Ab Initio, which historically charges by the peak SPECint consumed, rather than the amount actually used. But in a production environment, the amount varies: one minute it might be 100 times less than two minutes earlier and an hour later 200 times more (or less).

The challenge is in building a model which keeps count of what is consumed and then calculating the pay for this at the end of the week or month – much like conventional metering but applied to compute capacity. That this flexes with usage is fine. That one may pay a premium for unexpected volume is fine. Customers understand – this is how the conventional world works In Parkinson’s words, “If I use 1 million SPECints for an hour that is essentially the same as using one SPECint for 1 million hours. If we can find a way to model the demand profile we reach a point where we probably pay a little less but far more importantly we can connect what we are really using with the workloads. This means that each part of the business can understand much more clearly what its IT costs are. Now decisions become easier and more relevant.  It also enables IT to explore what is running efficiently and inefficiently and make improvements (something that, at my financial services client, we are already starting to do). About the consumption, with Ab Initio, we are about a year away from achieving this. Once that is proven we will be looking for the same capability from everyone. It will change our approach to other software vendors.”

Analysis

The software license tipping point beckons and with it an increased likelihood of customer push back and vendor resistance. As proven alternatives appear (open source being only one), the existing pricing model for large organization software will need change. The big question is whether current software providers are able to change – or will be left behind because they cannot adapt their business models and cost structures to the new world. (This is akin to Clayton Christensen’s description of what happened to the power digging tools in his Innovator’s Dilemma).

For traditional software vendors, the challenge is to evolve. The difficulty is that most either seem not to want to change (possibly because they cannot work out how to do this profitably) or do not believe it will occur (one exception, according to John Parkinson, is Microsoft – which well understands what can happen even though it has yet to produce a credible alternative). What is common is that none have good answers, which may lead to their products becoming quickly obsolete. After all, an ongoing characteristic of the IT sector is just how fast it evolves and changes – and leaves behind the lame and unwilling.

At the same time there are some heartening pointers. Recently IBM announced its Scale Out Network Attached Storage (SONAS) product. Among many interesting features is that the software is included in the price – much as it is in an iPod or iPhone. As Stephen Edel (of IBM) confirmed – this is what customers want.  Not all, necessarily, is sweetness and light – additional payments may be required in subsequent years in order to obtain the upgrades, etc. Nevertheless, it is an advance, albeit tightly tied to potentially tens of petabytes of hardware storage and controllers, which offers customers and vendors a sustainable path ahead.

Software license and maintenance charges are already the largest line item (after people) at many organizations. If nothing else, this must attract the attention of those who have to manage costs. In turn, this almost obliges organizations to consider no-conventional approaches – whether N-LLS or similar – in order to establish control. Arguably organizations like Amazon – with ECS, etc – have most of the pieces in place. Industrializing this is the hard part (it took the Linux community 10 years before Linux was accepted, as it has been, as a basis for commercial computing); but it is neither impossible nor that far out.

In the analysis of C3BC, the tipping point for traditional software vendors – including IBM, Microsoft, Oracle, SAP and a host of others – may be much nearer


20.    SAP Transaction Losers – Rejoice!

Last summer, C3BConsulting wrote about the difficulties of finding ‘lost transactions’ within SAP’s hugely successful enterprise application software – transactions that have gone AWOL in that they cannot be found but are ‘somewhere’ in an SAP system (if only one knew where). Whether for a bank, manufacturing company or a government agency, when this occurs there is a common cost when trying to find what is ‘missing’ that comes in three forms:

1. the immediate cost associated with (say) an order not going through;
2. the lost opportunity cost that can come with an unhappy customer; and
3. the direct cost of paying for skilled assistance to solve the problem.

The scale of the challenge was straightforward to describe in actual terms. A division at one major international SAP user has some 30 SAP instances that require 50+ SAP skilled people to keep everything operating: the price of retaining such expertise – which is what is needed to find transactions when they go ‘missing’ – costs that organization nearly $10M/year. At that time, we commented that we found it curious that, with tens of thousands of SAP implementations installed worldwide, such an opportunity had been neither addressed by SAP itself (whose tools in this area remain rudimentary) nor by a third party (given the potential market size).

At the time we said we expected some measure of relief to arrive via a startup called Conpertus, which was launching a product called ‘Periscope for SAP.’ Unfortunately, the bad news for SAP users was that a threat of legal proceedings against one individual participant meant that, within weeks, Periscope for SAP was no longer available from Conpertus (and Conpertus itself disappeared).

The good news for SAP users today is that this technology still exists, having never been transferred to Conpertus. Indeed, after the demise of Conpertus (which resolved all the legal issues), the developers took the opportunity to develop further their technology and to make key improvements. From early this month, InterfaceHandler (the new product name, q.v. www.interfacehandler.com) will be available from Frent IT BV, based in the Netherlands. The CEO is Fokko Jan Reinders and in discussions with Fokko Jan, he made the following points:

• the time since last summer was used to extend InterfaceHandler, particularly to support Windows 7 in both 32 and 64 bits implementations.
• MQSeries and TIBCO messaging support will further extend InterfaceHandler’s connectivity capabilities.
• there is support for both SAP XI and R/3.
• much work has been completed in improving the documentation.
• installation is simpler and faster.

He also mentioned that InterfaceHandler is already in use at several test customers and that their responses are positive. In particular, InterfaceHandler’s ability to free up highly skilled SAP staff by simplifying the discovery of the ‘missing transactions’ is regarded as a significant plus – because such people are so valuable.

Analysis

One of the key attractions to C3BC of InterfaceHandler is that it is non-intrusive. Essentially you install it on a PC attached to your SAP ‘network’ and then, using techniques derived from automated discovery, it identifies the path of a transaction through one or many SAP instances. From this discovery, the application builds a map for the selected business transaction.
Once you have the map, then one knows where to look for ‘errant transactions,’ which, once found, may be and repaired and recovered. Transactions (and messages) can be found by IDOC / RFC number or XI/PI GUID, and InterfaceHandler enables the details of every step of the transaction message flow to become visible.

InterfaceHandler achieves this by working from the bottom-up rather than from the more traditional top-down associated with work flow monitoring. Using Frent IT’s developers’ deep knowledge of SAP, InterfaceHandler examines the metadata and linkages associated with each designated transaction to build a picture through SAP’s application modules of what happens to the data in the transaction as it traverses SAP’s myriad of rules. In so doing, it identifies the interfaces used and captures the status of all the components. It can even show the transaction flow performance and measure this to see if the throughput thresholds satisfy service-level agreements.

There is, to C3BC, another compelling advantage to Frent IT’s non-invasive approach – SAP Certification. Many SAP users value certification, which is a process that SAP uses to ensure that a third-party offering does not prejudice the company’s own applications. As might be expected, SAP Certification can be expensive and is difficult to achieve because of the testing required. However, because Interface Handler is non-invasive it does not change anything in any part of SAP’s applications. According to FrentIT, InterfaceHandler’s SAP Certification is imminent.

To be expert today in software as complex as SAP’s offerings requires expensive skills. Increasingly, SAP user organizations need automated tools to find out (‘discover’) what is happening within and across their SAP applications. In C3BC’s analysis there will be two stages in the adoption of a tool like InterfaceHandler:

• the first will focus on immediate problem solving – locating ‘missing transactions’; this will produce significant day-to-day savings (whether measured in costs avoided or people skills released or both).
• the second will prove be even more valuable; it will rest in building an automated knowledge base of how business transactions function with SAP.

While Frent IT has applied its knowledge of SAP to a specific difficulty, C3B Consulting expects tools similar to InterfaceHandler to emerge – because enterprises simply cannot afford the costs of missing (but perhaps not lost) transactions or events. Meanwhile, from February 2010, SAP users who are ‘losing’ transactions should rejoice. 

19.   Message to Iberia: customer from IT communication needs to fly First Class

Information, information, information. This, when used intelligently with technology, is the key to limiting damages and even improving customer satisfaction in a crisis. Seems obvious – no? Not apparently to airlines (and, I suspect, many other organizations which assume a crisis will never hit them). Yet the solution is so simple and the benefits are so great.

Let me begin by saying that the experience (described below) does not qualify as anything special and is beneath pathetic compared to miseries in Haiti and elsewhere. Flight delays and execrable service describe the occasional lot of the modern air traveler (and many have seen far worse). But this experience set me to thinking about minimizing the impact of a crisis – on supplier, staff and customers.

Last week, the combination of bad weather and an airline ill-prepared (and apparently uninterested in its customers) meant that a journey with a connection scheduled to take 9 hours unnecessarily took 33 hours with much waste of time. This was only the second time I have travelled with an airline whose internal disorganization virtually incapacitated it (it had happened to me once before while flying Air Canada, which I now avoid when I can) This time it was Iberia at its headquarters airport in Madrid.

As is so often the case, it was not just snow (the reported reason) but multiple compounding factors, including that stunning lack of interest in, or understanding of, its passengers (and, I suspect, its staff) that made everything so bad. To understand the solution more clearly, let me offer some background.

At 0230 (all times are presented in 24 hour notation) on the day of travel, I received an SMS message from Iberia saying my flight from Alicante-Madrid (at 0715) was rescheduled (but with no details) and that I should call Iberia (with no number was given). Now awake, I tried the Web to find that Iberia’s web site said all was working as scheduled. Without a number to call, I had no choice but to leave for the airport as if all was working.

On trying to check in at 0600, I was told that not only was my flight canceled but so was the previous one – because the planes had been unable to leave from Madrid the evening before. In other words, Iberia knew it had problems many hours before it sent that SMS to me. More distressingly, I discovered that no attempt had been made to rebook me or the 200+ other people now queuing to talk with 2-3 stressed Iberia reservation staff. Eventually, I was rebooked for an 1150 flight to arrive in Madrid at 1300 with my connection nearly 12 hours later at 2330 that night.

I checked in and went through security when my ‘troubles’ really began. There were NO Iberia employees beyond security for passengers to consult. That meant relying on airport (not Iberia) TV monitors or the good will of a lounge. What information existed was almost always inaccurate. Even when I turned to the Internet, I found that Iberia’s own Arrivals and Departures pages lagged other travel sites (like Flightstats.com). Other contradictions abounded. For example, one kindly Iberia employee, who called to update me, said he had just been told I would not be flying at all that day – yet I was on a plane 30 minutes later (and even that was not full despite three or more flights having been canceled).

Eventually, I reached Madrid (barely 300 miles away) some 10 hours late to find Terminal 4 (Iberia’s 12 prestige home) in similar confusion. My misfortunes continued when all passengers had to disembark from the 2330 flight at 2350 into a deserted terminal because the plane would not work – again without any information. I finally reached my destination – merely 24 hours later than scheduled.

Analysis

Absent flights without delays or disruptions:

• What I wanted most – and I suspect most Iberia staff wanted – was accurate, timely and updated information, preferably with some context (like some ongoing explanations)
• What I presume an airline, when facing such dire straights, wants is time to solve its operational issues – in essence, it wants breathing space to address and correct its problem.

Without information, paying customers (passengers) become a cost rather than an asset. In wanting/seeking to know more, they lose sympathy and proceed to further stress the supplier with continuous questions. Furthermore, employees who are in the crossfire – in this case facing irate and frustrated passengers – often had no or incorrect information, which only compounds the confusion (they are asking the same questions). Last week was a recipe for trouble, which is exactly what Iberia duly served up.

The keys to preventing and correcting such situations, then, are information plus communication – which buys that breathing space. It does not take much wisdom to spot that if you deliver constantly updated information, you keep customers informed, which relieves the burden on customer-facing staff members, who in turn do not spend their time trying to obtain the impossible from the hard-pressed operations people trying to resolve the problems.

With constant information, especially for an overtly customer-unfriendly airline like Iberia, the odds are that customer (and even staff) relations will markedly improve. The irony is that implementing a solution for such circumstances is relatively trivial. All that is needed are: 1) a dedicated Web site, 2) a browser and 3) SMS – along with a specific ‘crisis’ information group.

Dedicating a Web site to the ‘crisis’ an organization:

• removes substantial query traffic from the main order/sales Web site (for Iberia, its reservations system), which organizations need to keep running in order to accept/support future business;
• establishes responsibility for, and some sense of awareness about, those affected by the crisis, along with the impression that someone is doing something, that the situation is evolving and improving and that the organization cares (as British Airways has done in the past)
• makes access to real-time information almost universal via a browser – whether on laptop, computer or phone, – either at the airport or away.

Of course, having a Web site is no good by itself. This has to have as much updated information as is practical. In an airline’s case, a small ‘crisis’ group should be tasked with keeping the Web site up to date (this includes liaising with operations swiftly and simply). This group should include the technologists who can rapidly process relevant information 13 onto the Web site. Personally, I would add a commentator to give a running update about the current position. A more superior Web site might even assist customers to do their own rescheduling online – after all, they probably know best what they need. All of which, if implemented, reduces the burden on other hard-pressed resources and staff.

The third technology is SMS (in this I include Blackberries, instant messaging, as well as social networking sites like Twitter, Ping! and other near-immediate communication media). Recall that SMS I received at 0230. Iberia would have helped much more if it said my flight was canceled due to bad weather (we all understand this happens), that I was rebooked on the XYZ later flight to connect with the 2330 flight and that I should call a given number if this was not acceptable. Had I had this information, I could have stayed at home until late afternoon (though it would not have helped when the plane at 2330 would not function).

As an aside about SMS, while Iberia’s performance was execrable, it is certainly not alone. About 18 months ago, I was talking with the IT integration staff within the operations group at a major US airline. The idea of alerting airline colleagues (never mind customers) by SMS was greeted with puzzlement – even though these able people knew of operational problems (like an aircraft leaving late) before anyone else. The notion of using operational data to alert passengers was regarded as abnormal.

Of course, not all affected passengers (and people waiting to meet them) have the necessary technologies to access such alerts or the Web site. But what I have noted time and again is how fellow customers share information generously. If you receive an SMS telling you that your flight (or order) has been rescheduled yet again, you will share this with others next to you. In this way (what might be called real-life social networking), customers help each other, but they can only do this if they are given the means to do so.

Conversely, customers (usually inadvertently) will hinder if left ignorant – as Iberia discovered. Looking back from the outside at what happened to me, Iberia made just about every mistake in the book. Had it done even a little prior planning, once it knew it had problems the previous evening, it could have easily forestalled the avalanche of bitter complaints that I overheard. Regardless of how bad the news is, passengers (customers) prefer being informed to being left in the dark.

The net of this is that:

• most organizations, with a little forethought and planning, can exploit simple technologies to make themselves work better and look better – even in a crisis when all appears to be going wrong.
• the alternative – failing to take simple steps – can prove very expensive, not just in lost customer satisfaction but in squandering client, as well as, staff loyalty (as Iberia will discover on its bottom line).

18.   AEP: Will it ride to the rescue of BI, as well as reduce the importance of the RDBMS?

One way (if a little provocative) to think about the differences between BI (business intelligence) and AEP (Analytical Event Processing) involves an analogy with aircraft ‘accidents’:

• BI is akin to crash investigating – looking at the causal evidence and circumstances after the crash event and then trying to explain what happened; this is valuable and necessary but hardly desirable.
• AEP, in contrast, is akin to ongoing engine and airframe systems monitoring, seeking to measure and anticipate, and thereby, preventing the incidents that aircraft crash investigators would have to research

In the Pund-IT edition of November 4th, Merv Adrian and I opened a dialogue: “Traditional RDBMSs Stagnate as AEP Emerges from the Shadows.” This provoked a range of comments (q.v. Merv's blog (www.mervadrian.wordpress.com) about various aspects of the futures of AEP, RDBMS (Relational DataBase Management System), and BI and RDBMS, all of which have inspired further consideration about BI and AEP – in addition to my original comments on SQL-based RDBMSs.

Strangely, since one of my opening comments was: “The first is that the relational database is a much more endangered species than I had thought. That is not to say that RDBMSs will disappear, or that their revenues will diminish.” Several of those commenting on Merv’s blog seemed to think that I was suggesting the opposite – that the demise of the RDBMS as we know it was imminent. . Of course, the RDBMS (and its associated revenues) will not evaporate – but I do believe that the central relevance of the RDBMS will likely decline relative to newer initiatives, one of which is AEP.

Other readers suggested that it was not helpful to create a new label or acronym, such as AEP (which I had not, having heard it from others). . Yet, while we agree that excess labeling is about as useful as percentage inflation – can 125% of effort truly be worth more than 100% effort? – our analysis supports the notion that AEP does indeed reflect a set of characteristics that assists understanding and therefore merits the differentiation.

BI is not AEP. . AEP is not BI. . RDBMS is neither. . Although they may one day complement or even fully reflect each other, today they are different. Why, or how, are they different?

BI is – at least traditionally – about the analysis of post-processed data – data that has been through the transaction or other preparatory mill (including ETL – Extract, Transform, Load). . For the most part, it is harbored or parked in data warehouses in a structured format ready for sophisticated analyses, most often delivered through some form of SQL-based tool.

Over the years an enormous amount on money has been poured into these sorts of data warehouses by large corporations seeking an analytical edge, which to some degree BI has provided. . But the reality is that much of the data that makes its way into data warehouses is out of date or, at best, stale by the time it has been through the process mill.

This does 10 not mean that practical conclusions cannot be drawn. They can be and are, but not in a timely manner. In effect, BI has survived and prospered primarily because there were few alternatives. A poor analytical solution was better than nothing, and was bought for this reason.

To understand why AEP is different, think about what business and government customers want from analytics in, say, a credit operation. Not just a Visa or MasterCard – this also includes telephone credit cards or even mobile phone operations. Credit providers want to ensure that the credit being requested by a customer has an acceptable or permissible probability that it will be repaid (and so can be granted). Credit providers wish to avoid issuing credit where the user has no credit, as well as seek patterns of past behavior that indicate potential fraud. The final objective is to issue ‘credit’ only when warranted and withhold it when it is not appropriate.

Such real-time processing (in-flight processing is possibly a better description) across one or many data streams is not what traditional BI offers. However, it is what AEP does. Ergo, traditional BI looks exposed when compared to something like AEP – in its many forms, whether on financial markets, in industrial production, in credit handling, in flight engineering operations, in ship and port scheduling (there is a short and simple example from the BBC describing the latter – see Note 1 below for URL) or whatever.

What makes AEP different? It enables analysis to be done as and when it matters by those who are most concerned. . In many instances this is long before the data ‘hits’ any formal RDBMS arrives for ETL-like activities prior to being put away in BI’s holding warehouses.

AEP is less structured than BI, yet it also offers analysis before any transactional implication becomes necessary. Whether it is analyzing financial events in the search for arbitrage possibilities before making trading decisions (thereby starting the transaction sequence all the way through to the data warehouse) or examining sensor-originated data before issuing an alert for remedial action, the analysis of events takes place in advance of traditional processing. Fundamentally this is what separates AEP from BI (and arguably both RDBMSs and even Business Activity Monitoring (BAM)).

Analysis

Though many may not like to consider or admit it, BI has stagnated over the past 20 years. Once fashionable (as AEP will no doubt prove to be at some point), BI became costly, complex and involved much IT expertise – not least, the magic arts of ETL. This, along with its inherent time delays, has distanced BI from common business users.

Instead, BI has become the preserve of specialists seeking to wring from past data precious conclusions that are relevant for now (and the future). The closest analogy is with forensic scientists (in this case ‘information scientists’) delving for insights. Or, put another way, BI analysts might be characterized as office-based examiners of what has happened, who hope to shape the future.

In contrast, AEP analysts are already much closer to the business. Look at Starview and Sybase’s Real-time Analytics Platform to understand AEP’s how’s and why’s. AEP is about ‘new’ (pre-conventionally processed) data rather than 'old' data (post-processed and only now ready for analysis).

It is significant that most of those looking to AEP are already front-line business people operating on or near the shop, pipeline or trading floor, who also possess practical business engineering insights and requirements. If you accept this differentiation, all can move forward.

That they are different does not mean that they cannot coexist. They can. But to achieve this, both approaches must be reconciled in terms of timing, storage and manipulation:

• pre-processed data – AEP's events – need not be 'thrown' away; events represent data points that can be mined for additional insights (they are not just on-the-fly happenings that have no longer term value)
• post-processed data – BI's traditional fare – too often used behind the times – can be spiced up by combining new and old data to bring revelations and possibly return the immediacy that BI seems to have lost.

At the moment, however, there is little in common in the two approaches, which means a significant division remains. In the analysis of C3B Consulting, BI will need to adapt to what the AEP and CEP and streaming vendors (from IBM to Oracle, from Streambase to Sybase, from Progress to TIBCO – among others) offer rather than the other way round. This position has been confirmed by consultations between BI vendors with C3BC about acquisitions, because they understand what it means for BI to lose out).

But do not confuse the fact that several AEP vendors are also RDBMS and BI vendors; what they sell for AEP is different from that sold for RDBMS/TP or BI. . As such AEP may yet rescue BI – if BI adapts. No, the RDBMS will not disappear, though it may lose relative importance. Yes, BI is in danger, but it can be rescued if vendors and specialists wake up. No, AEP will not either solve every problem or replace all other solutions. Yes, the overall market ‘pot’ will grow. But that growth will come from where in-flight data analysis can assist the business. We expect AEP/CEP to drive a part of that growth.

Note 1: Ships can now be tracked with smart software and GPS. The global shipping trade generates the data. And ships could be in and out of European ports much quicker thanks to smart software that now monitors their movements. (http:// news.bbc.co.uk/2/hi/technology/8413566.stm)

17.    Has IBM Made Cloud Comprehension Easier?

Cloud computing continues to attract much attention, to the point where it has become almost obligatory to mention it in every presentation (even if only to position something as ‘not being cloud-relevant). Meanwhile, as more and more crash the cloud party with ever more quixotic suggestions, clarity has been a principle victim.

However, IBM’s Connect 2009 was notable for its down-to-earth approach to cloud computing. Yes, IBM offers cloud services and products and will offer many more. Yes, IBM believes it will be a major player in private clouds as well as provisioning public ones and even in enabling the two to work together (so called hybrid clouds). What was, however, more interesting for C3B Consulting was IBM’s approach in trying to establish what clouds might do and what benefits are associated.

To IBM, cloud computing represents a new consumption and delivery model that has been inspired by consumer-oriented, Internet services that enable self-service, diverse sourcing options and potential economies-of-scale. In effect, cloud computing represents the industrialized delivery of IT supported services which are oriented to workload planning and management. In particular, one table Steve Mills (the SVP of IBM’s Software Group) used merits discussion. In it, Mills divided the main attractions of cloud computing into three:

• standardization
• Automation
• virtualization.

With standardization, IBM envisions:

• Internet access which is where services (like SaaS applications) are delivered - thereby enabling access anywhere and anytime.metering and billing, along with flexible pricing; this is where service usage is tracked with metrics to enable customer to exploit multiple payment models - thereby improving cost transparency and introducing more flexibility for pricing than has been the case in the past
• service catalog ordering where defined environments can be easily ordered by individuals or organizations; this is also where consumer concerns are abstracted from provider concerns (via service interfaces).

For automation IBM sees:

• elastic scaling where IT environments can scale up (and down) by large factors according to changing business requirements; this optimizes IT resources utilization as well as increases flexibility
• automated provisioning where IT resources rapidly be provisioned (or de-provisioned) on demand thus reducing IT’s provisioning cycle time and management cost.

Virtualization, arguably the most complex cloud-related process, allows IT resources to be shared between many applications:

• applications can run virtually anywhere
• more efficient utilization of IT resources by reducing hardware costs
• the introduction of the economies of scale for which cloud computing strives (though in a different form than popular, in-house virtualization

Analysis

The above features of IBM’s cloud vision are not mutually exclusive categorizations, just as workload planning and management is not the “be  all/end all” of everything related to cloud computing. That said, C3B Consulting sees IBM’s dissection of the cloud as being of use to customers – simply because the company so effectively isolates the reason(s) they are interested in cloud computing. Comprehending the particular attraction(s) that matters to you is, in our view, an imperative in a heavily over-marketed environment.

What IBM made clear at Connect 2009 (and C3B Consulting concurs) is that if public and hybrid clouds are to work, then they must be open, be standards-based, and use common components and processes which enable elastic scaling along with fault recovery. Without these features, cloud  computing will not offer as much as its promoters desire or its customers require. Indeed, even with private clouds, these ‘qualities’ represent worthwhile ‘tests’ against which to evaluate cloud offerings – whether they are offered by IBM or other vendors.


16 .   IBM’s Secret Product Sauce – Industry Frameworks

In the services area, it is still uncommon for consulting firms to analyze current and past projects in order to identify common aspects which can be re-used without the client having to pay for a learning cycle. When such analysis is done, clients benefit from continually evolving best practices, as well as from working with service professionals familiar with a technique or approach, rather than having to learn on the job. Accenture, IBM and Indra (of Spain) are three such organizations where such a discipline is methodically applied.

At IBM’s recent Connect 2009, the company demonstrated how this approach can be applied to software products. The essential approach is the same – looking for repeatable aspects associated with IBM’s software (and some services) – with a view to identifying common ground which can be reused. As IBM likes to describe it, developer/analysts are seeking to avoid ‘one of a kind’ projects, preferring to identify those where other organizations are likely to want the same or similar. The approach is straightforward:

• look for recurring pain points at customers;
• identify those areas where there are repeatable patterns; and
• create (a process IBM calls ‘hardening’) solutions that make the most of IBM Software Group’s products and assets.

But there is another essential point: IBM has overlaid this discipline with a second set of criteria – industry dimensions. In other words, search for recurring pain points and resulting features and solutions takes place within the context of selected industries. Through specializing by industry – for example, energy and utilities, telco, banking, healthcare, government (local and central), insurance and retail) – IBM has been able to create 10 frameworks that address specific types of problem that recur and recur in specific industries. The 10 priority frameworks outlined below cover 50 solutions areas:

• Solution Architecture For Energy and Utilities
• Insurance Process Acceleration Framework
• Product Development Integration Framework (for Automotive, Aerospace, Electronics, Petroleum and Chemical industries)
• Service Provider Delivery Framework (Telco)
• Banking Industry Framework
• Integrated Information Framework (Chemicals and Petroleum)
• Network Centric Operations Framework (Government - Military and Defense)
• Government Industry Framework (Government – Civilian)
• Health Integration Framework
• Retail Integration Framework

What do these Industry Frameworks comprise? In essence, they are collections of IBM software products that have been made to work together within specific frameworks that includes process models/flows, information models, design templates, re-usable code assets and common services. To this soup are added industry-specific standards, industry and subject matter-specific expertise and third-party specialty applications and tools.

For example, The Government Industry Framework addresses public safety issues by helping public safety professionals and teams to collaborate (a notorious failing during 9/11), derive intelligence from various sources of information (like CCTV and other devices), identify high-risk situations (whether perpetrators or potential victims in real time) and facilitate responses to natural disasters, as well as terrorist or other assaults). Indeed, the value of this was proven recently when a boy was shot on a bus in Chicago and the ‘event’ was analyzed by triangulation that accurately located the gun shot. As a result, the assailant was apprehended immediately with video evidence to back up the charges in court.

The Public Safety framework has software components drawn from IBM’s WebShere, Tivoli, Lotus, Information Management and Rational brands tied together using SOA disciplines, plus the WebSphere Enterprise Server Bus (ESB). ‘Above the line’ (the ESB) IBM works with partners to integrate specialty applications that can exploit the middleware plumbing that IBM has ‘hardened’ into the framework and products. The result is a complete specialized, pre-integrated offering which addresses common, recurring specific industry issues.

Analysis

The reason that this is attractive is straightforward: problem-specific frameworks that apply to industry-specific challenges make it much easier to solve customers’ problems efficiently and cost-effectively. This is ‘building from the problem out’ rather than the traditional ‘Mr. Customer, we provide the (software) parts, and you have the good fortune to have to assemble them’ approach, which has been too common in the past. Thus, the attraction for customers is great. For IBM’s partners in these frameworks, the attraction is similarly obvious. They can focus on what they know how to do best with their own specialized applications and seat this on IBM’s dependable and proven middleware stack (IBM remains adamant that it will not become an application vendor which competes with its partners).

For IBM, the attraction is larger sales over longer timeframes with the right software gathered together. IBM is, in effect, exploiting its own assets in ways that customers have always found it hard to do. The frameworks approach also makes it much harder for most competitors: not only do they have to compete with the depth of IBM’s software solutions (which they are accustomed to having to do) but they also have to compete with IBM’s specialty partners and the joint understanding that comes from acquiring and effectively leveraging deep industry-specific expertise. 

n effect, IBM has used its experience to find, through industry frameworks, specific solutions which multiply the company’s reach and capability  to sell. Rather than having different teams selling the advantages of Information Management, WebSphere and Tivoli middleware (on which applications still have to be made to run), industry frameworks enable one sale with all the component parts already assembled. This must be as near to a secret sauce for products as can exist and C3B Consulting fully believes that customers will love it and keep coming back to IBM for more.


15    A Dialogue: Traditional RDBMSs Stagnate as Analytical Event Processing (AEP) Emerges from the Shadows

Charles:

Over the past 2 weeks I have been traveling in California, in part to attend the High Performance Transaction Systems Workshop (an institution founded by the late Jim Gray who, alas, disappeared with his yacht off San Francisco some 3 years ago) and in part to visit a number of customer and vendor organizations. I am still musing on the implications and will be writing more. There were, however, two strong inputs that have impressed themselves on me.

The first is that the relational database (RDBMS) is a much more endangered species than I had thought. This is not to say that RDBMS will disappear, or that their revenues will diminish soon. But it is to say that future growth in the storage for processing of data is going toome from non-RDBMS storage solutions. Hadoop and the use of Map/Reduce techniques is a prime candidate for processing very large datasets (which may in turn feed conventional RDBMS). Specialized columnar databases – for example from Vertica – are acquiring their own following. In addition, for the huge datasets (measured in quintillions of bytes) that you find in ‘big’ science, Michael Stonebraker and others are working on SciDB.

Merv:

I’ve been on the road too: at Oracle Open World, Teradata Partners, IBM IoD, and most recent Data Warehouse Institute (TDWI) event. (Quick blog post about that at http://mervadrian.wordpress.com/2009/10/29/a-tale-of-three-cities-and-oracle-teradata-andibm-databases/ ) And I couldn’t disagree more with you about the future of RDBMS – while it is going to be supplemented by products highly tuned for some specific use cases, its ongoing growth will continue to be healthy. RDMBS has done well in the downturn, with growth in license revenues alone measured in the $billions, and there is no evidence of a slowdown, although price pressure from open source alternatives will continue.

Charles:

Merv’s view is that the RDBMS is safe. Financially, it probably is. But RDBMS will becoming a specialty, and will lose their dominant position. The database storage world is changing fast and there will be many alternatives, often far less expensive than conventional RDBMS, for customers to choose from, and covering faster as well as greater capabilities. Instead of a world where the traditional, transactional RDBMS has ruled supreme for 3 decades we willlikely see the RDBMS become a tool for specific tasks, and no longer be deployed for generalized consumption.

Merv:

In the near term, the value proposition of RDBMS remains sound – it’s a good general purpose engine for persisting data to capture transactions, to use it more than once for routine reporting and ad hoc analysis, etc. Use cases for more complex, high-performance, and special purpose applications have always existed, and in the past, such pretenders as object databases served them. They were zero billion dollar markets, and remained so, driving those vendors out of business.

While it’s likely to be different this time, alternatives don’t mean a market goes away. In fact, there are several RDMBS use cases to consider in this context: columnar stores for BI; streaming data, as Charles notes, scientific applications, and text analytics, which is getting its own play. Stonebraker is (surprise) active in most of them, as well in a new venture called VoltDB (some wags called it Horizontica) that will challenge the OLTP performance of “conventional” RDBMS. New workloads mean new sales, and new revenue, and maybe some new vendors will survive the shakeout – or become part of big vendor portfolios, as many already have. It is precisely for “generalized consumption,” for mainstream workloads at moderate cost, with existing skills in place, that RDBMS will – excuse the pun – persist.

Charles:

With what Merv says in the two paragraphs above, we concur. In part confirming this impression [that the RDBMS will become a specialized tool”] have been the discussions with various vendors about analyzing events. In financial markets, huge numbers of events are generated via data feeds (about stock, currency and commodity prices and their movements). On the shop floor in manufacturing, the same happens: most modern automated industrial devices produce events about what is happening. In medicine, it is similar: ECG, EKGs and numerous other devices all supply data points. In logistics, location information (for example from GPS units installed in thousands of vehicles) is copious.

What these four instances, and there are many more, have in common is the need for analysis of the ‘event’ data – whether in real time, near real-time or soon after. Large internet business firms (Google, eBay, Yahoo and many others) generate vast amounts of information from users visiting their sites. This data is proving to be a highly valuable resource for prediction and for determining and then optimizing around what customers want.

A new class of event processing or complex event processing is emerging. Oracle, Starview Technology and Sybase are three companies which are focusing on not only capturing events but also deploying specialist analytical platforms to accelerate the delivery of information obtained from the events data. Sybase has its feet firmly in the financial sector where its Real-time Analytical Platform is a complete (and expensive) solution. Oracle has combined its own CEP approach to data with much of the infrastructure for CEP that it acquired from BEA – to create a highly efficient platform. Starview, by contrast, has an industrial background and so is concentrating on what you can find out from the machines on the shop floor. All three of these merit additional descriptions (but this is for subsequent Pund-IT editions). Suffice it to say for now that each is distinctively different from each other and demonstrably superior to other competitors trying to occupy the same space.

Merv:

I agree that CEP, or stream-based data processing, is an exciting new class of applications that will create new business for the ultimately successful suppliers. But the mainstream firms like Oracle, and, by the way, IBM, whose InfoSphere Streams I wrote about in http://ervadrian.wordpress.com/2009/05/22/infosphere-streams-is-a-game-changer/ , see these opportunities as likely to be connected to other databases for follow-on processing.  These workloads will drive new RDBMS (or other new database engine) revenues as well as
their own.

One great example case is clickstream data. The problem of “sessionization” is a complex, multipass one – interpret a set of clicks of variable length, changing direction (back to previous page, off in a different direction), different type (read/transact/cancel) – and classify the resulting sessions. Then do analytics to determine what kinds of users had what kinds of sessions. Although a good deal of work can be done as the data flows by, much of the value add is downstream, operating on persisted data – often in an RDBMS. And lower-end alternatives like SQLStream show promise and are taking their own approach to the issue.

Summary Analysis

Charles:

What do the evolution of the data storage tools and Analytical Event Processing (AEP) have in common? They both need copious data as well as the means to analyze more than the streams of data that are generated by ticker feeds, GPS, machine tools, medical equipment or whatever. In different ways, in the analysis of C3B Consulting, both will be act as change agents for how data is used and stored. The old paradigm where the RDBMS (and before that the transactional database) reigned supreme is breaking down under the colossal volumes of data that are now being captured.

At HPTS, Wayne Duquaine in his presentation observed that there are c 6.7Bn people on the planet and there are already 25 Billion MCUs installed (plus 1 Bn WiFi devices and 5Bn sensors). These are already generating data points. What is happening is that these can now be captured for processing to improve almost anything – from potentially predicting heart attacks a day early through to better utilization of power. According to Wayne, by adding sensors to many mechanical devices and analyzing the output, energy efficiency can rise from today’s 55% to more than 90%. For this to happen, however, AEP has to come out of the shadows and data stores will change.

Merv:

We agree on this. Where we differ is on whether it implies that RDBMS is going to see a relative decay of importance (the revenue streams will probably continue, however). At Teradata Partners, I heard about important new opportunities customers are tackling with data volumes that were unheard of just a few years ago. And they, like IBM, Oracle, Netezza, Aster Data and others, are adding support for new tools like MapReduce into theirportfolios for hybrid applications that use both approaches.

In addition, moving logic into the (existing) database is seen as a way of tackling other problems that have suffered because of IO bottlenecks and the resulting cost in latency as well as the redundancy of moving data to other platforms. Several vendors have added direct support for SAS inside the database as a counter to IBM’s expected similar efforts with SPSS. However, using more specialty data copies, with multiple different engines, can lead down a slippery slope resulting in chaotic, ungoverned data proliferation.

What we are watching is a refactoring of the software portfolio. Specialty engines have taken the place of the monolithic transaction monitor, offering specialized services like rules processing, master data management, application services, etc. “One layer back” in the architecture, the persistence layer is undergoing a similar transformation, evolving engines that support specialized needs. As a result, rather than RDMBS shrinking, we believe that with the addition of AEP, the whole pie will grow.


14.    IBM – Helping Clients Escape Traditional, Organizationally-Based IT

On October 20th, IBM made a raft of announcements and product launches relating to what it calls ‘Dynamic Infrastructure.’ Amongst these were improved software, hardware, packaging and services, plus continuing re-integration of parts of its software businesses that it had previously separated. (In this case, the most notable announced were the ever closer ties between the Tivoli and Rationale brands).

Hidden in the details was what IBM refers to (less than pithily) as ‘Converged Service Management for Industries.’ Previous C3BC analyses suggested that this approach may be amongst the most significant changes for IT delivery in decades. Indeed, it may well threaten many conventional IT assumptions and responsibilities.

What is Converged Service Management for Industries? In IBM’s terms, it describes the cost effective delivery and management of services for selected industry environments, including:

• For energy and utilities, the IBM SAFE Framework seeks to provide integrated, real-time views across IP Networks, SCADA, IT and Smart Meter Infrastructures (with the objective to improve reliability and productivity);
• For health care, the provisioning of monitoring, security and compliance abilities for all sorts of related organizations/hospitals (with the objective to improve service quality, as well as patient focus and reduced costs);
• For banking, the monitoring of systems and IT infrastructures to measure how these perform against stated process indicators, plus the delivery of security and compliance (with the objective of consistent management of user identities, access to systems and security policies);
• For petrochemicals, the monitoring and event management applied across a converged infrastructure (with the objective of enabling participants to obtain a deeper and richer ‘view’ of business assets and their use on an enterprise basis than is possible today).

All this is about IBM delivering “smarter” services designed for specific industries and based on Dynamic Infrastructures, which include integrated enterprise and IT asset management. In IBM’s view, with all this can come integrated service desk and support functions with software that connects IT, ‘smart assets’ and ‘smarter services.’ Best of all (for IBM), success will leverage the company’s proven industry-specific expertise and efforts related to standards, best practices and compliance regulation. In other words, ‘IBM will choose the standards best for your industry.’

Analysis

Prima facie, IBM’s Converged Service Management for industries makes sense in today’s business environment where doing more with less is an explicit objective for most companies. In our analysis, however, it hides a much more profound change of emphasis – one that IBM likely spotted long ago and where it is now is aiming to grab as much initiative and advantage as possible.

There are at least two common ‘themes’ running through IBM’s Converged Service Management for Industries: industry specificity and scale. Neither one is new but considered in combination and in the context of the chosen industries, the greater opportunities become clear. For example, consider the scale of automation of metering (whether for electricity, gas, water or whatever). In any provider ‘network’ – for simplicity’s sake, let us take electricity – there exist a surprising number of participants (though how many depends on which country and which commodity is involved). In electricity, there are generators, distributors, transmission network providers, traders, meter owners and customers.

Some are huge by size and revenue (for example, generators) while others are huge by dint of large numbers (for example, customers). Then there is volume. If you have 30M customers with consumption readings (events) occurring every minute that means 1.31^12 readings per month. That is the raw information, which has to be communicated and processed somewhere – and preferably not multiple times, as could happen in each of generators, distributors, transmission network providers, traders, meter owners and customers – for each has their own interest.

Given this, what would be a more efficient way forward? We would suggest that some entity (call it ‘X’) step forward and say, ‘We will do the data gathering and aggregation processing for you,’ and you may pay us for the processing and provision of the relevant data/analyses. In effect, we will provide a data gathering and processing “utility” where the generator receives what is best for determining how to manage peaks and loads while customers analyze how they consume power and then optimize expenditures. The same will apply to other participants.

The point to understand is that key facets of IT currently undertaken – in part or 100% - by traditional IT would migrate to the X utility, giving it massive economies of scale. You might think this idea fanciful, but various organizations are already thinking about this.  C3BC has already written about Indra’s (from Spain) interest, and sees IBM envisioning exactly the same – though on a larger scale, potentially spanning multiple industries. If this happens (as it likely will in less than a decade, in C3BC’s analysis) the impact on traditional corporate IT will be considerable. In fact, much of IT as we know it will gradually be subsumed into a small number of mighty X-enabled services. This is already occurring in capital markets with the leaders in ETNs and in intermediary organizations like Stock Exchanges or DTC-equivalents. It can also happen in energy, healthcare, banking and petrochemicals – especially when scale of cost reductions represented by the efficiency delivery made possible by the scale of such outsourced IT is so great.

Power is shifting. IT’s future is not stable. Cloud computing is but an early manifestation of this disparity. This is why C3BC considers the  Converged Service Management for Industries element of IBM’s October 20th’s announcements to be critically important.

13.   Streambase: Social Networking Meets Complex Event Processing
(Published Pund-IT, October 14 , 2009)

Complex Event Processing (CEP) has long been associated with financial markets and the data that streams from exchanges and markets, including trade ticks, foreign exchange movements, commodity prices and many other “events” generated in New York, London and elsewhere. CEP is the name of the software – sold by companies like Aleri, Progress Software and Streambase, as well as by IBM, Oracle and TIBCO – which can examine and correlate huge numbers (measured in thousands or tens of thousands of events per second) looking for patterns or reasons to buy or sell. Traditionally, this type of processing seemed best suited for rocket scientists and other specialists who needed to have access to high cost, specialist data feeds from the likes of Thomson Financial/Reuters and Bloomberg, and stock exchanges. That is changing. One of the first and, in the view of C3B Consulting, most important harbingers of this change was the introduction of analysis of GPS events – using changing positional information to add to application capabilities and to extend this into social/criminal scenarios (probation monitoring, stalking, etc.).

Recently Streambase has added another – the matching of CEP with social networking – in this case Twitter. Tweets are now ‘events’ available for analysis. By adding social networking as such a source, CEP further extends its potential reach into and influence on organizations, large and small.

Tweets are like SMS messages; they are up to 140 characters and can be sent from a myriad of devices – from extended phones to PCs. With some estimates putting the number of active Twitter users at over 15M by the end of 2009, the capability to generate many messages is enormous. In financial markets, Twitter has become an alternative news service. For example, it is said that news from North Korea reached the markets via Twitter 45 minutes before it was broadcast on CNN and that Amazon’s purchase of Zappos reached the markets via Twitter sooner than conventional means. With traders in financial markets depending on the timely distribution of information, anything that seems to offer an edge is adopted. There is already a lively, though not necessarily accurate, volume of tweets amongst different trading communities.

So what did Streambase do? It took its Streambase Server event processing platform and added a Twitter-specific adapter. In practice this was not that difficult – after all, all tweets are already in digital form. By accepting tweets as an event source, the Streambase platform could now ‘listen to’ Twitter, then establish event processing rules in order to analyze tweets. And various customers have started to do this (although most are reluctant to talk in detail because they believe this capability provides a competitive advantage). As Mark Palmer, the CEO of Streambase, described: “Trading systems or operations support can (now) use Twitter
direct messaging to alert users of trading opportunities or system problems. Systems can also use Twitter messages to assess economic sentiment in real-time for trading systems or marketing analytics, through monitoring news headlines and popular sentiment transmitted via Twitter.” Richard Tibbetts, CTO, Streambase, pointed out to C3BC that “Adding adapters for other social networks – like Facebook – need not be difficult.”

C3BC Analysis

In the view of C3BC, this represents a significant change, in at least two different dimensions:

• For businesses (and governments) the ability to monitor and analyze Tweets and send alerts using social networking as a source (or sources) adds capabilities that previously were simply unavailable.
• For CEP software vendors, adding social networks builds on what GPS added and takes CEP deeper into non-financial specialty applications.
  

In the first instance, consider a ‘relatively’ trivial application – in public relations. A company or organization has a product or service that ‘goes bad.’ The longer it takes for the company or organization to know it has a problem, the harder it is to react appropriately. The sooner actions can commence, the less expensive the problem becomes (monetarily,and in managing distraction and damage to the firm).

By automating the monitoring of social networking for particular patterns, such as product or service issues, detailed proof-point data can be served up to management that recommends or demands action. Money, reputations and time are saved. But this ability to mine social networking data is not necessarily entirely positive. A potentially more sinister variant of this (that some governments are almost certainly already doing) is performing the same functions in the name of ‘security.’

This means that, in C3BC’s analysis, the marketplace for CEP is likely to expand and expand. If relatively mundane applications of CEP (as in the public relations example) achieve traction, the size of the opportunity will continue to grow. Indeed, the term CEP is probably surplus: event processing has become general purpose.

Plus, it has one other market expanding factor – event processing can exist alongside, in parallel to, existing IT infrastructure; thus it is relatively easy to introduce into existing schemas and business models. By adding social networking as a source of events, Streambase has broadened the scope of events processing to far wider market than specialty financial events or even GPS vendors ever envisioned. Who will benefit? IBM, Progress, Oracle and TIBCO should all do well, but so will specialists like Aleri, Streambase and others.

12.   40GB of DB Processing for $10/Day
(Published Pund-IT, September 30 , 2009)

The notion that the days of paying for infrastructure software – from which so many vendors, including IBM, Microsoft and Oracle, to name a few – are coming to an end still attracts much ridicule. Only recently, C3B Consulting (C3BC) was asked “Are we really movingto an ‘all you can eat for free’ software license environment and, if so, how will software companies sell licenses in the future?” This question implies two subsidiary questions. The first, in effect, sows doubt about whether or not it is practical to move to an ‘all you can eat for free’ software environment. The second one effectively questions whether or not there is a future for the traditional software license. Let us look at each of these.

Recently, C3BC had the opportunity to talk with Extrabux, a San Diego-based Web price comparison site which either offers a percentage back to the buyer from each purchase made through Extrabux or makes it easy for the buyer to donate all or some of that ‘discount’ to a selection of named charities. The volumes of data that Extrabux has to process each day are staggering, as is the minimal software and operations cost, the avoidance of traditional software licensing and the scalability the company has attained.

Currently Extrabux has to load and process 40GB+ of data every morning (and this volume is rising). The data comprises some 70 million items (and their prices) coming from multiple different sources in a variety of non-standardized feeds each with its own formats; i.e. there is minimal commonality. In only 90 minutes, Extrabux not only loads this 40GB into a data store manager (Hadoop), but it then uses MapReduce techniques to process all that data into a consistent comparison format and makes it available to the Extrabux Web site. The cost – for multiple processors and storage – is just US$10/day, payable to Amazon
Web Services (AWS). (An Insight-Spectra case study on Extrabux can be obtained by sending an email to: requestcopy@insight-spectra.com.)

While talking to Extrabux, C3BC discovered that its IT people had originally sought to use conventional RDBMS technology. Unfortunately, this proved almost impossible; not only was such a solution far more expensive (it requires a costly DBA) but the daily load times alone (for all the new item/price combinations) ran to multiple hours – and that was before processing could even begin. Even worse, queries could take minutes or even hours to run when analyses and answers were needed much faster.

In the end, using a combination of what C3BC calls Non-Linear Licensed Software (N-LLS) – where customers do not pay ongoing license fees by processor or core – with the Cloud offerings available from AWS (from Elastic MapReduce to S3, Elastic Block Store, SQS, SimpleDB, Elastic Load Balancing, Auto Scaling), Extrabux created a flexible environment which takes advantage of multiple processors and variable storage and yet costs less than US$5K/year.

Now think about what (say) a traditional license approach for (say) a commercial RDBMS (or even an application server) costs to maintain, never mind buy. Extrabux has produced a dependable, operational IT infrastructure using N-LLS that has minimal purchase and maintenance costs and which is scalable both up and down.

C3BC Analysis

For those who continue to argue that N-LLS is not practical, Extrabux stands as proof of how this model makes it possible to have IT which is scalable, flexible and inexpensive. The only real question that remains is why CIOs (and CFOs) are not jumping to adopt NLLS. We believe that one part of the answer is inertia while another is the cost of conversion. But probably the most base reason that N-LLS has not been more broadly adopted is that neither CFOs nor CIOs yet appreciate how much they can save (C3BC is currently in mid-preparation of an example-based model and Report examining this very issue).

Yet the success of Extrabux does not imply that software will always be free or that N-LLS–based solutions will dominate the market. Instead, the growth of N-LLS means that software can sell using the traditional model where it provides some capability or value proposition not otherwise available. Today this will be more about applications, but not necessarily exclusively. There will always be room for innovation in infrastructure – though probably not for a long-lasting, linear licensed software advantage. On the other hand, for the highest level of applications that manifestly provide innovation and/or value add, then the conventional
Linear Licensed Software (LLS) purchasing model will continue to be sustainable – at least until such point that what was an innovative application attracts N-LLS-based copycats.

This means that, to make money in software, innovation will matter more and more. Those who do something ‘new’ (which can mean providing new functions, better performance, improved user experience or whatever has value) will have an opportunity to continue leveraging the traditional pricing model. With time this advantage will, however, erode. Meanwhile, when thinking about software vendors, C3BC advises clients to start to assess what N-LLS alternatives are already available for functions for which they are paying on an LLS basis.

The results of such analysis may also prove to be an accurate guide to the long-term survivability of many software products, and even vendors.

11.   Google’s Splashy EPUB Books Announcement Misses a Big Point
(Published Pund-IT, September 2, 2009)

Prima facie, last week was a good one for ebooks and being able to read them. Sony announced new versions of its slender Reader Touch and Reader Pocket devices, along with various improvements, including support for the XML-based EPUB standard. Yet the announcement that Google is making more than 1 million public domain books accessible in the EPUB format—via its Google Books service—seemed to be the bigger of the announcements. Such availability would open up huge possibilities for people wanting to read around the world.

Many have argued that Google’s adoption of EPUB adds critical support for this increasingly popular standard, produced by the electronic book industry. In addition, the fact that Amazon’s Kindle does not support EPUB would seem to leave Amazon out in the cold. To C3B Consulting, this misrepresents the situation (though both Sony and Google might disagree). Speaking personally—as a consumer of ebooks (using both Sony’s Portable Reader System and Stanza on an iPhone—now owned by Amazon as well as Adobe’s PDF format), I initially thought the week’s announcements must be good news. But it did not take long to discover that I was being overly optimistic.

Over the past six months, since starting to ‘use’ ebooks seriously, I have been living in non-native English speaking countries where access to books in English is restricted. Having enough to read is a must for one’s sanity; however, carrying sufficient paper books is both expensive (excess luggage) and awkward. In addition, as a technology commentator and consultant, I would dearly love to carry manuals and other technological literature with me on a slim device that is not my PC. For one, I can work more efficiently without having to print. Secondarily, being able to have a manual available while trying to accomplish some obscure task/research on new software would be a positive boon.

Despite Sony and Google’s apparently positive announcements, the truth is that the ebook world remains just as confused and confusing today as it was a week ago. Let me demonstrate by example. Imagine you are looking for a selection of works of fiction in Google Books that you want to read (to download on, say, the Sony PRS device. To do this, you must first determine what formats the PRS can accept (EPUB, BBEB and PDF—thank you Wikipedia, for Sony does not made this wholly obvious). Once you know this, however, you would think moving forward would be simple. Wrong.

Google’s announcement of its 1 million downloadable books in EPUB format seemed like Nirvana. Finding 20-30 books before setting off on a new set of travels should now be straightforward. Sad to say, reality is a long way away from that sought-after destination. Go to Google Books and you will find plenty of books but most did not have the EPUB (or PDF) download symbol. Okay. But was there any apparent or meaningful way to search, never mind organize, listings of EPUB or PDF-only books so that you could choose from a list of downloadable options? Not that I could find.

For the record, Google is by no means the only ‘provider’ to make it difficult to find what you want. In the UK, Waterstones is a large bookseller owned by HMV (of music fame) and supports a web site, akin to Barnes and Noble or Chapters. It has ebooks to offer and a deal with Sony supporting the latter’s e-readers. However, Waterstones organizes its ebooks as a simple listing within a genre (science fiction, history, politics, etc.); there is no sophisticated search for ebook formats (or even by specific e-readers) .Indeed, using the advanced search means you obtain listings of books that on closer examination are only available for order in paper format—the reverse of the objective.

What both of these, and other ‘ebookstores’ lack is what Amazon already provides. Kindle-available books have their own logical Web site in which you search only for books downloaded to a Kindle. In this respect, Amazon is much further ahead, from the viewpoint of a book-consumer wishing to select a book.

Now, move away from EPUB—for it is not the only relevant format. Because EPUB is XMLbased, PDF books could still be attractive, not least because they are easer to create. This applies especially for technical literature, such as manuals and other short-shelf life publications. One of the great disappointments about the Sony PRS device is how poorly it handles PDF files. For example, there is a most interesting book that dispels any improbable eco-assertions and eco-claims (David McKay – Sustainable energy—without the hot air). This is downloadable for free.

But unlike PCs with normal Acrobat Readers, the PDF version does not display correctly the text and graphics on the Sony device, making a mockery of the assertion of PDF support. Instead the layout is lost or so thoroughly messed up and altered (for example, with data and graphics losing their relevant location) that the content becomes near impossible to read. This should be unacceptable for both Adobe and Sony.

C3BC Analysis

Google and Waterstones do not appear to have understood how people find and then download ebooks (by the way, I am prepared to pay for books, if not idiotically priced—which is one of the better sides of the evolution of ebooks). Being able to see (say) EPUB and non-EPUB books together is like going to buy a book from a library where non-borrowable and purchasable versions are intermingled: the buyer has to do the work to find out which can be bought.

This is ludicrous. You want to search for what you can download to your device—not see everything on a given site. As for PDF rendering, Adobe stands to lose out significantly if it does not work harder to ensure that devices which claim to support PDF do so in a complete and satisfactory way.

Do ebooks stand as an example of the difference between attempting to solve a problem from a market (Amazon) or a technological (Google and Sony) perspective? The former seems to develop/deliver a product to consumers, then creates a mechanism to do that; in contrast, the latter assume that developing/adopting “superior” technologies will eventually lead to success in the marketplace.

C3B Consulting’s net take is that the ebooks promise to be excellent. While they will not necessarily replace paper books (at least in the next decade or so), they do complement them. But the success of ebooks will not occur until the finding and downloading of what you want to read, and then the subsequent reading experience, all work together. In this, Amazon and its Kindle still lead by a significant margin. The idiocy is that it should not take much for competitors to catch up; but they are simply not there yet.

10.   Indra’s Holistic Re-Use Approach to Automated Metering
        and Energy Efficiency
(Published Pund-IT, July 29th, 2009)

 
Few organizations methodically analyze past projects and engagements to identify trendsand patterns. The result of such efforts are new projects that not only can be repeated but which, with subsequent iterations, can offer cheaper and more insightful solutions to customers and clients. IBM and Accenture are two of the most obvious vendors that do this in pursuit of creating ongoing value as part of their basic business processes.

Another, though rather less well known, is Indra. As this Pund-IT article describes, the company is using what it has proven elsewhere to address TAMSECE (C3B Consulting’s generic name for Total Automated Metering for Solutions for an Energy Conscious Environment). Indra today has more than 29000 professional staff members worldwide. Though based in Madrid, over one-third of its more than $3B of sales take place outside its Spanish home market—and this is growing. While 47% of sales come from two sectors (Defense and Security,plus Travel and Traffic), Indra also provides solutions to Energy and Industry, to Financial Services, to Public Administration/Healthcare, as well as to Telecom and Media.

In a recent briefing to C3B Consulting, much emphasis was placed on the growth prospects that the Energy and Industry arena offers, not least in relation to initiatives associated with TAMSECE. This is despite the fact that Indra has more than 130 energy utilities using an already comprehensive product set—from control systems to metering systems; from modeling and monitoring to technical consulting.
Take, for example, its BRISA product (brisa = breeze in Spanish). This provides the operation and control in real-time capabilities for windmill parks and substations, plus integration with system operators (including management of orders) and visualization (also in real-time) of how generation is occurring. BRISA is in use today at more than 50 wind farms that are collectively generating more than 1300MW of electricity.
Indra, however, wishes to go much further. It sees major growth prospects in the automation of metering together with all the associated systems needed to support this if energy control (and climate change) objectives are to be met. For example, it estimates that more than $17B will need to be spent worldwide in improving just electricity transmission and distribution alone.

In our view, Indra differs from BRISA in three distinct aspects:

1. It is adopting a holistic approach—From (electricity or gas) meter via intermediaries/utilities/generators and back to the customer; most players in the automation of metering seem to ignore ‘the poor, old customer,’ who is likely to be the true control point in the future
2. It has designed its own flexible block meters—These support unusual characteristics, in that the communications module can be changed for whatever communications technology is appropriate for a particular location or at a future time (if, for instance, WiMAX were to become widespread in the future, then GPRS or PLC or whatever was originally installed could be swapped out for a WiMax module)
3. Its foundation depends on an integrated, all embracing network architecture—Indra regards this as being fundamental to success.
   

The last of these is probably the most significant. Without such a network architecture/topology, Indra’s experience is that rigidity occurs yet for TAMSECE to deliver, flexibility is needed. As technologies improve (and emerge) coherent (and appropriate) network architectures enable change, adaptation and modification. It facilitates the introduction of control systems that are far finer grained than in the past. TAMSECE is not about a small number of utilities with tens of power stations and 100s of major distribution points; rather it is about millions of transformers and tens of millions of meters and as many customers. It is also about scalability of computing along with the intelligence and business systems that offer everybody concerned—from government to consumer, from generator to distributor—the detail required to make energy efficient decisions.
This is where Indra’s ability to re-use what it possesses already matters. The company provides the proven real-time control system that runs Spain’s highly successful high-speed (350KMh) trains—the AVE. It is proposing to take this system and to adapt it for the energy sector for, despite the apparent differences in industry, there are many similarities. The success of the AVE control system depends on highly reliable handling of tens of millions of events per day. The same approach will be required for TAMSECE solutions, which will need to handle (amongst other aspects) complex algorithms for optimizing day-to-day operations, contingency analysis and automated implementation, voltage stability (or sustained gas pressures), transformer coupling and distribution management—plus deliver relevant data (extracted from the raw events) to all interested parties.

C3BC Analysis

That Indra is re-using a proven, purpose built real-time control system to address more generic needs of the energy industry makes sense but is unusual. Most vendors to this space seem to want to re-invent the wheel by building a wholly new ‘system.’ But in doing this, Indra goes further. Its perception that the ‘network architecture is all important’ is different than most others. The company sees this architecture as being the key to future flexibility, as well as to satisfying the multiple different constituencies with an interest in how energy is consumed. Indeed, Indra’s architecture could, in C3B Consulting’s analysis, reshape how computing is deployed when energy is at stake.

This introduces the distinct possibility that the computing to support this may no longer be embedded in individual utilities or distributors or generators but may be openly accessed by and shared amongst them. This would add great efficiency because the ‘cloud’ (for want of a better term) would be scalable as well as share-able by all those interested as a common utility, including customers. For example, such a common utility (perhaps run by third-party services entities like Indra) would provide the sophisticated underlying analysis capabilities, delivered over the Web in an easy to use fashion that customers will need if they are to make sense of the data relevant to them.

This leads C3B Consulting to identify one area Indra has yet to emphasize—customer involvement. Indra clearly understands that the end customer plays a key role in energy saving, but this has yet to become an established focal point for the company—perhaps because the shape of the holistic computing environment to deliver TAMSECE is still evolving. Nevertheless, it is clear that Indra will be one of the significant players as energy management comes to the fore.

9.   Virtualization and the Candy Store Effect
      (Published Pund-IT, July 8th, 2009)

Server virtualization is fashionable and popular—for good reason. Delivered well, it produces big paybacks, fast, for organizations that adopt it. That said, all is not sweetness and light. There is a darker side, which is pernicious. IT, once it possesses virtualization, starts to behave like a kid with an unending supply of sweets (what C3B Consulting refers to as Virtualization’s Candy Store Effect).

If you want to find out if your organization ‘suffers’ from this malady, C3B Consulting can suggest an ‘acid test.’ Using some form of automated discovery (ADAD—see Pund-IT, April 22, 2009 or www.c3bconsulting.eu/page_1245237261117.html) or other relevant tools to find out how many virtual machines (VMs) are in use in your organization. Does this match the number expected or is it far greater? If the number of VMs is as expected, this is a broad indicator that good practices are in place. If the number is significantly higher (or lower), there is cause for immediate investigation.  In one IT professional services company—which really should have known better—there were almost as many VMs as there were employees. This was not because all PCs had been virtualized (employees had their own unique physically configured laptops and desktops);
they were simply more lax procedures and (in some measure) delighted about being able to offer flexibility on demand.

Why should this matter? The answer is simple: To install a new physical server (say) requires the discipline of officially approved company resources – CapExes, expenditure, physical locations, plus all the effort associated with the purchase/installation of the hardware
and software.

That is not true once virtualization has been deployed. C3B Consulting has watched organizations ‘bring up’ and tear down’ VMs at will. “You want a new VM? What size and with what resources? Hang on a moment. There it is done for you.” The sheer ease with which this is enabled by virtualization (i.e. the Candy Store Effect) is the root cause of the problem.

While the ease of commissioning a VM ‘on demand’ is one aspect of the challenge, the more worrying (to management) should be the subsequent operation and inevitable tearing down of VMs. What happens to data, or even processes, in a VM that is torn down? Were any backup or archival schemas created? Can recovery take place? Are compliance implications involved? Could a VM actively be used to dissimulate or worse? Might a renegade employee be running a “virtual” porn (or worse) site in ‘plain view’? What do you do if the virtualization specialist falls under the proverbial bus and his or her skills are no longer available? The big positives about virtualization remain its ease of use and flexibility. But those very same virtues can also pose potential threats—unless positive, managed actions are in place.

C3BC Analysis

It is not enough for organizations to introduce virtualization and then hope that all will, therefore, be well. Management must be proactive and implement both processes, as well as checks on those processes if an organization is to minimize or avoid exposure. The good news is that establishing such controls need not be difficult. The better news is that dedicated tools exist to automate delivery of good practices.

Take one example, VMware’s Lifecycle Manager (there are similar products from IBM and from Microsoft/partners). VMware’s Lifecycle Manager can track (via a Web-based interface) who owns which virtual machines in the virtualized environment. It maintains a formal record of when and how VMs are requested (with necessary justifications), authorized, created, deployed and operated, along with the backup requirements and the decommissioning schedule. It even lists denials (of a VM).

In effect, each request for a new VM is routed for formal approval within the organization. Once approved, the Life Cycle Manager deploys each VM because it possesses (by this time) all the required information about the virtualized resources needed. Lifecycle Manager can even automate the archiving of VM machine images prior to decommissioning; thereby, establishing the record in case this is needed for compliance or other purposes. It would be comforting to think that most organizations that have virtualized systems observe the disciplines encompassed above. Based on what it has seen, C3B Consulting is not convinced and fears that even though tools (like Lifecycle Manager) are purchased, they are
(at best) ‘underused.’

As described above, the quickest way to find out if there might be a problem is simply to discover how many VMs exist. If the number is not what is expected: INVESTIGATE. Furthermore, the ‘Cloud community’ should also take notice. These very same possible weaknesses are all too easy to repeat in cloud computing, whether in-house (private clouds), external (public clouds) or hybrids of the two. Just because a Cloud sounds good, it still rests on virtualization—and possesses all the associated Candy Store effects and implications.

8.  SAP Transactions Go Missing: What to Do?
     (Published Pund-IT, July 8th, 2009)

SAP has at least two parallel reputations. One is as a premier applications software provider to enterprises—encompassing a broad range of modules crossing many industries with diverse capabilities that combine to deliver the back office. The other is for complexity: SAP applications are neither simple to understand nor to work with. The need for highly qualified (and expensive) SAP-specialty consultants is well known—and feared—by many organizations. Among various other anomalies associated with SAP’s applications is the difficulty in finding ‘missing transactions’ that have gone AWOL—they cannot be found but are probably ‘somewhere in the SAP system, if only we knew ‘where.’

Whether it is for a bank, manufacturing company or a government department, when this occurs there is a common sinking feeling when a transaction is reported missing. All know that finding out what happened will be difficult—and expensive. Furthermore, this expense is multi-faceted. There is:

• The immediate cost associated with (say) an order not going through;
• The opportunity cost associated with unhappy customers; and
• The direct cost of paying for skilled assistance to solve the problem.
  

All are unwelcome. Yet it is curious that, with tens of thousands of SAP implementations installed, this is an opportunity area that has been neither addressed by SAP itself (whose tools in this area are rudimentary) nor by a third party (given the potential market size).

The Scale of the Challenge

The scale of the challenge is straightforward. One major international SAP user has a division with no less than 30 SAP instances that need 50+ SAP-skilled people to keep everything operating at a cost of nearly $10M/year. With an average of 225,000 transactions flowing through ‘only’ 9 SAP instances every month, and with most transactions comprising 4-5 messages each, this division’s success depends on the accurate delivery 1M+ messages monthly. For a transaction to complete, the messages that comprise each step of the transaction must be generated, processed correctly and delivered in a prescribed sequence within specified time parameters. Inevitably (especially in multi-instance SAP environments) a small percentage of messages will stop, disappear or become delayed as messages must move back and forth between SAP instances. Each time a message is dropped, hung or goes AWOL, those problems must be investigated,tracked down and resolved. With approximately 100 incidents on an average day, resolution can become overwhelming—and the SAP team is often forced to apply triage. But this leaves many problems not only unresolved but likely to recur at substantial cost to the business.

Relief Arrives

In early July, some measure of relief arrives for this type of problem. A new company—Conpertus (www.conpertus.com) —launches its Periscope for SAP product. Using techniques derived from automated discovery (or ADAD – see below) with deep SAP knowledge, Periscope can ‘divine’ the path of a transaction through one or many SAP instances and build a map for that particular type of transaction. Armed with this, it becomes relatively easy for Periscope to go looking for errant SAP transactions. Once found, repair and recovery is comparatively simple (users can find transactions and messages by IDOC/RFC number or XI/PI GUID, plus, see the details of every step of the transaction message flow in the landscape).

Where Periscope is different from most approaches to this problem is that it works from the bottom-up, rather than from the more traditional top-down methodologies associated with flow monitoring. During this process, Periscope looks at the metadata and the linkages associated with a given, named transaction. Not only does it discover the path through the various SAP modules but it also:

• Builds a picture of what happens to the data in the transaction as it traverses SAP’s myriad rules and identifies the interfaces used.
• Captures the status of all the components that make up the transaction flow.
• Determines the transaction flow performance thresholds needed to satisfy service level agreements.
  

That information is then stored in what Conpertus calls a TxMDB or Transaction Management Database (not dissimilar to the concept adopted in a CMDB). This data can then be aggregated, correlated and analyzed in the search for missing transaction answers. Longer term (for users), this can form the basis for establishing a detailed map and description of how all of an enterprise’s transactions process through SAP.

Somewhat startlingly, Conpertus Periscope is non-intrusive. Nothing has to be installed on a working SAP instance. The application currently runs in .NET (it is written in C#) on a separate server device that monitors what is happening in the SAP environment and then stores the discovered data in its own TxMDB. According to Conpertus, Periscope requires a minimum of configuration – with the name of the SAP transactions as the starting point. It is already in use in a global manufacturing organization - one of SAP’s larger customers—which has reported potential savings of some US$27,000/week or almost $1.5M/year—never mind releasing skilled SAP staff for more productive tasks.

C3BC Analysis

Lord Kelvin, the eminent Victorian era scientist, once remarked that “if you cannot measure (something) you cannot improve it.” Understanding what is happening in modern systems, especially at the application level, is becoming ever harder as systems become more and more sophisticated and complex. To be expert today in enterprise applications is an order of magnitude more difficult, narrower and more expensive for client organizations. SAP is not the exception—this is a norm.

Increasingly organizations need automated tools to find out (‘discover’) what is happening within their systems. Initially these tend to be installed for problem solving. But their larger value lies, first, in building a knowledge base that can be mined and, second, in monitoring what occurs and in looking out for exceptions and creating immediate, location-specific alerts. This pattern is being seen across many industry segments, not just in systems management or SAP—think of utilities and similar occurrences in other applications. Conpertus is different in that it has applied its knowledge of SAP to a specific difficulty. C3B Consulting expects multiple other tools similar to Conpertus Periscope for SAP eventually to emerge because enterprises simply cannot afford the ongoing costs of missing (but perhaps not lost) transactions or events.

7.   Businesses Fail to Take Virtualization Far Enough – Part 2
      (Published Pund-IT, July 1st, 2009)

Last week’s Pund-IT introduced C3B Consulting’s analytical framework, which defines four different degrees of virtualization complexity (from VirtForm1 - basic virtualization; Virt-
Form2 - improved virtualization; VirtForm3 - extended virtualization and VirtForm4 - ITaltering virtualization). The discussion of the first two indicated that while these represent
probably 80%+ of virtualization efforts to date, for most customers, the latter pair represents a potential opportunity to obtain additional benefits from virtualization.

One of the curious aspects about virtualization, once introduced, is how little subsequently changes in the way businesses utilize it. All too many organizations, having completed the
transition to virtualization (perhaps because this is rarely as simple as the way it was sold) seem to lose interest in obtaining additional benefits – and it does not seem to matter whether they start with VirtForm1 or VirtForm2. The result is that VirtForm3 implementations are relatively rare. VirtForm3 is about exploiting the capabilities – irrespective of the underlying virtualization
technology or platform – that virtualization offers. This can be from:

• introducing a formal process continually to review how increased use of an existing virtualization investment can be extended – for example to exploit functionalities added by vendors (or the Open Source community, in the case of KVM and others);
• aggressively standardizing more elements of the software stack (including changing applications) and/or simplifying the operational/systems management environment.
  

C3B Consulting’s experience indicates that, of these two ‘extremes,’ the latter is a particularly rewarding (in financial and efficiency terms). That said, the opposite also seems to apply: companies which fail to fully exploit their virtualization deployments instead (often unwittingly) introduce new inefficiencies.

Too many organizations, once they have introduced virtualization, seem to lose a certain discipline. Perhaps the most obvious example is the readiness of IT to relax basic procedures. C3B Consulting has come across such instances where IT staff act like kids in a candy factory. Given the opportunity opened up by virtualization, IT staff add, and then delete, VMs ‘on demand’ – with little attention paid to formal processes – never mind ensuring that even temporary VMs are suitably backed up or are recoverable or that data is not destroyed (think of the Sarbanes-Oxley implications). This is retrograde behavior and not a constructive form of VirtForm3 (though some seem to believe it is).

While VirtForm3 is, therefore, essentially a progression from VirtForm 1 and 2, VirtForm4 is rarely found – because it should and can be ‘IT-altering.’ Such fundamental change really only occurs when the technological capabilities of virtualization are understood as being but one element when addressing a larger problem. Take software versioning (just one of many different possibilities that might be chosen). Software versioning is an ongoing development and operational nightmare in large IT organizations. Recall how many times some piece of software will not install or run because a required library or sub-routine or application is not available in the right version at the right time. Think how each and every server, never mind desktops or laptops, is subtly different in its installation image.

Virtualization can provide formidable assistance in such situations. One financial institution determined to introduce IT-altering VirtForm4 processes when it decided to place only one copy of every version of every executable it possessed in a common cache accessible to all. This included software libraries, executables, defined OS images, middleware, applications, etc. Because this organization possessed a high-speed network it went even further. When someone requests a system to start (server or PC, using Linux or Windows or other predefined virtualized environments), the starting system - using PXE Boot - looks for and retrieves whatever is required from the cache. Furthermore, because everything loaded from the cache (replicated for performance and operational reliability), previous ‘library not found’ failures disappeared. By removing one serious problem via VirtForm4, both server and user environments were simplified and, by being predetermined, corporate policy has become easier to observe and document. The result is a dramatic reduction in traditional software failures, as well as radical simplification of what had previously been an enterprise scale muddle.

Primary facie, the key to this was virtualization, for this is the one fundamental which enables the VirtForm4 approach to work. But as important were three management disciplines:

1. 1. To identify, document and install all known ‘executables’ (including libraries);
   
2. 2. To institute a rigorous set of naming conventions that ensured every element was uniquely named; and
3. 3. To introduce tight, formal, controls as to what can be added to or deleted from the cache (thus ensuring absolute integrity and completeness).
   

Analysis

Discipline matched to invention is what enables virtualization to become IT-altering. The above-mentioned financial institution went even further in its IT-altering – with the result that it moved to being what IT should be about - operating lower cost, simpler and less error-prone IT for customers. Logically, such a result should not be surprising to virtualization customers or vendors.
The surprise is, however, that virtualization is under-exploited by so many when so much more can be achieved.

Why is this not happening? Perhaps it is because IT customers are not adventurous enough and are content to do just what was intended. Perhaps it is because vendors, once they have the virtualization licenses (which produce ongoing revenue streams) are comfortable to move onto the next opportunity. Perhaps it is because imagination is not always an IT strength.

Whatever the explanation, what is clear to C3B Consulting is that significant opportunities and cost savings are being ignored – not a good omen in financially stretched times. Or oganizations and even vendors, by understanding what and how each of C3B Consulting’s ‘VirtForms’ work, can access additional benefits that might not otherwise occur. Plus they can avoid unintended operational and other expenses, as well as risks.

6.   Businesses Fail to Take Virtualization Far Enough – Part 1  
      (Published Pund-IT, June 24, 2009)

The concept of the virtual machine, originally invented by Burroughs, was infinitely refined by IBM on its mainframes with its VM operating system and subsequently with its more recent LPAR capabilities. However, when most people hear ‘virtualization’ today, they tend to think of server virtualization solutions from VMware, from Citrix (Zen), from Microsoft (Hyper V) and KVM (Open Source, as adopted by Red Hat and others). Usually this is in the context of x86-based server or IT infrastructure consolidation. However, this narrow view substantially undervalues what modern virtualization technologies can bring to IT (and to businesses).

In C3B Consulting’s analysis, there are four categories that define virtualization strategies and scenarios. For simplicity these can be called:

• VirtForm1 - basic virtualization
  
• VirtForm2 - improved virtualization
  
• VirtForm3 - extended virtualization
  
• VirtForm4 - IT-altering virtualization.
  

Most organizations do not progress much past either of the first two. But the real meat of virtualization can be found in the latter pair, if discipline and an open mind are applied in combination. Let us consider these approaches in some detail: VirtForm1 exploits the simple fact that commodity X86-based server hardware is inexpensive, as well as simple to buy and install. The justification is that (say) 100 discrete servers can have their operating environments consolidated onto (say) 10 multi-core servers in a data center. Instead of having to support 100 individual servers in up to a 100 locations with often <20% utilization, consolidation introduces fewer servers concentrated in a few locations with consequent utilization rates that are 3-4 times higher. Combined with reduced operating costs (from decreased power and cooling requirements), VirtForm1is a win-win – which is why this approach to virtualization is so popular.

But think about what occurs in these situations: In moving the 100 OS images onto the 10 multi-core virtualized servers, the system images have simply been relocated – without any consideration being given as to whether additional efficiency gains are obtainable. Balancing this, however, is simplicity; everything that already works ‘just’ moves – without altering what ready functions. In other words, while the size of the server infrastructure is drastically reduced, its design remains fundamentally intact.

C3B Consulting’s VirtForm2 goes one step further. It delivers improved virtualization by undertaking more preparation prior to deployment. While the underlying server virtualization principles are the same as in VirtForm1, organizations introducing Virt2 go further by refining the (same) 100 system images - in so far as is possible. Typically the opportunity is taken to standardize on a specific software version for the operating system, middleware, databases, applications and related solutions. The advantage of this preparation is that, once virtualization is achieved, operations become simpler because not only are the 100 servers consolidated to 10 but consistency of development, upgrading, systems management, planning and day-to-day operations deliver additional benefits.

So far as C3B Consulting can identify, some 80%+ of all virtualization efforts fall into the VirtForm1 and VirtForm2 categories, with only relatively few organizations moving onto what VirtForm3 and VirtForm4 can offer. This means that deeper and even more cost effective use of virtualization is largely being missed. For organizations, this represents a huge opportunity. What these might be will be discussed in greater detail in next week’s Pund-IT Review. But are vendors concerned? That is an interesting issue – which we will also address.

4.   GPS + GPRS + Event Processing = A New Magic Application Triangle
      (Published April 29, 2009, Pund-IT)

The Global Positioning System (GPS) has been around for a long time, and is now firmly embedded in both commercial and consumer devices (including the iPhone). The General Packet Radio Service (GPRS)—a packet-oriented, mobile data service available to users of 2G and 3G GSM networks—has been available across the globe for many years. Event processing (EP) has also been around since computing started (think real-time systems) but in recent years it has made huge strides in handling complexity, and many of the big vendors have responded by buying the innovators (Progress bought Apama, IBM bought Aptsoft, Oracle bought BEA, etc.).

GPS, GPRS and EP have largely existed as disconnected technologies but recently C3BC sees a ‘magic application triangle’ emerging. Old challenges are attracting new solutions which combine the position generating capability of GPS with the two-way, almost universal communications coverage of GPRS and EP’s ability to analyze, in near real-time, the implications of tens of thousands of events being generated a second. The implications of this development are profound.

These technologies have been tapped to address a broad range of ‘social problems’— everything from stalking to pedophilia, from drink/driving to probation monitoring, from prisoner location tracking to preventing domestic violence (to list but a few). In the past there have been many attempts to use technologies to address these problems – for example using RF ankle devices for probation observance. But such approaches possessed many downsides, including cost and complexity of operation, as well as often unreasonable restrictions on the individual. Worst of all, these solutions were simply inadequate to the task.<o:p>

This situation is changing for what we believe is the better. In Spain, for example, the Ministerio de Igualdad (Ministry of Equality) has recently put out to tender for a system to address Violencia de Género (gender, or domestic, violence). In many Latin countries, including Spain and South America, there are far too many women being beaten up (or killed) by male partners. In Spain alone, some 2000+ women are currently at high risk (and this does not include their children or close relatives). Thousands more may not be officially registered or are at a lesser perceived risk that could escalate unpredictably. How does the GPS/GPRS/EP magic triangle apply? Consider one potential supplier to the Spanish requirements, a Madrid company called Navento (www.navento.com) The company offers a commercial solution combining GPS/GPRS/EP where:

•       The ‘aggressor’ wears a bracelet that links to a GPS receiver communicating that person’s position by GPRS to a special EP platform.

•       The ‘victim’ either carries a location device (with GPS/GPRS and a panic button) or just a commercial GPS/GPRS smart phone that  communicates her or his position to the EP platform(this can also receive alerts from the EP platform).
  
  
•             A platform running EP software analyzes the relative positions of bracelet pairs (or trios—or more—if children or others are included) and can communicate not only with the victim (to warn about possible danger) but with the aggressor and with the forces of law and order, as well as social security.

Thus, once set up by court order, the positions of the aggressor/victim pair can be actively monitored by software and only generate alerts when predefined thresholds are crossed (such as if the two come to close together or the aggressor appears to be lurking near the victims home or place of work, etc.). Indeed, the combination of the above characteristics enables a form of “socialization” around prevention by providing different enforcement authorities with real-time information and risk assessment via constant yet automated position access.

But this is not all. Along with up-to-the second location information, including pictures and locations positioned on Google Maps, speed can also be taken into account. If victim and aggressor are 5Kms apart and the aggressor is walking at 5Kmh, there is minimal imminent danger; but if the aggressor starts to move at 25Kmh on a bike or 60Kmh on a motorbike, the threat horizon could change significantly and requires more urgent action. It is the EP software that analyzes and automates this threat escalation process in a hugely more sophisticated way than has been possible in the past.

This application of the GPS/GPRS/EP magic application triangle is not limited to social issues.  In one discussion between Rámon Fernández (the CEO of Navento) and C3BC, he described at least another 10 wholly different scenarios where the GPS/GPRS/EP combination enables the delivery of super high-added value, whether measured monetarily or otherwise. What is also clear from C3BC’s ongoing work in this area is that relatively few vendors (spread across distinctly different technology areas like GPS or GPRS or EP) understand that it is not any singular technology but the combination that creates the greatest opportunity. Right now, for the most part GPS vendors do not relate to GPRS and/or EP; the mobile telephone companies have nonsensical GPRS pricing aimed at discouraging or preventing usage, and EP vendors have concentrated on Wall Street, largely ignoring what they might achieve.

C3BC believes that the GPS/GPRS/EP combination will establish a magic application triangle. We will be investigating these efforts in several countries and markets. What also seems likely is that it will be the nimble and imaginative smaller vendors that will drive these new markets, at least until the major vendors wake up. It will be interesting to see who wins the Ministerio de Igualdad contract in Spain and the consequences of that choice. C3BC is watching closely.

© 2010 / 2009 C3B Consulting. All rights reserved