Rationalizing the Tyranny of Visual Anarchy

It has become evident from the advancements we’ve seen in the business analytics market that the use of visualization is now becoming mainstream. In my analysis of the market last year I wrote about the pathetic state of dashboards, where the assumption in the business intelligence software industry is that placing four to six charts or tables of data in a screen and publishing to business users can create business intelligence. That assumption has yet to be proven and is completely irrational, as presenting analytics in basic charts does little to provide context and a guide for taking actions and making decisions.

Now enter the new age of visual discovery, drawn from advanced data vr_bigdata_big_data_capabilities_not_availablevisualization tools with interactive analytic capabilities that enable users to present large volumes of data in an appealing manner. This technology does not need to just operate against big data technology where the volume and velocity of data also need a better method of interacting with the data and is a top capability not available today in 37 percent of organizations, according to our big data benchmark research. These types of data visualization go beyond the doldrums of overused pie and bar charts, bringing heat maps, scatter plots and other forms into the standard presentation of business analytics. However, such visualizations are not designed for the average business user, since most are not trained in visual interpretation of data. Rather, they are designed for the analyst who can interpret what the visualization means and then take some set of interactive steps to investigate and discover what is behind the presentation. Sometimes called visual discovery, this method is one of four main approaches used to maximize the outcomes of the analytical discovery process, along with data, event and information discovery, as I have previously outlined. Also realize that analytical discovery is just one of many types of business analytics that organizations need to be successful. And while many in the industry are calling this segment visual analytics, the scope is the same in terms of the proper use of visualization.

Despite the skills gap in business and IT in interpretation of visualization and the ability to know what type of visualization should be used when, where and why, adoption of visual discovery software is growing, mostly in business to complement business intelligence software efforts in IT. Technology providers have seen this growth in recent years, and now most business analytics vendors have expanded upon existing products, created new ones or acquired technology to enter the rapidly growing market. This frenzy has just about every business intelligence software provider touting existing or new visualization like it is the recipe for living forever.

Let’s get a little reality check that visual discovery is an important step in the business analytics market, as it provides more intelligent representation of data than just simple bar and pie charts. However, we have to also realize that just providing sophisticated visualization to business users is not going to magically help them interpret the data and make a decision or take an action. Interpretation of visualization is as much art as it is science unless an analyst makes notations of the key points the business user should focus on. This concept of annotation is lacking in most analytics tools, which is why analysts continue to copy and paste visualization into Microsoft PowerPoint to highlight bullet points of interest for the business users and then publish to business users that in many cases is converted to Adobe Acrobat format. I pointed this out in a previous piece titled “Why Business Intelligence is Failing Business,” noting that technology vendors should place more intelligence in their analytics software to generate observations and places to focus on using expert systems, personalization and maybe even methods of artificial intelligence.

Presenting hundreds of points in a scatter plot or 25 shades of green and red in a heat map is not an intuitive approach for business users trying to determine where they should start and where they should stop. In fact, sometimes performing visual discovery on data points that look obvious for interacting might not be the best place to focus as the data could be blinding areas of opportunity or concern that have been aggregated or masked out by the data and visualization. Visualization of what should be interacted with and discovered needs to be more intelligently presented. For example, the visual discovery tool might provide a link to a document, report or even the actual events that impacted the metric being represented. Most important, electronic and collaborative discussion forums where analysts and business users can discuss and address issues and opportunities are needed to guide the business in the discovery-to-actions process. The current approach of interacting via telephone or email is insufficient, as the complete context of issues is often lost using these methods.

Until visual discovery can graduate to being more than visuals or tables of data and relate to how business actually works and meet the real competencies of the larger scope of users, we will see over-adoption of visual discovery. Until organizations realize that this type of tool is good for only a specific type of individual, one with a high levelvr_bti_br_technology_innovation_priorities of analytic and data competency, we will see growing frustration among users, as has occurred with dashboards. What we need is to have more visual presentation methods will expand to formats that are more digestible by business users, including geographic maps, paths of activity traffic, workflow of business processes and even presenting the key facts inside of a business centric infographic, all of which today are available and found in separate products. As I pointed out in my analysis of information optimization, the point of visualization is to provide information that is optimized for effective utilization across any role and competency in the organization. I do believe that visual discovery is critical to larger technology innovation within the business analytics market: According to our business technology innovation benchmark research, it is the top-ranked technology innovation priority in 39% of organizations. I believe we will continue to see advancements in the application of visualization and even where it can be more effectively reviewed on tablets and integrated with other technologies that are used frequently.

For business users and analysts, it is essential to assess visual discovery tools based not just on the role of the users, but also their competency in analytics to ensure a full return on investment. Take heed of the best practices in discovery analytics that my colleague Tony Consentino discussed recently and do advanced visualization in your organization if it can be used properly. If you are passionate about visualization and presenting the real meaning of information, then a read through Edward Tufte books might help provide a larger perspective, as his work looks at the history and importance of presenting information properly through visualization is well recognized. Rationalizing the potential tyranny of visual anarchy is critical to the success of your business analytics and big data investments.


Mark Smith

CEO & Chief Research Officer

Datawatch Acquires Panopticon for Big Data Discovery and Visualization across Business Processes

Business analytics can help organizations use data to find insightsVR_leadershipwinner that lead to new opportunities and address issues unrecognized before. One player in this market is Datawatch, known for its tools for information optimization and harvesting value from big data including content and documents. I assessed the company earlier this year, and recently our firm recognized its customers’ achievements with 2013 Ventana Research Leadership Awards for Information Optimization with Phelps County Regional Medical Center and Governance, Risk and Compliance (GRC) with The Fauquier Bank.

Datawatch has made news with its acquisition of Panopticon, a Swedish provider of business visualization and analytics for not only data but events and messages as well. Panopticon’s innovations in analytical discovery are not widely recognized yet but look forward to them getting recognition moving forward. Of the four types of analytical discovery I have described (data, visual, event and information), Panopticon addresses the first three and Datawatch the last though they already have experience with data. The combined company could simplify the portfolio of tools business people and analysts need to produce analytics and insights. Their capabilities enable users to go far beyond the reporting and dashboard approach of conventional business intelligence tools.

Panopticon has been steadily advancing its technology, developing interactive visual discovery of data in real time from many sources, including complex event processing, message queues and polling. Its tools also take data from relational databases and big data technologies like SAP HANA and blend it with proprietary sources to meet users’ specific strategic and operational needs. They use in-memory computing and analytics to process data and events on the fly, creating specialized time windows in which to analyze and visualize information. Panopticon Designer provides access to various sources of data that can be brought into its environment vr_grc_value_of_a_better_approach_to_grcusing prebuilt connectors. Panopticon’s technology operates on the Web and offers a mobile capability that connects to a server-based environment. It also can be embedded in other environments, which its software partners CallidusCloud, Deltek and NICE Systems do to value add to their applications. I am impressed that Panopticon also is on the cutting edge of integrating to sources such as SAP HANA. It goes beyond what SAP and others offer today, offering full integration of SAP in-memory computing called SAP HANA and can simultaneously integrate real-time events from SAP Sybase ESP and others. SAP itself highlighted Panopticon and these capabilities at its press conference in 2012. However, since SAP began working on its own technology for this, SAP Lumira, it has been less eager to welcome partners in business analytics, as I have pointed out.

Panopticon had made progress in customer deployments in North America, and that should increase as part of Datawatch and help expand it further globally. Panopticon’s approach has appealed to financial services companies and specific risk management needs. Our research into governance, risk and compliance finds large potential in new technology to identify and manage risks faster, which 79 percent of organizations are seeking and which Panopticon can supply. Advanced analytics that handle more information can more easily address risk and fraud issues than do older tools for reporting or monitoring.

Panopticon’s support of technology events that are flowing acrossvr_oi_goals_of_using_operational_intelligence a network can also addresses a growing demand for what we call operational intelligence. Our research finds that 63 percent of organizations consider visualizing events important, and as the chart shows, their primary goals for operational intelligence, both named by 59 percent of organizations, are to manage performance better and detect fraud or security problems. As well, our latest research into the technology innovations of big data and business analytics finds that visual discovery is a much desired capability not available in most organizations today; beyond that businesses need access to all types of data, events and information to provide insights for analysts or management. Panopticon delivers more than data visualization which is why this company and products are significant opportunity for Datawatch to shape and grow the full potential of this acquisition.

Datawatch is addressing my colleague’s view on the four pillars of big data analytics which is part of our research and helping organizations gain business value from big data. For Datawatch this step moves them to help organizations realize the full value of information optimization including from big data investments as I have pointed out that are essential for almost every organization. By acquiring Panopticon and integrating to Datawatch will introduce a new breed of technology to handle the broad spectrum of information for discovery (including the use of visualization) and to generate insights that will bring value to big data investments and business processes.


Mark Smith

CEO & Chief Research Officer