MarkLogic Revs Up Information Applications with New Energy and Leadership

At this year’s user conference, it was clear that change is afoot at MarkLogic, whose technology platform enables users to access information more easily accessible within applications and devices. Last month the board of directors appointed a new CEO, Ken Bado, created the new position of chief marketing officer (CMO) and named a head of global services and alliances, all within three weeks. The Silicon Valley software company has been growing in the last several years but appears not fast enough for its board members. There have been a lot of advancements since my in-depth analysis in 2010 and at last years conference.

I had a chance to meet with MarkLogic’s new executives, who have extensive experience in operating large-scale technology organizations; they said they will reset some priorities to ensure it can meet its goal of being a half-billion-dollar company in five years. In our conversation they emphasized building a better and larger community for developers and launching more viral communications about their technology. I agree that MarkLogic needs to expand and be more consistent in use of its own technology for providing information to the industry; it also needs to make it easier for customers to onboard and try the technology, which provides the ability to assemble and deploy what we call information applications. Moving in this direction, already MarkLogic has started to supply more information and examples to its developer community and announced support for deployments to Amazon EC2, which it didn’t from my opinion communicate well over the last year.

To its user conference the company brought customers in publishing and government who have been using its products to meet a growing demand for access to and searching of information within both the enterprise and the Internet. Numerous customer presentations discussed applications built on MarkLogic Server 4.2, which was announced last fall. Our benchmark research on information applications and underlying platforms for them found that only 11 percent of organizations are very satisfied with their existing efforts in this area. The conference sessions suggest users are making progress using the MarkLogic technology to handle huge volumes of data and large-scale information management deployments; some are integrating MarkLogic Server with Hadoop to take advantage of that open source community. The interfacing to Hadoop is something my colleague has been researching in great detail. In fact MarkLogic is accessing petabytes of content that is indexed and accessible through its search methods to its XML information store. Early information about its next major release indicates it will be able to store more binaries and content. The use of XML and Xquery, along with interfacing to SQL, is adding versatility to the platform’s ability to integrate semistructured information across the enterprise. Our research confirms that SQL and XML are the top two standards that more than 55 percent of organizations must interface to in providing an information platform for assembling applications.

I think MarkLogic has been too secretive about the upcoming major release and not specific enough on its roadmap even to its customers. But its conference keynotes did offer some information. Jason Monberg, VP of product management, discussed three types of usage of its technology, which he called enterprise intelligence, situational awareness and information applications. They cover the bulk of how customers are using or integrating its technology. His discussion outlined a range of what is possible but was short on what capabilities are new or are being improved. It appears that the company is struggling with what to call these customer accomplishments, but the key point is that the applications are information-centric. Ron Avnur, VP of engineering, discussed what will be the three technological pillars of its next major release: a shared-nothing architecture, concurrency and indexing, along with clustering and integration with Hadoop. My conclusion is that this is an information infrastructure that can adapt to existing architectures in the enterprise for the purpose of building information applications.

One of the big challenges for MarkLogic in regard to technology and integration is how to support mobility in smartphones and tablet computers. Many potential customers and partners are adopting such products, and MarkLogic needs to lay out a clear product roadmap across the disparate Apple, Google Android, HP webOS, Microsoft Mobile and RIM platforms. The buzz at the user conference showed that customers and partners are demanding more from MarkLogic in this area, and the new executives seem to understand that they need to respond. Our research in information applications confirms the importance of supporting the gamut of mobile devices for access to information.

For the future, the company has to make its server easier to manage in terms of the information life cycle and storage and able to manage the large-scale distributed demands for data. Our research into information applications found little confidence among business people that their organization’s current skills and resources can meet the demands for information internally and even less confidence in providing it to customers and consumers.  MarkLogic and other technology providers in this area should realize that for potential customers inclined to consider others than IBM, Microsoft, Oracle and SAP, the drivers are to increase workforce productivity, improve operational efficiency and gain a competitive advantage. Making it simpler to assemble and use information applications can support these goals, and thus they should make this a priority.

MarkLogic has enlisted consulting firms to help reach more users, but according to our research not many organizations want to hand off this responsibility; 42 percent want business analysts to work with IT to build new systems, and only 7 percent want to work with consultants, while 36 percent prefer building in IT and 15 percent would purchase packages. MarkLogic will have to be careful not to slow the time to value in getting applications deployed by pushing lengthy consulting engagements but rather deliver them in an iterative approach based on revision.

Despite the challenges, MarkLogic has a large opportunity for growth, as competing larger vendors like IBM, Oracle and SAP are focused on middleware and IT’s agenda, paying less attention to helping lines of business get the information-centric applications they desire. The point of information applications is to get the information you need, both inside and outside of the enterprise, to be better informed; they can provide direct access to the value chain of consumers, customers and suppliers, and that’s what business people want.


Mark Smith – CEO & EVP Research

Datawatch Offers Shorter Path from Data to Information

Turning data into information for taking actions and making decisions has bedeviled businesses throughout the computer age. Many organizations have data in dozens of applications and legacy systems along with many reports in various business intelligence systems. The challenge is to get data from each of the reports and assemble it into contextualized views of information for particular business needs. In our benchmark research on what we call information applications, only 11 percent of organizations said they are satisfied with their existing efforts to do this; more than half of organizations see the current process as too slow and not adaptable to the changes that necessarily occur in assembling actionable information.

Against this background I have been reviewing the efforts of Datawatch, which provides capabilities to gain access to data transparently from reports and other information generated in common formats such as Adobe Acrobat, HTML and XML from existing ERP, CRM and other applications and legacy systems. Its products also can process Microsoft Office documents including Microsoft Excel and spreadsheets, which is the technology most commonly used for business analytics and for business intelligence, according to our benchmark research on those topics. Helping organizations reduce the copy and paste activity and provide consistency of information is essential for improving the quality of information used across the enterprise. Instead of focusing on data integration and movement of data to a centralized database, Datawatch provides the ability to access data that has already been generated such as from transactional systems and applications. Our recent research in information applications found that leveraging reports from BI systems is important to 57 percent of organizations and getting to the data from source ERP, CRM and other applications is important to 71 percent. This ability to harvest existing reports and information that has already been created not only enables companies to utilize existing investments but also reduces the time it takes to get to information that people know already exists.

Datawatch is known for this capability in its Monarch platform and with a customer base of forty thousand users around the world since its original release from more than a decade ago, which generates a consistent view of information from any format of data or report. Datawatch has continued to refine its ability to dynamically integrate data from reports and output from across the enterprise into a common report, dashboard or application. Its current platform and set of tools called Monarch Enterprise Server can customize the output and generate information applications from it. The company recently announced a new release of the server that provides more flexibility for the business user and assembler of information to navigate among sources and interact. New capabilities for administration and storage of the actual reports allow easier access to the source data. The new release also addresses the top two vendor and product considerations for purchasers identified in our benchmark research in information applications: usability and reliability. Datawatch also has been making it easier to apply analytics to the data its tools bring together, which should be attractive to users in the lines of business and vertical industries. If Datawatch adds mobile access from smartphones and tablet computers and perhaps makes its products available through cloud computing, it could find even more new customers.

Datawatch also provides a document management system called Datawatch Business Document Server (BDS) that can help organizations maintain a repository of source data for downstream information needs. Datawatch BDS also provides the ability to provide notification, workflow, email archiving and records management that can help manage the information life cycle from storage and retention to overall governance through policy management. Datawatch also recently released a new version of this technology that adds operating system support for Red Hat Linux and deeper security for data across enterprise deployments; it also expands support for language translation along with report and document redaction.

A public company, Datawatch recently announced that it has appointed a new CEO, Michael Morrison, who is a veteran of many successful companies in the information and analytics domains including IBM, Applix and Cognos. This executive change is important as the company has not grown significantly in recent years and was not often mentioned in the industry and buyer dialogue about information management and business intelligence or the emerging market in business analytics. Morrison already has begun to simplify the company’s marketing and sales processes and is focusing on additional types of buyers. Though Datawatch does not compete directly in terms of functionality with BI technologies, it does compete in the larger buying agenda to improve information accessibility in the enterprise. If the company can gain recognition as a harvester of existing investments in applications and systems by capturing and utilizing data in them, it could increase in size and recognition.

Datawatch’s customers range from the Fortune 500 to the midsize. It is looking to expand by touting a faster, more cost-effective method of increased benefit from existing systems, including business intelligence systems and reporting tools. Now it has to make it easier for enterprises on both the IT and business sides to see how Monarch Enterprise Server can be the platform they need for rapid assembly of information for any line of business’s requirements. Our business analytics benchmark revealed that in the analytics processes people spend 69 percent of their time waiting for, preparing and reviewing data rather than actually analyzing it. We also found that the capability analysts in more than half of organizations need most for business analytics is sourcing data. This suggests that Datawatch could play a valuable role in addressing the accessibility and time pressures for supporting business analytics. It might find a niche in business analytics initiatives by filling the business technology gap in rapidly assembling information applications.


Mark Smith – CEO & EVP Research