Some Notes on the Development of the New Version of Omniviz

Last year Instem Scientific became actively involved in planning and developing the new version of OmniViz – our advanced data analytics platform. In particular we decided to address long standing concerns of compatibility with latest versions of Microsoft Windows such as Windows 7 and Windows 8  (64-bit versions), compatibility with Linux and Mac/OSX, as well integration with the latest versions of Microsoft Office 2007 and 2010. This is the behind-the-scenes story….

When Instem acquired BioWisdom

When Instem acquired BioWisdom at the start of 2011 there was a period of review, during which we decided which products would undergo development. One of the products needing some care and attention was OmniViz, Instem Scientific’s tool for exploratory analysis, data projection, text analytics, and more.

OmniViz had seen over a decade of development, which used a ‘bespoke build system’, a term to strike fear into the heart of many a software developer. The build system was a mixture of Makefiles, shell scripts and native applications that were required to run on specifically configured machines with Cygwin and a ClearCase client installed. Then on top of this, there were also additional dependencies on specific network servers. All of this build system needed some attention.

In comparison the source code had had a lucky break, as there had been an effort to move the code and the build system into subversion and so preparatory steps had been taken to extract the code from the legacy build system. This comprised a mixture of scripts, java code and pre-compiled binaries and did mean that the source code had survived.

A third key component of OmniViz is the installer. OmniViz users will know that it does not install as a single executable but is dependent on a whole supporting cast of configuration files and externally managed applications, one of which being the bundled java virtual machine itself. However the fate of the installer was not as fortunate as that of the code base, and so to simplify matters we decided to graft in a new installer.

Find a build system

An initial survey of the landscape showed quite quickly that a new build process needed to be found. Having successfully used the Maven build system on many projects, I decided to give it a try.

So into a clean eclipse workspace I poured the OmniViz source code and all. Then after picking out and smoothing over jar and file dependencies, I added a magic Maven pom and stirred over a low heat until I had some artefacts that would compile.

It turned out that Maven was a great choice for this project. As ever the winning feature of Maven was the dependency management, but this time the code base also had some special compilation requirements which were taken care of, and the modularisation helped to break down a mammoth compilation stage.

Find an install tool

So OmniViz was now compiled, but it still required an installer. OmniViz is quite demanding about its supporting files, and changing the code on this scale was not an option so the demands on the installer were high. After a some experimentation I settled on the open source izpack.

Izpack is a fully java base installed with support for large amounts of customisation, the ability to run ant scripts and lots of plug-in modules. Then to round this off there is a Maven plug-in to support izpack to the whole installer building process could be integrated into the build system without too much blood being spilt.

A welcome side effect of adopting a new installation tool was that the installer could take care of ensuring a valid version of the JRE was installed, while not requiring that the application come bundled with a fixed JRE version.

The first deployable

So now we finally had a deployable application, with optional components. Also using izpack a collection of separate importers were built that could be installed to expand the import features of the application. Also now that OmniViz was using the system JRE the code was updated to support both JRE 6 and 7, bringing it up to date with the current java releases and taking advantage of better operating system integration.

At this point we were also able to run OmniViz properly on Windows 7 and even Windows 8 beta.

32 to 64

The next challenge was to break the 32bit barrier. With computing power ever growing, the restrictions of 32bit applications are getting tight. Users now demand more processing power as data volumes increase and more complex processing is required. But OmniViz had some limitations, as it was dependent on some native 32bit libraries, for features such as OpenGL rendering. So the goal was set to remove the 32bit dependencies, which involved moving to full java2d support, avoiding the use of native libraries and replacing native library functionality with pure java code.

Once these ties were cut we could fire up OmniViz with a 64bit JRE. This was a big step forward as we could start to access data sets that had previously been too big for OmniViz to handle, and our application scientists were starting to demand more powerful 64bit machines with lots of memory to crunch data.

Linux and Mac

As I am not a Windows user through choice and since the shackles of native executables had been cut, the challenge of alternative operating systems lay ahead. So due to the power of java and the clever design of izpack, after a short time OmniViz was up an running on 64bit Linux with a good amount of memory to draw upon when required. After some further installer configuration we also had a version running in-house on the Mac.  While we’re not currently supporting OmniViz on the Linux and Mac, if there’s sufficient demand who knows, as the groundwork has been laid…

Let the trials begin

So, to cut a long story short, back in October 2012 we were able to let the new puppy out on a leash for beta testing. While OmniViz is used daily in-house it has many features that need testing and each user seems to want to use it for different things, and has their own preference for tools that they use regularly. Once beta-testing was complete and the final changes made, OmniViz 6.1 was released in November 2012.  As I have started to learn since dusting off the code base, there is a large user base out there who discovered OmniViz long before I did, and love the functionality and capabilities OmniViz offers. But there’s still room for at least one more. So if you would like to find out more about the tamed beast that is OmniViz get in touch.