Mumps is dead? Long live Mumps?

I've been doing quite a bit more thinking recently about the role, position and future of Mumps, particularly based on issues raised in the many blogs I've been reading. There's a lot of naive and uninformed bashing of Mumps, particularly with respect to its role and relevance in healthcare, and also in terms of its role in the modern world.

So I thought I'd pull together some thoughts and provide a fairly in-depth analysis.

Firstly, and crucially, we need to separate out the two components of Mumps: the database and the language.

The database is actually an interesting beast. It's a schema-less hierarchical database engine: characteristics which are coming back into vogue, as witnessed by the likes of Google's AppEngine, Amazon's simpleDB and Apache couchDB. This style of database is being driven by the demands of the so-called Internet-scale database. See:

http://www.25hoursaday.com/weblog/PermaLink.aspx?guid=576aecc0-607b-4f42-9cf5-fda2b42a74fa

and, for our analysis with respect to the Mumps *database*, see:

http://www.slideshare.net/george.james/mumps-the-internet-scale-database-presentation

With GT.M providing a free open-source implementation of the Mumps database engine (http://www.fis-gtm.com), I believe that there's great merit in the kind of modern developers who are being attracted by this style of database taking a look at just what the Mumps database engine is capable of. For some background on the Mumps database and how you can use it, see our paper at:

http://gradvs1.mgateway.com/download/extreme1.pdf

In the "olden days", a lot of the bad press encountered by Mumps was due to its closed nature, its lack of a schema and the associated inability to apply SQL to search the database. Now let's be clear about two things:

1) this was originally due to restrictions in the Mumps *language* which used to be the only way in which you could interact with a Mumps *database*.

2) these days the Mumps database is much more readily accessible, eg:

- InterSystems have added object/relational projections, allowing the core Mumps database to be queried by their embedded SQL and also (and crucially), allowed their objects to be accessible natively from within the more modern mainstream languages such as C++, .Net and Java. Additionally, their web gateways (the result of pioneering work by our company) allow both the Cache object/relational projections *and* any legacy Mumps database records to be accessible via a browser interface, allowing much of the client-side scripting to be written using Javascript.

- our own company's gateway technology (MGWSI) further allows the Mumps database in both Cache and, perhaps more interestingly, GT.M to be fully accessible and integrated with any modern mainstream scripting language: eg Java, .Net, C++, PHP, and, in our next forthcoming release, Ruby, Python, and Perl.

So, in fact, the reality is that there is no reason why the very interesting features and capabilities of the Mumps database can't now be explored and experienced by a modern generation of programmers using their favourite pet language.

It is absolutely true that the Mumps *language* can appear very cryptic and arcane to an outsider, and unfortunately it's a language in which it very easy to write unmaintanable code - another of the themes that has come up in this debate. With the move away from the old "green screen" days to client/server and, more latterly. web/ajax applications, there's actually very little need to write much in the way of Mumps code any more - most of the scripting is done in other languages. Most of my work, for example, is now in the Javascript domain. So the statement "it’s widely-regarded outside of healthcare IT that MUMPS is relatively unmaintainable compared to modern languages/platforms" is true only if you're talking about applications written entirely using the Mumps *language*. Modern healthcare applications written using Cache or GT.M rely very little on the Mumps language.

Let's move to a second issue:

Something that comes up time and time again in these debates is the suggestion that it's about time that Healthcare abandoned the old fashioned Mumps technology and that something more modern and maintainable was used for applications in healthcare. I wish people who harp on about this would actually examine and ponder the track record of previous attempts to migrate from Mumps to something more modern and mainstream. The fact is that in over 25 years of watching these attempts, I'm not aware of a single one that's been successful, despite the best endeavours of some of the biggest names in IT armed with, in some cases, astronomically large sacks of cash. You need only look at the embarrassingly snail-like rate of progress within the UK's NHS IT Strategy where they're all still waiting for the much vaunted replacements products. They're years behind schedule and there's a trail of some very large companies who have become casualties in this multi-billion pound exercise. eg see:

http://www.computerweekly.com/Articles/2008/01/22/229021/fujitsu-may-quit-nhs-national-programme-for-it.htm

http://www.guardian.co.uk/society/2008/aug/10/nhs.computersystem

Now discussion of this issue normally descends into another futile debate about whether or not there must be something inherently different about the data within healthcare that makes the Mumps database suitable and the RDBMS unsuitable. My opinion is that this is far too naive an argument. I believe there's a lot more to it than this, but I'd summarise the situation by saying that the demands within the healthcare environment have an interesting similarity to those being experienced in the Internet-scale arena. For example:

- healthcare IT projects have a terrible habit of suffering from scope drift, often due to changing statutory requirements for data collection as political strategies related to healthcare come and go during the development of the applications. Suddenly that originally nice and tidy database schema needs a bunch of other stuff added to it. This is certainly an area where the schemaless databases come into their own and one of the features that Amazon promote for simpleDB (see the paragraph titled "Flexible" in http://www.amazon.com/gp/browse.html?node=342335011). Conversely it can play hell with an RDBMS implementation.

- cost-effective scalability: Healthcare systems for an entire metropolitan area have to be extremely large, with high peak loads and the need for high performance by busy healthcare professionals. Whilst it's possible for the mainstream RDBMS's to be scaled up to vast sizes and deliver high performance, it requires a *lot* of hardware, expensive licenses and DBAs. In many cases it's just not a cost-effective solution for the highly budget-conscious healthcare community.

- rapid development: Healthcare systems are far from simple in scope. They include a lot of functionality, and if systems are to be delivered speedily to the healthcare community, that requires extremely rapid development. I have to say, based on experience of working on all kinds of languages and platforms, the mainstream view of what constitutes rapid development is a long way removed from what I've seen possible in Mumps and Cache. I think that forms part of the great frustration of Mumps devotees: they see great claims being made about the latest and greatest new technologies in terms of their abilities for rapid application development, but they seem like a sad joke compared with what they know they've been able to achieve with their old Mumps stuff. Why this should be so is beyond the scope of this discussion, but something worthy of further analysis elsewhere. Anyway the result is that even the fastest mainstream rapid development techniques prove inadequate, so development just ends up taking too long.

So, the net result is that it's clearly not as simple as the many IT consultants, advisors and pundits would have us believe. It's not as simple as just getting rid of that old Mumps stuff and rewriting those applications in something more modern and maintainable. If it was that simple, that old Mumps stuff would have been long gone, but it's still there, and there in droves. In my opinion a lot more people should be analysing in more depth just why such an apparently anomalous situation should exist, and comparison with the Internet-scale database requirements goes some way to explaining it.

The third issue that I've seen discussed relates to the lack of a thriving community ("the weak MUMPS ecosystem"). This is an interesting one. I totally agree that one of the great problems is the dearth of people creating value-added features and libraries of modules and until recently I was of the belief that it was a terminal problem, but now I'm not so sure. I think it is partly a historical/timing issue. The great emergence of massive thriving communities has really been a result of the Internet, and therefore a phenomenon of the last 10 years. That coincides with exactly the time when all but one of the original Mumps implementations was bought out by InterSystems and buried into their proprietary Cache product. I think it's fair to say that InterSystems are a company that likes to tightly control and determine the use of their product, and I think as a result, there's been little incentive for third parties to create a wealth of add-odd products. It's also greatly limited because Cache is not a free, open-source product.

Now the interesting factor is GT.M. It was a somewhat unusual implementation of Mumps and never had a particularly big following. So when InterSystems snapped up the other vendors, it acquired most of the then Mumps developer community as users. most of whom would have found the GT.M implementation just too different for their liking and comfort.

Since then, a couple of things have happened. GT.M has been released as free open-source, around about the very time when the demands of Internet-scale databases are causing people to look beyond the relational model. And of course we have the Internet which can, as we've seen with Ruby, allow an explosive growth in a hitherto little-known technology.

Contrary to the views that I see expressed about the future for Mumps, I therefore believe that conditions are now pretty much right for a new-found interest in it. Not, I suspect, in the Mumps language, but I believe there's a great opportunity for a new community to emerge, discovering the potential of the Mumps database, and adding value, features and libraries to the free, open-source GT.M Mumps database engine to fill in its deficiencies etc. Our company has already provided the gateways that open it out to the mainstream languages, and I would love to see new people discovering the potential of the Mumps database as a persistence layer for their particular language.

Will this happen? Let's wait and see. I would love to see it happen and there's great treasures for new people to discover. The kind of things I can do with ease, speed and simplicity in Mumps would be a real eye-opener to a great many of the new kids out there.

Very interesting read Rob. I

Very interesting read Rob. I completely agree ith your points and have for many years been looking at the need for lightweight DB applications in the emerging market. This is becoming incresingly important with various cross platform devices that applications are expected to run on. One additional and arguably hardest barrier to overcome is the provision of skills and resource availible. We need to address this and ensure confedence that there is in fact a strong community thriving who will support the technology now and in the future.

An interesting paper

An interesting paper here:

http://www.mit.edu/~dna/vldb07hstore.pdf

A very interesting paper on

A very interesting paper on 'Next-Generation Object Database Standardization'
from OMG/ODMS.
http://www.odbms.org/download/033.01%20Card%20Next-Generation%20Object%2...

The future seems to be bright!?

Hierarchical data

How's this for irony! Quote

How's this for irony!

Quote from Jeff Barr, evangelist at Amazon on its Web Services team, at AjaxWorld today, describing Amazon SimpleDB: "a Post-Relational Database".

Now where did I hear that word before?......

Hello Rob: Interesting post.

Hello Rob:

Interesting post. Now being the open minded person that I am, suppose I am seeking to implement a reference implementation for connecting our software through MUMPS. How would one go about doing this?

I have tried to setup GT.M MUMPS, but there seems to be no straightforward documentation.
There is no README or HOWTO, etc.

Is there a wiki out there that shows how to setup an M environment? Its so easy to setup SimpleDB, etc. There is so much information about these technologies. I can't say I've found much information on M to help me set things up. As a newbie to MUMPS, I couldn't find enough info to set up the environment. How about a walk through to get a "hello world" app? (I usually can figure things out, as I'm an accomplished computer scientist experienced with almost all languages, databases, and platforms...even Cobol and Vax!)

Our software platform is accessed through REST over HTTPS. We want to build sample implementations for a variety of languages/platforms including MUMPS. Is this possible?

I may not be a fan (yet) because of the obscurity, but I know it is important to have MUMPS hooks if we want to interface with with Epic, Cerner, etc.

If there is to be a renewed community and interest then there must be plentiful resources and examples online.

Help this "new kid" (not really..I'm in my 30's) find a resource to get started that won't break the bank?

Alan Viars

Regarding access to your

Regarding access to your REST-based system from GT.M, the answer is yes, it can be done, but right now, for GT.M, it's a slightly qualified Yes as you'll see...

My previous response recommended you download the EWD Virtual Appliance. In the EWD suite of routines, you'll find a routine named ^%zewdGTM (you can see the source in /usr/local/gtm/ewd/_zewdGTM.m)

This routine includes a bunch of utility functions and methods (which may be useful to you in their own right), but the one to look at in particular is httpGet. You can use this as a raw HTTP client to make GET requests from within GT.M and to get hold of the response from a REST-based system. For example:

Assuming you have the VM running, get into the GT.M interactive shell:

cd /usr/local/gtm/ewd
$gtm

Now type:

s url="http://www.google.co.uk"
s ok=$$httpGET^%zewdGTM(url,.html)

Assuming it can make an Internet connection from the server hosting the VM, you'll now have the HTML in a GT.M local array named "html". To simply list it, type:

zwrite html

You should see all the HTML from Google's home page.

If you want the entire, raw HTTP response, type:

s ok=$$httpGET^%zewdGTM(url,.html,,,,.rawResponse)
zwrite rawResponse

Now, you'll need to parse the response to make it usable, but you can see how it's possible fto access REST services from within GT.M. Also, I've only implemented the HTTP GET. I haven't had the time (or so far, the need) to implement the HTTP POST, DELETE, PUT etc that are also needed by REST services, but hopefully the source code for ^%zewdGTM will allow you (or any other volunteer) to see how it could be done.

In fact I've already implemented a complete Amazon S3 client which you'll find on our web site (see the Misc tab). This is limited to working on Caché right now, mainly because it makes use of its built-in encryption functions. However it's otherwise all standard Mumps code that could be pretty easily adapted to GT.M. So, I can tell you that full-blown REST is entirely feasible from within GT.M. The S3 Client works beautifully on Caché and it's been adapted by another customer to allow them to build a caché-based REST-based system that uses pretty much the same security/encryption mechanism as Amazon S3 (I helped them reverse-engineer it), and I also helped them build a client test harness that allows them to test their services from within Caché. It could all be done in GT.M too.

Note that the EWD Virtual Appliance includes an SSL proxy called Delegate, so you should be able to access SSL/HTTPS services from within GT.M - just use the relevant https:// URL. What actually happens (as you'll see in the httpGET() source is that SSL requests are routed via a different port (89) which passes them to Delegate to handle. One of the other advantages of the Virtual Appliance - I've done all the tedious, hard work getting all that kind of stuff pre-installed, configured and properly working for you.

if you want the reverse - to make GT.M methods available as REST services, take a look at EWD. It includes a very easy-to-use mechanism to expose GT.M methods as REST services. It's all documented in the EWD web interface (known as ewdMgr) which is part of the web-based portal interface for the VM. You should find a specific tab in the ewdMgr interface on REST.

Oh, one other alternative to the Virtual Appliance - if you have an Asus Eee, there's a download kit on our web site which installs GT.M and EWD on these machines, so for 200 UK pounds or so you'll have a fully operational system :-) Just follow the EWD Download tabs and you'll see it listed as one of the options.

Rob

Alan Your observations are

Alan

Your observations are well made and I sympathise. It's one of my gripes that the initial experience of GT.M is very poor. The installation process is obscure and complex - it had me baffled for quite a long time, despite my long experience in Mumps systems. As you point out, the alternative "new kids on the block" always make sure that you can be up and running in next to no time with minimal effort. One thing that InterSystems have, to their credit, achieved is a very quick and simple installation and getting started process for Caché. If GT.M is to be taken seriously, this is one very important area that needs to be addressed - these days nobody new to a technology will bother with it if it can't be easily installed, configured and used. This self-inflicted obscurity is one clear reason that perpetuates the problem of Mumps remaining one of the IT World's great mysteries.

In the meantime, what I have done to make the experience better is to package up a VMWare Virtual Machine, known as the EWD Virtual Appliance. This includes a pre-built, pre-installed and fully operational GT.M system inside a Ubuntu Linux JEOS VM. You can download it for free from our website (http://www.mgateway.com Click the Enterprise Web Developer tab, then select Download EWD). This VM was originally designed as a pre-packaged system to showcase our EWD technology, but I'm more than happy for people to use it as a quick and easy way to get a working GT.M environment.

Here's a simple GT.M "Hello World" with the Virtual Appliance:

- Download it and VMWare Player
- Install VMWare Player
- Unzip the Virtual Appliance files
- Start VMWare Player and start the Virtual Appliance
- When it completes its boot sequence, log on as ewd (password 1234567)
- Type the following commands into the console
- cd /usr/local/gtm/ewd
- $gtm
- You're now in the GT.M Mumps interactive shell. Type:
- s ^helloWorld="Hello world!"
- w ^helloWorld

You've just created a global, saved it to disc and read it back.

To finish type:

- halt
You're now back into the Linux shell.

Hope this helps!

Rob

Hey Rob...thanks for the

Hey Rob...thanks for the reply. I'm just not checking back on this. I'm going to give it a whirl. Maybe I'll post something when I get things going called, building a RESTful interface in MUMPS, and a step-by-step howto on getting MUMPS running.

cheers,

alan

Very interesting post. I

Very interesting post.

I guess one question. If Mumps database should be thought of in the same way as BigTable or SimpleDB, how does it actually compare with them? For example, is it amenable to being used / sold as a cloud service like those?

I'm also very struck by your point about rapid development. Agree, after working with Cache for a bit over a year, there are ways that it's impressively quick to work with. And it would be very interesting to see some more analysis of that. I'd guess it's because of a) direct db access in the language, and b) the always-on environment. But worth exploring further.

Just announced yesterday by

Just announced yesterday by me at the CAMTA meeting in London: M/DB

M/DB is a "plug-compatible" alternative to Amazon's SimpleDB, using the same REST APIs and security as SimpleDB. Just like SimpleDB, it's a "black box" database that you access only through REST. The only difference in use will be that the HTTP requests will be sent to a different domain name: otherwise it looks and behaves identically to SimpleDB.

At the heart of M/DB is GT.M (although we'll make the components available for anyone wanting to run it on Caché). However nobody will ever really need to know or care since it will be accessed through REST calls only.

M/DB will be entirely amenable to being used and sold as a cloud service. Indeed one of the incarnations that I'm intending to make available is as a pre-built public Amazon EC2 AMI. It will also be possible to run locally - I'll be making a VMWare Virtual Appliance for this purpose.

Watch our website (http://www.mgateway.com) for details very soon.

>If Mumps database should be

>If Mumps database should be thought of in the same way as BigTable or SimpleDB, how does it actually compare with them?

I guess someone needs to find out! Volunteers please?

Bhaskar didn't mention the

Bhaskar didn't mention the great opportunity he's providing next month for us to further debate the issues I've raised, particularly with respect to the opportunities afforded by the free, Open Source version of Mumps called GT.M.

For full details of this meeting, see: http://www.fidelityinfoservices.com/FNFIS/Markets/NonFinancialIndustries...

See also my posting at http://groups.google.com/group/comp.lang.mumps/browse_thread/thread/071e...

Hope to see you all there!

Speaking for GT.M, yes, it

Speaking for GT.M, yes, it is very amenable to being used in cloud. The license is Affero GPL v3 (http://www.gnu.org/licenses/agpl-3.0.txt) so there is no licensing obstacle to its use. Support is available on a commercial basis with service levels, if your application goes commercial. You can deploy GT.M based services under the control of inetd/xinetd and other normal security mechanisms that UNIX/Linux sysadmins are used to. There is also a server for a network protocol, so that you can deploy GT.M as a pure schemaless hierarchical data store.

GT.M also has support for full ACID transactions (where the Consistency includes Serialization). Also, you can also create logical multi-site application configurations that can continue to be available not only in the face of unplanned events, but also in the face of planned events. In an always-on world, it's not the unplanned events that get you - it's the unplanned ones.

Never having used BigTable or SimpleDB, I am unable to compare GT.M to them.

Regards
-- Bhaskar

Excellent points and counter

Excellent points and counter point Rob. I would include another tragic attempt at moving from a Mumps database to another platform. That would be the Department Veterans Affairs CoreFLS, this was an attempt to move Fiscal and Asset Management out of the Mumps arena into Oracle, the result was catostrophic from both a technical and fiscal perspective.

Oracle almost always

Oracle almost always introduces unnecessary complexity and high costs.

Most of the time, customers simply don't need it, they have just been convinced they do.

Same goes for many "Big" commercial web services. I highly recommend reading the new book, "RESTful Web Services" on how to build a real SOA and reduce complexity.

Let's face it, for many business models SOA and Oracle means more complexity more consulting, more integration, and greater revenue for integrators.

Simple is better.

Just my 2 cents.

- Alan Viars

Rob et al Thanks for all the

Rob et al
Thanks for all the comments. When I was first introduced to M in 1995 (SAIC) I thought - man...this is rough. But I was amazed at the amount of instructions I could pack into single line of code (shades of BASIC) and the blistering speed on a fully loaded database was phenomenal. I now have an opportunity to use the GT.M compiler with the VistA code for a demo in China - and hope "Hello World" works (displays) effortlessly in simplified Chinese. Everything here is 4-5x PLUS the size of things in the USA. A typical municipal hospital is 1-2k beds - sees 2-3k outpatients per day, and typically operates at 110% bed capacity (beds in the hallways). Disadvantage (or opportunity) is I'm probably the ONLY M developer (and incredibly rusty) in China. Since VistA is mostly text-based - I see incredible opportunity for outputting XML or similar for front-end interface developers - and China LOVES open source (boo Microsoft & Intersystems).