The following is a reprint of an article that first appeared in IDG’s TECH(Talk) online community forum.

Chapter 4

You need a smarter database. Yep, I hate to break the news to you but you’re probably working with a database that’s, relatively, dumb as a rock. As if you didn’t have enough to worry about.

What does that mean – a smarter database? As covered in the previous chapter, we need to migrate our business logic out of the middle-tier. Doing so requires a database that is capable of expressing, storing and securing our business logic. But, business logic extends far beyond simple string manipulation, adding numbers and comparing dates.

As mentioned in the conclusion of chapter 3, business logic includes things like IOT integration, hardware device integration (e.g. 3D Printers, CNC, CAD/CAM, Telecom, factory floor automation), AI/ML, metadata analysis and so forth. In these scenarios, low-level code frequently has to integrate with operating system primitives and device drivers, do bit manipulation and other complex functions. Here, the de-facto language of choice is usually C or C++ and the logic is implemented in the middle-tier.

Data-layer languages however do not have these abilities. In the alternative, we rely upon the flexibility and inherent logical extensibility of the middle-tier to accomplish these tasks. Another kink in the works is that ‘complex’ logic frequently operates on unstructured data. So, you can see we are caught in a Catch-22 situation – we need to move our logic and resources out of the middle-tier but there’s still aspects that can not be expressed at the data-layer with present techniques and patterns.

A Smarter Database? Yes, You Need One

How do we make the database smarter? To answer that question I’ll draw upon something that all programmers are familiar with.

Remember when you first started programming and wrote your the traditional ‘hello world’ program that displayed a simple message on the screen. Did you ever stop to think how your ‘print’ statement caused the phrase ‘hello world’ to show up on your screen? You didn’t write the code that actually made that happen – the operating system did that for you. You just made a function call and like magic, the words appeared on your screen. This is an example that shows how your program can take advantage of another program’s intelligence in order to accomplish a task – in this case displaying something on the screen.

Now what’s happening under the covers is your program makes an Inter-Process Communications (IPC) call to the operating system and the O/S, also using IPC techniques, calls the display driver to print your message on the screen. It may seem as though your program is smart enough to draw things on the screen but in reality, it’s just knows how to call somebody else that knows how to do it!

To draw the analogy, your data-layer program written in a data-layer language, running within the database, needs to make something akin to an IPC call to another database process that has the ‘smarts’ to accomplish the task. The other process can be written in C, C++, or any language you like. The called process then merely has to return its results via IPC or another mechanisms.

The secret here is that your program doesn’t have to actually do the work, it just has to know how to trigger another process to perform the desired function.

Do databases have IPC capabilities? Yes, the more advanced databases do – it’s called message queues. By implementing a data-communication mechanism based on message queues, we can create an architecture that allows database processes to communicate. This is essentially the same thing that is taking place within the operating system when one executable process communicates with another.

Practical Examples of a Smarter Databases

In theory, this all sounds very interesting but we need some practical examples of how these techniques can be used in real-world use-cases. Here are some examples. One thing to take note of is how each example leverages complex logic and unstructured data. As you can see, making the database smart enough to understand any data-type and flexible enough to store any data type is critical when taking a data-layer-centric approach.

Artificial Intelligence / Machine Learning

AI/ML is a perfect fit for a smarter database. Examples of AI/ML include face/object recognition, spectral analysis and complex iterative processes. This field is rich with unstructured data-types that are best stored in a database.

In addition, the ML models produced by frameworks such as TensorFlow are valuable corporate assets that express business logic as it pertains to the analysis that takes place. This too needs to be stored in the database.

Finally, the scripts, that are written in higher languages such as Python, can also be stored in the database. You would then use a database-aware runtime environment to execute the scripts. Here, you have the benefit of not only storing the data, but the models and scripts that implement the AI/ML process.

Complex Hardware Device Control and Integration

Interfacing to device drivers and complex apparatus from the database allows organizations to migrate critical infrastructure functionality out of the middle-tier. Here, what we are talking about are things like controlling complex devices such as robots, 3D printers, plasma-cutters, factory-floor automation devices and so forth. Just consider what would happen if a hacker got control of a robotic arm on a factory-floor; total mayhem could easily ensue.

The operational requirements here are much like those for AI/ML systems. For example, the devices are controlled by a data-driven logic layer. The data that the logic considers in order to control the device can be stored in the database. An example is the CAD drawing that tells a CAM or CNC device how to build part.

With a smarter database, all this apparatus can move to the data-layer.

Metadata Harvesting & Analysis

Unstructured data objects such as images, video and audio frequently have embedded metadata contained within them. Once again, as you might suspect, data-layer logic does not know how to extract these valuable metadata items.

Using a data-layer IPC mechanism, one can build a ‘plugin’ that knows how to read the unstructured data and return the metadata items. With our new data-layer oriented approach, the unstructured data is stored and accessed directly from the database.

No Data, No Logic – What’s a Middle-Tier to Do?

Now, you may be wondering, what will become of our middle-tier if we migrate all of our business logic and user resources to the data-layer? Is there still a role for the middle-tier? As you may recall from Chapter-1, the middle-tier becomes an elastic security isolation layer, responsible for handling the transformation between HTTP and SQL. The details and benefits of the new middle-tier in the Modern Mainframe Architecture will be the subject of our next chapter.