Reply

Has the Objects/Relational Impedance Mismatch been Solved?

At 3:19 PM on Nov 20, 2005, David Chamberlin wrote:

At last with JDO and EJB, with Hibernate and TopLink, with Kodo and OJB, there is no longer any need to write SQL, or even use a relational database. So where's the catch?

Relational databases originally won out over hierarchical or file-based databases not because they were more performant, more efficient or had better support, but because of the qualities of the relational model, which has since the early days geen accessed through the (arguably broken) Structured Query Language (SQL). SQL is a victim of its own history and success. It is overly verbose. It lacks symmetry. It struggles to be effective without resorting to extensions to the agreed standard. It's in a different league compared to Java and C#, for example. These object oriented languages are so much more clean and exact in the way they represent and manipulate data as well as being infinitely more flexible and frankly, let's face it, more beautiful.

The fact is that this beauty hides a mess of ugliness beneath, whereas the ugliness of SQL hides a quiet beauty that is not even noticed by most casual writers of select statements.

So what is object-oriented programming's dark secret? Simply put, the strength of OO derives from the fact that it generates structures in programming languages that are akin to the way human minds happen to process information. We love to classify things, put boxes around them, create abstractions where a complex concept can be treated in a way that doesn't need to include the details. We use these structures very effectively: we can classify all of the organisms on earth into Kingdoms, Families and Species: one huge inheritance hierarchy of evolution. We can simultaneously understand the nature of an aeroplane as being like a bus (except that it flies) or like a bird (except that it's man-made, and bigger), both being different abstract ways of understanding the same object. And object oriented programming helps us all the way: for every abstraction we can think of, there is an interface. For every 'thing' that exists in our simulated world, there is an object class. This is 'common sense injection' and we can use it to structure our very code, so that things that don't make sense can't even be expressed in our language.

There is a problem however, with common sense injection, and it is this. This approach to modelling systems is only one possible way of doing things, which doesn't work very well in many cases. Einstein did not develop special relativity by thinking about space an time using his common sense, rather he took a simple observation (that light needs no frame of reference to limit its speed) and then used mathematics to derive the unavoidable, but distinctly non-common sense conclusions. Neither did Keynes come up with his ideas about economics by thinking in accepted common sense terms. Sometimes our common sense does not permit us to reach a deeper understand of an underlying model. The reason for us organizing our thoughts in the way we do has troubled philosophers since Plato , but the fact is, that despite two and a half centuries of effort, nobody has come up with a convincing explanation. There is nothing intrinsically correct about organizing the world in this way, and modelling programs as such is only useful because that's how most people's minds work. Formal, correct and complete as it may seem, object oriented programming has its basis in arbitrary human behaviour.

SQL is different. Behind SQL, and somewhat obscured by its clunky syntax, sits a body of number theory called relational calculus which, like the rules that tell us what to do when adding, subtracting or multiplying two numbers, provide us with a set of operations we can apply to 'relations' which equate roughly to tables in a relational database. SQL as a language evolved as a de facto standard in the era of COBOL and has been slow to modernize its syntax and clean up its idiosyncrasies because, to be honest, it isn't in the commercial interests of any of the database vendors to do it. There has been agonizingly slow progress with the ANSI process and now at least some of the features introduced by vendors in the early 90's are starting to become available through this standard - at the same time as the vendors are further extending their database's capabilities. Of course there have long been calls for a proper symbolic relational calculus as an agreed standard to access relational databases, but that's another story.

The bottom line is that the relational model is a formal mathematical representation for data. Mathematical abstractions are useful because they help us to solve problems that common sense finds difficult to access.

Alice starts north from Abbotsville going at 50mph and Bob starts from Boxford at 40mph at the same moment. Boxford is 20miles directly north of Abbotsville. Ignoring relativistic effects, and assuming both travel in a straight line north, how far from Boxford are they when Alice passes Bob.

Go on, solve it, you know you want to. The fact is, most people solve this kind of problem by writing down an equation and solving the equation. While solving the mathematics, they are not thinking about what the variable in the equation represent, they are going through a logical process to reach an answer that is difficult to access using a common sense approach (and even harder if you do take relativity into account).

Relational databases are a formal mathematical representation that is useful when solving data problems. It so happens that SQL is the 'standard' for accessing the relational algebra needed to manipulate this relational data. This highlights the true impedance mismatch between object oriented programming and relational databases. The former is an arbitrary, but common sense model of the world wheras the latter is a formal mathematical abstraction of the data. The problem is not helped by the idiosyncrasies and dialects of SQL, but this fact actually hides the true nature of what is required to reconcile these two views.

So much yada yada, so where are the examples? The problem with coming up with approaches to dealing with complex systems, is that the simple examples don't really shed much light on the problem. However, let's give it a go. Suppose we are building a stock tracking system. We have stock items that have a stock index, a colour, a shape and a price. We have clients in the database with a name, address and telephone, and purchase records that record which clients bought which stock items for which price.

The system is pure common sense, there are 'person' things representing clients, 'stockitem' things representing lines of stock, and 'purchase' items representing a transaction. You have implemented this system in an object oriented language, using an object oriented, XML, or other non-relational database. Everything has been working well for years, and you little company has gone from strength to strength, reaping in the profits. So well, in fact, that you decide to buy out another company and leverage your loyal client base. You now have a new requirement for your little database for doing cross selling:

• Give me a list of all the people who bought blue triangular items in Boxford.
• Give me a list of people called Smith who paid more than $80 in a single purchase.

What has happened here is that the common sense injection in your original application has let you down. This application had a single purpose: keeping stock records and recording client addresses and payments. Now you want to use it for data mining: a purpose for which it wasn't designed, and you end up having to do a major rewrite to have it answer these simple questions. Had the application been created with a relational database, these questions would be very simple to answer, though it might be difficult, particularly with a real-world problem, to map the answers into your object model using an Object/Relational mapping layer that does not give you full access to the underlying SQL.

So moving back to the original question: has the impedance problem been solved? The risk is that many solutions that map object data to relational databases and vice versa sacrifice the power of the relational model in doing so. By eliminating SQL, or offering a solution that is agnostic as to whether it is using a relational, object or XML database under the covers, you lose the flexibility to change your mind, in the future, about what you want your database to do. And that's got to be a bad thing.