Bridging the divide between Java and the database with Liquibase
The refactoring movement has largely passed unnoticed in a critical part of the Java domain; the database. Whereas the last couple of years have brought great improvements in Java refactoring, databases still subscribe to the same top-down design model. Worse, the denizens of the two worlds hardly cooperate.
The consequence? Java developers maintain the Java application, whereas the database is maintained by Database Administrators. Any unit of work that crosses the boundary has to be picked up by two different roles. Sometimes, the roles combine in one person. More often than not, this is not the case, introducing communication overhead to fulfill a single unit of work. Communication is further aggravated by a cultural difference between the two roles, each generally finding their own silo superior to the other. Having two distinct, logically unrelated, though functionally related, ultimate sources (ie, the domain model and the database design) does not make things easier.
The situation would not be too bad if there were to be a clear separation of concerns between Java application and database, such as a service interface. Alas, this is not the case. Since the entanglement looks likely to continue for some time, even with the advent of sound alternatives like NoSQL, any cross-border operation is bound to be expensive and error-prone. We better have a decent way to deal with this mess.
Fortunately, we do. Thanks to Tim Berglund, the database refactoring tool Liquibase is now available as a Groovy plugin, even with a Domain Specific Language (DSL), allowing Groovy language constructs to be executed during the migration script run. Without Tim it would not have been possible to do what we are doing now. So a big thank you Tim!
From a process point-of-view, what we want to do is create a single well defined unit-of-work, not too small and not too big. Our philosophy is that a single developer should be able to handle most of the units-of-work. Only specialized work then remains for specialists in the literal sense. Think of very complex Hibernate queries, database performance issues, advanced security requirements etc.
When a change in the Java model takes place, the developer also takes care of the change in the database. This is done by adding a changeset which contains the corresponding changes of the Java model for the database. Liquibase has a nifty way of maintaining the changes by keeping track of all previous changesets. So, effectively, when the deployment of the new Java application takes place, the database changes are executed during the same deployment window.
Now I hear a number of you DBA’s shouting aloud: “Java developers in my database? Never!”. I would love to contradict you, but I won’t. Instead, I hope to reassure you that you will stay in charge of the process. In our company, before the release is deployed, professional DBA’s get the chance to review the changesets and look for inefficiencies and errors. Most of the time, the fixes are just that—fixes. Rarely do we see huge changes that require going back to the drawing board after such a review.
That said, you need developers who have a feel for the database. Within 42, Java developers specialize either in the frontend or the backend. The latter group must know about the database. Hiding behind the ORM is not an option. This means that our backend developers feel at ease with the database and know how to effect non-complex changes.
The advantages of this database refactoring approach are as follows:
- Self-reliant developers; by being able to pick up most of the units-of-work independently, the developer becomes self-reliant
- Synchronized model and database changes; when we deploy a release, the actual deployment and the changes to the schema are run during the same deployment window
- Better In-memory database value proposition; the usual way of starting up the in-memory database is by running the HBM2DDL “create” which dreams up the database schema on the basis of the ORM mapping. This is not even close to the target database schema. With Liquibase and some adjustments, we now use this to create the HSQLDB in-memory database as well. HBM2DDL is still used to validate the schema against the mapping
- Earlier feedback; when model and database do not integrate, the indicator will be raised earlier during the feedback cycle than usual. Hence, the defect will be cheaper to fix
- Technology enabler; the way we use Liquibase allows us to bridge more gaps between Java and database, among them the way we are dealing with test data, views and constraint information. More on this in later blog posts.
To summarize, Liquibase allowed us to make giant leaps in bridging the divide between Java application and database. If you are not familiar with this technology and what it accomplishes, definitely give it a glance to see if it is worthwhile for your endeavors. We compared a number of database refactoring frameworks and Liquibase came out top dog.
In another 42 blog post we describe how you can bootstrap your own project with Liquibase. Following this guide will show you how Liquibase can be integrated into your Spring project.
Robert Bor – has written 24 posts on this site.
Rob is Chief Technology Officer for 42 B.V. His main focus points are to improve the quality of the products being developed by the company and export the ideas, experience, processes and tools related to quality control to the outside world. Rob is very outspoken on the need for bridging the gaps between the traditional frontend, mid-tier and backend. He insists on searching for better ways of getting things done and is fanatical about operational stability and performance of applications. Growing an organization that thrives on excellence in craftmanship and a deep understanding of all the aspects involved is also very important to him.
Before working at 42 B.V., Rob stood at the cradle of the music distribution platform Fuga, where quality of service is held to a high standard. Among other metrics it contained a code coverage of well over 99%. Before that, Rob worked for ABN AMRO in a Quality Control department to support Java projects with achieving high quality under architecture. Rob is an outside the box thinker with a good feel for the merits of the orthodox. He is always looking for new ways to combine what is already proven with innovative technology and ideas.
Rob still cannot resist the urge of working with code. Every now and then he is pulled back inside, where he feels most at home. Whenever there is a production issue, Rob is likely to be involved in the process of analyzing and fixing the problem.