Before we dive into microservices, first a little history lesson on Coolblue’s system architecture and how it has developed itself through the years. This should give you an idea of the landscape we’re dealing with, what challenges were facing and how we believe we can overcome them.
In the beginning
Not long after Coolblue was founded 15 years ago, development began on our primary systems: the back-office application named Vanessa, the CMS named Fleur – which was later renamed to Flirt – and the webshops.
Vanessa is a Windows application written in Delphi and uses an Oracle database. Both the CMS and the webshops are written in PHP. And whilst the former shares its database with Vanessa, the latter uses a MySQL database with a subset of the data housed in the Oracle database. All three applications follow the same 2-tier, monolithic design.
To publish content from the back-office and CMS to the websites, a process called Coolsync synchronises data from the Oracle database to the MySQL database. It also synchronises data entered on the website (such as orders and reviews) back to the Oracle database for processing.
The Coolsync process works pretty well, but is asynchronous. When the need arose for real-time access to data in the back-office database, we introduced our first web services which would expose the needed data (customer login, orders) to the websites. As we wanted to move from Delphi to C#, we took this as an opportunity to introduce .NET into our technology stack. We rewrote Coolsync in PHP to phase out our use of Perl, as we had little knowledge of it left in-house.
With this change, Coolsync was now unidirectional, only synchronising data from Oracle to MySQL. All communication from the websites to the back-office was now being done through web services.
With this architecture in place, for a long time, we were able to develop pretty quickly. With all the code for an application in a single codebase and all data available in the same database, everything you needed was always in one place and development as well as deployment was a fairly straight forward process.
As Coolblue grew, so did these systems. More customers, more products, more traffic, more developers, more changes – more everything. And that’s when our architecture started to become a bottleneck.
On one hand, there’s just the cognitive load of dealing with a large and complex system. Trying to make sense of it all just takes more time. This is especially true for developers new to Coolblue.
Then there’s the simple fact that changes to a big system just take more effort and time. Pulling in new changesets, running tests, integrating your modifications, code reviews, builds; they all take longer. And let’s not even mention hunting bugs or doing performance optimisations.
One final consequence of our architecture is an abundance of duplicate logic. After all, the webshops share a large chunk of the data with the back-office Oracle database. Because the data has to be treated in the same way in both systems (applying the same validation, for example) and they’re written in different languages, the logic has to be re-implemented and kept up to date with changes in the other system. This is of course very error prone and scales terribly.
Time then, to take a different approach. One with small, reusable components with a single purpose: microservices. Each microservice will be responsible for a small domain, like products, orders, customers, shopping carts, etc. They’ll be fully isolated from one another, with separate datastores. Besides allowing us to use the right type of datastore (key-value, relational, document oriented) for each use case, it also prevents coupling the services on a datastore level – as is currently the case with the back-office and webshops.
To be able to act on domain events – such as products being added to the assortment, orders being created, etc. – all services dispatch event notifications through a message bus. Other services can then act on these events, such as an order confirmation service acting on messages sent by the order service.
All microservices have a uniform, RESTful interface which is available to the back-office and our webshops. Any future applications that we might build as well as third parties can also consume this API.
Because all services are small in size and scope, they’re easy to work on and will change less often. Due to their isolation and uniform interface, we’re able to use various technologies under the hood, so we’re less likely to get locked in. We can scale resources up or down per service and deploy them more easily in various environments (bare metal, cloud, local Vagrant boxes).
Now, despite these clear advantages to developing microservices, it takes some solid engineering to keep everything running smooth and scalable. Over the next few months we’ll publish articles on this blog to tell you all about how we’re taking on these challenges. Sign up for the newsletter to stay up to date!