Author of the article: Tautvydas Versockas, Chapter Lead at Danske Bank
Every software system starts as the most innocent greenfield project. After all those legacy systems that you have worked on, you believe that this time it’s going to be different. This time it’s finally going to be better. A couple of weeks in and you feel like a new person, you feel proud — it’s your gorgeous baby after all. Suddenly, a few unexpected business requirements later, you realize that it’s no longer your beautiful child. It’s the Kraken that has been released! By you! What have you done? Well… Off you go… To the next project!
If you have ever developed a software system in an inherently complex business domain, I bet you know how complicated it could get. As a software engineer, I had an epiphany about dealing with complex business domains when I heard about DDD (domain-driven design) for the first time. A few years later, working in the financial field, I met some brilliant people using a DDD pattern called Event Sourcing. It took me quite a few days to get a grasp of it. It was so different yet so elegant. My only regret now is that I haven’t heard about Event Sourcing before.
I spent quite some time reading books, articles, having discussions with experts in this field, and implementing the pattern myself. As a result, I wrote this guide, which, I believe, is one of the quickest ways to start with Event Sourcing.
What is Event Sourcing?
We’re used to the “classic” way of storing all application state changes as the latest state of our business model aggregates such as an Account or an Order. It’s not unusual to find those kinds of systems to be CRUD-based and built around relational databases. Unlike the “classic” approach, when using the Event Sourcing pattern, we persist an application state changes as a sequence of state-changing events — immutable business-related information about things that happened in the past. In a nutshell, Event Sourcing is just a different way to store the data. Let’s take a hypothetical payments related application, for example. Whenever we make a new payment, a new PaymentMade event is appended to the Account aggregate events list and persisted within a single atomic operation. Later, the Account balance can be reconstructed by replaying persisted PaymentMade events.
At the same time, the ability to check what happened to a brand new smartphone that was supposed to arrive on your doorstep one week ago is something that most people would certainly appreciate. Moreover, we can use events to reconstruct past aggregate states to solve complicated issues or roll back unwanted changes.
If you’re like me before I knew anything about Event Sourcing, by now, you should have more questions than answers.
Why would one use Event Sourcing?
Event Sourcing is by no means the silver bullet for software systems. Yet it can be an incredible tool if you know all the whens and the hows.
Event Sourcing provides a complete audit trail of business transactions. You can’t delete a transaction or change it once a transaction has happened. You can use those stored transactions to determine the system’s state at any previous point in time by querying the events up to that point in time. It enables you to answer historical questions from the business about the system.
Troubleshooting is another fantastic Event Sourcing advantage. You can troubleshoot problems in a production system by copying the production event store and replaying it in a test environment. By troubleshooting, you might discover source code errors. Rather than performing risky manual data adjustments, you can fix the code and replay the event stream letting the system recalculate values correctly based on the new version of the code.
Event Sourcing often leads to better performance and scalability than complex relational storage models since you only use append-only operations for small immutable event objects. Having append-only operations allows you to reap the benefits of databases such as Cassandra. Moreover, Event Sourcing is usually combined with CQRS, which could be a crazy improvement on its own compared to the “classic” relational storage models.
Event Sourcing is an excellent fit in distributed systems (i.e., microservices architecture). You can publish events to notify other subsystems of changes to the application’s state in a considerably decoupled manner. Even though, strictly speaking, you can do that without Event Sourcing, having an event-based system makes it a lot easier.
Yes, yes, yes, but where is the code?
I wouldn’t be surprised if, at this point, you weren’t completely sold on the idea. The concept might seem difficult to grasp and hard to evaluate whether all of these benefits are real, or I am just a snake oil salesman trying to sell you my goods. Hopefully, the code will provide you with some answers.
The code examples are written in C# because its syntax is relatively simple and should not be challenging to understand for most developers. However, you could implement the same thing in many different programming languages, even the functional ones (see resources section).
Event Sourcing is not all fun and games
The Event Sourcing pattern has a lot of good parts. However, there are some bad ones as well. If you think you need Event Sourcing — please read on.
Most of the developers are not familiar enough with the Event Sourcing concept. Having to explain Event Sourcing to less experienced colleagues takes much time. Moreover, explaining only Event Sourcing is usually not enough. DDD, CQRS, eventual consistency model often goes hand-in-hand with Event Sourcing. It’s a very steep learning curve!
Most of the domains are not that complex. There, I said it. Event sourcing, on the other hand, is very complex. You have to consider snapshotting algorithms, not being able to change the data in the database in case there’s an urgent production issue, handling event schema changes, and that’s just the tip of the iceberg. Using Event Sourcing for a simple CRUD-based application is like using a chainsaw instead of a butter knife to make a sandwich — they both cut things, but one of them is a bit of an overkill. I will let you decide which one.
Even though the Event Sourcing pattern is great, you have to thoroughly consider whether it’s worth using it for your specific problem.
Frequently asked questions
As it turns out, Event Sourcing isn’t as straightforward as it might seem from first sight. However, I find that some of the questions about Event Sourcing are asked more frequently than others.
Does Event Sourcing hurt the performance?
If done well, Event Sourcing doesn’t hurt the performance. It’s often quite the opposite since Event Sourcing usually comes together with the CQRS pattern, which solves many performance issues. Of course, there are cases where aggregates tend to have long and complex lifetimes (having many events), making the querying part very slow, but that’s solvable with snapshots.
What’s the difference between domain and integrational events?
People often split events into two categories — integration and domain, especially when working with microservices-based architecture. These categories are very similar since both types of events are state change notifications. However, the reasoning behind them is quite different. Integration events are for notifying other services about the state changes inside your service. Therefore, they must be changed as rarely as possible and might contain additional information that domain events don’t. Domain events are for notifying aggregates inside your domain boundaries (and restoring the state in Event Sourcing). Therefore, they can be changed easier and have only a minimal amount of relevant information.
What data stores can I use to implement an event store?
We can implement an event store using SQL Server, MySQL, Cassandra, Event Store, and many other data stores. We should be able to query events in the same order as we produced them for a specified aggregate ID, save events atomically, utilize snapshotting. Other event store capabilities are mostly related to functional and non-functional application requirements. It’s worth mentioning that there are many debates on the internet discussing whether we could use streaming platforms such as Kafka to implement an event store. I’ve seen it work for specific business problems where there wasn’t much data stored, and heavy in-memory caching was used. However, in most cases, you’re better off with a different datastore since you would most likely need to query events for a specified aggregate ID in a reasonable amount of time.
Can I update stored events?
One day our event schemas are inevitably going to change. Therefore, it’s crucial to think about dealing with it ahead of time. It becomes very tempting to change the stored data when the inevitable happens. However, most of the time, we shouldn’t. By doing that, we’re messing with historical data, which eliminates any guarantees of the data being authentic and makes it less accurate. What we should do instead is to version our event schemas. Event schema versioning allows us to adjust the code according to new business requirements while keeping the stored data unchanged. We could hide the mapping between old and new schemas somewhere in the infrastructure layer of our application.
Can I use Event Sourcing without CQRS?
While it’s possible to use Event Sourcing without CQRS, it’s often not very practical. We usually need to query data from multiple aggregates combined. Moreover, it is relatively simple to build custom query models by subscribing to produced events. Event Sourcing and CQRS combination is like a pair of socks — you could go with one sock instead of two, but why on earth would you do that?
Resources
If you’re interested in learning more about Event Sourcing, here are some of the best resources you should start with.
Exploring CQRS and Event Sourcing: A journey into high scalability
I can’t stress enough how good this book is! This free e-book is hands down the single best resource available right now about Event Sourcing. It covers a team’s journey to CQRS and Event Sourcing in a very detailed fashion with plenty of code examples and explanations of every decision. All of the code used in the book is available on GitHub.
Event Storming and Event Sourcing are entirely different things. However, they work exceptionally well together. We model our business domain on a whiteboard in aggregates, commands, and events using the Event Storming method. We can transform the whiteboard model to the source code with relative ease if done right.
Simple CQRS is a sample project created by Greg Young to illustrate CQRS and Event Sourcing patterns. While the code is not quite production-ready, the complexity is extremely low, making it an excellent example for beginners.
CQRS & EventSourcing is a sample project I created to illustrate CQRS and Event Sourcing patterns. The implementation is slightly more complex than Simple CQRS. However, it uses EventSoreDB and MySQL databases and is somewhat closer to what you might see in the actual production environment. I would recommend running the project and playing around with it to get the feel of Event Sourcing.
If you’re not a big fan of OOP, you might enjoy trying the pattern in a functional language such as F#.
Final words
To use or not to use Event Sourcing, that is the question. Event Sourcing pattern indeed brings more additional complexity than the classic “store the current state” approach. On the other hand, it tends to bring many fantastic benefits such as performance gain, the ability to restore the past states of your application, do state rollbacks, and, most importantly, tell exactly what happened if things were to go south. If used in the right place at the right time, Event Sourcing is going to be your best friend that you wish you had met sooner.
Author: Tautvydas Versockas, Chapter Lead at Danske Bank, writer at thehonestcoder.com 
About the Author: I joined Danske Bank as a software engineer 3.5 years ago. During those years, I was building financial trading software and modernizing bank APIs. Now I am leading a chapter at Markets Front Office tribe. Every week there's a new challenge to tackle, a new puzzle to solve - that's why it's fun working at Danske Bank!
Areas of Interest: Back-end technologies, Distributed Systems, Domain Driven Design.
Curious to know more about our Tech @Danske? Visit this page.
