Handling Microservice transactions with Oracle MicroTX

Maintaining data consistency in todays complicated "digital highway", and with regards to this article, across multiple microservices is one of the significant challenges today. Each microservice has its local transactions and databases, which may result in data inconsistencies if one microservice fails, or its transaction is rolled back without the others following suit. This problem becomes even more complicated in distributed and asynchronous environments where communication failures and network latency can occur.

Another challenge is how to perform queries that involve data from several microservices without causing too much overhead or coupling. This means that microservices should expose their data in a way that is easy to consume and aggregate by other services or clients without affecting their autonomy or performance. 

Consistency helps maintain reliable operations and ensures that once a transaction is committed, the effects are permanent and visible to all subsequent transactions. If consistency is not maintained, later transactions might see outdated or incorrect data, leading to incorrect operations and results.

In distributed systems where data is stored across multiple nodes or locations, consistency ensures that a change made in one location is reflected across all others. This synchronisation is vital for the system to function as a coherent whole, rather than a collection of disjointed parts.

Inconsistent data might not only lead to operational problems but also legal issues, especially in industries that have regulatory requirements to ensure that data is handled accurately and consistently. It can also erode users' trust in a system and lead to a loss of reputation and business for the company operating the system.

Consistency helps in avoiding various types of anomalies like lost updates, temporary inconsistencies, and uncommitted data being read. It also facilitates collaboration in systems where multiple users might be working with the same data simultaneously.

Consistency can be achieved by implementing consistency handling mechanisms, such as:

Manual reconciliation 

Manual reconciliation is a process that is often used to ensure data consistency, accuracy, and integrity between different systems or within different parts of a single system. It is typically applied in scenarios where automated reconciliation might not be feasible or where discrepancies have been detected that require human intervention to resolve.

• Inconsistent data view for a period
• Potential financial losses due to loss of business and customer dissatisfaction 
• Resource intensive task, which increases cost of operations 

Develop transaction logic

Developers building transaction management logic in apps 

• Requires developers to have advanced skills
• Takes valuable time away from app developers
• Can be error prone; increases testing complexity
• Increases time and cost to market 

Use of existing Transaction Managers

Which will elaborated further in this article.

In summary, data consistency is fundamental to the correct, reliable, and lawful operation of databases and distributed systems. It's what allows different parts of a system to work together coherently and provides users with accurate and reliable information. Without it, systems can become unreliable, confusing, and prone to errors and misuse. Transaction patterns are essential for ensuring data consistency and reliability in distributed systems, where multiple services interact with each other and with external resources. Oracle provides a comprehensive framework that supports various transaction models, such as SAGA, 2 phase commit and XA, and allows you to choose the best option for your use case. Let's take a closer look at each of these patterns and how Oracle MicroTX can facilitate their management.

ACID

ACID (Atomicity, Consistency, Isolation, Durability) is a set of properties that ensure reliable processing in a database system. These properties are essential for the proper functioning of a database system, particularly when handling transactions - sequences of operations that read or write database items. Here's what each property means:

1. Atomicity: This property ensures that a transaction is treated as a single unit, which either completes entirely or not at all. If a transaction is interrupted (for example, due to a system failure), any changes it made are rolled back, and the database is left in a consistent state as though the transaction had never occurred.

2. Consistency: This property ensures that any transaction will bring the database from one valid state to another while maintaining all defined rules and constraints. It guarantees that the database will not be left in a contradictory or conflicting state after the transaction. If a transaction might violate a constraint, it will be rolled back, and an error will be reported.

3. Isolation: Isolation ensures that concurrent execution of transactions leaves the database in the same state as though the transactions were executed sequentially. In other words, other transactions cannot see the results of a transaction until it has been committed. This prevents temporary inconsistencies and protects ongoing transactions from seeing partial results from other concurrently running transactions.

4. Durability: Once a transaction has been committed, it is permanent, and the changes made by the transaction will persist even in the face of system failures. Durability is typically ensured by storing transaction data in non-volatile storage, and often a transaction log is used to replay changes if a failure occurs after a transaction is committed but before all its changes are physically written to disk.

The ACID properties are a set of principles that work together to provide a strong and dependable framework for processing transactions. By making sure that transactions are atomic, consistent, isolated, and durable, ACID helps prevent data corruption, maintain data integrity, and offer predictable and correct behavior even when there are multiple users and potential system failures.

Traditional relational database systems are built to follow ACID compliance to ensure data consistency, durability, isolation, and atomicity. However, in distributed systems, achieving all four ACID properties at the same time can be challenging. Therefore, some models prefer to relax one or more of these properties to improve performance or availability. For example, some NoSQL databases prioritize eventual consistency over strict consistency to achieve higher availability and partition tolerance.

In conclusion, ACID transactions are fundamental to ensuring that database operations are reliable, and they are a core concept in database management and design.

Transaction patterns and solutions

In this section a few of the common patterns are discussed, however there are a few more, all with their own characteristics and use-cases.

SAGA is a transaction pattern that consists of a sequence of local transactions, each performed by a different service, that together achieve a global business goal. If one of the local transactions fails, the SAGA executes a series of compensating actions to undo the effects of the previous transactions and restore the system to a consistent state. SAGA is suitable for long-running and complex transactions that involve multiple services and resources, where locking or blocking them for the duration of the transaction is not feasible or desirable. SAGA also provides more flexibility and resilience than traditional atomic transactions, as it allows partial failures and retries.

To illustrate how the SAGA pattern works, let's consider an example of a travel booking system that consists of three microservices: flight service, hotel service and payment service. The global business goal is to book a flight and a hotel for a customer and charge their credit card accordingly. The SAGA workflow for this scenario could be as follows:

- The customer initiates the booking request by providing their travel details and payment information.

- The flight service receives the request and tries to reserve a flight ticket for the customer. If successful, it returns a confirmation code to the customer and notifies the coordinator service. If not, it returns an error message to the customer and aborts the transaction.

- The hotel service receives the request and tries to reserve a hotel room for the customer. If successful, it returns a confirmation code to the customer and notifies the coordinator service. If not, it returns an error message to the customer and executes a compensating action to cancel the flight reservation by calling the flight service with the confirmation code.

- The payment service receives the request and tries to charge the customer's credit card for the total amount of the booking. If successful, it returns a receipt to the customer and notifies the coordinator service. If not, it returns an error message to the customer and executes two compensating actions to cancel both the flight and the hotel reservations by calling the respective services with their confirmation codes.

As you can see, each local transaction has a corresponding compensating action that reverses its effect in case of failure. The coordinator service is responsible for orchestrating the execution of the local transactions and the compensating actions, as well as handling failures and timeouts.

Oracle MicroTX supports the SAGA pattern by providing a coordinator service that orchestrates the execution of the local transactions and the compensating actions. The coordinator service communicates with the participating services through a standard interface, which defines the business logic and the compensation logic for each service. The coordinator service also maintains a log of the transaction state and handles failures and timeouts. Oracle MicroTX allows you to define your SAGA workflows using declarative annotations or XML configuration files, which simplifies the development and maintenance of your microservices.

2 phase commit (2PC) is a transaction pattern that ensures atomicity and consistency across multiple resources, such as databases, message queues or web services. 2PC involves two phases: a prepare phase and a commit phase. In the prepare phase, each resource is asked to vote on whether it can commit or abort the transaction. If all resources vote to commit, the transaction moves to the commit phase, where each resource is instructed to finalize the transaction. If any resource votes to abort or fails to respond, the transaction moves to the abort phase, where each resource is instructed to roll back the transaction.

Prepare Phase – The coordinator asks the participating nodes whether they are ready to commit the transaction. The participants returned with a yes or no. 

Commit Phase – If all the participating nodes respond affirmatively in phase 1, the coordinator asks all of them to commit. If at least one node returns negative, the coordinator asks all participants to roll back their local transactions.

Oracle MicroTX supports the 2PC pattern by providing a transaction manager service that coordinates the voting and the finalization of the transactions across multiple resources. The transaction manager service uses a standard protocol, such as JTA or WS-AT, to communicate with the resources and ensure their agreement on the outcome of the transaction. Oracle MicroTX also provides APIs and tools for integrating various types of resources with the transaction manager service, such as JDBC drivers, JMS providers or REST clients.

XA is a specification that defines how distributed transactions can be managed by a transaction manager service and multiple resource managers. XA is based on the 2PC pattern, but it adds some additional features, such as recovery mechanisms, timeout settings and heuristic decisions(decisions made in unusual circumstances, such as communication failures). XA is widely adopted as a standard for distributed transactions in heterogeneous environments, where different types of resources need to be coordinated by a common transaction manager service.

Oracle MicroTX supports the XA specification by providing an XA-compliant transaction manager service that can interoperate with any XA-compliant resource manager. Oracle MicroTX also provides XA drivers for various Oracle products, such as Oracle Database, Oracle WebLogic Server or Oracle Coherence, which enable them to participate in XA transactions. Oracle MicroTX also offers advanced features for monitoring and managing XA transactions, such as performance tuning, diagnostic tools and recovery options.

Other well-known patterns are: Try-Confirm-Cancel(TCC), Long Running Actions(LRA), Eventual Consistency, Optimistic/Pessimistic Locking.

Oracle MicroTX Architecture

The Oracle MicroTX architecture is a distributed transaction management system that enables data consistency across microservices deployed in Kubernetes and/or other environments. The MicroTX architecture consists of two main components: the transaction coordinator and the MicroTX library.

The transaction coordinator is a microservice that runs on Kubernetes and coordinates the outcome of distributed transactions among the participating microservices. The transaction coordinator supports different transaction protocols, such as XA, SAGA, LRA, and TCC, depending on the consistency and performance requirements of the application.

The MicroTX library is a client library that is integrated with the application microservices and provides the APIs and annotations to manage the distributed transactions. The MicroTX library communicates with the transaction coordinator and the database to perform the transaction operations, such as begin, commit, rollback.

Additional, supported OpenSource tools are provided for observability of transaction coordinations such as Prometheus, Jaeger, ELK and Kiali.

The MicroTX architecture simplifies the application development and operations by providing capabilities that make it easier to develop, deploy, and maintain microservices-based applications.

As you can see, Oracle MicroTX provides a comprehensive framework for handling different transaction patterns in microservice architectures. Whether you need to implement SAGA, 2PC or XA transactions(and many other transaction patterns).

***