假设我们有一个用户、钱包REST微服务和一个API网关,它将这些东西粘合在一起。当Bob在我们的网站上注册时,我们的API网关需要通过user微服务创建一个用户,并通过wallet微服务创建一个钱包。

下面是一些可能出错的情况:

用户Bob创建失败:没关系,我们只是向Bob返回一个错误消息。我们使用SQL事务,所以没有人在系统中看到Bob。一切都很好:) 用户Bob已经创建,但是在我们的钱包创建之前,我们的API网关硬崩溃了。我们现在有一个没有钱包的用户(数据不一致)。 创建了用户Bob,当我们创建钱包时,HTTP连接断开。钱包的创建可能成功,也可能失败。

有什么解决方案可以防止这种数据不一致的发生?是否存在允许事务跨越多个REST请求的模式?我读过维基百科上关于两阶段提交的页面,它似乎涉及到这个问题,但我不确定如何在实践中应用它。这篇原子分布式事务:一篇RESTful设计论文看起来也很有趣,尽管我还没有读过。

或者,我知道REST可能不适合这个用例。处理这种情况的正确方法可能是完全放弃REST并使用不同的通信协议,如消息队列系统?或者我应该在我的应用程序代码中强制一致性(例如,通过有一个后台作业来检测不一致并修复它们,或者通过在我的用户模型上有一个“state”属性与“creating”,“created”值等)?


当前回答

有什么解决方案可以防止这种数据不一致的发生?

Traditionally, distributed transaction managers are used. A few years ago in the Java EE world you might have created these services as EJBs which were deployed to different nodes and your API gateway would have made remote calls to those EJBs. The application server (if configured correctly) automatically ensures, using two phase commit, that the transaction is either committed or rolled back on each node, so that consistency is guaranteed. But that requires that all the services be deployed on the same type of application server (so that they are compatible) and in reality only ever worked with services deployed by a single company.

是否存在允许事务跨越多个REST请求的模式?

对于SOAP(好吧,不是REST),有WS-AT规范,但我曾经集成过的任何服务都不支持该规范。对于REST, JBoss已经准备好了一些东西。否则,“模式”是要么找到一个可以插入到您的体系结构中的产品,要么构建您自己的解决方案(不推荐)。

我已经为Java EE发布了这样一个产品:https://github.com/maxant/genericconnector

根据您引用的论文,还有来自Atomikos的Try-Cancel/Confirm模式和相关Product。

BPEL引擎使用补偿处理远程部署服务之间的一致性。

或者,我知道REST可能不适合这个用例。处理这种情况的正确方法可能是完全放弃REST并使用不同的通信协议,如消息队列系统?

将非事务资源“绑定”到事务中有很多方法:

As you suggest, you could use a transactional message queue, but it will be asynchronous, so if you depend on the response it becomes messy. You could write the fact that you need to call the back end services into your database, and then call the back end services using a batch. Again, async, so can get messy. You could use a business process engine as your API gateway to orchestrate the back end microservices. You could use remote EJB, as mentioned at the start, since that supports distributed transactions out of the box.

或者我应该在我的应用程序代码中强制一致性(例如,通过有一个后台作业来检测不一致并修复它们,或者通过在我的用户模型上有一个“state”属性与“creating”,“created”值等)?

玩魔鬼的拥护者:为什么要建立这样的东西,当有产品为你做这件事(见上文),可能比你做得更好,因为他们是经过试验和测试的?

其他回答

最终的一致性是这里的关键。

选择其中一个服务作为事件的主要处理程序。 该服务将处理单个提交的原始事件。 主处理程序将负责将次要效果异步地传递给其他服务。 主处理程序将对其他服务调用进行编排。

指挥官负责分布式事务并进行控制。它知道要执行的指令,并协调执行它们。在大多数情况下,只有两条指令,但它可以处理多条指令。

指挥官负责保证所有指令的执行,这意味着退休。 当指挥官试图执行远程更新而没有得到响应时,它没有重试。 通过这种方式,系统可以配置为更不容易出现故障,并自我修复。

当我们进行重试时,我们有幂等性。 幂等性是这样一种性质,即可以做某事两次,最终结果与只做一次一样。 我们需要远程服务或数据源上的幂等性,以便在它多次接收指令的情况下,它只处理一次指令。

最终一致性 这解决了大多数分布式事务的挑战,但是我们需要考虑以下几点。 每一个失败的事务都将被重试,尝试重试的数量取决于上下文。

一致性是最终的,即在重试期间系统处于不一致的状态,例如,如果客户订购了一本书,并支付了费用,然后更新了库存数量。如果库存更新操作失败,并且假设这是最后一个可用的库存,那么在库存更新的重试操作成功之前,这本书仍然可用。重试成功后,您的系统将保持一致。

有什么解决方案可以防止这种数据不一致的发生?

Traditionally, distributed transaction managers are used. A few years ago in the Java EE world you might have created these services as EJBs which were deployed to different nodes and your API gateway would have made remote calls to those EJBs. The application server (if configured correctly) automatically ensures, using two phase commit, that the transaction is either committed or rolled back on each node, so that consistency is guaranteed. But that requires that all the services be deployed on the same type of application server (so that they are compatible) and in reality only ever worked with services deployed by a single company.

是否存在允许事务跨越多个REST请求的模式?

对于SOAP(好吧,不是REST),有WS-AT规范,但我曾经集成过的任何服务都不支持该规范。对于REST, JBoss已经准备好了一些东西。否则,“模式”是要么找到一个可以插入到您的体系结构中的产品,要么构建您自己的解决方案(不推荐)。

我已经为Java EE发布了这样一个产品:https://github.com/maxant/genericconnector

根据您引用的论文,还有来自Atomikos的Try-Cancel/Confirm模式和相关Product。

BPEL引擎使用补偿处理远程部署服务之间的一致性。

或者,我知道REST可能不适合这个用例。处理这种情况的正确方法可能是完全放弃REST并使用不同的通信协议,如消息队列系统?

将非事务资源“绑定”到事务中有很多方法:

As you suggest, you could use a transactional message queue, but it will be asynchronous, so if you depend on the response it becomes messy. You could write the fact that you need to call the back end services into your database, and then call the back end services using a batch. Again, async, so can get messy. You could use a business process engine as your API gateway to orchestrate the back end microservices. You could use remote EJB, as mentioned at the start, since that supports distributed transactions out of the box.

或者我应该在我的应用程序代码中强制一致性(例如,通过有一个后台作业来检测不一致并修复它们,或者通过在我的用户模型上有一个“state”属性与“creating”,“created”值等)?

玩魔鬼的拥护者:为什么要建立这样的东西,当有产品为你做这件事(见上文),可能比你做得更好,因为他们是经过试验和测试的?

恕我直言,微服务架构的一个关键方面是事务被限制在单个微服务中(单一责任原则)。

在当前示例中,User创建将是一个自己的事务。用户创建将把USER_CREATED事件推入事件队列。钱包服务将订阅USER_CREATED事件并创建钱包。

为什么不使用支持脚本/编程的API管理(APIM)平台?因此,您将能够在APIM中构建复合服务,而不会干扰微服务。为此,我使用APIGEE进行设计。

不合理的事情:

使用REST服务的分布式事务。REST服务根据定义是无状态的,因此它们不应该是跨多个服务的事务边界的参与者。您的用户注册用例场景是有意义的,但是使用REST微服务创建用户和钱包数据的设计并不好。

什么会让你头疼:

EJBs with distributed transactions. It's one of those things that work in theory but not in practice. Right now I'm trying to make a distributed transaction work for remote EJBs across JBoss EAP 6.3 instances. We've been talking to RedHat support for weeks, and it didn't work yet. Two-phase commit solutions in general. I think the 2PC protocol is a great algorithm (many years ago I implemented it in C with RPC). It requires comprehensive fail recovery mechanisms, with retries, state repository, etc. All the complexity is hidden within the transaction framework (ex.: JBoss Arjuna). However, 2PC is not fail proof. There are situations the transaction simply can't complete. Then you need to identify and fix database inconsistencies manually. It may happen once in a million transactions if you're lucky, but it may happen once in every 100 transactions depending on your platform and scenario. Sagas (Compensating transactions). There's the implementation overhead of creating the compensating operations, and the coordination mechanism to activate compensation at the end. But compensation is not fail proof either. You may still end up with inconsistencies (= some headache).

最好的选择是什么:

Eventual consistency. Neither ACID-like distributed transactions nor compensating transactions are fail proof, and both may lead to inconsistencies. Eventual consistency is often better than "occasional inconsistency". There are different design solutions, such as: You may create a more robust solution using asynchronous communication. In your scenario, when Bob registers, the API gateway could send a message to a NewUser queue, and right-away reply to the user saying "You'll receive an email to confirm the account creation." A queue consumer service could process the message, perform the database changes in a single transaction, and send the email to Bob to notify the account creation. The User microservice creates the user record and a wallet record in the same database. In this case, the wallet store in the User microservice is a replica of the master wallet store only visible to the Wallet microservice. There's a data synchronization mechanism that is trigger-based or kicks in periodically to send data changes (e.g., new wallets) from the replica to the master, and vice-versa.

但是如果需要同步响应呢?

重构微服务。如果使用队列的解决方案不能工作,因为服务使用者需要立即得到响应,那么我宁愿重新构造User和Wallet功能,将其配置在同一个服务中(或者至少在同一个VM中,以避免分布式事务)。是的,它离微服务更近了一步,也更接近了一个庞然大物,但它会让你免去一些头疼的事情。