6

We have started using a "micro service" architecture and we have multiple services. Each service owns the data and the business logic for a domain and only entity ids are shared across services.

Everything was perfect until 1 requirement came in...

On the UI, when a user is about to delete an entity, a warning listing where this entity is used must be displayed.

An fictitious example, let's say you have a billing service and a user service. If a manager is about to delete a user, the UI has to display the list of active subscriptions for this user.

The user service doesn't know where the user id is referenced and it's probably in multiple services.

How can we achieve this?

Obviously the user service or the UI can't ask each service to return all entities related to this user id. That means every service would have to provide an endpoint for each entity and it would be very clumsy.

Marc
  • 545
  • 2
    That means every service would have to provide an endpoint for each entity -- No, that's not true. Each service would need the ability to respond to a question, like "do you have any subscriptions for this user id?" – Robert Harvey Jul 02 '19 at 14:33
  • @RobertHarvey Yes exactly. You need an endpoint to search subscriptions based on some filters. This can get tedious. – Marc Jul 02 '19 at 15:46
  • Also it would mean that the user service has to be aware of all other services – Marc Jul 02 '19 at 15:46
  • If it's a requirement, tedious is irrelevant. Have you considered using a gateway? That would give you a single point of access for all of your microservices. – Robert Harvey Jul 02 '19 at 15:47
  • Or, create a single microservice that attaches to all of the databases in question. – Robert Harvey Jul 02 '19 at 15:51
  • We do have Kong api gateway. Not sure I understand how that would solve the problem? Can you call multiple api with 1 route and merge the results? – Marc Jul 02 '19 at 16:09
  • I don't know the answer to that. You would have to check the Kong API documentation. – Robert Harvey Jul 02 '19 at 16:09

3 Answers3

7

Yes, querying each microservice is unavoidable in this case. That's one of the main trade offs of microservices. However, I think if you actually sat down and designed the endpoints, you would find it's not as burdensome as you think.

For example, it would surprise me very much if your billing service doesn't already have a way to list all the subscriptions for a given user. If that is important in a deletion context, it's most likely important for other use cases.

I think your main hangup is you're trying to make this too generic and decentralized. If you want a good UX, you're going to want to be able to display referenced subscriptions differently from results from other services. That means there's no easy way getting around your front end needing to know everywhere a user might be referenced.

Karl Bielefeldt
  • 147,435
2

My team once had to deal with this exact problem and I've written an article about how we came up with a solution for this.

You can read it here.

I know it has been almost 3 years since this question was made, but I hope it can help people that might have this problem in the future.

In short, we added a new statement to all the write (create, update, delete) transactions in all our services. That statement is nothing more than an emission of an outbox event (a write in a table/collection named "outbox" that describes what is being written). Then, that event gets caught by a Kafka Source Connector and put into a topic that is being consumed by a new microservice which has a Neo4j database and builds the knowledge graph of the whole system. That service can then provide an API (REST, GraphQL, ...) to tell whether an entity is being used and/or referenced by others, even if that reference is not direct, e.g. A->B vs A->B->C, in the former case, A uses B, in the latter, A uses B directly and C indirectly because B uses C, and it works with even a greater relationship length (e. g. A->B->C->D->...) thanks to the potentialities of a graph database.

thegoncalomartins
  • 21
0

There is no way to magically track the use of an entity across microservice boundaries. You can create a description of each of your microservices, much like you would describe a web service in WSDL or a database in an EDMX diagram, and then find everything that references the entity you want to delete. Not very wieldy since you have to update it every time you change something.

Another thing you can do is have each microservice define an endpoint for checking where a specific entity is referenced. Same downside as the previous approach.

The simplest thing you can do is not physically delete anything. You can then have a purge service/tool that periodically moves/physically deletes everything that is marked for deletion. This removes the need for showing warnings when someone deletes something.

devnull
  • 3,045
  • Well, actually, this is software. So yes, there is a way, and it's not magic. You would just have to build it. – Robert Harvey Jul 02 '19 at 14:34
  • @robertharvey - But you shouldn’t. That creates a circular dependency between the microservices, greatly harming their “independently deployable” requirement. – Telastyn Jul 02 '19 at 14:38
  • @telastyn I never said it was a good idea. I only said it was possible. – Robert Harvey Jul 02 '19 at 14:39
  • @RobertHarvey what I meant was that there is no readily available piece of code (at least I'm not aware of one) that you can point at your microservice mesh and detect dependencies, there are too many variables involved. the only way is, as you pointed out, to create the magic yourself. – devnull Jul 02 '19 at 14:44
  • We already do "soft delete" but sadly we still need to list the references in a warning – Marc Jul 02 '19 at 15:35