Orchestration services combine multiple foundation or processing services to perform a complex logical operation. Their main responsibilities are multi-entity logical operations and delegating the dependencies of those operations to downstream processing or foundation services.
Orchestration services' primary responsibility is encapsulating operations requiring two or three business entities.
public async ValueTask<LibraryCard> CreateStudentLibraryCardAsync(LibraryCard libraryCard) =>
TryCatch(async () =>
{
ValidateLibraryCard(libraryCard);
await this.studentProcessingService
.VerifyEnrolledStudentExistsAsync(libraryCard.StudentId);
return await this.libraryCardProcessingService.CreateLibraryCardAsync(libraryCard);
});In the above example, the LibraryCardOrchestrationService calls both the StudentProcessingService and LibraryCardProcessingService to perform a complex operation. First, we verify the student's existence and enrollment, then create the library card.
Creating a library card for a given student cannot be performed by simply calling the library card service because the library card service (processing or foundation) needs access to all the details about the student. Therefore, a combination logic is required to ensure that a proper flow is in place.
It's important to understand that orchestration services are only required if we need to combine multi-entity operations, which can be primitive or higher-order. In some architectures, orchestration services might not even exist. That's simply because some microservices might be merely responsible for applying validation logic and persisting and retrieving data from storage, no more or no less.
Orchestration services are one of the core business logic components in any system, positioned between single entity services (such as processing or foundation) and advanced logic services such as coordination services, aggregation services, or simply exposers such as controllers, web components, or anything else. Here's a high-level overview of where orchestration services may live:
As shown above, Orchestration services have quite a few dependencies and consumers. They are the core engine of any software. On the right-hand side, you can see an orchestration service's dependencies. Since a processing service is optional based on whether a higher-order business logic is needed, orchestration services can combine multiple foundation services as well.
The existence of an Orchestration service warrants the presence of a Processing service. But that's only sometimes the case. In some situations, all orchestration services need to finalize a business flow to interact with primitive-level functionality.
However, an Orchestration service could have several consumers, such as coordination services (orchestrators of orchestrators), aggregation services, or an exposer. Exposers are like controllers, view services, UI components, or another foundation or processing service in case of putting messages back on a queue - which we will discuss further in our Standard.
In general, orchestration services are concerned with combining single-entity primitive or higher-order business logic operations to execute a successful flow. But you can also think of them as the glue that ties multiple single-entity operations together.
Just like Processing services, the language used in Orchestration services defines the level of complexity and the capabilities it offers. Orchestration services usually combine two or more primitive or higher-order operations from multiple single-entity services to execute a successful operation.
Orchestration services have a common characteristic regarding the language of their functions. Orchestration services are wholistic in most of the language of its function. You will see functions such as NotifyAllAdmins where the service pulls all users with an admin type and then calls a notification service.
Orchestration services offer functionality that inches closer to a business language than primitive technical operations. You may see almost an identical expression in a non-technical business requirement matching a function name in an orchestration service. The same pattern continues as one goes to higher and more advanced categories of services within a specific realm of business logic.
Orchestration services can also be a pass-through for some operations. For instance, an orchestration service could allow an AddStudentAsync to be propagated through the service to unify the source of interactions with the system at the exposer's level. In this case, orchestration services will use the same terminology a processing or foundation service may use to propagate the operation.
Orchestration services mainly combine multiple operations supporting a particular entity. So, if the primary entity is Student and the rest of the entities are just to support an operation mainly targeting a Student entity, then the name of the orchestration service would be StudentOrchestrationService.
Enforcement of naming conventions ensures that any orchestration service stays focused on a single entity's responsibility concerning multiple other supporting entities.
For instance, creating a library card requires the school enrollment of the student referenced in that library card. In this case, the Orchestration service name will reflect its primary entity, LibraryCard. Our orchestration service name would then be LibraryCardOrchestrationService.
The opposite is also true. If enrolling a student in a school has associated operations such as creating a library card, then, in this case, a StudentOrchestrationService must exist to create a Student and all other related entities.
The same idea applies to all exceptions created in an orchestration service, such as StudentOrchestrationValidationException and StudentOrchestrationDependencyException.
As we mentioned above, Orchestration services might have a more extensive range of dependencies, unlike Processing and Foundation services, because Processing services are optional. Therefore, Orchestration services may have dependencies ranging from foundation services or optional processing services to cross-cutting services such as logging or other utility brokers.
A fundamental rule governing the consistency and balance of orchestration services is the 'Florance Pattern', which dictates that any orchestration service may not combine dependencies from different categories of operation.
That means an Orchestration service cannot combine Foundation and Processing services. The dependencies have to be either all Processings or all Foundation services. That rule doesn't apply to utility broker dependencies, however.
Here's an example of an unbalanced orchestration service dependency:
An additional processing service is required to give a pass-through to a lower-level foundation service to balance the architecture - applying 'Florance Pattern' for symmetry would turn our architecture into the following:
Applying the 'Florance Pattern' might be very costly initially, including creating an entirely new processing service (or multiple) to balance the architecture. However, its benefits outweigh the cost from maintainability, readability, and pluggability perspectives.
The 'Two-Three' rule is a complexity control rule. This rule dictates that an Orchestration service may have up to three or less than two Processing or Foundation services to run the orchestration. This rule, however, doesn't apply to utility brokers. Orchestration services may have a DateTimeBroker or a LoggingBroker without restriction. However, an orchestration service may not have an entity broker, such as a StorageBroker or a QueueBroker, which feeds directly into the core business layer of any service.
This rule, like most of the patterns and concepts in The Standard, is inspired by nature. You can see how the trees branch into twos and threes - the same thing for thunder, blood vessels, and so many other creations around, within, and above us follow the same pattern.
A tree branches as it grows upwards but also in its very roots. And so is the case with Orchestration and Orchestration-Like services. They can branch further upwards, as I will explain here shortly, but also downwards through patterns like the Cul-De-Sac pattern.
The 'Two-Three' rule may require a layer of normalization to the categorical business function. Let's talk about the different mechanisms of normalizing orchestration services.
There are frequently situations where the current architecture of any given orchestration service ends up with one orchestration service with three dependencies. A new entity processing or foundation service is required to complete an existing process.
For instance, let's say we have a StudentContactOrchestrationService, which has dependencies that provide primitive-level functionality for each student Address, Email, and Phone. Here's a visualization of that state:
Now, a new requirement, 'SocialMedia', is added to 'Student', to gather more contact information about how to reach a student. We can go into full-normalization mode by finding common ground that equally splits the contact information entities. For instance, we can break out regular contact information versus digital contact information as in Address and Phone versus Email and SocialMedia. This way, we split four dependencies into two, each for their orchestration services as follows:
In the figure above, we modified the existing StudentContactOrchestrationService into StudentRegularContactOrchestrationService and removed one of its dependencies on the EmailService.
Additionally, we created a new StudentDigitalContactOrchestrationService to have two dependencies on the existing EmailService and the latest SocialMediaService. Consequently, we need an advanced business logic layer, like a coordination service, to provide student contact information to upstream consumers.
Normalization is more complex than the example above, especially when a core entity has to exist before creating or filling in additional information about related entities.
For instance, let's say we have a StudentRegistrationOrchestrationService which relies on StudentProcessingService, LibraryCardProcessingService, and BookProcessingService as follows:
But now, we need a new service called' ImmunizationProcessingService' to handle students' immunization records. We need all four services, but we already have a StudentRegistrationOrchestrationService that has three dependencies. At this point, a semi-normalization is required for the re-balancing of the architecture to honor the 'Two-Three' rule and eventually control the complexity.
In this case, a further normalization or a split is required to re-balance the architecture. We must think conceptually about the common ground between the primitive entities in a student registration process. Student requirements contain identity, health, and materials. We can, in this scenario, combine LibraryCard and Book under the same orchestration service as books and libraries are somewhat related. So we have StudentLibraryOrchestrationService, and for the other service, we would have StudentHealthOrchestrationService as follows:
To complete the registration flow with a new model, a coordination service must pass in advanced business logic to combine these entities. But more importantly, you will notice that each orchestration service has a redundant dependency of StudentProcessingService to ensure no virtual dependency on any other orchestration service, creating/providing a student record exists.
Virtual dependencies are very tricky. It's a hidden connection between two services of any category where one service implicitly assumes that a particular entity will be created and present. Virtual dependencies are very dangerous and threaten the proper autonomy of any service. Detecting virtual dependencies early in the design and development process could be a daunting but necessary task to ensure a clean, Standardized architecture is in place.
Just like model changes require database structure migrations and additional logic and validations, a new requirement for a new entity might require restructuring an existing architecture or extending it to a new version, depending on which stage the system receives these new requirements.
Adding another dependency to an existing orchestration service may be very enticing - but that's where the system starts to diverge from 'The Standard'. And that's when the system begins to become an unmaintainable legacy system. But more importantly, this scenario tests the design principles and standards of craftsmanship of the engineers involved in designing and developing the system.
Everything, everywhere, is somehow connected. Yet, there are scenarios where higher levels of normalization are challenging to achieve. Sometimes, it might be incomprehensible for the mind to group multiple services under one orchestration service.
Because it's hard for me to come up with an example for multiple entities that have no connection to each other, it couldn't exist. I'm going to rely on some fictional entities to visualize a problem. So, let's assume there are AService and BService orchestrated together with an XService. The existence of XService is important to ensure that both A and B can be created with an assurance that a core entity X does exist.
Now, let's say a new service, CService, must be added to the mix to complete the existing flow. So, now we have four different dependencies under one orchestration service, and a split is mandatory. Since there's no relationship whatsoever between A, B, and C, a 'No-Normalization' approach becomes the only option to realize a new design as follows:
The above primitive services will be orchestrated with a core service, X, and then gathered under a coordination service. This case above is the worst-case scenario, where normalization of any size is impossible. Note that the author of this Standard couldn't come up with a realistic example unlike any others to show you how rare it is to run into that situation, so let a 'No-Normalization' approach be your very last solution if you run out of options.
Regardless of the type of normalization you follow, you must ensure that your grouped services represent a common meaning. For instance, putting together a StudentProcessingService and LibraryProcessingService must require a functional commonality. An excellent example of that would be StudentRegistrationOrchestrationService. The registration process requires adding a new student record and creating a library card for that student.
Implementing orchestration services without an intersection between two or three entities per operation defeats the whole purpose of having an orchestration service. This condition is satisfied if at least one intersection between two entities has occurred. An orchestration service may have other 'Pass-Through' operations where we propagate certain routines from their processing or foundation origins if they match the same contract.
Here's an example:
public class StudentOrchestrationService
{
public async ValueTask<Student> RegisterStudentAsync(Student student)
{
Student addedStudent =
await this.studentProcessingService.AddStudentAsync(student);
LibraryCard libraryCard =
await this.libraryCardPorcessingService.AddLibraryCardAsync(
addedStudent.Id);
return addedStudent;
}
public async ValueTask<Student> ModifyStudentAsync(Student student) =>
await this.studentProcessingService.ModifyStudentAsync(student);
}In the example above, our StudentOrchestrationService had an orchestration routine that combined adding a student and creating a library card for that student. It also offers a 'Pass-Through' function for a low-level processing service routine to modify a student.
'Pass-Through' routines must have the same contract as the other routines in any orchestration service. Our 'Pure Contract' principle dictates that any service should allow the same contract as input and output or primitive types.
Orchestration services may combine two or three different entities and their operations to achieve a higher business logic. There are two scenarios for contract/models for orchestration services: One that stays true to the primary entity's purpose and one that is complex - a combinator orchestration service that tries to expose its inner target entities explicitly.
Let's talk about these two scenarios in detail.
Some orchestration services are still single-purposed, even though they may combine two or three other higher-order routines from multiple entities. For instance, an orchestration service that reacts to messages from some queue and persists in these messages is a single-purposed and single-entity orchestration service.
Let's take a look at this code snippet:
public class StudentOrchestrationService
{
private readonly IStudentEventProcessingService studentEventProcessingService;
private readonly IStudentProcessingService studentProcessingService;
public StudentOrchestrationService(
IStudentEventProcessingService studentEventProcessingService,
IStudentProcessingService studentProcessingService)
{
this.studentEventProcessingService = studentEventProcessingService;
this.studentProcessingService = studentProcessingService;
ListenToEvents();
}
public void ListenToEvents() =>
this.studentEventService.ListenToEvent(UpsertStudentAsync);
public async ValueTask<Student> UpsertStudentAsync(Student student)
{
...
await this.studentProcessingService.UpsertStudentAsync(student);
...
}
}In the above example, the orchestration service still exposes functionality that honors the physical model Student and internally communicates with several services that may provide completely different models. These are the scenarios where a single entity has a primary purpose, and all other services are supporting services to ensure a successful flow for that entity.
In our example, the orchestration services listen to a queue for new student messages and use that event to persist any incoming new students in the system. So, the physical contract Student is the same language the orchestration service explicitly uses as a model to communicate with upper stream services/exposers or others.
However, there are other scenarios in which a single entity is not the only purpose/target for an orchestration service. Let's discuss that in detail.
In some scenarios, an orchestration service may be required to create non-physical contracts to complete a particular operation. For instance, consider an orchestration service required to persist a social media post containing a picture. The requirement is to persist the picture in one database and the actual post (comments, authors, and others) in a different database table in a relational model.
The incoming model might be significantly different from the actual physical models. Let's see what that would look like in the real world.
Consider having this model:
public class MediaPost
{
public Guid Id {get; set;}
public string Content {get; set;}
public DateTimeOffset Date {get; set;}
public IEnumerable<string> Base64Images {get; set;}
}The above contract, MediaPost, contains two separate physical entities. The first is the actual post, including the Id, Content, and Date, and the second is the list of images attached to that post.
Here's how an orchestration service would react to this incoming virtual model:
public async ValueTask<MediaPost> SubmitMediaPostAsync(MediaPost mediaPost)
{
...
Post post = MapToPost(mediaPost);
List<Media> medias = MapToMedias(mediaPost);
Post addedPost =
await this.postProcessingService.AddPostAsync(post);
List<Medias> addedMedias =
await this.mediaProcessingService.AddMediasAsync(medias);
return MapToMediaPost(addedPost, addedMedias);
}
public Post MapToPost(MediaPost mediaPost)
{
return new Post
{
Id = mediaPost.Id,
Content = mediaPost.Content,
CreatedDate = mediaPost.Date,
UpdatedDate = mediaPost.Date
};
}
public List<Media> MapToMedias(MediaPost mediaPost)
{
return mediaPost.Base64Images.Select(image =>
new Media
{
Id = Guid.NewGuid(),
PostId = mediaPost.Id,
Image = image,
CreatedDate = mediaPost.Date,
UpdatedDate = mediaPost.Date
});
}The above code snippet shows the orchestration service deconstructing a given virtual model/contract, MediaPost, into two physical models. Each one has its separate processing service that handles its persistence. There are scenarios where the virtual model gets deconstructed into one single model with additional details used for validation and verification with downstream processing or foundation services.
In hybrid situations, the incoming virtual model may have nested physical models, which we can only allow with virtual models. Physical models shall always stay anemic (contains no routines or constructors) and flat (contains no nested models) to control complexity and focus responsibility.
In summary, Orchestration services may create their contracts, which may be physical or virtual. A virtual contract may be a combination of one or many physical (or nested virtual) contracts or simply have its own flat design in terms of properties.
Sometimes, Orchestration services and their equivalent (coordination, management, etc.) may not need an exposer component (controller, for instance). That's because these services may listen to specific events and communicate them back into a Processing or a Foundation service at the same level where the event started or was received.
For example, incoming messages can be received from a subscription to an event service or a queue. In this case, the input for these services isn't necessarily through an exposer component anymore. Imagine building a simple application that gets notified with messages from a queue and then maps these messages into some local model to persist it in storage. In this case, the orchestration service would look something like the following:
The StudentEventOrchestrationService listens to messages for new students and immediately converts them into models that can be persisted in the database.
Here's an example:
Let's start with a unit test for this pattern as follows:
[Fact]
private void ShouldListenToProfileEvents()
{
// given . when
this.profileEventOrchestrationService.ListenToProfileEvents();
// then
this.profileEventServiceMock.Verify(service =>
service.ListenToProfileEvent(
this.profileEventOrchestrationService.ProcessProfileEventAsync),
Times.Once);
this.profileEventService.VerifyNoOtherCalls();
this.profileServiceMock.VerifyNoOtherCalls();
this.loggingBrokerMock.VerifyNoOtherCalls();
}
[Fact]
private async Task ShouldAddProfileAsync()
{
// given
ProfileEvent randomProfileEvent =
CreateRandomProfileEvent();
ProfileEvent inputProfileEvent =
randomProfileEvent;
this.profileServiceMock.Setup(service =>
service.AddProfileAsync(inputProfileEvent.Profile));
// when
await this.profileEventOrchestrationService
.ProcessProfileEventAsync(inputProfileEvent);
// then
this.profileServiceMock.Verify(service =>
service.AddProfileAsync(inputProfileEvent.Profile),
Times.Once);
this.profileServiceMock.VerifyNoOtherCalls();
this.loggingBrokerMock.VerifyNoOtherCalls();
this.profileEventServiceMock.VerifyNoOtherCalls();
}The test here indicates that an event listening has to occur first, and then persistence logic in the student service must match the outcome of mapping an incoming message to a given student.
Let's make this test pass.
public partial class ProfileEventOrchestrationService : IProfileEventOrchestrationService
{
private readonly IProfileEventService profileEventService;
private readonly IProfileService profileService;
private readonly ILoggingBroker loggingBroker;
public ProfileEventOrchestrationService(
IProfileEventProcessingService profileEventService,
IProfileProcessingService profileService,
ILoggingBroker loggingBroker)
{
this.profileEventService = profileEventService;
this.profileService = profileService;
this.loggingBroker = loggingBroker;
}
public void ListenToProfileEvents() =>
TryCatch(() =>
{
this.profileEventService.ListenToProfileEvent(
ProcessProfileEventAsync);
});
public ValueTask ProcessProfileEventAsync(ProfileEvent profileEvent) =>
TryCatch(async () =>
{
...
await this.profileService.AddProfileAsync(profileEvent.Profile);
});
}In the above example, the Orchestration service constructor subscribes to the events that would come from the ProfileEventProcessingService. When an event occurs, the orchestration service calls the ProcessProfileEventAsync function to persist the incoming student into the database through a foundation or a processing service at the same level as the event service.
This pattern or characteristic is called the Cul-De-Sac. An incoming message will turn and head in a different direction for a different dependency. This pattern is typical in large enterprise-level applications where eventual consistency is incorporated to ensure the system can scale and become resilient under heavy consumption. This pattern also prevents malicious attacks against your API endpoints since it allows processing queue messages or events whenever the service is ready to process them. We will discuss the details in 'The Standard Architecture.
Orchestration services provide advanced business logic. It orchestrates multiple flows for multiple entities/models to complete a single flow. Let's discuss in detail what these responsibilities are:
Orchestration services can only exist by combining multiple routines from multiple entities. These entities may differ in nature but share a standard flow or purpose. For instance, a LibraryCard model fundamentally differs from a Student model. However, they both share a common purpose regarding the student registration process. Adding a student record is required to register a student, but assigning a library card to that student is required for a successful student registration process.
Orchestration services ensure the correct routines for each entity are integrated and called in the proper order. Additionally, orchestration services are responsible for rolling back a failing operation. These three aspects constitute an orchestration effort across multiple routines, entities, or contracts.
Let's talk about those in detail.
We spoke earlier about orchestration services combining multiple routines to achieve a common purpose or a single flow. This aspect of orchestration services can serve as both a fundamental characteristic and a responsibility. An orchestration service without at least one routine combining two or three entities is not considered an orchestration. Integrating multiple services without a common purpose is a better-fit definition for aggregation services, which we will discuss later in this services chapter.
However, within the flow combination comes the unification of the contract. I call it mapping and branching. Mapping an incoming model into multiple lower-stream service models and then branching the responsibility across these services.
Just like the previous services, during their flow combination, Orchestration services are responsible for ensuring the purity of the exposed input and output contracts, which becomes a bit more complex when combining multiple models. Orchestration services will continue to be responsible for mapping incoming contracts to their respective downstream services. They will also map back the results from these services into the unified model.
Let's bring back a previous code snippet to illustrate that aspect:
public async ValueTask<MediaPost> SubmitMediaPostAsync(MediaPost mediaPost)
{
...
Post post = MapToPost(mediaPost);
List<Media> medias = MapToMedias(mediaPost);
Post addedPost =
await this.postProcessingService.AddPostAsync(post);
List<Medias> addedMedias =
await this.mediaProcessingService.AddMediasAsync(medias);
return MapToMediaPost(addedPost, addedMedias);
}
private Post MapToPost(MediaPost mediaPost)
{
return new Post
{
Id = mediaPost.Id,
Content = mediaPost.Content,
CreatedDate = mediaPost.Date,
UpdatedDate = mediaPost.Date
};
}
private List<Media> MapToMedias(MediaPost mediaPost)
{
return mediaPost.Base64Images.Select(image =>
new Media
{
Id = Guid.NewGuid(),
PostId = mediaPost.Id,
Image = image,
CreatedDate = mediaPost.Date,
UpdatedDate = mediaPost.Date
});
}
private MediaPost MapToMediaPost(Post post, List<Media> medias)
{
return new MediaPost
{
Id = post.Id,
Content = post.Content,
Date = post.CreatedDate,
Base64Images = medias.Select(media => media.Image)
}
}As you can see in the above example, the mapping and branching don't just happen on the way in. But a reverse action has to be taken on the way out. It violates The Standard to return the same input object that was passed in. That takes away any visibility on potential changes to the incoming request during persistence. The duplex mapping should substitute the need to dereference the incoming request to ensure no unexpected internal changes have occurred.
Note that breaking out the mapping logic into its own aspect/partial class file is also recommended�something like StudentOrchestrationService.Mappings.cs�to ensure the only thing left is orchestration's business logic.
Calling routines in the correct order can be crucial to any orchestration process. For instance, a library card cannot be created unless a student record is created first. Enforcing the order here can be divided into two different types. Let's discuss those here for a bit.
The natural order here refers to specific flows that cannot be executed unless a prerequisite of input parameters is retrieved or persisted. For instance, imagine a situation where a library card cannot be created unless a student's unique identifier is retrieved first. In this case, we don't have to worry about testing that certain routines were called in the correct order because it comes naturally with the flow.
Here's a code example of this situation:
public async ValueTask<LibraryCard> CreateLibraryCardAsync(LibraryCard libraryCard)
{
Student student = await this.studentProcessingService
.RetrieveStudentByIdAsync(libraryCard.StudentId));
return await this.libraryCardProcessingService
.CreateLibraryCardAsync(libraryCard, student.Name);
}In the example above, having a student Name is a requirement to create a library card. Therefore, the orchestration of order here comes naturally as part of the flow without additional effort.
Let's talk about the second type of order - Enforced Order.
Imagine the same example above, but instead of the library card requiring a student name, it just needs the student Id already enclosed in the incoming request model. Something like this:
public async ValueTask<LibraryCard> CreateLibraryCardAsync(LibraryCard libraryCard)
{
await this.studentProcessingService.VerifyEnlistedStudentExistAsync(
libraryCard.StudentId);
return await this.libraryCardProcessingService.CreateLibraryCardAsync(libraryCard);
}Ensuring a verified enrolled student exists before creating a library card might become a challenge because there is no dependency between the return value of one routine and the input parameters of the next. In other words, the VerifyEnlistedStudentExistAsync function returns nothing that the CreateLibraryCardAsync function cares about in terms of input parameters.
In this case, an enforced type of order must be implemented through unit tests. A unit test for this routine would require verifying not just that the dependency has been called with the correct parameters but also that they are called in the correct order let's take a look at how that would be implemented:
[Fact]
private async Task ShouldCreateLibraryCardAsync()
{
// given
Student someStudent = CreateRandomStudent();
LibraryCard randomLibraryCard = CreateRandomLibraryCard();
LibraryCard inputLibraryCard = randomLibraryCard;
LibraryCard createdLibraryCard = inputLibraryCard;
LibraryCard expectedLibraryCard = inputLibraryCard.DeepClone();
Guid studentId = inputLibraryCard.StudentId;
var mockSequence = new MockSequence();
this.studentProcessingServiceMock.InSequence(mockSequence).Setup(service =>
service.VerifyEnlistedStudentExistAsync(studentId))
.Returns(someStudent);
this.libraryCardProcessingServiceMock.InSequence(mockSequence).Setup(service =>
service.CreateLibraryCardAsync(inputLibraryCard))
.ReturnsAsync(createdLibraryCard);
// when
LibraryCard actualLibraryCard = await this.libraryCardOrchestrationService
.CreateLibraryCardAsync(inputLibraryCard);
// then
actualLibraryCard.Should().BeEquivalentTo(expectedLibraryCard);
this.studentProcessingServiceMock.Verify(service =>
service.VerifyEnlistedStudentExistAsync(studentId),
Times.Once);
this.libraryCardProcessingServiceMock.Verify(service =>
service.CreateLibraryCardAsync(inputLibraryCard),
Times.Once);
this.studentProcessingServiceMock.VerifyNoOtherCalls();
this.libraryCardProcessingServiceMock.VerifyNoOtherCalls();
this.loggingBrokerMock.VerifyNoOtherCalls();
}In the example above, the mock framework is being used to ensure a certain order is enforced when calling these dependencies. This way, we enforce a certain implementation within any given method to ensure that non-naturally connected dependencies are sequentially called in the intended order.
It's more likely that the type of ordering leans more towards enforced than natural when orchestration services reach the maximum number of dependencies.
This responsibility is very similar to flow combinations. Except in this case, orchestration services unify all the exceptions that may occur from any dependencies into one unified categorical exception model. Let's start with an illustration of what that mapping may look like:
In the illustration above, you will notice that validation and dependency validation exceptions, thrown from downstream dependency services, map into one unified dependency exception at the orchestration level. This practice allows upstream consumers of that same orchestration service to determine the next course of action based on one categorical exception type instead of four, or in the case of three dependencies, it would be six categorical dependencies.
Let's start with a failing test to materialize our idea here:
public static TheoryData DependencyValidationExceptions()
{
string exceptionMessage = GetRandomMessage();
var innerException = new Xeption(exceptionMessage);
var studentValidationException =
new StudentValidationException(
message: "Student validation error occurred, fix errors and try again.",
innerException);
var studentDependencyValidationException =
new StudentDependencyValidationException(
message: "Student dependency validation error occurred, fix errors and try again.",
innerException);
var libraryCardValidationException =
new LibraryCardValidationException(
message: "Library card validation error occurred, fix errors and try again.",
innerException);
var libraryCardDependencyValidationException =
new LibraryCardDependencyValidationException(
message: "Library card dependency validation error occurred, fix errors and try again.",
innerException);
return new TheoryData<Xeption>
{
studentValidationException,
studentDependencyValidationException,
libraryCardValidationException,
libraryCardDependencyValidationException
};
}
[Theory]
[MemberData(nameof(DependencyValidationExceptions))]
private async Task ShouldThrowDependencyValidationExceptionOnCreateIfDependencyValidationErrorOccursAndLogItAsync(
Xeption dependencyValidationException)
{
// given
Student someStudent = CreateRandomStudent();
var expectedStudentOrchestrationDependencyValidationException =
new StudentOrchestrationDependencyValidationException(
message: "Student dependency validation error occurred, fix errors and try again",
dependencyValidationException.InnerException as Xeption);
this.studentServiceMock.Setup(service =>
service.AddStudentAsync(It.IsAny<Student>()))
.ThrowsAsync(dependencyValidationException);
// when
ValueTask<Student> addStudentTask =
await this.studentOrchestrationService.AddStudentAsync(someStudent);
StudentOrchestrationDependencyValidationException
actualStudentOrchestrationDependencyValidationException =
await Assert.ThrowsAsync<StudentOrchestrationDependencyValidationException>(
addStudentTask.AsTask);
// then
actualStudentOrchestrationDependencyValidationException.Should()
.BeEquivalentTo(expectedStudentOrchestrationDependencyValidationException);
this.studentServiceMock.Verify(service =>
service.AddStudentAsync(It.IsAny<Student>()),
Times.Once);
this.loggingBrokerMock.Verify(broker =>
broker.LogError(It.Is(SameExceptionAs(
expectedStudentOrchestrationDependencyValidationException))),
Times.Once);
this.libraryCardServiceMock.Verify(service =>
service.AddLibraryCardAsync(It.IsAny<Guid>()),
Times.Once);
this.studentServiceMock.VerifyNoOtherCalls();
this.loggingBrokerMock.VerifyNoOtherCalls();
this.libraryCardServiceMock.VerifyNoOtherCalls();
}Above, we verify that any of our four exception types are mapped into a StudentOrchestrationDependencyValidationException. We maintain the original localized exception as an inner exception. But we unwrap the categorical exception at this level to keep the original issue as we go upstream.
These exceptions are mapped under a dependency validation exception because they originate from a dependency or a dependency of a dependency downstream. For instance, if a storage broker throws an exception, it is a dependency validation (something like DuplicateKeyException). The broker-neighboring service would map that into a localized StudentAlreadyExistException and then wrap that exception in a categorical exception of type StudentDependencyValidationException. When that exception propagates upstream to Processing or an Orchestration service, we lose the categorical exception as we have already captured it under the proper scope of mapping. Then, we continue to embed that very localized exception under the current service dependency validation exception.
Let's try to make this test pass:
public partial class StudentOrchestrationService
{
private delegate ValueTask<Student> ReturningStudentFunction();
private async ValueTask<Student> TryCatch(ReturningStudentFunction returningStudentFunction)
{
try
{
return await returningStudentFunction();
}
catch (StudentValidationException studentValidationException)
{
throw await CreateAndLogDependencyValidationExceptionAsync(studentValidationException);
}
catch (StudentDependencyValidationException studentDependencyValidationException)
{
throw await CreateAndLogDependencyValidationExceptionAsync(studentDependencyValidationException);
}
catch (LibraryCardValidationException libraryCardValidationException)
{
throw await CreateAndLogDependencyValidationExceptionAsync(libraryCardValidationException);
}
catch (LibraryCardDependencyValidationException libraryCardDependencyValidationException)
{
throw await CreateAndLogDependencyValidationExceptionAsync(libraryCardDependencyValidationException);
}
}
private async ValueTask<StudentOrchestrationDependencyValidationException>
CreateAndLogDependencyValidationExceptionAsync(Xeption exception)
{
var studentOrchestrationDependencyValidationException =
new StudentOrchestrationDependencyValidationException(
message: "Student dependency validation error occurred, fix errors and try again",
exception.innerException as Xeption);
await this.loggingBroker.LogErrorAsync(studentOrchestrationDependencyValidationException);
return studentOrchestrationDependencyValidationException;
}
}Now we can use the TryCatch as follows:
public async ValueTask<Student> AddStudentAsync(Student student) =>
TryCatch(async () =>
{
...
Student addedStudent = await this.studentService.AddStudentAsync(student);
LibraryCard libraryCard = await this.libraryCard.AddLibraryCard(addedStudent.Id);
return addedStudent;
});In the implementation, you can see that we mapped all four different types of external downstream services validation exceptions into one categorical exception and then maintained the inner exception for each one.
The same rule applies to dependency exceptions. Dependency exceptions can be both Service and Dependency exceptions from downstream services. For instance, in the above example, calling a student service may produce StudentDependencyException and StudentServiceException. These categorical exceptions will be unwrapped from their categorical layer and have their local layer wrapped in one unified new orchestration-level categorical exception under StudentOrchestrationDependencyException. The same applies to all other dependency categorical exceptions like LibraryCardDependencyException and LibraryCardServiceException.
It's crucial to unwrap and wrap localized exceptions from downstream services with categorical exceptions at the current service layer to ensure consistency with the Exposers layer. These exceptions can be easily handled and mapped into whatever the nature of the exposer component dictates. In the case of an Exposer component of type API Controller, the mapping would produce HTTP Status Codes. In the case of UI Exposer components, it would map to text meaningful to end users.
We will discuss further upstream in this Standard when to expose localized inner exceptions details where end-users are not required to take any action exclusive to dependency and service level exceptions.
Orchestration services vary depending on their position in the overall low-level architecture. For instance, an orchestration service that relies on downstream orchestration services is called a Coordination service. An Orchestration service working with multiple Coordination services as dependencies is called a Management Service. These variants are Orchestration services with uber-level business logic.
Let's take a look at the possible variants for orchestration services and where they would be positioned:
In my personal experience, I've rarely had to resolve to an Uber Management service. The limitation here in terms of dependencies and variations of orchestration-like services is to help engineers rethink the complexity of their logic. But admittedly, there are situations where complexity is an absolute necessity. Therefore, Uber Management services exist as an option.
The following table should guide the process of developing variants of orchestration services based on the level:
| Variant | Dependencies | Consumers | Complexity |
|---|---|---|---|
| Orchestration Services | Foundation or Processing Services | Coordination Services | Low |
| Coordination Services | Orchestration Services | Management Services | Medium |
| Management Services | Coordination Services | Uber Management Services | High |
| Uber Management Services | Management Services | Aggregation, Views or Exposer Components | Very High |
Working beyond Uber Management services in an orchestration manner would require a more profound discussion and a serious consideration of the overall architecture. Future versions of The Standard might be able to address this issue in what I call "The Lake House," but that is outside of the scope of this version of The Standard.
With the variations of orchestration services, I recommend staying true to the concept of unit of work. Every request can do one thing and one thing only, including its prerequisites. For instance, if you need to register a student in a school, you may also need to add a guardian, contact information, and other details. Eventing these actions can significantly decrease the complexity of the flow and lower the risk of failures in downstream services.
Here's a visualization for a complex single-threaded approach:
The solution above is a working solution for registering a student. We needed to include guardian information, library cards, classes, etc. These dependencies can be broken down into eventing, allowing other services to pick up where the single-threaded services leave off to continue the registration process. Something like this:
Above, the incoming request is turned into events, each of which would notify its orchestration services in a cul-de-sac pattern, as discussed in section 2.3.2.2. That means that a single thread is no longer responsible for the success of each dependency in the system. Instead, every event-listening broker would handle its process in a simplified way.
This approach does not guarantee an immediate response of success or failure to the requestor. It's an eventual consistency pattern where the client would get an Accepted message or its equivalent based on the communication protocol to let them know that a process has started. Still, results are only guaranteed once all event logic has been executed.
Note that we can add an extra layer of resiliency to these events by temporarily storing them in Queue-like components or memory-based temporary storages; depending on the criticality of the business.
However, an eventual consistency approach is only sometimes a good solution if the client on the other side is waiting for a response, especially in critical situations where an immediate response is required. One solution to this problem is Fire-n-Observe queues, which we will discuss in the future version of The Standard.
[*] Introduction to Orchestration Services