Getting Started with EF Process Manager: Setup, Configuration, and TipsEF Process Manager is a lightweight orchestration pattern and library that helps coordinate long-running business processes by managing state transitions and interactions between services or components. This guide walks through core concepts, installation and setup, configuration and usage patterns, error handling, testing strategies, and practical tips to help you integrate EF Process Manager into your applications.
What is an EF Process Manager?
A process manager (also called a saga in some ecosystems) coordinates a sequence of operations across multiple services or components to achieve a larger business goal. Unlike simple command handlers, a process manager keeps state between steps and reacts to events, ensuring that the overall business process completes or compensates on failure.
EF Process Manager typically uses Entity Framework (EF) or an EF-like persistence mechanism to store process state (the “process instance”), allowing the process to survive application restarts and enabling querying, auditing, and recovery.
Key responsibilities:
- Maintain state of long-running processes.
- React to incoming events and dispatch commands or actions.
- Implement compensation logic for failures.
- Provide visibility into process progress and status.
When to use a Process Manager
- Orchestrating multi-step workflows that span services or bounded contexts.
- Coordinating eventual consistency where transactions across services are not possible.
- Implementing retries, timeouts, or human approvals within a workflow.
- Handling sagas where steps may succeed or require compensating actions on failure.
Avoid adding a process manager for trivial synchronous flows or when simple domain events and eventual consistency are sufficient without central orchestration.
Installation and project setup
This section assumes a .NET project using Entity Framework Core. The EF Process Manager pattern can be implemented in different ways; below is a practical setup using a process manager library or a custom implementation storing state with EF Core.
-
Create or use an existing .NET solution (ASP.NET Core service, worker, or console app).
-
Add EF Core packages and a database provider (example uses SQL Server):
dotnet add package Microsoft.EntityFrameworkCore dotnet add package Microsoft.EntityFrameworkCore.SqlServer dotnet add package Microsoft.EntityFrameworkCore.Design -
Add any process-manager-specific package if using a third-party library (replace with the library name you choose). If implementing yourself, create a ProcessInstance entity and supporting infrastructure.
-
Configure DbContext and migrations:
- Define a DbContext that includes a DbSet
. - Add migrations and update the database.
Example ProcessInstance entity (simplified):
public class ProcessInstance { public Guid Id { get; set; } public string ProcessType { get; set; } = default!; public string State { get; set; } = default!; public string DataJson { get; set; } = "{}"; public DateTime CreatedAt { get; set; } public DateTime? UpdatedAt { get; set; } public bool IsCompleted { get; set; } } DbContext setup:
public class ProcessDbContext : DbContext { public ProcessDbContext(DbContextOptions<ProcessDbContext> options) : base(options) { } public DbSet<ProcessInstance> ProcessInstances { get; set; } = default!; } Designing the process model
A clear process model makes implementation simpler and more robust.
- Identify process boundaries: define when a process starts and when it should finish.
- Define states and transitions: map business steps to discrete states (e.g., Created → WaitingForApproval → Processing → Completed → Compensated).
- Events and commands: list the events that move the process forward and the commands/actions the manager must issue.
- Data model: decide what part of the process data you store in the process instance versus other domain stores.
Example state enum:
public enum OrderProcessState { Created, PaymentPending, PaymentConfirmed, ShippingPending, Shipped, Completed, Compensating, Failed } Implementing the process manager
Core components:
- ProcessInstance persistence (EF).
- An orchestrator/service that receives events, loads the process instance, applies transitions, and persists changes.
- A message/command dispatcher to send commands to other services or queues.
- Timeout and retry handlers (scheduling future events).
- Compensation handlers for rollback scenarios.
Flow example:
- Receive Event: OrderPlaced.
- Create ProcessInstance with state Created and initial data.
- Dispatch command: ReserveInventory.
- On InventoryReserved event: transition to PaymentPending and dispatch ProcessPayment.
- On PaymentFailed: transition to Compensating and dispatch ReleaseInventory.
Pseudo implementation snippet:
public class OrderProcessManager { private readonly ProcessDbContext _db; private readonly ICommandDispatcher _dispatcher; public async Task HandleOrderPlaced(OrderPlaced evt) { var process = new ProcessInstance { Id = Guid.NewGuid(), ProcessType = "Order", State = OrderProcessState.Created.ToString(), DataJson = JsonSerializer.Serialize(evt), CreatedAt = DateTime.UtcNow }; _db.ProcessInstances.Add(process); await _db.SaveChangesAsync(); await _dispatcher.SendAsync(new ReserveInventory { OrderId = evt.OrderId }); } public async Task HandleInventoryReserved(InventoryReserved evt) { var process = await _db.ProcessInstances .FirstOrDefaultAsync(p => /* match by order id in DataJson */ true); // update state, persist, dispatch next command... } } Concurrency and consistency
When multiple events might act on the same process instance concurrently, protect updates:
- Use optimistic concurrency with a rowversion/timestamp column in EF.
- Use pessimistic locking if supported and necessary.
- Design idempotent handlers so repeated events don’t break logic.
Example optimistic concurrency:
public class ProcessInstance { public byte[] RowVersion { get; set; } = null!; } Configure in DbContext:
modelBuilder.Entity<ProcessInstance>() .Property(p => p.RowVersion) .IsRowVersion(); Handle DbUpdateConcurrencyException to reload, reapply logic, or retry.
Timeouts, delays, and scheduled steps
Long-running processes often need time-based transitions:
- Store next action due timestamps in the process instance.
- Use a background worker that scans for due processes and triggers events.
- Integrate with a scheduler (Hangfire, Quartz) or message broker delayed messages.
Example: set NextRetryAt and have a recurring worker query for processes where NextRetryAt <= now.
Error handling and compensation
Plan for failures and compensation:
- Log errors and capture failure reasons on the process instance.
- Implement compensating actions (e.g., refund payment, release inventory).
- Mark processes as Failed or Compensating and track progress of compensation.
Keep compensation idempotent and resilient to partial failures.
Observability and administration
Provide tools to inspect and manage running processes:
- Admin UI or dashboards to query ProcessInstances by state, age, or correlation id.
- Expose endpoints to retry, cancel, or force-complete processes.
- Emit metrics (counts per state, failure rate, processing time) and structured logs.
Example useful queries:
- Processes stuck in a non-terminal state for > X minutes.
- Recently failed processes with failure reason.
Testing strategies
- Unit test state transition logic in isolation from persistence and messaging.
- Integration test with an in-memory or test database and a fake dispatcher.
- End-to-end tests using real message brokers and databases to validate real-world flows.
- Test concurrency scenarios and compensation paths explicitly.
Performance considerations
- Keep the process instance data compact; store large payloads in external storage with references.
- Index frequently queried columns (correlation id, state, next due date).
- Use batching for scanning scheduled items.
- Monitor transaction times and tune EF ChangeTracker usage (AsNoTracking for read-only queries).
Security considerations
- Validate and sanitize incoming events and data before persisting the process instance.
- Secure administrative endpoints and UI.
- Encrypt sensitive data stored in DataJson or store sensitive details in a protected secret store and reference their ids.
Practical tips and patterns
- Correlation IDs: always include a correlation id in events to locate process instances reliably.
- Idempotency: design event handlers and outgoing commands to be idempotent.
- Thin process instances: keep the persisted process record small; reference larger data.
- Clear terminal states: ensure every started process can reach a success or failure terminal state.
- Observability from day one: implement basic dashboards for state counts and stuck processes.
Example: Small end-to-end flow (summary)
- OrderPlaced event → create process instance (Created).
- Dispatch ReserveInventory → InventoryReserved event → update state (PaymentPending).
- Dispatch ProcessPayment → PaymentConfirmed → update state (ShippingPending).
- Dispatch ShipOrder → Shipped → mark process Completed.
On any failure, transition to Compensating and dispatch corresponding compensation commands.
Further reading and resources
- Saga / process manager patterns in microservices literature.
- EF Core concurrency and transaction documentation.
- Messaging and scheduling libraries for .NET (RabbitMQ, Azure Service Bus, Hangfire, Quartz).
If you want, I can provide:
- A complete sample repository layout and full code examples.
- A step-by-step tutorial integrating a specific message broker or scheduler.
- Sample SQL schema and EF migrations.
Leave a Reply