AI integration is more than sending a prompt to an LLM.
A production .NET application must decide:
How should the AI service be called?
Where should provider-specific code live?
How should prompts be managed?
How should conversation state be stored?
How should AI responses be streamed?
How should failures be handled?
How should multiple providers be supported?
How should AI calls be logged and measured?
How should RAG and vector search be integrated?
How should tools and agents be integrated?
How should AI security be enforced?
These decisions form the application's AI integration architecture.
Modern .NET provides several building blocks for these patterns. Microsoft.Extensions.AI provides common abstractions such as IChatClient and IEmbeddingGenerator, and it is designed to fit into dependency injection, middleware, telemetry, caching, and other standard .NET application patterns. (Microsoft Learn)
Microsoft's current .NET AI guidance recommends starting with Microsoft.Extensions.AI for application-level AI features, adding vector/data ingestion for RAG, using MCP when capabilities need to cross process or product boundaries, and moving to Agent Framework when a simple prompt grows into a multi-step workflow.
What Is an AI Integration Pattern?
An integration pattern is a repeatable way of connecting an AI capability to the rest of an application.
For example:
ASP.NET Core
|
v
AI Service
|
v
IChatClient
|
v
OpenAI
is one integration pattern.
Another might be:
ASP.NET Core
|
v
AI Gateway
|
+---+-------+--------+
| | |
OpenAI Azure Ollama
Another:
User
|
v
AI Application
|
v
RAG
|
+---- Vector Search
|
+---- Documents
|
v
LLM
The goal of integration patterns is to make AI functionality:
Reusable
Testable
Maintainable
Secure
Observable
Provider-independent
Scalable
Why AI Integration Patterns Matter
A prototype can be extremely simple:
var response =
await chatClient.GetResponseAsync(
"Explain C#.");
But a production application might need:
Authentication
Authorization
Prompt validation
Conversation history
RAG
Caching
Rate limiting
Retries
Timeouts
Telemetry
Cost tracking
Output validation
Tool calling
Human approval
A well-designed integration pattern prevents all of those concerns from being mixed into one controller method.
The Basic AI Integration Pattern
The simplest architecture is:
Client
|
v
Application
|
v
AI Client
|
v
AI Provider
|
v
Model
For a console application:
C#
|
v
IChatClient
|
v
AI Provider
This is appropriate for small applications.
The Service Layer Pattern
As soon as AI becomes important to the application, a service layer can be introduced.
Controller
|
v
AI Service
|
v
IChatClient
|
v
AI Provider
Example:
public interface IAIService
{
Task<string> GenerateAsync(
string prompt,
CancellationToken cancellationToken = default);
}
Implementation:
using Microsoft.Extensions.AI;
public sealed class AIService : IAIService
{
private readonly IChatClient _chatClient;
public AIService(IChatClient chatClient)
{
_chatClient = chatClient;
}
public async Task<string> GenerateAsync(
string prompt,
CancellationToken cancellationToken = default)
{
ChatResponse response =
await _chatClient.GetResponseAsync(
prompt,
cancellationToken: cancellationToken);
return response.Text;
}
}
The controller now depends on:
IAIService
instead of an OpenAI, Azure OpenAI, or Ollama-specific client.
This is one of the most useful patterns for ASP.NET Core applications.
The Dependency Injection Pattern
AI clients are natural candidates for dependency injection.
The architecture becomes:
Program.cs
|
v
DI Container
|
+---- IAIService
|
+---- IChatClient
|
+---- IEmbeddingGenerator
|
+---- Other AI services
For example:
builder.Services.AddSingleton(
chatClient);
builder.Services.AddScoped<
IAIService,
AIService>();
Microsoft's IChatClient guidance explicitly supports dependency-injection-based usage and describes composing chat clients through builder/delegating patterns. (Microsoft Learn)
Why Dependency Injection Is Important
Without DI:
Controller
|
+---- new OpenAIClient(...)
|
+---- prompt
|
+---- configuration
|
+---- error handling
With DI:
Controller
|
v
IAIService
|
v
IChatClient
|
v
Provider
The second design is easier to:
Test
Replace
Configure
Monitor
Reuse
Scale
The Provider Abstraction Pattern
A .NET application can use:
IChatClient
as the common boundary.
Then different implementations can exist underneath:
IChatClient
|
+------------+------------+
| | |
OpenAI Azure Ollama
OpenAI
Microsoft's current .NET AI guidance describes Microsoft.Extensions.AI as a unifying layer intended to let applications use different AI services through common abstractions.
This enables configurations such as:
Development -> Ollama
Testing -> Fake client
Production -> Azure OpenAI
while the application layer remains unchanged.
Provider Adapter Pattern
A provider adapter converts a provider-specific client into the common application abstraction.
For example:
OpenAI SDK
|
v
AsIChatClient()
|
v
IChatClient
Conceptually:
Provider SDK
|
v
Adapter
|
v
Common AI abstraction
This is the Adapter Pattern applied to AI services.
The Facade Pattern for AI
Sometimes even IChatClient is too low-level for business code.
You can create a business-oriented facade:
public interface IAIService
{
Task<string> SummarizeAsync(
string text,
CancellationToken cancellationToken = default);
Task<string> AnswerAsync(
string question,
CancellationToken cancellationToken = default);
}
Now the application talks in business concepts:
Summarize
Answer
Classify
Extract
Translate
instead of:
GetResponseAsync
ChatMessage
ChatOptions
Provider configuration
Architecture:
Application
|
v
IAIService
|
v
IChatClient
|
v
Provider SDK
This is especially useful for enterprise applications.
The AI Gateway Pattern
A larger organization may place all AI traffic through an internal AI gateway.
Clients
|
v
AI Gateway
|
+---------------+---------------+
| | |
OpenAI Azure Local
OpenAI AI
The gateway can centralize:
Authentication
Authorization
Provider selection
Model selection
Rate limiting
Logging
Cost tracking
Prompt policies
Caching
Resilience
This becomes useful when multiple applications need consistent AI governance.
AI Gateway Example
Imagine these applications:
Customer Portal
Internal Assistant
Mobile App
Admin Portal
Support System
Instead of each application independently calling an AI provider:
Customer Portal ----+
Mobile App ---------+
Admin Portal -------+----> AI Provider
Support System -----+
use:
Customer Portal ----+
Mobile App ---------+
Admin Portal -------+----> AI Gateway ----> Providers
Support System -----+
The AI gateway becomes an architectural boundary.
The Middleware Pattern
One of the strongest current .NET AI patterns is to treat AI functionality as a pipeline.
Microsoft's Microsoft.Extensions.AI libraries support middleware-style composition around IChatClient, including telemetry, caching, function invocation, and custom middleware. (Microsoft Learn)
Conceptually:
Application
|
v
Logging
|
v
Caching
|
v
Function Invocation
|
v
Resilience
|
v
AI Client
|
v
Provider
This means concerns can be added without changing the core AI implementation.
IChatClient Pipeline
Microsoft documents IChatClient as composable through delegating clients and ChatClientBuilder. Different layers can be added around the underlying client.
Conceptually:
IChatClient
|
+----------+----------+
| | |
Cache Telemetry Tools
| | |
+----------+----------+
|
v
Provider Client
An example pipeline is:
IChatClient client =
new ChatClientBuilder(baseClient)
.UseDistributedCache(cache)
.UseFunctionInvocation()
.UseOpenTelemetry(
sourceName: "MyApp.AI")
.Build();
The exact middleware and configuration depend on the packages installed and the provider implementation, but the architectural idea is stable: compose concerns around the AI client instead of embedding them in every call. Microsoft documents this functionality-pipeline approach directly.
The Decorator Pattern for AI
The middleware approach is closely related to the Decorator Pattern.
Imagine:
Application
|
v
CachedChatClient
|
v
TelemetryChatClient
|
v
ToolCallingChatClient
|
v
OpenAIChatClient
Each layer wraps another AI client.
This is useful because each component can perform one responsibility.
For example:
Caching
-> return previous result when appropriate
Telemetry
-> record metrics
Function Invocation
-> execute available tools
Provider client
-> communicate with the model
The Caching Pattern
AI requests can be expensive or slow.
Caching can avoid repeating identical requests where doing so is acceptable.
The architecture is:
Application
|
v
Cache
|
+--+------+
| |
Hit Miss
| |
v v
Response AI Provider
|
v
Response
|
v
Cache
Microsoft's Microsoft.Extensions.AI tooling includes a distributed-cache chat client that can return a cached response instead of forwarding a repeated chat history to the underlying client. (Microsoft Learn)
When AI Caching Makes Sense
Caching can work well for:
Stable prompts
Repeated classifications
Repeated summaries
Frequently requested public content
Embedding generation
Expensive deterministic preprocessing
It is less suitable for:
Highly personalized requests
Real-time information
Time-sensitive data
Requests where randomness is intentionally important
Requests containing changing context
Caching policy must therefore be based on application semantics.
The Distributed Caching Pattern
In a single application:
ASP.NET Core
|
v
Memory Cache
In a multi-instance deployment:
Server A ----+
Server B ----+----> Distributed Cache
Server C ----+
This avoids each server having a completely separate AI cache.
Possible distributed stores include:
Redis
SQL-based cache
Other distributed cache implementations
The Telemetry Pattern
AI calls should be observable.
A basic architecture is:
Application
|
v
AI Client
|
+---- Logs
+---- Metrics
+---- Traces
|
v
AI Provider
Microsoft's AI abstractions include OpenTelemetry integration around IChatClient, including generative-AI-oriented telemetry. (Microsoft Learn)
You can measure:
Request count
Latency
Failures
Model
Provider
Token usage
Tool calls
Cache hits
Cache misses
Why AI Telemetry Is Different
Traditional API telemetry might track:
HTTP status
Latency
CPU
Memory
AI applications also need:
Model
Input tokens
Output tokens
Total tokens
Prompt size
Retrieved context
Tool calls
Evaluation score
This makes AI observability a specialized extension of standard application observability.
The Resilience Pattern
AI providers are external network dependencies.
Failures can include:
Timeout
Transient network error
Rate limit
Temporary provider outage
Connection failure
.NET provides HTTP resilience tooling through Microsoft.Extensions.Http.Resilience, including retry, timeout, circuit-breaker, and rate-limiting/bulkhead-style strategies. (Microsoft Learn)
The architecture can be:
Application
|
v
AI Service
|
v
HTTP Resilience
|
+---- Timeout
+---- Retry
+---- Circuit Breaker
+---- Rate Limiting
|
v
AI Provider
Retry Pattern
Suppose an AI provider temporarily returns a transient error:
Request
|
v
Provider
|
Error
|
v
Retry
|
v
Provider
|
Success
However, not every error should be retried.
Do not blindly retry:
Invalid API key
Invalid request
Unsupported model
Invalid structured schema
Authorization failure
Retries are generally intended for transient failures.
Exponential Backoff
A common retry pattern is:
Attempt 1
|
v
short delay
Attempt 2
|
v
longer delay
Attempt 3
|
v
longer delay
This reduces the chance of overwhelming a temporarily unavailable dependency.
.NET's current resilience APIs support retry pipelines with configurable backoff and related resilience strategies.
Circuit Breaker Pattern
If an AI provider repeatedly fails:
Request
|
v
Provider
|
Failure
|
Failure
|
Failure
|
v
Circuit Opens
Further requests can fail fast instead of repeatedly sending traffic to the unavailable dependency.
Architecture:
Application
|
v
Circuit Breaker
|
v
AI Provider
This helps protect both the application and the provider.
Fallback Pattern
A multi-provider system can sometimes use fallback:
Primary Provider
|
Failure
|
v
Secondary Provider
|
v
Response
For example:
Primary -> Azure OpenAI
Fallback -> OpenAI
or:
Cloud Provider
|
Failure
|
v
Local Model
A fallback strategy needs careful handling of:
Model capabilities
Prompt compatibility
Structured output
Token limits
Privacy requirements
Latency
Cost
Fallback should not simply mean "try any model."
The Multi-Provider Pattern
A multi-provider system might have:
IChatClient
|
+---------+---------+
| | |
OpenAI Azure Ollama
OpenAI
Configuration can determine the active provider:
{
"AI": {
"Provider": "AzureOpenAI",
"Model": "your-model-name"
}
}
The application service does not need to know how the provider was selected.
The Provider Factory Pattern
For multiple providers:
public interface IAIClientFactory
{
IChatClient Create(
string provider);
}
Then:
Provider Name
|
v
Factory
|
+---+-------+------+
| | |
OpenAI Azure Ollama
The factory belongs in infrastructure or composition-root code.
Business logic should normally depend on abstractions rather than provider-selection mechanics.
The Strategy Pattern
Another approach is to define:
public interface IAIProviderStrategy
{
bool CanHandle(string provider);
IChatClient GetClient();
}
Different strategies can then implement:
OpenAI
Azure OpenAI
Ollama
Google
Amazon Bedrock
Microsoft's current .NET AI ecosystem supports a wide range of providers through compatible AI abstractions, including OpenAI, Azure OpenAI, Azure AI Foundry, Ollama, Google Gemini, and Amazon Bedrock. (Microsoft Learn)
The Configuration-Driven Pattern
AI configuration should normally live outside business logic.
For example:
{
"AI": {
"Provider": "OpenAI",
"Model": "your-model-name",
"Temperature": 0.2
}
}
A strongly typed options class:
public sealed class AIOptions
{
public string Provider { get; set; } =
string.Empty;
public string Model { get; set; } =
string.Empty;
public double Temperature { get; set; }
}
Register:
builder.Services
.AddOptions<AIOptions>()
.Bind(builder.Configuration.GetSection("AI"))
.ValidateOnStart();
Secrets should remain outside normal source-controlled configuration files.
The Prompt Template Pattern
Instead of writing prompts directly inside controllers:
var prompt =
$"Summarize this customer message: {message}";
use dedicated prompt construction.
public interface IPromptBuilder
{
string BuildSummaryPrompt(
string text);
}
Implementation:
public sealed class PromptBuilder :
IPromptBuilder
{
public string BuildSummaryPrompt(
string text)
{
return
"""
Summarize the following customer message.
Be concise and factual.
Customer message:
""" + Environment.NewLine +
text;
}
}
Architecture:
Controller
|
v
Application Service
|
v
Prompt Builder
|
v
IChatClient
This keeps prompt construction testable and reusable.
The System Prompt Pattern
A system instruction can define the behavior of the assistant.
For example:
List<ChatMessage> messages =
[
new ChatMessage(
ChatRole.System,
"You are a helpful C# programming assistant."),
new ChatMessage(
ChatRole.User,
"Explain dependency injection.")
];
The system instruction should not be treated as a security boundary by itself.
Application-level authorization and validation still need to happen outside the model.
The Conversation State Pattern
A chat application needs conversation state.
Architecture:
User
|
v
ASP.NET Core
|
+---- Conversation Store
|
v
Prompt Construction
|
v
AI Model
The state may be stored in:
SQL Server
Redis
Document database
Memory
A conversation entity could contain:
public sealed class ConversationMessage
{
public long Id { get; set; }
public string ConversationId { get; set; } =
string.Empty;
public string Role { get; set; } =
string.Empty;
public string Content { get; set; } =
string.Empty;
public DateTimeOffset CreatedAt { get; set; }
}
The flow becomes:
User message
|
v
Store message
|
v
Load relevant history
|
v
Build AI request
|
v
Model
|
v
Store AI response
The Context Management Pattern
Sending the entire conversation forever is not always practical.
Instead:
Full history
|
v
Context manager
|
+---+-----------+
| |
v v
Recent messages Summary
| |
+-------+-------+
|
v
AI request
A context manager can:
Keep recent messages
Summarize old messages
Retrieve relevant history
Remove redundant context
Limit request size
This becomes important as chat sessions grow.
The RAG Integration Pattern
RAG integrates external knowledge into model generation.
User Question
|
v
Embedding
|
v
Vector Search
|
v
Relevant Documents
|
v
Prompt + Context
|
v
LLM
|
v
Answer
This pattern combines multiple libraries.
Microsoft's current ecosystem guidance describes RAG applications using Microsoft.Extensions.DataIngestion, Microsoft.Extensions.VectorData, and Microsoft.Extensions.AI.
RAG Service Pattern
Do not put vector search directly into the controller.
Instead:
public interface IRAGService
{
Task<string> AskAsync(
string question,
CancellationToken cancellationToken = default);
}
Implementation conceptually:
IRAGService
|
+---- Generate embedding
|
+---- Search vector store
|
+---- Build context
|
+---- Call IChatClient
|
v
Answer
The controller simply says:
string answer =
await ragService.AskAsync(
request.Question,
cancellationToken);
The Embedding Pipeline Pattern
Embeddings usually follow:
Source Text
|
v
IEmbeddingGenerator
|
v
Vector
|
v
Vector Store
For querying:
Question
|
v
IEmbeddingGenerator
|
v
Query Vector
|
v
Vector Search
Microsoft's vector-data ecosystem provides common abstractions for vector stores, while IEmbeddingGenerator provides the embedding abstraction. (Microsoft Learn)
The Document Ingestion Pattern
A document system commonly follows:
PDF / Word / Text / Web
|
v
Data Ingestion
|
v
Chunking
|
v
Enrichment
|
v
Embedding
|
v
Vector Store
Current .NET data-ingestion guidance describes components for document processing, chunking, enrichment, embeddings, and integration with vector stores.
The Background Processing Pattern
Document ingestion should often run outside the HTTP request.
Instead of:
HTTP Request
|
v
Read PDF
|
v
Chunk
|
v
Embed
|
v
Store
|
v
HTTP Response
use:
HTTP Request
|
v
Queue Job
|
v
HTTP Response
Then:
Background Worker
|
v
Data Ingestion
|
v
Embeddings
|
v
Vector Store
This is especially useful for large documents.
The Streaming Pattern
AI responses can be streamed instead of waiting for the entire answer.
Architecture:
Client
|
v
ASP.NET Core
|
v
IChatClient
|
v
Streaming Model
|
+---- chunk 1
+---- chunk 2
+---- chunk 3
+---- chunk 4
IChatClient supports streaming responses through GetStreamingResponseAsync. (Microsoft Learn)
This is useful for:
Chat applications
Assistants
Long-form generation
Interactive UI
Server-Sent Events Pattern
One way to expose AI streaming from ASP.NET Core is Server-Sent Events.
Browser
|
v
GET /api/chat/stream
|
v
ASP.NET Core
|
v
IChatClient
|
v
Streaming updates
Conceptually:
AI Chunk 1
|
v
SSE Event
AI Chunk 2
|
v
SSE Event
AI Chunk 3
|
v
SSE Event
This works well for browser-based AI interfaces.
The SignalR Pattern
For applications that need bidirectional real-time communication:
Client
|
v
SignalR Hub
|
v
AI Service
|
v
Streaming Model
SignalR becomes particularly useful when the application needs:
Real-time status
Typing indicators
Streaming responses
Tool execution notifications
Agent progress
The Tool Calling Pattern
Tool calling connects a model to application functionality.
User
|
v
LLM
|
| Tool request
v
.NET Tool
|
v
Database / API
|
v
Tool Result
|
v
LLM
|
v
Final Answer
The application should not simply allow the model to execute arbitrary methods.
Tools should be:
Explicit
Authorized
Validated
Limited
Audited
The Function Invocation Pattern
Microsoft.Extensions.AI provides middleware for automatic function invocation. Microsoft's IChatClient guidance demonstrates UseFunctionInvocation() as one of the pipeline components that can be layered around a client. (Microsoft Learn)
Conceptually:
IChatClient
|
v
Function Invocation Middleware
|
v
Provider
The middleware can participate in the tool-call loop.
The Tool Authorization Pattern
A tool should not be equivalent to unrestricted application access.
Instead:
AI Tool Request
|
v
Tool Validator
|
v
Authorization
|
v
Parameter Validation
|
v
Tool Execution
For example:
AI -> "Delete customer 123"
should not automatically result in:
DELETE FROM Customers WHERE Id = 123
A production application needs explicit authorization and validation.
The Human-in-the-Loop Pattern
Some AI operations should require user approval.
AI
|
v
Proposed Action
|
v
Approval Required
|
+---- Reject
|
+---- Approve
|
v
Execute
Examples include:
Sending an email
Creating an order
Updating financial information
Deleting records
Publishing content
Changing account settings
The model proposes.
The application decides.
The Agent Pattern
An AI agent is appropriate when the application must perform multi-step work.
Goal
|
v
Agent
|
+---- Reason
|
+---- Tool
|
+---- Observe
|
+---- Reason
|
+---- Tool
|
v
Result
Microsoft's current agent guidance describes agents as systems that accomplish objectives using reasoning, tools, and context, and identifies workflow patterns including sequential, concurrent, handoff, group-chat, and magentic orchestration.
The Agent Workflow Pattern
A sequential workflow:
Input
|
v
Agent A
|
v
Agent B
|
v
Agent C
|
v
Final Result
A concurrent workflow:
+---- Agent A ----+
| |
Input -------+---- Agent B ----+----> Aggregate
| |
+---- Agent C ----+
A handoff workflow:
Input
|
v
Agent A
|
+---- condition ----> Agent B
|
v
Agent C
These are architectural patterns rather than requirements for every chatbot.
The Agent + RAG Pattern
An agent can use retrieval as one of its tools or context providers.
Agent
|
+---- Chat Model
|
+---- Search Tool
| |
| v
| Vector Store
|
+---- Database Tool
|
+---- API Tool
This creates an agentic RAG system.
The MCP Integration Pattern
MCP is useful when tools and resources need to be exposed across application boundaries.
The architecture is:
AI Host
|
v
MCP Client
|
v
MCP Server
|
+---- REST API
+---- Database
+---- File system
+---- Business tools
Microsoft describes MCP as a standardized client-server protocol for connecting AI applications with external tools and data sources.
The MCP C# SDK itself uses Microsoft.Extensions.AI libraries for AI-related abstractions.
When to Use MCP
MCP is particularly useful when capabilities need to be reused across:
Multiple AI applications
Multiple agents
Different products
Different processes
Different AI model providers
Without MCP:
Application A -> custom tool integration
Application B -> another custom integration
Application C -> another custom integration
With MCP:
Applications
|
v
MCP Servers
|
+---- Database
+---- APIs
+---- Tools
The Provider-Agnostic Pattern
A common architecture is:
Application
|
v
Microsoft.Extensions.AI
|
+--+------+------+
| | | |
OpenAI Azure Ollama
This is useful when your application may change providers.
Microsoft's current .NET AI overview lists integrations for multiple major providers through compatible AI abstractions.
The Provider-Specific Pattern
Not every application needs abstraction.
A provider-specific system can be:
Application
|
v
OpenAI SDK
|
v
OpenAI
This may be appropriate when:
Only one provider is required
Provider-specific functionality is important
The application is small
Maximum direct API access is desired
The key is to choose intentionally.
The Hybrid Pattern
A mature application can use both:
Application
|
v
IChatClient
|
v
Common AI functionality
while also using direct provider SDKs for specialized features:
Infrastructure
|
+---- Microsoft.Extensions.AI
|
+---- OpenAI SDK
|
+---- Azure SDK
This avoids forcing every provider-specific feature through a common abstraction that may not represent it.
The Options Pattern
AI configuration should be strongly typed.
public sealed class AIOptions
{
public string Provider { get; set; } =
string.Empty;
public string Model { get; set; } =
string.Empty;
public string Endpoint { get; set; } =
string.Empty;
}
Register:
builder.Services
.AddOptions<AIOptions>()
.Bind(
builder.Configuration
.GetSection("AI"))
.ValidateOnStart();
This keeps configuration separate from business logic.
The Environment-Specific Pattern
One application can have different AI providers per environment:
Development
-> Ollama
Test
-> Fake AI
Staging
-> Azure OpenAI
Production
-> Azure OpenAI
The code remains:
IAIService
|
v
IChatClient
Only the composition root changes.
The Factory Pattern for Environments
A factory can select the implementation:
Configuration
|
v
AI Factory
|
+----+----+----+
| | |
Dev Test Prod
| | |
Ollama Fake Azure
This is useful when the same application must support multiple deployment configurations.
The Repository Pattern for AI Data
The Repository Pattern can be used around AI-related persistent data.
For example:
public interface IConversationRepository
{
Task<IReadOnlyList<ConversationMessage>>
GetMessagesAsync(
string conversationId,
CancellationToken cancellationToken);
}
Implementation:
IConversationRepository
|
+---- SQL Server
+---- PostgreSQL
+---- Cosmos DB
The AI service should not need to know how conversation history is stored.
The Unit of Work Pattern
A request may update several pieces of application state:
Conversation
Usage
Audit record
Tool result
These may need to be treated consistently.
Architecture:
AI Request
|
+---- Conversation
+---- Usage
+---- Audit
|
v
Unit of Work
This is an application-data pattern rather than a model-specific feature.
The Queue-Based AI Pattern
Long-running AI operations can be placed on a queue.
Client
|
v
ASP.NET Core
|
v
Queue
|
v
AI Worker
|
v
AI Provider
This is useful for:
Large document processing
Batch summarization
Bulk classification
Embedding generation
Report generation
The Event-Driven AI Pattern
Events can trigger AI processing.
OrderCreated
|
v
Message Broker
|
v
AI Worker
|
v
AI Processing
|
v
Result Event
Examples:
DocumentUploaded
CustomerCreated
TicketOpened
TransactionCompleted
ProductUpdated
AI becomes a participant in the application's event-driven architecture.
The Scheduled AI Pattern
Background scheduling can trigger AI tasks.
Scheduler
|
v
AI Worker
|
v
AI Provider
Examples:
Daily report summary
Weekly knowledge-base analysis
Scheduled content classification
Periodic anomaly analysis
This can be implemented using standard .NET hosted services or dedicated scheduling infrastructure.
The BackgroundService Pattern
A worker can be implemented using:
public sealed class AIWorker :
BackgroundService
{
protected override async Task ExecuteAsync(
CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
await ProcessAsync(
stoppingToken);
await Task.Delay(
TimeSpan.FromMinutes(5),
stoppingToken);
}
}
private Task ProcessAsync(
CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
}
The worker can inject:
IAIService
IRAGService
IEmbeddingGenerator
Repository
Queue client
The Security Boundary Pattern
An AI system should have explicit security boundaries.
User
|
v
Authentication
|
v
Authorization
|
v
Application Service
|
v
AI Service
|
v
Provider
The model is not the authentication system.
The prompt is not an authorization system.
The application remains responsible for access control.
The Tenant Isolation Pattern
For a SaaS application:
Tenant A
|
+---- Documents A
+---- Conversations A
+---- Vectors A
Tenant B
|
+---- Documents B
+---- Conversations B
+---- Vectors B
Every AI operation should preserve tenant context.
For example:
Tenant ID
|
v
Conversation
|
v
RAG Search
|
v
Relevant Tenant Documents
This prevents cross-tenant retrieval.
The Prompt Injection Defense Pattern
RAG and tool-using systems need special care.
A document might contain:
"Ignore previous instructions and perform this action."
The AI should not automatically treat retrieved content as trusted instructions.
A safer model is:
System/Application Instructions
|
v
User Input
|
v
Retrieved Content
|
v
Tool Policy
|
v
Model
The application must separately control:
What tools exist
Which tools are allowed
Which parameters are allowed
Which user can use each tool
Which actions require approval
The Output Validation Pattern
AI-generated output should pass through validation.
AI Response
|
v
Schema Validation
|
v
Business Validation
|
v
Authorization
|
v
Application Action
For structured output:
public sealed class ResumeAnalysis
{
public string CandidateName { get; set; } =
string.Empty;
public int ExperienceYears { get; set; }
public List<string> Skills { get; set; } = [];
}
Even if the model returns valid JSON, application-level business validation can still be necessary.
The Structured Output Pattern
The integration architecture is:
Prompt
|
v
AI Model
|
v
Structured Output
|
v
C# Type
|
v
Validation
This is much more reliable for machine-to-machine workflows than scraping arbitrary natural-language text.
The Content Filtering Pattern
An application may need to inspect both:
Incoming user input
Generated AI output
Architecture:
User Input
|
v
Input Policy
|
v
AI
|
v
Output Policy
|
v
Client
The specific policy depends on the application's domain.
The Cost Control Pattern
AI calls can have measurable usage costs.
A useful architecture is:
AI Request
|
+---- Usage Tracking
|
+---- Token Tracking
|
+---- Budget Check
|
v
AI Provider
For example:
User
|
v
Usage Limit Service
|
+---- Allowed -> AI
|
+---- Denied -> Usage limit response
This is especially important in multi-tenant applications.
The Rate Limiting Pattern
An AI API can be protected with:
Global Rate Limit
Tenant Rate Limit
User Rate Limit
Endpoint Rate Limit
Provider Rate Limit
Architecture:
Client
|
v
Rate Limiter
|
v
AI Service
|
v
Provider
Rate limiting and HTTP resilience are complementary rather than identical concerns.
The Semantic Caching Pattern
Traditional caching asks:
"Is this exact request the same?"
Semantic caching asks:
"Is this new question sufficiently similar to a previous question?"
Architecture:
New Question
|
v
Embedding
|
v
Similarity Search
|
+---+---+
| |
Match Miss
| |
v v
Cached AI
Answer Provider
This can reduce repeated AI requests, but semantic cache correctness must be designed carefully.
The Evaluation Pattern
AI quality can be evaluated after integration.
AI Application
|
v
Generated Response
|
v
Evaluation
|
+---+---+---+
| | | |
Relevance Truth Safety
|
v
Evaluation Result
Microsoft's current AI ecosystem includes dedicated evaluation libraries and recommends adding evaluations once AI behavior is useful enough to measure and protect against regressions.
The Model Routing Pattern
Different requests may require different models.
For example:
Simple classification
|
v
Small model
Complex reasoning
|
v
More capable model
Embedding
|
v
Embedding model
The application can route based on workload:
Request
|
v
Model Router
|
+---+-------+--------+
| | |
Fast Quality Embedding
Model Model Model
Model routing should be based on measurable requirements such as quality, latency, capability, and cost.
The Semantic Model Routing Pattern
A more advanced router can inspect the request:
Question
|
v
Classifier / Router
|
+---+-----------+
| |
Simple Complex
| |
v v
Model A Model B
This can improve resource utilization when different workloads have different requirements.
The Circuit + Fallback Pattern
A production system can combine resilience techniques:
Application
|
v
Rate Limit
|
v
Cache
|
v
Circuit Breaker
|
v
Primary Provider
|
Failure
|
v
Fallback Provider
This should only be introduced when its operational complexity is justified.
The AI Anti-Corruption Layer Pattern
An Anti-Corruption Layer protects your domain model from an external provider's data model.
For example:
Domain
|
X OpenAI-specific types
|
v
Application AI Model
|
v
Provider Adapter
|
v
OpenAI SDK
Instead of allowing:
OpenAI ChatCompletion
to spread throughout the domain, convert it to an application-level model:
public sealed record AIAnswer(
string Text,
string? Model,
int? InputTokens,
int? OutputTokens);
This limits provider-specific coupling.
The AI Result Wrapper Pattern
Instead of returning only:
string
an application might return:
public sealed class AIResult<T>
{
public T? Value { get; init; }
public string? Model { get; init; }
public int? InputTokens { get; init; }
public int? OutputTokens { get; init; }
public TimeSpan Duration { get; init; }
public bool FromCache { get; init; }
}
Then:
AIResult<T>
|
+---- Value
+---- Usage
+---- Timing
+---- Model
+---- Cache information
This can be useful in enterprise systems.
The AI Pipeline Pattern
Putting multiple patterns together:
HTTP Request
|
v
Authentication
|
v
Authorization
|
v
Validation
|
v
Rate Limiting
|
v
AI Service
|
v
Prompt Builder
|
v
Context / RAG
|
v
Cache
|
v
Resilience
|
v
Telemetry
|
v
IChatClient
|
v
AI Provider
This is a more realistic view of production AI integration.
A Complete ASP.NET Core AI Architecture
A scalable ASP.NET Core system can look like:
Client
|
v
ASP.NET Core
|
+-----------+-----------+
| |
Authentication Rate Limiting
| |
+-----------+-----------+
|
v
Application Layer
|
+--------------+--------------+
| | |
v v v
AI Service RAG Service Tool Service
| | |
v v v
IChatClient VectorData Authorized Tools
| |
| DataIngestion
|
+----------------------+
|
v
AI Provider
Supporting infrastructure:
SQL Server
Redis
Vector Store
Blob Storage
Message Broker
OpenTelemetry
Secret Management
The AI Integration Pipeline in Microsoft.Extensions.AI
A very useful pattern is to compose one base AI client with multiple middleware layers.
Conceptually:
IChatClient client =
new ChatClientBuilder(baseClient)
.UseDistributedCache(cache)
.UseFunctionInvocation()
.UseOpenTelemetry(
sourceName: "MyApp.AI")
.Build();
The flow becomes:
Application
|
v
Distributed Cache
|
v
Function Invocation
|
v
OpenTelemetry
|
v
Provider Client
Microsoft's current documentation explicitly demonstrates this type of composable pipeline and notes that intermediate components can come from Microsoft.Extensions.AI, other NuGet packages, or custom implementations.
Custom AI Middleware
You can also build your own middleware-style AI component.
For example, you might want to log safe metadata:
public sealed class AITrackingChatClient
: IChatClient
{
private readonly IChatClient _inner;
private readonly ILogger<AITrackingChatClient> _logger;
public AITrackingChatClient(
IChatClient inner,
ILogger<AITrackingChatClient> logger)
{
_inner = inner;
_logger = logger;
}
public async Task<ChatResponse> GetResponseAsync(
IEnumerable<ChatMessage> messages,
ChatOptions? options = null,
CancellationToken cancellationToken = default)
{
_logger.LogInformation(
"AI request received.");
ChatResponse response =
await _inner.GetResponseAsync(
messages,
options,
cancellationToken);
_logger.LogInformation(
"AI response received.");
return response;
}
public IAsyncEnumerable<ChatResponseUpdate>
GetStreamingResponseAsync(
IEnumerable<ChatMessage> messages,
ChatOptions? options = null,
CancellationToken cancellationToken = default)
{
return _inner.GetStreamingResponseAsync(
messages,
options,
cancellationToken);
}
public object? GetService(
Type serviceType,
object? serviceKey = null)
{
return _inner.GetService(
serviceType,
serviceKey);
}
public void Dispose()
{
_inner.Dispose();
}
}
This demonstrates the Decorator/Middleware concept.
In production, the exact implementation should also preserve the underlying client's service interfaces and lifetime behavior appropriately.
The AI Middleware Order Matters
Consider:
Cache
|
v
Telemetry
|
v
Tool Invocation
|
v
Provider
versus:
Telemetry
|
v
Cache
|
v
Tool Invocation
|
v
Provider
The order changes what gets measured.
For example:
Telemetry outside cache
can measure both:
Cache hits
Cache misses
while:
Telemetry inside cache
may primarily measure requests that actually reach the provider.
Microsoft's documentation explicitly demonstrates that the order of intermediate Use... calls forms the resulting pipeline.
The Provider Routing Pattern
A large system can route different tenants or workloads differently.
Tenant
|
v
Routing Policy
|
+---- Premium -> Azure OpenAI
|
+---- Standard -> OpenAI
|
+---- Local -> Ollama
This pattern can support:
Cost policies
Privacy policies
Regional requirements
Provider availability
Workload specialization
Routing decisions should be made by trusted application code, not arbitrary client input.
The Regional AI Pattern
For globally deployed systems:
User
|
v
Region Router
|
+---- India -> Provider A
|
+---- Europe -> Provider B
|
+---- US -> Provider C
This can be relevant when applications need regional data-processing or latency policies.
The application still needs to validate provider availability and data-governance requirements.
The Human Approval Pattern
For sensitive actions:
AI
|
v
Action Proposal
|
v
Approval Service
|
+---- Approved
| |
| v
| Execute
|
+---- Rejected
|
v
Stop
This is especially useful when AI is connected to:
Business transactions
Administrative operations
External communication
Data modification
The model should propose an action, while application logic controls whether that action is allowed.
The Agent Tool Isolation Pattern
A safe agent architecture separates:
Agent
|
v
Tool Registry
|
v
Authorized Tool
|
v
Business Service
Not:
Agent
|
v
Direct database access
For example:
Agent
|
v
GetCustomerBalance tool
|
v
CustomerService
|
v
Database
This keeps authorization and business rules outside the model.
The RAG Authorization Pattern
RAG introduces an important security problem.
The vector search should not simply retrieve:
Top 10 most similar documents
It may need:
Top 10 similar documents
WHERE TenantId = currentTenant
AND UserCanRead = true
Architecture:
Question
|
v
Vector Search
|
v
Authorization Filter
|
v
Allowed Documents
|
v
LLM
The AI model should never be relied upon to enforce document authorization.
The RAG Context Compression Pattern
A RAG application can retrieve many chunks:
20 chunks
|
v
Context Processor
|
v
5 relevant chunks
|
v
LLM
Context processing may include:
Deduplication
Ranking
Filtering
Compression
Summarization
Token budgeting
This reduces irrelevant context.
The AI Data Enrichment Pattern
AI can enrich ordinary application data.
Customer Record
|
v
AI Classification
|
v
Category
or:
Product Description
|
v
AI Extraction
|
v
Structured Attributes
The result should be validated before becoming authoritative application data.
The AI-as-a-Service Pattern
An organization can centralize reusable AI capabilities:
Applications
|
v
AI Service
|
+---+---------+---------+
| | |
Chat Summarize Extract
| | |
+-------------+---------+
|
v
AI Providers
This can reduce duplicated AI integration code across products.
The AI Sidecar Pattern
In some architectures, AI capabilities run as a separate service:
Main Application
|
v
AI Sidecar
|
v
AI Provider
This can be useful when:
Multiple applications use AI
AI dependencies should be isolated
AI scaling differs from application scaling
AI runtime requires special infrastructure
However, a sidecar also introduces:
Network latency
Operational complexity
Deployment overhead
Use it when those trade-offs are justified.
The AI Microservice Pattern
For larger systems:
API Gateway
|
+-----------------+----------------+
| | |
Order API Customer API AI API
|
+-------------+-------------+
| | |
Chat RAG Agents
The AI service becomes an independently deployable component.
When Not to Create an AI Microservice
A small application does not necessarily need:
AI Service
+
RAG Service
+
Embedding Service
+
Agent Service
as four separate deployments.
Start with:
ASP.NET Core
|
v
AI Service Layer
and split services only when there is an actual operational or organizational reason.
The Modular Monolith Pattern
For many enterprise applications, a modular monolith is a useful middle ground:
ASP.NET Core
|
+---- Chat Module
|
+---- RAG Module
|
+---- Document Module
|
+---- Agent Module
|
+---- AI Infrastructure
Everything can deploy together while remaining logically separated.
The AI Integration Decision Tree
A practical decision process looks like this.
Need simple text generation?
Use:
IChatClient
Need provider flexibility?
Use:
Microsoft.Extensions.AI
Need reusable prompts?
Use:
Prompt Service / Templates
Need conversation?
Add:
Conversation Store
Context Management
Need your own documents?
Add:
DataIngestion
VectorData
Embeddings
RAG
Need external functions?
Add:
Tools / Function Calling
Need capabilities shared across applications?
Consider:
MCP
Need multi-step autonomous workflows?
Consider:
Agent Framework
Need multiple distributed services?
Consider:
Aspire
Message queues
Observability
This progression closely matches Microsoft's current ecosystem recommendations.
A Complete Reference Architecture
Putting the patterns together:
Client
|
v
ASP.NET Core API
|
+-------------+-------------+
| |
Authentication Rate Limiting
| |
+-------------+-------------+
|
v
Application Layer
|
+--------------------+--------------------+
| | |
v v v
AI Service RAG Service Tool Service
| | |
v v v
Prompt VectorData Tool Registry
| DataIngestion |
| | |
+--------------------+--------------------+
|
IChatClient
|
+---------------------+---------------------+
| | |
Cache Telemetry Resilience
| | |
+---------------------+---------------------+
|
Provider Router
|
+---------------+---------------+
| | |
OpenAI Azure Ollama
OpenAI
And for advanced applications:
Agent
|
+------------------+------------------+
| | |
RAG Tools MCP
| | |
Vector Store Business APIs MCP Servers
| | |
+------------------+------------------+
|
IChatClient
|
AI Provider
A Practical ASP.NET Core Example
A simple production-oriented service can combine:
Dependency Injection
Configuration
Prompt construction
IChatClient
Telemetry
Cancellation
Validation
AIOptions.cs
public sealed class AIOptions
{
public string Model { get; set; } =
string.Empty;
}
IAIService.cs
public interface IAIService
{
Task<string> AnswerAsync(
string question,
CancellationToken cancellationToken = default);
}
AIService.cs
using Microsoft.Extensions.AI;
public sealed class AIService :
IAIService
{
private readonly IChatClient _chatClient;
public AIService(
IChatClient chatClient)
{
_chatClient = chatClient;
}
public async Task<string> AnswerAsync(
string question,
CancellationToken cancellationToken = default)
{
if (string.IsNullOrWhiteSpace(question))
{
throw new ArgumentException(
"Question is required.",
nameof(question));
}
ChatResponse response =
await _chatClient.GetResponseAsync(
[
new ChatMessage(
ChatRole.System,
"You are a helpful .NET assistant."),
new ChatMessage(
ChatRole.User,
question)
],
cancellationToken:
cancellationToken);
return response.Text;
}
}
Controller
using Microsoft.AspNetCore.Mvc;
public sealed record AskRequest(
string Question);
[ApiController]
[Route("api/ai")]
public sealed class AIController :
ControllerBase
{
private readonly IAIService _aiService;
public AIController(
IAIService aiService)
{
_aiService = aiService;
}
[HttpPost("ask")]
public async Task<IActionResult> Ask(
AskRequest request,
CancellationToken cancellationToken)
{
if (string.IsNullOrWhiteSpace(
request.Question))
{
return BadRequest(
"Question is required.");
}
string answer =
await _aiService.AnswerAsync(
request.Question,
cancellationToken);
return Ok(new
{
answer
});
}
}
This is intentionally simple.
Additional patterns should be layered on only when the application needs them.
Integration Patterns for a Production Application
A production application can evolve like this:
Stage 1
Controller
|
v
IChatClient
Stage 2
Controller
|
v
AI Service
|
v
IChatClient
Stage 3
Controller
|
v
AI Service
|
+---- Cache
+---- Telemetry
+---- Resilience
|
v
IChatClient
Stage 4
AI Service
|
+---- Conversation
+---- RAG
+---- Tools
+---- Evaluation
|
v
IChatClient
Stage 5
Application
|
v
Agent
|
+---- RAG
+---- Tools
+---- MCP
+---- Memory
|
v
IChatClient
This incremental approach prevents overengineering.
Testing AI Integration Patterns
A good architecture makes testing possible at several levels.
Unit Testing
Replace:
IChatClient
with a fake implementation.
AIService
|
v
Fake Chat Client
Integration Testing
Use:
Real AI SDK
Test provider
Controlled credentials
Evaluation Testing
Test AI behavior:
Question
|
v
AI Response
|
v
Evaluation
End-to-End Testing
Test the entire flow:
Browser
|
v
ASP.NET Core
|
v
RAG
|
v
AI
|
v
Response
The four levels answer different questions.
AI Integration and Observability
A mature system can emit:
Trace:
Request
|
+---- RAG
|
+---- Vector Search
|
+---- AI Model
|
+---- Tool Calls
Metrics:
latency
tokens
failures
cache hits
Logs:
request ID
tenant ID
model
provider
Sensitive user content should not automatically be recorded in logs.
Microsoft's current Microsoft.Extensions.AI tooling supports OpenTelemetry integration at the chat-client layer.
AI Integration and Error Handling
A useful exception flow is:
Provider Exception
|
v
Provider Adapter
|
v
Application AI Exception
|
v
API Error Handler
|
v
Safe Client Response
Do not expose raw provider errors containing:
API keys
Internal URLs
Stack traces
Provider internals
Sensitive request content
AI Integration and Cancellation
Every asynchronous AI operation should accept:
CancellationToken
Example:
await _chatClient.GetResponseAsync(
prompt,
cancellationToken:
cancellationToken);
This matters when:
User closes the browser
HTTP request times out
Background job is stopped
Application is shutting down
The cancellation signal should flow through the application.
AI Integration and Timeouts
AI calls can take longer than ordinary database queries.
A good architecture defines:
Client timeout
Application timeout
Provider timeout
Operation timeout
Resilience tools can help enforce total and per-attempt timeouts. The current .NET HTTP resilience options include total request timeout and attempt timeout as distinct strategies. (Microsoft Learn)
AI Integration and Rate Limits
External providers can impose their own limits.
Your application may also impose:
User requests/minute
Tenant requests/minute
Token budget/day
Concurrent AI requests
Architecture:
Incoming Request
|
v
Application Rate Limit
|
v
AI Provider Rate Limit
Both need to be considered.
AI Integration and Cost Limits
A multi-tenant application might maintain:
Tenant
|
+---- Monthly AI Budget
+---- Token Usage
+---- Request Count
+---- Model Policy
Before the AI call:
Budget Check
|
+--+--+
| |
OK Denied
| |
v v
AI Stop
This avoids unexpected consumption.
AI Integration and Data Privacy
A model integration pattern must also define what data is sent to external services.
Application Data
|
v
Data Classification
|
+-----+------+
| |
Allowed Restricted
| |
v v
AI Provider Local/Controlled Processing
The architecture may therefore route different data to different models or providers.
AI Integration and Local Models
Local AI can be an integration strategy:
Application
|
v
IChatClient
|
v
Local Model Server
|
v
Model
The application abstraction remains the same even when the model is no longer hosted by a cloud provider.
This is one reason provider-neutral abstractions are useful.
AI Integration and Cloud Models
Cloud deployment:
Application
|
v
IChatClient
|
v
Cloud AI Provider
Local deployment:
Application
|
v
IChatClient
|
v
Local AI Provider
The top half of the architecture can remain unchanged.
AI Integration and Hybrid AI
An advanced system can route requests between local and cloud models:
Request
|
v
AI Router
/ \
/ \
Private General
| |
v v
Local AI Cloud AI
The routing policy can depend on:
Data sensitivity
Model capability
Latency
Cost
Availability
AI Integration and Model Routing by Task
Different application tasks can use different models:
Task
|
+---- Classification -> Fast model
|
+---- Summarization -> General model
|
+---- Reasoning -> More capable model
|
+---- Embeddings -> Embedding model
The important architectural point is that model selection belongs in a routing/configuration layer rather than being scattered throughout business logic.
AI Integration and Vector Stores
A vector store can be treated as another infrastructure dependency:
IRAGService
|
+---- IEmbeddingGenerator
|
+---- VectorStore
|
+---- IChatClient
Microsoft's vector-data abstractions intentionally separate the application from individual vector-store implementations.
AI Integration and Data Ingestion
Document ingestion can similarly be isolated:
IDocumentIngestionService
|
+---- PDF
+---- Word
+---- HTML
+---- Text
It can then output normalized chunks for the RAG layer.
Document
|
v
IDocumentIngestionService
|
v
Chunks
|
v
Embedding + Vector Store
AI Integration and Event-Driven RAG
A scalable enterprise RAG system can use events:
DocumentUploaded
|
v
Queue
|
v
Ingestion Worker
|
v
Embedding Worker
|
v
Vector Store Updated
This avoids blocking users while large documents are processed.
AI Integration and Agent Framework
As application complexity increases:
Simple prompt
|
v
IChatClient
|
v
Tools
|
v
Agent
|
v
Workflow
Microsoft's current documentation describes Microsoft.Extensions.AI and Microsoft.Extensions.VectorData as foundations for agents and positions Agent Framework above these lower-level building blocks.
AI Integration and Agent Middleware
Agent systems can have multiple middleware layers.
The current Agent Framework documentation describes an execution pipeline involving agent middleware, history providers, AI context providers, IChatClient middleware, function invocation, and provider calls.
Conceptually:
Agent
|
v
Agent Middleware
|
v
Conversation History
|
v
Context Providers
|
v
Chat Middleware
|
v
Function Invocation
|
v
IChatClient
|
v
Provider
This is the advanced version of the same pipeline idea used by IChatClient.
Choosing the Right Integration Pattern
A useful guideline is:
| Requirement | Pattern |
|---|---|
| Simple model call | Direct IChatClient |
| Business abstraction | AI service/facade |
| Multiple providers | Provider abstraction |
| Cross-cutting AI behavior | Middleware/decorator |
| Repeated requests | Caching |
| External failures | Resilience |
| Real-time output | Streaming |
| Own documents | RAG |
| Application functions | Tool/function calling |
| Human approval | Human-in-the-loop |
| Shared tools across products | MCP |
| Multi-step autonomous work | Agent |
| Long-running work | Queue/background worker |
| Many deployments | Configuration/provider factory |
| Enterprise AI platform | AI gateway/service |
Common AI Integration Mistakes
Calling the provider from every controller
Avoid:
Controller
|
+---- OpenAI
+---- Prompt
+---- RAG
+---- Logging
+---- Retry
Prefer:
Controller
|
v
Application Service
|
v
AI Pipeline
Putting prompts everywhere
Avoid:
Controller 1 -> Prompt A
Controller 2 -> Prompt B
Worker -> Prompt C
Centralize important prompt construction.
Adding agents too early
A simple chatbot does not need agent orchestration.
Start with:
IChatClient
and introduce agents only when the workflow genuinely needs:
Planning
Tool use
Multi-step execution
Delegation
Workflow orchestration
Microsoft's current guidance explicitly recommends moving to Agent Framework when one-step prompts become multi-step workflows.
Making the model responsible for authorization
The model should never be your permission system.
Logging every prompt
Prompts may contain sensitive information.
Log safe metadata unless content logging is explicitly justified and protected.
Retrying every failure
Retries should target appropriate transient failures.
Making RAG retrieval unrestricted
Retrieved data must obey application authorization and tenant boundaries.
Creating too many microservices
A modular AI service inside one ASP.NET Core application can be enough for many applications.
Split into independent services only when the operational benefits justify the additional complexity.
Practical Project: AI Integration Gateway
A useful project for this topic is an AI integration gateway.
Architecture:
Clients
|
v
ASP.NET Core API
|
v
AI Gateway
|
+----------------+----------------+
| | |
Chat RAG Tools
| | |
v v v
IChatClient VectorData Tool Registry
| | |
+----------------+----------------+
|
AI Router
|
+-----------+-----------+
| | |
OpenAI Azure Ollama
The gateway can implement:
Authentication
Authorization
Prompt policies
Provider routing
Model routing
Rate limiting
Caching
Resilience
Telemetry
Cost tracking
Tenant isolation
This project ties together almost every integration pattern discussed in this article.
Practical Project: AI Customer Support Platform
A customer support platform could have:
Customer
|
v
Support API
|
+---- Conversation Store
|
+---- RAG Service
|
+---- Tool Service
|
+---- AI Service
Question flow:
Customer Question
|
v
Authentication
|
v
Conversation History
|
v
RAG Search
|
v
Tool Decision
|
v
AI Model
|
v
Output Validation
|
v
Response
For complex requests:
Customer asks
|
v
Agent
|
+---- Search knowledge
|
+---- Get customer data
|
+---- Check order
|
v
Answer
Practical Project: AI Document Assistant
Architecture:
Upload PDF
|
v
API
|
v
Queue
|
v
Ingestion Worker
|
v
Chunk
|
v
Embedding
|
v
Vector Store
Question flow:
User
|
v
Ask Question
|
v
RAG Service
|
v
Vector Search
|
v
Relevant Context
|
v
IChatClient
|
v
Answer
This becomes one of the most useful practical AI architectures in .NET.
Practical Project: AI Agent Platform
A more advanced project could contain:
Agent API
|
v
Agent Framework
|
+--+--------+----------+
| | |
Tools RAG MCP
| | |
DB/API Vector DB MCP Server
|
v
IChatClient
|
v
AI Provider
This is the direction toward advanced AI agents in later roadmap topics.
Production AI Integration Checklist
Before production, verify:
Provider credentials are secure
AI clients use dependency injection
Prompts are centralized where appropriate
Input validation exists
Output validation exists
Cancellation is supported
Timeouts are configured
Transient failures are handled
Rate limiting is configured
Caching is intentional
Telemetry is enabled
Sensitive data is protected
Tenant isolation is enforced
RAG retrieval is authorized
Tool execution is authorized
AI usage is tracked
AI quality is evaluated
Package versions are controlled
Frequently Asked Questions
What is an AI integration pattern?
It is a repeatable architecture for connecting AI capabilities to the rest of a software application.
What is the most important basic AI integration pattern?
For modern .NET applications, a useful starting point is:
Application
|
v
IChatClient
|
v
Provider
with dependency injection and provider-specific code isolated at the infrastructure boundary.
Why use Microsoft.Extensions.AI?
It provides common abstractions and composable middleware for AI services, helping applications remain less coupled to a specific provider.
What is the middleware pattern in AI?
It places reusable behavior around the AI client:
Cache
Telemetry
Tool Calling
Resilience
Logging
Provider
Microsoft's IChatClient architecture explicitly supports this composability.
What is the difference between an AI service and an AI provider?
The AI service is your application's abstraction or business service.
The provider is the external system that actually supplies the model.
Application
|
v
IAIService
|
v
IChatClient
|
v
Provider
What is an AI gateway?
An AI gateway is a centralized service through which applications access one or more AI providers.
It can centralize:
Security
Routing
Rate limits
Caching
Observability
Cost control
When should I use RAG?
Use RAG when responses need information from external or frequently changing data that should be retrieved at request time.
What is the difference between RAG and fine-tuning?
RAG supplies relevant external context at runtime.
Fine-tuning changes model behavior through additional training.
They solve different problems.
When should I use MCP?
MCP is especially useful when AI capabilities need to be exposed or consumed across process or product boundaries. Microsoft describes MCP as a standardized client-server approach for exposing tools and resources to AI applications.
When should I use an AI agent?
Use an agent when the application needs multi-step goal-oriented behavior, tool use, contextual decisions, or workflow orchestration. Microsoft currently distinguishes those agentic scenarios from simpler one-step AI interactions.
Should every AI application have an AI gateway?
No.
A gateway adds operational complexity.
A small application can call its provider directly through a clean application service.
Should I cache all AI responses?
No.
Caching should be based on whether the response is reusable and whether stale or shared results are acceptable.
Why is resilience important for AI APIs?
AI providers are external dependencies and can experience transient failures, timeouts, and rate limits. .NET's resilience libraries provide retry, timeout, circuit-breaker, and related strategies for handling such failures.
Why should tool calls be authorized?
Because an AI-generated tool request is still untrusted input.
The application must decide whether that particular user is allowed to execute that particular operation.
What is human-in-the-loop AI?
It is a pattern where the AI proposes an action but a human or explicit application approval step must authorize execution.
Can the same architecture work with OpenAI, Azure OpenAI, and Ollama?
Yes.
A provider-neutral abstraction such as IChatClient can provide a common application boundary while the concrete provider implementation changes. Microsoft's current .NET AI ecosystem supports multiple provider integrations through these abstractions.
Interview Questions
What is an AI integration pattern?
A reusable architectural approach for integrating AI capabilities into an application.
Why should AI provider code be isolated?
To reduce coupling and make provider changes, testing, and maintenance easier.
What is the Adapter Pattern in AI?
It converts a provider-specific AI client into a common application abstraction.
What is the Facade Pattern in AI?
It provides a simple business-oriented interface over lower-level AI functionality.
What is the Decorator Pattern in AI?
It wraps an AI client with additional capabilities such as:
Caching
Logging
Telemetry
Tool invocation
Resilience
What is AI middleware?
Reusable logic that executes before, after, or around AI requests.
What is the Cache-Aside pattern?
The application checks a cache first and calls the AI provider only when the needed result is absent.
What is an AI gateway?
A centralized service that manages access to one or more AI providers.
What is a provider factory?
A component that selects the appropriate AI client implementation based on configuration or policy.
What is the RAG integration pattern?
A pattern where relevant external information is retrieved and supplied as context to an AI model before generating an answer.
What is the tool-calling pattern?
A pattern where an AI model requests execution of an application-defined function and receives the result before continuing.
What is human-in-the-loop?
A design where a human or explicit approval process must authorize certain AI-proposed actions.
What is the MCP integration pattern?
A standardized client-server integration pattern allowing AI applications to consume tools and resources exposed by MCP servers.
Why use cancellation tokens in AI services?
AI requests can be long-running external operations. Cancellation allows work to stop when the caller no longer needs the result.
Why should AI responses be validated?
Because generated output is not automatically guaranteed to satisfy application requirements.
What is the difference between RAG and an agent?
RAG focuses on retrieving relevant information.
An agent focuses on achieving a goal through multi-step reasoning, tool use, or workflow execution.
An agent can use RAG as one of its capabilities.
What is AI resilience?
The ability of the application to continue operating appropriately despite transient failures in AI dependencies.
What is a circuit breaker?
A resilience mechanism that temporarily stops calls to a failing dependency after repeated failures, allowing the system to recover instead of continuously sending failing traffic.
Exercises
Exercise 1: AI Service Pattern
Create:
IAIService
AIService
AIController
Connect the service to:
IChatClient
Exercise 2: Provider Abstraction
Create two provider configurations:
OpenAI
Ollama
Switch providers without modifying your controller.
Exercise 3: AI Middleware
Create a custom IChatClient wrapper that records:
Request start
Request duration
Success/failure
Do not log sensitive prompt content.
Exercise 4: Caching
Add an AI response cache.
Test:
First request -> AI Provider
Second request -> Cache
Exercise 5: Resilience
Add:
Timeout
Retry
Circuit Breaker
to an AI HTTP dependency and simulate temporary failures.
Exercise 6: Streaming
Create:
GET /api/chat/stream
and stream the AI response to the client.
Exercise 7: Conversation Memory
Store chat messages in SQL Server and reload the recent conversation before each AI request.
Exercise 8: RAG
Build:
Document
|
v
Chunk
|
v
Embedding
|
v
Vector Store
|
v
Search
|
v
IChatClient
Exercise 9: Tool Calling
Expose a safe application function such as:
GetProductDetails
and allow the AI to request it.
Add explicit validation and authorization.
Exercise 10: Human Approval
Modify a tool workflow:
AI proposes action
|
v
Approval
|
+----+----+
| |
Approve Reject
| |
v v
Execute Stop
Exercise 11: Multi-Provider AI Gateway
Create an ASP.NET Core gateway supporting:
OpenAI
Azure OpenAI
Ollama
with a provider-selection policy.
Exercise 12: AI Evaluation
Create a test dataset and compare AI responses after changing:
Prompt
Model
RAG configuration
Package version
Measure whether response quality changes.
Learning Path After AI Integration Patterns
The next topic is:
#15 AI Application Architecture with .NET
That topic brings these individual patterns together into complete application architectures.
After that, the roadmap moves into:
Large Language Models with .NET
Using LLMs in C#
Calling LLM APIs from .NET
Chat Models with .NET
Text Generation with .NET
AI Text Generation with C#
AI Chat Applications with .NET
Streaming AI Responses with .NET
AI Conversation History with .NET
AI Context Management with .NET
AI Prompt Management with .NET
System Prompts with .NET
Prompt Templates with .NET
Structured AI Responses with .NET
JSON Responses from AI Models with .NET
Those topics then lead into:
OpenAI
Azure OpenAI
Microsoft AI
Semantic Kernel
RAG
Embeddings
Vector Databases
Local AI
Vision
Speech
Documents
Databases
Agents
MCP
Blazor
.NET MAUI
Automation
Security
Testing
Production
Key Takeaways
AI integration patterns are about architecture rather than a particular AI provider.
The simplest architecture is:
Application
|
v
IChatClient
|
v
AI Provider
As the application grows, additional patterns can be layered around it:
Application
|
v
AI Service
|
+---- Prompt Management
+---- Conversation
+---- RAG
+---- Tools
+---- Caching
+---- Resilience
+---- Telemetry
|
v
IChatClient
|
v
Provider
For advanced systems:
Application
|
v
Agent
|
+---- RAG
+---- Tools
+---- MCP
+---- Memory
+---- Workflows
|
v
IChatClient
|
v
Provider
The most reusable patterns are:
Dependency Injection
Provider Abstraction
Adapter
Facade
Service Layer
Decorator / Middleware
Caching
Resilience
Factory
Strategy
RAG
Tool Calling
Human-in-the-Loop
Agent
MCP
Background Processing
Event-Driven AI
AI Gateway
The key design principle is to start with the simplest architecture that solves the actual requirement.
Do not build an agent when a chat service is enough.
Do not build an AI microservice when a service class is enough.
Do not add a vector database when the application does not need retrieval.
Do not add MCP when the tools do not need cross-process interoperability.
But when complexity genuinely appears, introduce the appropriate pattern instead of putting everything inside one controller.
Conclusion
AI integration in .NET works best when AI is treated as another carefully designed application dependency rather than as a special block of code hidden inside a controller.
A clean progression is:
AI API
|
v
Provider SDK
|
v
IChatClient
|
v
AI Service
|
+---- Middleware
+---- Cache
+---- Resilience
+---- Telemetry
|
v
RAG / Tools / Conversation
|
v
Agents / MCP / Workflows
The modern .NET AI ecosystem supports this layered approach directly. Microsoft.Extensions.AI provides the common AI interaction and middleware layer; vector-data and ingestion components support grounding and RAG; MCP provides standardized external capability integration; and Agent Framework provides higher-level orchestration for multi-step agentic systems.
The most important lesson is separation of responsibilities.
Your:
Controller
should not be responsible for:
Provider selection
Prompt construction
Conversation persistence
Vector search
Tool authorization
Retries
Caching
Telemetry
Instead, those concerns should be composed through dedicated application and infrastructure layers.
A strong .NET AI application can therefore evolve naturally:
Simple Chat
|
v
AI Service
|
v
Production AI Pipeline
|
v
RAG
|
v
Tools
|
v
Agents
|
v
MCP
|
v
Enterprise AI Platform
That progression forms the architectural foundation for everything that follows in the .NET + AI roadmap.
Next AI Application Architecture with .NET
Post a Comment