Building an AI feature is relatively easy.
Building a maintainable AI application is a different problem.
A simple application can start with:
User
|
v
.NET Application
|
v
AI API
|
v
Response
But a production AI application may eventually contain:
Authentication
Authorization
AI services
Prompt management
Conversation history
Databases
RAG
Embeddings
Vector search
Document ingestion
Caching
Resilience
Streaming
Tools
MCP
Agents
Background workers
Queues
Evaluation
Observability
Cost tracking
Without a clear architecture, all of these concerns can quickly end up inside controllers, services, and database code.
A better approach is to treat AI as another architectural capability inside the application.
Microsoft's current .NET AI guidance recommends Microsoft.Extensions.AI as the starting layer for application-level AI behavior, then adding data ingestion and vector storage for RAG, MCP when capabilities need to cross application boundaries, Agent Framework when workflows become genuinely agentic, and Aspire when the system grows into multiple distributed services.
What Is AI Application Architecture?
AI application architecture describes how the different parts of an AI-enabled system are organized and how they communicate.
A simple view is:
AI Application
|
+------------+------------+
| | |
Client Business AI
Logic Services
|
v
AI Model
A production architecture is more complete:
Clients
|
v
ASP.NET Core API
|
+--------------+--------------+
| |
Authentication Rate Limiting
| |
+--------------+--------------+
|
v
Application Layer
|
+-------------------------+-------------------------+
| | |
v v v
AI Service RAG Service Tool Service
| | |
v v v
IChatClient Vector Search Tools
| | |
| Data Ingestion APIs
| |
+-------------+-----------+
|
v
AI Provider
The important point is that the AI model is only one part of the system.
Why AI Architecture Is Different
Traditional applications generally operate around:
UI
API
Business Logic
Database
AI applications add probabilistic and external components:
UI
API
Business Logic
AI Services
Model
Prompt
Context
Vector Store
Tools
Evaluation
A normal database query tends to return data according to a deterministic query.
An AI model generates output based on the supplied instructions, context, model behavior, and configuration.
Therefore, AI applications require additional architectural concerns such as:
Prompt management
Context management
Output validation
Model selection
Evaluation
Token usage
Provider reliability
The Core Principle: AI Is an Application Dependency
The biggest architectural mistake is treating AI as the entire application.
Instead:
Application
|
+---- Database
|
+---- Cache
|
+---- AI
|
+---- Messaging
|
+---- External APIs
AI should normally be one capability among several.
A useful mental model is:
AI is a dependency.
It is not the architecture itself.
Clean Architecture for AI Applications
Microsoft's ASP.NET Core architecture guidance recommends separating applications according to responsibilities and describes Clean Architecture as a way to keep business logic independent from infrastructure concerns.
A useful AI version of Clean Architecture is:
Presentation
|
v
Application
|
v
Domain
^
|
Infrastructure
More accurately, the compile-time dependency direction should point inward:
+----------------+
| Domain |
+----------------+
^
|
+----------------+
| Application |
+----------------+
^
|
+----------------+
| Infrastructure|
+----------------+
^
|
+----------------+
| API |
+----------------+
Infrastructure implements interfaces defined by the inner layers.
AI Clean Architecture
For an AI application:
API
|
v
Application
|
+---- IAIService
+---- IRAGService
+---- IConversationRepository
+---- IToolService
|
v
Domain
Infrastructure then implements those abstractions:
Infrastructure
|
+---- OpenAI
+---- Azure OpenAI
+---- Ollama
+---- SQL Server
+---- Redis
+---- Vector Store
+---- File Storage
This keeps provider-specific technology outside the business core.
Recommended Project Structure
A strong starting point is:
DotNetAI.sln
src
|
+-- DotNetAI.Api
|
+-- DotNetAI.Application
|
+-- DotNetAI.Domain
|
+-- DotNetAI.Infrastructure
|
+-- DotNetAI.Worker
tests
|
+-- DotNetAI.UnitTests
|
+-- DotNetAI.IntegrationTests
+-- DotNetAI.EvaluationTests
Each project has a specific purpose.
DotNetAI.Domain
The Domain project contains business concepts.
For example:
Entities
ValueObjects
Domain Services
Business Rules
Domain Exceptions
It should normally not reference:
OpenAI
Azure.AI.OpenAI
Semantic Kernel
Vector databases
ASP.NET Core
The domain should remain independent of infrastructure.
DotNetAI.Application
The Application project contains use cases and application interfaces.
Examples:
IAIService
IRAGService
IConversationRepository
IDocumentService
IToolAuthorizationService
IUsageService
It can also contain:
DTOs
Application Services
Commands
Queries
Validation
Business workflows
DotNetAI.Infrastructure
Infrastructure contains technical implementations.
For example:
AI
|
+-- OpenAI
+-- AzureOpenAI
+-- Ollama
Persistence
|
+-- SQL Server
+-- Redis
Vector
|
+-- Qdrant
+-- Azure AI Search
Documents
|
+-- Storage
+-- Data Ingestion
Provider-specific NuGet packages belong here where practical.
DotNetAI.Api
The API project contains:
Controllers
Middleware
Authentication
Authorization
Dependency Injection
HTTP configuration
API endpoints
The controller should not contain:
Prompt construction
Vector search
OpenAI client creation
SQL queries
Tool authorization
Retry logic
Those belong in appropriate services.
DotNetAI.Worker
Long-running operations can live in a Worker Service:
Document ingestion
Embedding generation
Bulk processing
AI summarization
Queue processing
Scheduled tasks
The worker can reuse application abstractions.
Dependency Direction
A recommended dependency graph is:
DotNetAI.Api
|
v
Application
|
v
Domain
Infrastructure
|
+---- Application
+---- Domain
Worker
|
v
Application
At runtime:
API
|
v
Application
|
v
Infrastructure implementations
|
+---- AI
+---- Database
+---- Vector Store
This follows the Clean Architecture idea that application interfaces should not depend on infrastructure implementation details.
The AI Abstraction Layer
One of the most useful current .NET building blocks is:
Microsoft.Extensions.AI
It provides common abstractions such as:
IChatClient
IEmbeddingGenerator<TInput,TEmbedding>
AIFunction
and fits naturally into dependency injection and application architecture. Microsoft currently positions it as the normal first AI layer for many .NET applications.
The architecture can be:
Application
|
v
IAIService
|
v
IChatClient
|
v
Provider Adapter
|
v
AI Provider
Why Use Two Abstraction Levels?
You may wonder why an application needs both:
IAIService
and:
IChatClient
They solve different problems.
IChatClient is a technical AI abstraction.
IAIService can represent application-specific behavior.
For example:
public interface IAIService
{
Task<string> SummarizeAsync(
string text,
CancellationToken cancellationToken = default);
Task<string> AnswerAsync(
string question,
CancellationToken cancellationToken = default);
}
The application therefore talks about:
Summarize
Answer
Classify
Extract
rather than:
ChatMessage
ChatOptions
Provider SDK
Application Service Pattern
A clean AI service might look like:
using Microsoft.Extensions.AI;
public sealed class AIService : IAIService
{
private readonly IChatClient _chatClient;
public AIService(IChatClient chatClient)
{
_chatClient = chatClient;
}
public async Task<string> AnswerAsync(
string question,
CancellationToken cancellationToken = default)
{
ChatResponse response =
await _chatClient.GetResponseAsync(
[
new ChatMessage(
ChatRole.System,
"You are a helpful .NET assistant."),
new ChatMessage(
ChatRole.User,
question)
],
cancellationToken: cancellationToken);
return response.Text;
}
public async Task<string> SummarizeAsync(
string text,
CancellationToken cancellationToken = default)
{
ChatResponse response =
await _chatClient.GetResponseAsync(
$"Summarize the following text:\n\n{text}",
cancellationToken: cancellationToken);
return response.Text;
}
}
The controller only sees IAIService.
Dependency Injection Architecture
In Program.cs:
builder.Services.AddSingleton(chatClient);
builder.Services.AddScoped<
IAIService,
AIService>();
Then:
Controller
|
v
IAIService
|
v
IChatClient
|
v
Provider
This architecture supports:
Testing
Provider replacement
Configuration
Telemetry
Caching
Middleware
Provider Independence
The application can use:
OpenAI
Azure OpenAI
Ollama
Other compatible providers
without changing the application service interface.
Microsoft's current .NET AI ecosystem supports multiple providers through compatible abstractions, including OpenAI, Azure OpenAI, Azure AI Foundry, Ollama, Google Gemini, and Amazon Bedrock.
For example:
Development
|
v
Ollama
Production
|
v
Azure OpenAI
while:
IAIService
IChatClient
remain unchanged.
Application Configuration
AI configuration should be separated from business logic.
Example:
{
"AI": {
"Provider": "OpenAI",
"Model": "your-model-name"
}
}
A configuration object:
public sealed class AIOptions
{
public string Provider { get; set; } =
string.Empty;
public string Model { get; set; } =
string.Empty;
}
Registration:
builder.Services
.AddOptions<AIOptions>()
.Bind(
builder.Configuration
.GetSection("AI"))
.ValidateOnStart();
Secrets should come from secure configuration mechanisms rather than source-controlled files.
AI Infrastructure Boundary
A useful rule is:
Application knows WHAT it needs.
Infrastructure knows HOW to provide it.
For example:
Application:
IChatClient
IConversationRepository
IVectorSearchService
IFileStorage
Infrastructure:
OpenAIChatClient
SqlConversationRepository
QdrantVectorSearch
AzureBlobStorage
This is the Dependency Inversion Principle applied to AI.
The RAG Architecture
RAG adds an additional data layer.
Microsoft's current .NET guidance separates the process into data ingestion, vector storage/retrieval, and AI model interaction.
The architecture is:
RAG System
|
+---------------+---------------+
| |
Ingestion Query
| |
v v
Documents Question
| |
v v
Chunking Embedding
| |
v v
Enrichment Vector Search
| |
v v
Embeddings Relevant Chunks
| |
+---------------+---------------+
|
v
IChatClient
|
v
Answer
RAG Data Flow
Ingestion flow
PDF
|
v
Extract
|
v
Normalize
|
v
Chunk
|
v
Enrich
|
v
Embed
|
v
Vector Store
Microsoft's current DataIngestion documentation describes an ETL-style process of extracting source data, transforming it through cleaning/chunking/enrichment, and loading it into a destination such as a vector store.
Query flow
User Question
|
v
Embedding
|
v
Vector Search
|
v
Relevant Context
|
v
Prompt
|
v
LLM
|
v
Answer
Data Ingestion Layer
A clean RAG application should isolate ingestion.
public interface IDocumentIngestionService
{
Task IngestAsync(
Stream document,
string fileName,
CancellationToken cancellationToken = default);
}
Implementation:
IDocumentIngestionService
|
v
Microsoft.Extensions.DataIngestion
|
+---- Chunking
+---- Enrichment
+---- Embedding
|
v
Vector Store
The current .NET ingestion ecosystem is specifically designed to provide reusable preparation pipelines rather than requiring each application to reinvent document processing.
Vector Data Layer
Vector storage should also be isolated.
public interface IVectorSearchService
{
Task<IReadOnlyList<SearchResult>>
SearchAsync(
string query,
CancellationToken cancellationToken = default);
}
Implementation:
IVectorSearchService
|
v
Microsoft.Extensions.VectorData
|
v
Vector Store
Microsoft.Extensions.VectorData provides common abstractions for vector stores, collections, records, and search.
Vector Database as Infrastructure
Do not allow the domain model to depend directly on:
Qdrant SDK
Azure AI Search SDK
MongoDB vector APIs
Redis vector APIs
Instead:
Application
|
v
IVectorSearchService
|
v
Infrastructure
|
v
Vector Store
This makes migrations easier.
Conversation Architecture
AI chat applications need conversation state.
A useful architecture is:
Chat Request
|
v
Conversation
|
+----------+----------+
| |
v v
Conversation Store Context Manager
|
v
IChatClient
|
v
LLM
Conversation storage can use:
SQL Server
PostgreSQL
Redis
Document database
depending on the application's needs.
Conversation Entity
For example:
public sealed class Conversation
{
public Guid Id { get; set; }
public string UserId { get; set; } =
string.Empty;
public DateTimeOffset CreatedAt { get; set; }
public DateTimeOffset UpdatedAt { get; set; }
}
Message:
public sealed class ConversationMessage
{
public long Id { get; set; }
public Guid ConversationId { get; set; }
public string Role { get; set; } =
string.Empty;
public string Content { get; set; } =
string.Empty;
public DateTimeOffset CreatedAt { get; set; }
}
The AI service can then load the appropriate history.
Context Management
It is rarely desirable to send unlimited conversation history.
A context manager can:
Keep recent messages
Summarize older messages
Retrieve relevant history
Limit token usage
Remove redundant data
Architecture:
Conversation History
|
v
Context Manager
|
+----+----+
| |
Recent Summary
| |
+----+----+
|
v
Prompt
|
v
LLM
This becomes particularly important for long-running assistants.
Prompt Architecture
Prompt management should not be spread randomly across controllers.
A better design is:
Application Service
|
v
Prompt Builder
|
v
Prompt Template
|
v
Context
|
v
IChatClient
For example:
public interface IPromptService
{
string BuildCustomerSupportPrompt(
string question,
IReadOnlyList<string> context);
}
Implementation:
public sealed class PromptService :
IPromptService
{
public string BuildCustomerSupportPrompt(
string question,
IReadOnlyList<string> context)
{
string contextText =
string.Join(
Environment.NewLine,
context);
return
$"""
Answer the user's question using the
supplied context.
Context:
{contextText}
Question:
{question}
""";
}
}
This separates prompt construction from transport.
Structured Output Architecture
When AI output is consumed by application code:
LLM
|
v
Structured Output
|
v
Deserializer
|
v
Schema Validation
|
v
Business Validation
|
v
Application
Example:
public sealed class ResumeAnalysis
{
public string CandidateName { get; set; } =
string.Empty;
public int ExperienceYears { get; set; }
public List<string> Skills { get; set; } = [];
}
The model's output should still be validated before it becomes authoritative application data.
Tool Architecture
Tools should sit between the AI service and business services.
AI Model
|
v
Tool Registry
|
v
Tool
|
v
Business Service
|
v
Database / API
For example:
AI
|
+---- GetCustomer
| |
| v
| CustomerService
|
+---- GetOrder
|
v
OrderService
The tool should not directly bypass business rules.
Tool Authorization
A secure architecture is:
AI Tool Request
|
v
Tool Authorization
|
v
Parameter Validation
|
v
Business Service
|
v
Database
The model does not decide whether the user has permission.
The application does.
MCP Architecture
Microsoft currently describes MCP as a standardized client-server protocol for exposing and consuming capabilities such as tools and resources. The official .NET MCP SDK integrates with Microsoft.Extensions.AI, and Agent Framework can connect to MCP servers.
Architecture:
AI Application
|
v
MCP Client
|
v
MCP Server
|
+---+---------+---------+
| | |
Database API Files
This is particularly useful when capabilities need to be reused across applications or processes.
If everything is local and in-process, ordinary function calling may be simpler. Microsoft explicitly distinguishes those use cases in its current .NET AI guidance.
Agent Architecture
Not every AI application needs an agent.
A simple chat request can be:
Question
|
v
IChatClient
|
v
Answer
An agent is different:
Goal
|
v
Agent
|
+---- Reason
|
+---- Tool
|
+---- Observe
|
+---- Decide
|
+---- Tool
|
v
Result
Microsoft currently describes Agent Framework as the orchestration layer for systems that pursue goals across multiple steps, make decisions, use tools, and potentially coordinate multiple agents.
Agent Layer in the Architecture
A mature architecture can therefore become:
API
|
v
Application
|
v
Agent Service
|
+---- AI Model
+---- RAG
+---- Tools
+---- MCP
+---- Memory
|
v
Provider
The Application layer still controls business rules.
The agent is not a replacement for the entire application architecture.
Agent Workflow Architecture
Agent Framework supports orchestration patterns including:
Sequential
Concurrent
Handoff
Group Chat
Magentic
according to Microsoft's current agent documentation.
Sequential
Request
|
v
Agent A
|
v
Agent B
|
v
Agent C
|
v
Result
Concurrent
+---- Agent A ----+
| |
Request -----+---- Agent B ----+----> Aggregate
| |
+---- Agent C ----+
Handoff
Request
|
v
Agent A
|
+----> Agent B
|
+----> Agent C
These are application workflow choices, not requirements for ordinary chat.
Background AI Architecture
Some AI operations should not execute inside an HTTP request.
For example:
Large PDF
Bulk embeddings
Thousands of records
Large summarization jobs
Use:
API
|
v
Queue
|
v
Background Worker
|
v
AI Processing
Architecture:
ASP.NET Core
|
v
Queue
|
+----------+----------+
| |
v v
AI Worker Document Worker
| |
v v
AI API Vector Store
Queue-Based Architecture
A typical flow:
POST /documents
|
v
Store File
|
v
Create Job
|
v
Queue Message
|
v
Return 202 Accepted
Then:
Worker
|
v
Read Job
|
v
Process Document
|
v
Generate Embeddings
|
v
Store Vectors
|
v
Mark Complete
This is more appropriate than making the user wait for a potentially long-running processing pipeline.
Event-Driven AI
AI can also be triggered by application events.
DocumentUploaded
|
v
Message Broker
|
v
AI Processing
|
v
DocumentIndexed
Or:
TicketCreated
|
v
AI Classification
|
v
TicketClassified
AI then becomes part of a broader event-driven architecture.
Caching Architecture
AI responses can sometimes be cached.
Application
|
v
Cache
|
+--+--+
| |
Hit Miss
| |
v v
Resp AI
|
v
Cache
Caching can be applied to:
AI responses
Embeddings
Document metadata
Search results
Expensive preprocessing
Caching should be used only when reuse is semantically safe.
Distributed Cache
For a single server:
ASP.NET Core
|
v
Memory Cache
For multiple servers:
Server A ----+
Server B ----+----> Redis
Server C ----+
A distributed cache prevents each application instance from having an isolated cache.
Resilience Architecture
AI providers are external dependencies.
A production architecture should include:
Timeout
Retry
Circuit Breaker
Rate Limiting
Cancellation
Fallback where appropriate
Architecture:
Application
|
v
AI Service
|
v
Resilience Layer
|
+--+-------+--------+
| | | |
Timeout Retry Circuit
|
v
Provider
.NET's resilience libraries provide standardized pipeline strategies such as retry, timeout, and circuit breaker for network-based dependencies.
Provider Fallback
A system may have:
Primary
Azure OpenAI
|
X
|
v
Secondary
OpenAI
But fallback should only be used when the secondary provider supports the required:
Model capability
Structured output
Tool calling
Context length
Privacy requirements
Latency requirements
Otherwise, fallback can produce unexpected behavior.
Model Routing Architecture
Different workloads may need different models.
AI Request
|
v
Model Router
/ | \
/ | \
Fast Model Quality Embedding
Model Model
Routing can consider:
Task
Latency
Quality
Cost
Data sensitivity
Provider availability
This logic belongs outside business entities.
AI Gateway Architecture
Multiple applications may share an internal AI gateway:
Customer App ----+
Mobile App -------+
Admin App --------+----> AI Gateway
Support App ------+ |
|
+-------------+-------------+
| | |
OpenAI Azure Local
OpenAI AI
The gateway can enforce:
Authentication
Authorization
Rate limits
Model policies
Provider routing
Usage tracking
Cost controls
Observability
This architecture is particularly useful in larger organizations.
Modular Monolith Architecture
Not every AI solution needs microservices.
A modular monolith can contain:
DotNetAI.Api
|
+-- Chat Module
+-- RAG Module
+-- Document Module
+-- Agent Module
+-- AI Module
+-- Usage Module
Everything deploys together.
Each module still has clear boundaries.
This often provides a practical middle ground between:
One large codebase
and:
Many independent services
AI Microservices Architecture
A larger platform may eventually split into:
API Gateway
|
+---- Chat Service
|
+---- RAG Service
|
+---- Document Service
|
+---- Agent Service
|
+---- Evaluation Service
Each service can scale independently.
But microservices introduce:
Network calls
Deployment complexity
Service discovery
Distributed tracing
Message handling
Configuration
Operational overhead
Therefore, microservices should be introduced because of actual architectural requirements rather than simply because AI is involved.
When to Choose a Monolith
A single ASP.NET Core application is often sufficient when:
Small team
Moderate traffic
One product
One deployment
Simple AI workflows
Architecture:
ASP.NET Core
|
+---- Application
+---- Domain
+---- Infrastructure
+---- AI
This is often the best starting point.
When to Choose a Modular Monolith
Use a modular monolith when:
Multiple business areas
Growing codebase
Several AI capabilities
Need strong boundaries
Still manageable as one deployment
Example:
Application
|
+---- Customer
+---- Documents
+---- AI
+---- RAG
+---- Billing
When to Choose Microservices
Microservices become more reasonable when:
Independent scaling
Different deployment cycles
Different teams
High operational boundaries
Very different workloads
For example:
Chat Service
-> low latency
Embedding Worker
-> batch processing
Document Service
-> CPU/storage intensive
Agent Service
-> long-running workflows
These workloads may eventually justify separate deployments.
Aspire for Distributed AI Applications
Microsoft currently positions Aspire as the orchestration, service-wiring, and observability layer for distributed .NET applications. It can model services, databases, queues, caches, containers, and cloud dependencies in one application model.
This is especially relevant when AI evolves beyond a single application.
For example:
Aspire
|
+---- ASP.NET Core API
+---- AI Service
+---- RAG Service
+---- Worker
+---- Redis
+---- SQL Server
+---- Vector Store
+---- MCP Server
Aspire is not the AI model runtime.
It is the application infrastructure layer around the distributed system.
Aspire-Based AI Architecture
A distributed AI system might look like:
Aspire AppHost
|
+----------------+----------------+
| | |
v v v
API Worker AI Service
| | |
+----------------+----------------+
|
+-----------+-----------+
| |
v v
SQL Server Redis
|
v
Vector Store
The architecture becomes easier to develop and observe when the services are modeled together.
AI Observability Architecture
A production AI application needs observability at several levels.
AI Request
|
+-----------+-----------+
| | |
Logs Metrics Traces
| | |
+-----------+-----------+
|
v
Observability
Useful measurements include:
Request latency
Model
Provider
Token usage
Cache hits
Cache misses
Retrieval latency
Tool calls
Failures
Retry counts
Evaluation scores
Token and Cost Tracking
The AI architecture can include a usage service.
AI Request
|
+---- Model
+---- Input tokens
+---- Output tokens
+---- Duration
+---- Cost estimate
Then:
Tenant
|
+---- Monthly tokens
+---- Requests
+---- Estimated cost
This is especially useful for SaaS systems.
Multi-Tenant AI Architecture
For a multi-tenant application:
AI Application
|
+------------+------------+
| |
Tenant A Tenant B
| |
+------+-------+ +------+-------+
| | | | | |
Chat RAG Data Chat RAG Data
Every request should carry tenant context.
Request
|
v
Tenant Resolution
|
v
Authorization
|
v
AI Service
|
v
Tenant-Specific Data
Tenant-Isolated RAG
A vector search should not merely ask:
Find similar documents.
It should effectively operate within the user's allowed data scope:
Find similar documents
WHERE TenantId = currentTenant
AND UserCanRead = true
The exact implementation depends on the vector store.
The architectural rule is:
Authorization happens before context reaches the model.
The LLM is not a data-isolation mechanism.
Security Architecture
A production AI application should have explicit boundaries:
Client
|
v
Authentication
|
v
Authorization
|
v
Input Validation
|
v
Application Service
|
v
AI Service
|
v
Provider
Tool calls add another boundary:
AI
|
v
Tool Registry
|
v
Authorization
|
v
Business Service
|
v
Data
RAG adds another:
Question
|
v
Retriever
|
v
Authorization Filter
|
v
Context
|
v
LLM
The AI Security Principle
Never assume:
AI-generated instruction
is equivalent to:
Authorized application command
The application controls:
Identity
Permissions
Tools
Data
Actions
The model provides intelligence within those boundaries.
Evaluation Architecture
AI systems can regress even when the software builds successfully.
For example:
Prompt Update
|
v
AI Response Changes
|
v
Application Still Compiles
|
v
Quality Regresses
Evaluation provides a separate quality layer.
Microsoft's current .NET AI ecosystem includes Microsoft.Extensions.AI.Evaluation specifically for repeatable quality measurement and regression protection.
Architecture:
Test Cases
|
v
AI Application
|
v
Responses
|
v
Evaluation
|
v
Quality Results
AI Evaluation in CI/CD
A mature pipeline can become:
Commit
|
v
Restore
|
v
Build
|
v
Unit Tests
|
v
Integration Tests
|
v
AI Evaluation
|
v
Security Checks
|
v
Deploy
The purpose is to test both:
Software correctness
and:
AI behavior
AI Architecture and Testing Boundaries
The solution can therefore have:
DotNetAI.UnitTests
DotNetAI.IntegrationTests
DotNetAI.EvaluationTests
Unit tests
Test:
Business rules
Prompt builders
Validation
Routing
Authorization
using fake AI services.
Integration tests
Test:
Database
Vector store
AI provider
API
Queue
Evaluation tests
Test:
Answer relevance
Groundedness
Completeness
Tool behavior
RAG quality
AI Architecture and Failure Isolation
One service failing should not necessarily bring down every other capability.
For example:
Document Worker
X
should not necessarily stop:
Chat API
A distributed architecture can isolate workloads:
API
|
+---- Chat Service
|
+---- Document Worker
|
+---- Agent Worker
Each can have independent failure handling.
AI Architecture and Timeouts
Different operations can have different expected durations:
Chat
-> seconds
Embedding
-> seconds
Document ingestion
-> minutes
Agent workflow
-> potentially much longer
Therefore, architectural boundaries should use operation-specific timeout policies.
AI Architecture and Long-Running Agents
An agent should not always be tied directly to an HTTP request.
Instead:
HTTP Request
|
v
Create Agent Job
|
v
Queue
|
v
Agent Worker
|
v
Workflow
|
v
Store Result
The client can then query status:
GET /api/agent-jobs/{id}
This is much safer for workflows that can run for a long time.
AI Job Architecture
Example:
POST /api/agents/jobs
|
v
Job Created
|
v
Queue
|
v
Agent Worker
|
+---- RAG
+---- Tools
+---- MCP
+---- Model
|
v
Completed
This architecture also gives better operational control.
AI Architecture for Blazor
A Blazor application should generally separate the UI from provider credentials.
Blazor
|
v
ASP.NET Core API
|
v
Application
|
v
AI Service
|
v
AI Provider
The server controls:
Authentication
Authorization
Provider credentials
Rate limits
Usage
AI Architecture for .NET MAUI
Similarly:
.NET MAUI
|
v
ASP.NET Core API
|
v
AI Service
|
v
AI Provider
For local AI scenarios, a different architecture may be appropriate, but production cloud credentials should not simply be embedded in a mobile client.
AI Architecture for APIs
For a public AI API:
Client
|
v
API Gateway
|
v
Authentication
|
v
Rate Limit
|
v
AI API
|
v
AI Service
A public API should not directly expose unrestricted provider functionality.
AI Architecture for SaaS
A SaaS AI platform might contain:
Client
|
v
API Gateway
|
+-------------+-------------+
| |
Authentication Tenant Context
| |
+-------------+-------------+
|
v
Application Layer
|
+----------------+----------------+
| | |
Chat RAG Agents
| | |
+----------------+----------------+
|
AI Platform
|
+------------------+------------------+
| | |
OpenAI Azure Local
OpenAI AI
Data:
SQL Server
Redis
Vector Store
Blob Storage
Operations:
Telemetry
Evaluation
Usage Tracking
Cost Tracking
Auditing
AI Architecture for Enterprise Systems
A larger enterprise AI platform may evolve into:
Users
|
v
API Gateway
|
+-------------------+-------------------+
| | |
Customer Internal Admin
App App Portal
| | |
+-------------------+-------------------+
|
v
Application APIs
|
+----------------------+----------------------+
| | |
v v v
AI Service RAG Service Agent Service
| | |
v v v
IChatClient VectorData Agent Framework
| | |
v v v
Providers Vector Store Tools
|
v
Data Ingestion
Infrastructure:
SQL Server
Redis
Blob Storage
Message Queue
Vector Store
Key Vault
OpenTelemetry
Aspire
Reference Architecture for a Complete .NET AI Application
A practical architecture for the rest of this course is:
Client
|
v
ASP.NET Core API
|
+-----------------+-----------------+
| |
Authentication Rate Limiting
| |
+-----------------+-----------------+
|
v
Application Layer
|
+-----------------------+-----------------------+
| | |
v v v
AI Service RAG Service Tool Service
| | |
| v v
| VectorData Layer Authorization
| | |
| DataIngestion Business APIs
| |
+------------+----------+
|
v
IChatClient
|
+-----------+-----------+
| | |
Cache Telemetry Resilience
| | |
+-----------+-----------+
|
AI Router
|
+---------+---------+
| | |
OpenAI Azure Ollama
OpenAI
And advanced agent functionality:
Agent
|
+------------------+------------------+
| | |
RAG Tools MCP
| | |
Vector Store Business APIs MCP Servers
| | |
+------------------+------------------+
|
IChatClient
|
Provider
A Complete Solution Structure
A production-oriented solution might look like:
DotNetAI.sln
src/
DotNetAI.Api/
Controllers/
Middleware/
Extensions/
DotNetAI.Application/
AI/
IAIService.cs
IPromptService.cs
RAG/
IRAGService.cs
Documents/
IDocumentService.cs
Conversations/
IConversationRepository.cs
Tools/
IToolAuthorizationService.cs
Common/
Results/
Validation/
DotNetAI.Domain/
Entities/
ValueObjects/
Services/
Exceptions/
DotNetAI.Infrastructure/
AI/
OpenAI/
AzureOpenAI/
Ollama/
RAG/
Vector/
Ingestion/
Persistence/
SqlServer/
Redis/
Documents/
Tools/
Observability/
DotNetAI.Worker/
Jobs/
Consumers/
Services/
tests/
DotNetAI.UnitTests/
DotNetAI.IntegrationTests/
DotNetAI.EvaluationTests/
The exact project count can be smaller or larger depending on the application.
Example Domain
Suppose we are building an AI customer support platform.
Domain entities:
Customer
Ticket
Conversation
Subscription
Product
Application services:
AIService
RAGService
TicketService
ConversationService
Infrastructure:
OpenAI
Azure OpenAI
SQL Server
Redis
Vector Store
Blob Storage
The architecture:
Customer
|
v
Support API
|
v
Application
|
+--+--------------------+
| |
v v
Conversation AI Service
|
v
RAG
|
+----------+----------+
| |
Vector Search Documents
|
v
IChatClient
|
v
Model
Example Request Flow
User asks:
"Why was my order delayed?"
Request flow:
1. Authenticate user
|
v
2. Resolve tenant
|
v
3. Load conversation
|
v
4. Search authorized knowledge
|
v
5. Retrieve order data
|
v
6. Build context
|
v
7. Call AI
|
v
8. Validate answer
|
v
9. Store conversation
|
v
10. Return response
This is much closer to a production AI application than simply sending:
"Why was my order delayed?"
to an LLM.
Architecture for AI Search
An AI search application may look like:
Search Request
|
v
Query Understanding
|
v
Embedding
|
v
Vector Search
|
v
Keyword / Filter Search
|
v
Hybrid Ranking
|
v
Optional LLM
|
v
Search Result
This architecture separates:
Retrieval
Ranking
Generation
rather than assuming the LLM should perform the entire search.
Architecture for Document Intelligence
A document-analysis system can use:
Document
|
v
Upload API
|
v
Storage
|
v
Queue
|
v
Document Worker
|
+---- OCR
+---- Text extraction
+---- Chunking
+---- Enrichment
+---- Embedding
|
v
Vector Store
Questioning the document then becomes:
Question
|
v
RAG
|
v
Relevant Chunks
|
v
LLM
Architecture for AI SQL Assistants
An AI database assistant requires an extra security layer.
Unsafe:
User
|
v
LLM
|
v
SQL Database
A safer architecture:
User
|
v
AI Service
|
v
SQL Generation
|
v
SQL Validation
|
v
Authorization
|
v
Read-only Database Connection
|
v
Result
|
v
AI Explanation
The generated SQL must not automatically receive unrestricted database privileges.
Architecture for AI Coding Assistants
A coding assistant may use:
Source Repository
|
v
Code Indexer
|
v
Embeddings
|
v
Vector Store
|
v
RAG
|
v
LLM
Tools may then expose:
Search files
Read file
Run tests
Inspect diagnostics
Each tool must have explicit permissions.
Architecture for AI Email Assistants
A production email assistant might use:
Email
|
v
Authentication
|
v
Email Reader
|
v
AI Classifier
|
+---- Important
+---- Normal
+---- Requires action
|
v
Draft Generator
|
v
Human Approval
|
v
Send
The AI should not necessarily be permitted to send messages automatically.
Architecture for AI Interview Assistants
A mock interview platform might contain:
Interview Session
|
+---- Conversation Store
|
+---- Question Generator
|
+---- Evaluation
|
+---- Resume Context
|
v
IChatClient
|
v
AI
An interviewer agent can orchestrate:
Ask
|
v
Listen
|
v
Evaluate
|
v
Follow-up
|
v
Score
The score itself may also need evaluation and business rules.
Architecture for AI Customer Support
A support AI can combine:
Customer data
Order data
Knowledge base
Conversation history
AI
Tools
Architecture:
Customer
|
v
Support API
|
+-----------+-----------+
| | |
CRM RAG Orders
| | |
| Vector Store |
| | |
+-----------+-----------+
|
v
AI Agent
|
+------+------+
| |
Search Tools
| |
+------+------+
|
Answer
Architecture and Business Rules
Business rules should remain outside the model.
For example:
AI:
"Customer appears eligible for refund."
does not mean:
Application:
"Refund customer automatically."
Instead:
AI Recommendation
|
v
Business Rule Engine
|
v
Eligibility Check
|
v
Authorized Action
The model assists.
The application decides.
Architecture and Human Approval
For higher-risk operations:
AI Recommendation
|
v
Validation
|
v
Human Approval
|
v
Business Action
This is one of the most useful patterns for enterprise AI.
Architecture and Audit Logging
AI actions should sometimes be auditable.
An audit event might contain:
User ID
Tenant ID
Request ID
Action
Tool
Timestamp
Decision
Approval state
Result
Be careful about storing full prompts or sensitive data in audit logs.
Architecture and Correlation IDs
A request can produce multiple downstream calls:
HTTP Request
|
+---- SQL
|
+---- Vector Search
|
+---- AI
|
+---- Tool
Use a correlation identifier:
Request ID
so all related logs and traces can be connected.
This becomes increasingly important when the system is distributed.
Architecture and Observability
For a distributed AI system:
API
|
+---- RAG Service
| |
| +---- Vector Store
|
+---- AI Service
| |
| +---- Provider
|
+---- Worker
|
+---- Queue
distributed tracing can show:
API 120ms
|
+-- Vector Search 35ms
|
+-- AI Provider 820ms
|
+-- SQL 15ms
This allows performance problems to be diagnosed rather than guessed.
Architecture and Performance
AI applications can be slow because:
Network
Model inference
Large prompts
Vector search
Document retrieval
Tool execution
Agent loops
A good architecture can reduce unnecessary work with:
Caching
Streaming
Parallel operations
Context filtering
Model routing
Background jobs
Connection reuse
Parallel Retrieval
RAG may retrieve from multiple sources:
Question
|
+---- SQL
|
+---- Vector Store
|
+---- Search API
|
v
Aggregate
|
v
LLM
Independent retrieval operations can potentially run concurrently.
The application should only parallelize operations that are truly independent and safe to execute concurrently.
Architecture and Cost Optimization
A system can route operations:
Simple task
|
v
Lower-cost model
Complex task
|
v
More capable model
Likewise:
Repeated embedding
|
v
Embedding cache
and:
Repeated answer
|
v
Response cache
Cost management should be designed at the application architecture level rather than added after deployment.
Architecture and Model Abstraction
Keep model selection outside the domain.
Instead of:
new ProviderClient(
"specific-model-name");
everywhere, use:
Configuration
|
v
Model Selection
|
v
AI Client
This allows model changes without modifying business services.
Architecture and Version Independence
An AI application should minimize the amount of code that directly depends on rapidly changing SDK APIs.
For example:
Application
|
v
IAIService
|
v
IChatClient
|
v
Provider SDK
If a provider SDK changes, the impact is largely confined to the infrastructure boundary.
AI Architecture and Package Boundaries
A useful package boundary is:
Application
|
+---- Microsoft.Extensions.AI.Abstractions
while Infrastructure contains:
Microsoft.Extensions.AI
OpenAI
Azure.AI.OpenAI
OllamaSharp
VectorData provider
DataIngestion
The exact package arrangement depends on the application's requirements, but the architectural principle is:
Provider-specific dependencies should not spread everywhere.
Choosing the Right Architecture
Small AI application
ASP.NET Core
|
v
AI Service
|
v
IChatClient
|
v
Provider
Medium RAG application
ASP.NET Core
|
v
Application
|
+---- AI Service
+---- RAG Service
|
+---- SQL
+---- Vector Store
Advanced AI application
API
|
v
Application
|
+---- AI
+---- RAG
+---- Tools
+---- Agents
+---- MCP
Distributed enterprise application
Aspire
|
+---- API
+---- AI Service
+---- RAG Service
+---- Worker
+---- Agent Service
+---- SQL
+---- Redis
+---- Vector Store
+---- Message Queue
+---- MCP Server
Aspire's current role is specifically to orchestrate and observe such distributed applications rather than replace the AI libraries themselves.
The Most Important Architectural Rule
Do not start with the most complex architecture.
Start here:
ASP.NET Core
|
v
Application Service
|
v
IChatClient
Then add:
RAG
when external data is required.
Add:
Tools
when application actions are required.
Add:
MCP
when capabilities need standardized interoperability.
Add:
Agents
when multi-step goal-oriented orchestration is actually needed.
Add:
Aspire
when the solution becomes distributed.
This progression closely reflects Microsoft's current .NET AI ecosystem guidance.
Complete AI Architecture Evolution
A useful way to understand the complete course is:
Level 1
Simple AI
|
v
IChatClient
Level 2
Application AI
|
v
AI Service
|
v
IChatClient
Level 3
Production AI
|
+---- Cache
+---- Resilience
+---- Telemetry
+---- Evaluation
Level 4
RAG
|
+---- Ingestion
+---- Embeddings
+---- Vector Search
Level 5
Tools
|
+---- Functions
+---- APIs
+---- Databases
Level 6
Agents
|
+---- Planning
+---- Tools
+---- RAG
+---- Memory
Level 7
Interoperability
|
+---- MCP
Level 8
Distributed AI
|
+---- Workers
+---- Queues
+---- Microservices
+---- Aspire
Complete Reference Architecture
The resulting architecture can be summarized as:
Users
|
v
Client Apps
|
v
ASP.NET Core API
|
+------------+------------+
| |
Authentication Rate Limiting
| |
+------------+------------+
|
v
Application Layer
|
+-----------------------+-----------------------+
| | |
v v v
AI Service RAG Service Agent Service
| | |
v v v
IChatClient VectorData Agent Framework
| | |
| DataIngestion Tools
| | |
+-----------------------+-----------------------+
|
AI Middleware Pipeline
|
+-----------------+-----------------+
| | |
Cache Telemetry Resilience
| | |
+-----------------+-----------------+
|
AI Provider
|
+---------------+---------------+
| | |
OpenAI Azure Local
OpenAI AI
Distributed infrastructure:
Aspire
|
+-------------------------+--------------------------+
| | |
API Worker AI Service
| | |
+-------------------------+--------------------------+
|
+------------+------------+
| | |
SQL Redis Vector Store
|
Queue
|
Document Worker
A Practical Reference Solution
For the projects that follow in this series, a reusable solution structure can be:
DotNetAI.sln
src/
DotNetAI.Api
DotNetAI.Application
DotNetAI.Domain
DotNetAI.Infrastructure
DotNetAI.Worker
tests/
DotNetAI.UnitTests
DotNetAI.IntegrationTests
DotNetAI.EvaluationTests
The responsibilities are:
Api
HTTP
Authentication
Authorization
Middleware
Application
Use cases
AI interfaces
RAG interfaces
Tool interfaces
Validation
Domain
Entities
Business rules
Domain services
Infrastructure
AI providers
Database
Vector store
Cache
Storage
External APIs
Worker
Queues
Ingestion
Background processing
Tests
Unit tests
Integration tests
AI evaluations
Complete Dependency Flow
Presentation
|
v
Application
|
v
Domain
Infrastructure
|
+---- Application
+---- Domain
API
|
+---- Application
+---- Infrastructure
Worker
|
+---- Application
+---- Infrastructure
This creates clear architectural boundaries.
AI Architecture Checklist
Before declaring an AI application production-ready, verify:
Application boundaries are clear
Provider code is isolated
Dependency injection is used
Configuration is externalized
Secrets are protected
Input validation exists
Output validation exists
Authentication exists
Authorization exists
Tenant isolation exists where required
RAG retrieval is authorized
Tool execution is authorized
Timeouts exist
Cancellation is supported
Retries are controlled
Circuit breaking is considered
Rate limits exist
Caching is intentional
AI usage is tracked
AI quality is evaluated
Observability is enabled
Background jobs exist for long-running work
Package versions are controlled
Security auditing is enabled
Common Architecture Mistakes
Putting everything in Program.cs
Avoid:
Program.cs
|
+---- OpenAI
+---- Prompts
+---- SQL
+---- Vector Search
+---- Tools
+---- Business Logic
Use separate layers.
Putting AI code in controllers
Avoid:
Controller
|
+---- Prompt
+---- RAG
+---- Database
+---- OpenAI
Use:
Controller
|
v
Application Service
|
v
AI Infrastructure
Letting the Domain depend on AI SDKs
Avoid:
Domain
|
+---- OpenAI
The domain should remain focused on business concepts.
Creating microservices too early
Do not split every AI component into its own process simply because the application uses AI.
Start with a modular architecture.
Using agents for simple requests
A direct AI call is often enough for:
Summarization
Classification
Simple question answering
Agents become valuable when the problem actually involves multi-step goal pursuit and orchestration.
Letting the model enforce permissions
Never rely on:
System prompt
as the application's authorization mechanism.
Allowing unrestricted tools
Tools need:
Authorization
Validation
Auditing
Business rules
Sending all data to the model
Retrieve only what is needed.
This improves:
Privacy
Latency
Cost
Context quality
Ignoring evaluation
An application that successfully returns JSON is not necessarily an application that produces high-quality answers.
Frequently Asked Questions
What is AI application architecture?
It is the organization of the application components that provide AI capabilities, data, business logic, infrastructure, security, and operations.
Should AI code be inside the Application layer?
The application layer should generally define AI abstractions and use cases.
Provider-specific implementations should normally live in Infrastructure.
What is the role of IChatClient?
It provides a common abstraction for AI chat interactions and allows application code to remain less coupled to a particular model provider. Microsoft currently positions Microsoft.Extensions.AI and IChatClient as a core application-level AI layer.
What is the role of an IAIService?
It provides an application-specific abstraction over AI behavior.
For example:
IAIService
|
+---- Answer
+---- Summarize
+---- Classify
Why use Clean Architecture for AI?
It keeps business logic independent of changing AI providers, databases, vector stores, and other infrastructure concerns. Microsoft's ASP.NET Core architecture guidance recommends separation of concerns and dependency inversion for maintainable applications.
Is RAG part of the domain layer?
Usually no.
RAG is generally an application/infrastructure capability:
Application
|
v
IRAGService
|
v
Infrastructure
Where should embeddings live?
Embedding generation is generally an AI infrastructure capability exposed to application code through an abstraction such as IEmbeddingGenerator.
Where should vector databases live?
Normally in Infrastructure.
The application should depend on an abstraction rather than a particular vector database SDK.
Microsoft.Extensions.VectorData provides common vector-store abstractions for this purpose.
Where should document ingestion live?
Usually in Infrastructure or a dedicated document-processing module.
The application can depend on an abstraction such as:
IDocumentIngestionService
while the implementation uses the ingestion libraries.
Where should prompt templates live?
They can live in an application-level prompt service, prompt repository, or dedicated prompt module.
The correct choice depends on how prompts are managed and versioned.
Where should conversation history live?
Normally in an application-owned persistence layer such as SQL Server or another appropriate data store.
Should conversation history be passed entirely to the model?
Not necessarily.
A context-management layer should decide which messages or summaries belong in the current context.
What is the best architecture for a small AI application?
A simple layered architecture is usually enough:
API
|
v
Application
|
v
AI Service
|
v
IChatClient
|
v
Provider
What is the best architecture for RAG?
A useful architecture is:
API
|
v
Application
|
+---- AI Service
+---- RAG Service
|
+---- DataIngestion
+---- Embeddings
+---- VectorData
|
v
AI Provider
Microsoft's current .NET AI ecosystem recommends the MEDI + MEVD + MEAI combination for RAG applications.
When should I introduce Agent Framework?
When a simple model call or tool-calling loop becomes a genuinely multi-step workflow involving goal pursuit, routing, handoffs, or multiple agents.
When should I introduce MCP?
When tools or capabilities need standardized interoperability across different AI clients, applications, or processes. If everything is in one application, ordinary in-process function calling can be simpler.
When should I use Aspire?
When the AI application becomes a distributed system involving multiple services, workers, databases, queues, caches, or other infrastructure. Microsoft currently positions Aspire as the orchestration and observability layer for such distributed .NET applications.
Does Aspire replace Microsoft.Extensions.AI?
No.
They operate at different levels.
Microsoft.Extensions.AI
-> AI interaction
Aspire
-> Distributed application orchestration
Microsoft explicitly describes Aspire as the multi-service application layer around AI systems rather than the AI runtime itself.
Should I build an AI microservice?
Not necessarily.
A modular monolith may be better when the application is still small or medium-sized.
How should AI applications handle long-running operations?
Move them into:
Queue
|
v
Worker
|
v
AI Processing
instead of holding an HTTP request open for long operations.
How should AI applications handle security?
Separate:
Authentication
Authorization
Input validation
Tool authorization
RAG authorization
Output validation
from the model.
How should AI applications handle multiple providers?
Use a provider-neutral abstraction:
IChatClient
and select provider implementations through configuration or policy.
Can local and cloud AI use the same architecture?
Yes.
The application layer can remain provider-neutral while Infrastructure chooses:
Ollama
OpenAI
Azure OpenAI
Other supported provider
Interview Questions
What is Clean Architecture?
An architectural approach that keeps core business logic independent from infrastructure and implementation details.
How can Clean Architecture be applied to AI?
Place AI abstractions and use cases in Application while keeping provider SDKs, vector stores, databases, and external services in Infrastructure.
Why should the Domain project not reference OpenAI?
Because OpenAI is an infrastructure dependency.
The business domain should not depend on a particular AI provider.
What is the role of Microsoft.Extensions.AI?
It provides common AI abstractions and middleware for .NET applications, including chat and embeddings.
What is the role of IChatClient?
It is the abstraction used to interact with chat-oriented AI services without requiring the application to be hard-coded to one provider.
What is an AI service layer?
An application-level layer that translates business operations into AI operations.
What is RAG architecture?
A system that retrieves relevant information from external data sources and supplies it to an AI model as context.
What is the role of a vector store?
It stores embeddings and supports similarity or semantic retrieval for relevant data.
What is data ingestion?
The process of extracting, transforming, chunking, enriching, and storing data so that it can be used effectively by AI applications.
What is the role of MCP?
It standardizes how AI applications consume or expose tools, resources, and capabilities across process boundaries.
What is an AI agent?
An AI system that can pursue a goal across multiple steps, make decisions, use tools, and participate in workflows.
What is the difference between an AI application and an AI agent?
An AI application may simply call a model.
An agent introduces goal-oriented multi-step execution and orchestration.
What is an AI gateway?
A centralized service that controls access to one or more AI providers.
Why use a modular monolith?
It provides strong module boundaries without immediately introducing distributed-system complexity.
Why use microservices?
When services genuinely require independent scaling, deployment, ownership, or operational boundaries.
What is Aspire?
Aspire is a code-first orchestration and observability layer for distributed applications.
Why is observability important for AI systems?
Because AI requests can involve multiple external services, long model latency, retrieval, tools, and variable token usage.
Why is evaluation part of AI architecture?
AI behavior can change when prompts, models, data, or tools change. Evaluation helps detect regressions.
Why should tool calls be authorized outside the model?
Because the model is not an application security boundary.
Why should RAG retrieval be authorization-aware?
Because semantic similarity does not imply that the current user has permission to access the retrieved document.
Exercises
Exercise 1: Clean Architecture
Create:
DotNetAI.Api
DotNetAI.Application
DotNetAI.Domain
DotNetAI.Infrastructure
DotNetAI.Tests
Add references so that provider-specific dependencies stay in Infrastructure.
Exercise 2: AI Service
Create:
IAIService
with:
AnswerAsync
SummarizeAsync
Implement it using IChatClient.
Exercise 3: Provider Switching
Support:
OpenAI
Azure OpenAI
Ollama
without changing the Application project.
Exercise 4: Conversation Architecture
Create:
Conversation
ConversationMessage
IConversationRepository
Store conversations in SQL Server.
Exercise 5: RAG Architecture
Create:
IDocumentIngestionService
IVectorSearchService
IRAGService
and implement:
Document
|
v
Ingestion
|
v
Embedding
|
v
Vector Store
|
v
AI
Exercise 6: Tool Architecture
Create:
IToolService
ToolAuthorizationService
Expose one read-only business operation.
Exercise 7: Human Approval
Create:
AI Proposal
|
v
Approval Record
|
+---- Approved
|
+---- Rejected
Only execute the action after approval.
Exercise 8: Queue-Based Processing
Move document ingestion into a Worker Service.
Use:
API
|
v
Queue
|
v
Worker
Exercise 9: AI Evaluation
Create an evaluation project and test:
RAG answer relevance
Answer completeness
Groundedness
Exercise 10: Aspire
Create a distributed development environment containing:
ASP.NET Core API
Worker
SQL Server
Redis
Then add your AI service.
Aspire's current role is to model and observe these distributed application components together.
Practical Project: Production AI Assistant
A complete project for this topic is an enterprise AI assistant.
Features
User authentication
Conversation history
RAG
Document upload
Semantic search
AI tools
Streaming
Usage tracking
Evaluation
Admin monitoring
Architecture
Web / Mobile
|
v
ASP.NET Core API
|
+----------------+----------------+
| |
Authentication Authorization
| |
+----------------+----------------+
|
v
Application Layer
|
+-----------------------+-----------------------+
| | |
v v v
Chat Service RAG Service Tool Service
| | |
v v v
IChatClient VectorData Business APIs
| |
| DataIngestion
| |
+-----------+-----------+
|
v
AI Provider
Supporting infrastructure:
SQL Server
Redis
Blob Storage
Vector Database
Message Queue
OpenTelemetry
AI Evaluation
Practical Project: Complete AI RAG Platform
A full RAG system can be organized into two pipelines.
Offline pipeline
Documents
|
v
Storage
|
v
Queue
|
v
Ingestion Worker
|
v
Chunking
|
v
Enrichment
|
v
Embeddings
|
v
Vector Store
Online pipeline
User
|
v
API
|
v
Question
|
v
Query Embedding
|
v
Vector Search
|
v
Authorization
|
v
Context Construction
|
v
IChatClient
|
v
Answer
This separation is one of the most important patterns for scalable RAG systems.
Practical Project: Multi-Agent Platform
An advanced platform could be:
User
|
v
Agent Host
|
+---------+---------+
| |
Coordinator Memory
|
+----------+----------+----------+
| | | |
v v v v
Research Database Support Reporting
Agent Agent Agent Agent
| | | |
+----------+----------+----------+
|
v
Tools
|
v
MCP
|
v
External Systems
Microsoft's current Agent Framework documentation supports several workflow styles for multi-agent systems, including sequential, concurrent, handoff, group-chat, and magentic orchestration.
A Practical Enterprise Deployment Architecture
A production deployment could eventually resemble:
Internet
|
v
Load Balancer
|
v
API Gateway
|
+-------------+-------------+
| | |
v v v
API AI Service Admin
| |
| +---- RAG
| +---- Tools
| +---- Agents
|
+--------+---------+
| | |
v v v
SQL Redis Object Storage
|
v
Document Worker
|
v
Vector Store
Additional components:
Message Queue
OpenTelemetry
Secret Store
AI Evaluation
CI/CD
Architecture Evolution for This Series
The purpose of this architecture topic is to establish the foundation for the rest of the roadmap.
The next topics will progressively add capabilities.
Architecture Foundation
|
v
LLMs with .NET
|
v
OpenAI
|
v
Azure OpenAI
|
v
Microsoft AI
|
v
Semantic Kernel
|
v
ASP.NET Core AI
|
v
RAG
|
v
Embeddings
|
v
Vector Databases
|
v
Local AI
|
v
Vision
|
v
Speech
|
v
Documents
|
v
Databases
|
v
Agents
|
v
MCP
|
v
Blazor / MAUI
|
v
Automation
|
v
Security
|
v
Testing
|
v
Production
Architecture Decision Guidelines
A useful way to make architecture decisions is to ask:
Does the application only need model generation?
Use:
IChatClient
Does the application need reusable business AI operations?
Add:
IAIService
Does it need external documents?
Add:
DataIngestion
VectorData
RAG
Does it need actions?
Add:
Tools
Function Calling
Authorization
Do capabilities need interoperability between applications?
Consider:
MCP
Does the system need multi-step goal-oriented workflows?
Consider:
Agent Framework
Does the application have multiple services?
Consider:
Aspire
Queues
Distributed tracing
This keeps complexity proportional to requirements.
The Architecture Principle to Remember
A strong AI application follows this progression:
Business Requirement
|
v
Application Use Case
|
v
Application Abstraction
|
v
Infrastructure Implementation
|
v
Provider / Database / External System
For AI:
Business Requirement
|
v
AI Use Case
|
v
IAIService
|
v
IChatClient
|
v
Provider SDK
|
v
AI Model
For RAG:
Business Requirement
|
v
IRAGService
|
+---- Embeddings
+---- Vector Search
+---- Context
|
v
IChatClient
For agents:
Business Requirement
|
v
Agent Workflow
|
+---- RAG
+---- Tools
+---- MCP
+---- Memory
|
v
AI Model
Key Takeaways
AI application architecture is about organizing AI functionality inside a broader software architecture.
The foundation is:
API
|
v
Application
|
v
IAIService
|
v
IChatClient
|
v
Provider
RAG adds:
DataIngestion
VectorData
Embeddings
Vector Store
Tools add:
Tool Registry
Authorization
Business Services
Agents add:
Multi-step orchestration
Tools
RAG
Memory
Workflows
MCP adds:
Standardized capability interoperability
Distributed AI systems can add:
Workers
Queues
Microservices
Aspire
The architecture should evolve gradually.
A small application should remain small.
A RAG application should introduce only the data components it needs.
An agent should be introduced when multi-step orchestration is genuinely required.
Microservices should appear when independent deployment or scaling makes them worthwhile.
Aspire should appear when the system becomes distributed enough that service wiring and observability become significant concerns.
Microsoft's current .NET AI guidance follows this same progression: Microsoft.Extensions.AI for application-level AI, DataIngestion and VectorData for RAG, MCP for capability interoperability, Agent Framework for genuinely agentic workflows, and Aspire for distributed AI applications.
Conclusion
AI application architecture with .NET is best understood as a layered system.
The core business application should remain independent of the particular AI provider.
Business Application
|
v
AI Use Cases
|
v
AI Layer
|
IChatClient
|
+-------------+-------------+
| | |
OpenAI Azure Ollama
OpenAI
When the application needs its own knowledge:
Documents
|
v
DataIngestion
|
v
Embeddings
|
v
VectorData
|
v
RAG
|
v
AI
When it needs actions:
AI
|
v
Tools
|
v
Authorization
|
v
Business Services
When it needs multi-step autonomous workflows:
Agent
|
+---- Tools
+---- RAG
+---- MCP
+---- Memory
+---- Workflows
When it becomes distributed:
Aspire
|
+---- API
+---- AI Service
+---- RAG Worker
+---- Agent Worker
+---- SQL
+---- Redis
+---- Queue
+---- Vector Store
The most important architectural lesson is that AI should fit into your application's architecture rather than dictate the entire architecture.
Keep business rules independent.
Keep provider-specific code isolated.
Use abstractions where they provide real value.
Separate online request processing from long-running background work.
Treat RAG as a data pipeline.
Treat tools as privileged application capabilities.
Treat agents as orchestration rather than ordinary chat.
Treat MCP as interoperability.
Treat Aspire as distributed application infrastructure.
And most importantly, introduce each architectural layer only when the application's actual requirements justify it.
That foundation now prepares the series for the next major section:
LLMs with .NET
where the course moves from architecture into the actual mechanics of large language models, model requests, chat messages, text generation, context, prompts, streaming, and structured responses.
Post a Comment