This blog post demonstrates what it takes to build a capable, CLI-style agent using .NET and Microsoft Agent Framework's batteries-included harness, a concept the framework calls claw. In other words, claw is a capable, CLI-style agent that can plan, use tools, remember things, and safely on your behalf.
A harness, in Microsoft's own framing, is what's left once you strip away the polished CLI or chat UI from a coding agent: a loop that wires a language model up with tools, planning, memory, approval gates, and observability, with a path to deployment at the end. Microsoft Agent Framework ships these as composable building blocks, so developers don't have to hand-roll every piece of the loop themselves.
The blog post starts from a basic Console App and Aspire AppHost project setup, progressively layering Ollama support, explicit launch capability for the console app from the Aspire dashboard, and observability to view logs, metrics, end-to-end traces.
Here's a glimpse of the final app, built up incrementally throughout this post.
Prerequisites
- Docker for Desktop - Docker Desktop enhances your development experience by offering a powerful, user-friendly platform for container management.
- Microsoft Agent Framework - Microsoft Agent Framework (MAF) is an open, multi-language framework for building production-grade AI agents and multi-agent workflows in .NET and Python.
- Aspire - Aspire gives you a unified, code-first toolkit to compose, debug, and deploy distributed apps and agents, all from a single AppHost.
- Arize Phoenix - The open-source platform for agent development and evaluation. It helps you to understand and improve AI applications by giving you a workflow for debugging and iteration.
DEMO App
The demo application consists of two projects, named agent-harness-agent.AppHost and agent-harness.Console and solution structure should look similar to the image given below.
The demo app provides three different scenarios: finance, incident, and release. Each scenario includes samples for every capability available through Microsoft Agent Framework, allowing the same harness features to be compared across distinct domains. This makes it possible to observe how tools, memory, approvals, shell execution, background agents, loop evaluation, and context compaction behave in realistic workflows.
The graph below shows the agent runtime after the console process has started.
Let's begin
With the initial structure in place, let us start building out the rest beginning with the external dependencies.
AppHost
Aspire's AppHost is the code-first place where you declare your application's services and their relationships. In this demo, it orchestrates the starting Ollama, loading the chat model, enabling observability, while keeping the Console App (Agent Harness or Claw) on an explicit-start for on-demand execution.
AppHost.cs
var builder = DistributedApplication.CreateBuilder(args);
const string modelName = "llama3.1:latest";
var repositoryDirectory = Path.GetFullPath("..", builder.AppHostDirectory);
var consoleProjectPath = Path.Combine(repositoryDirectory, "agent-harness.Console", "agent-harness.Console.csproj");
// Ollama LLM & Model
var ollama = builder.AddOllama("ollama")
.WithDataVolume()
.WithUrlForEndpoint("http", url =>
{
url.Url = "/";
url.DisplayText = "Ollama LLM";
});
// Model
var chatModel = ollama.AddModel("model", modelName);
// Arize Phoenix
var phoenix = builder.AddContainer("phoenix", "arizephoenix/phoenix", "latest")
.WithHttpEndpoint(targetPort: 6006, name: "http")
.WithEndpoint(targetPort: 4317, name: "grpc")
.WithHttpHealthCheck("/readyz")
.WithEnvironment("PHOENIX_WORKING_DIR", "/mnt/data")
.WithVolume("phoenix-data", "/mnt/data")
.WithUrlForEndpoint("http", url =>
{
url.Url = "/";
url.DisplayText = "Arize Phoenix";
})
.WithUrls(context =>
{
foreach (var url in context.Urls)
{
if (url.Endpoint?.EndpointName == "grpc")
{
url.DisplayLocation = UrlDisplayLocation.DetailsOnly;
}
}
});
// Agent Harness
builder.AddExecutable("agent-harness", "osascript", builder.AppHostDirectory, "-e", CreateTerminalLauncherScript(repositoryDirectory, consoleProjectPath))
.WithReference(chatModel, "ollama")
.WithEnvironment("OTEL_SERVICE_NAME", "agent-harness-console")
.WithEnvironment("PHOENIX_OTLP_TRACES_ENDPOINT", $"{phoenix.GetEndpoint("http")}/v1/traces")
.WithEnvironment("PHOENIX_OTLP_TRACES_PROTOCOL", "http/protobuf")
.WithOtlpExporter()
.WaitFor(chatModel)
.WaitFor(phoenix)
.WithExplicitStart();
builder.Build().Run();
static string CreateTerminalLauncherScript(string workingDirectory, string projectPath)
=> $$"""
tell application "Terminal"
set commandText to "cd " & quoted form of {{ToAppleScriptString(workingDirectory)}} & " && "
set commandText to commandText & my envAssignment("ConnectionStrings__ollama")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_ENDPOINT")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_METRICS_ENDPOINT")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_LOGS_ENDPOINT")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_PROTOCOL")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_TRACES_PROTOCOL")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_METRICS_PROTOCOL")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_LOGS_PROTOCOL")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_HEADERS")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_TIMEOUT")
set commandText to commandText & my envAssignment("OTEL_EXPORTER_OTLP_COMPRESSION")
set commandText to commandText & my envAssignment("PHOENIX_OTLP_TRACES_ENDPOINT")
set commandText to commandText & my envAssignment("PHOENIX_OTLP_TRACES_PROTOCOL")
set commandText to commandText & my envAssignment("OTEL_METRIC_EXPORT_INTERVAL")
set commandText to commandText & my envAssignment("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT")
set commandText to commandText & my envAssignment("OTEL_SERVICE_NAME")
set commandText to commandText & my envAssignment("OTEL_RESOURCE_ATTRIBUTES")
set commandText to commandText & my envAssignment("OTEL_DOTNET_EXPERIMENTAL_OTLP_EMIT_EXCEPTION_LOG_ATTRIBUTES")
set commandText to commandText & my envAssignment("OTEL_DOTNET_EXPERIMENTAL_OTLP_EMIT_EVENT_LOG_ATTRIBUTES")
set commandText to commandText & my envAssignment("SSL_CERT_DIR")
set commandText to commandText & my envAssignment("SSL_CERT_FILE")
set commandText to commandText & "dotnet run --project " & quoted form of {{ToAppleScriptString(projectPath)}}
set harnessTab to do script commandText
set custom title of harnessTab to "agent-harness"
activate
repeat while busy of harnessTab
delay 1
end repeat
end tell
on envAssignment(variableName)
set variableValue to system attribute variableName
if variableValue is missing value then
return ""
end if
if variableValue is "" then
return ""
end if
return variableName & "=" & quoted form of variableValue & " "
end envAssignment
""";
static string ToAppleScriptString(string value)
=> "\"" + value.Replace("\\", "\\\\").Replace("\"", "\\\"") + "\"";Console
Scenarios\Finance\Finance.cs
using System.ComponentModel;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
public static class FinanceScenario
{
public static readonly SamplePrompt[] Samples =
[
new("What's the price of MSFT?", "Function invocation"),
new("My name is Ajay. Then ask: What is my name?", "Per-service-call history persistence"),
new("Look up MSFT, AAPL, and NVDA prices, then summarize the portfolio context briefly.", "Compaction"),
new("Create a todo list with research MSFT, compare NVDA, and summarize next steps.", "Todo provider"),
new("Remember that my risk limit is 2%.", "File memory provider"),
new("Create a file named auto-watchlist.md with MSFT and NVDA.", "File access provider"),
new("Create a file named watchlist.md with MSFT and NVDA.", "Tool approval"),
new("Search the web for the latest Microsoft stock news and summarize the top themes.", "Web search"),
new("Use the portfolio-risk-review skill to review MSFT and NVDA.", "Skills provider"),
new("Ask the MSFT and NVDA background agents for one catalyst and one risk each, then compare the results.", "Background agents"),
new("Use the shell to print the current working directory and OS name.", "Shell environment"),
new("Use looping to produce a two-sentence MSFT risk note.", "Looping"),
];
public const string HarnessInstructions = """
You are a compact end-to-end harness demonstration agent.
You are also a small personal finance assistant, modeled after the Claw blog sample.
Keep responses short. Use tools only when useful.
Use get_msft_price for the exact sample request "What's the price of MSFT?"
For multi-ticker lookup requests, use get_stock_price for each requested ticker before summarizing.
Use the active AgentSession to answer follow-up questions such as "What is my name?"
For multi-step planning requests, use the todo tools to create, inspect, and complete plan items.
For file creation, reading, and editing requests, call the provided file access tools.
For current news or other current-information requests, use the hosted web search tool.
If hosted web search is unavailable, say that search could not be executed instead of printing tool-call JSON.
For portfolio or watchlist risk reviews, use the portfolio-risk-review Agent Skill when it is available.
Do not call skill resources or scripts unless the loaded skill explicitly lists them.
For parallel ticker research, delegate independent MSFT and NVDA checks to the available background agents.
For shell environment requests, use the shell tool and rely on the injected OS, shell, working-directory, and CLI probe context.
For the exact shell environment sample, run exactly: printf 'cwd=%s\n' "$PWD"; if command -v sw_vers >/dev/null 2>&1; then v="$(sw_vers -productVersion)"; case "$v" in 26.*) release='Tahoe' ;; *) release='' ;; esac; if [ -n "$release" ]; then printf 'os=macOS - %s %s\n' "$release" "$v"; else printf 'os=%s %s\n' "$(sw_vers -productName)" "$v"; fi; else printf 'os=%s\n' "$(uname -s)"; fi
On macOS, do not report uname -s as the user-facing OS name because it returns the Darwin kernel family.
For the exact looping sample, produce exactly two sentences about MSFT risk. Do not mention todos, todo lists, the loop, attempts, process, or instructions. On the first attempt, omit LOOP_DONE. If loop feedback asks for completion, revise the answer and append LOOP_DONE at the end.
Keep file reads and writes scoped to the configured harness file access root.
After creating or editing a file, respond with the relative path and a brief summary.
""";
public const string ChatInstructions = """
Demonstrate the harness in a simple way:
- for "What's the price of MSFT?", call get_msft_price;
- for "Look up MSFT, AAPL, and NVDA prices, then summarize the portfolio context briefly.", call get_stock_price for MSFT, AAPL, and NVDA, then summarize the three prices and context briefly;
- for "My name is Ajay.", acknowledge the user's name briefly;
- for "What is my name?", use the chat history and answer Ajay;
- for "Create a todo list with research MSFT, compare NVDA, and summarize next steps.", use the todo tools to add those three todos, then summarize the remaining todo list;
- for "Create a file named auto-watchlist.md with MSFT and NVDA.", write auto-watchlist.md with MSFT and NVDA using the file access tools, then say which relative file path was created;
- for "Create a file named watchlist.md with MSFT and NVDA.", write watchlist.md with MSFT and NVDA using the file access tools, then say which relative file path was created;
- for "Search the web for the latest Microsoft stock news and summarize the top themes.", use the hosted web search tool and summarize the current themes briefly;
- for "Use the portfolio-risk-review skill to review MSFT and NVDA.", follow the portfolio-risk-review Agent Skill structure;
- for "Ask the MSFT and NVDA background agents for one catalyst and one risk each, then compare the results.", start one task on msft-researcher and one task on nvda-researcher, wait for both results, then compare them briefly;
- for "Use the shell to print the current working directory and OS name.", call the shell tool with exactly: printf 'cwd=%s\n' "$PWD"; if command -v sw_vers >/dev/null 2>&1; then v="$(sw_vers -productVersion)"; case "$v" in 26.*) release='Tahoe' ;; *) release='' ;; esac; if [ -n "$release" ]; then printf 'os=macOS - %s %s\n' "$release" "$v"; else printf 'os=%s %s\n' "$(sw_vers -productName)" "$v"; fi; else printf 'os=%s\n' "$(uname -s)"; fi
- when interpreting shell OS output on macOS, report the macOS product/version, not Darwin;
- for "Use looping to produce a two-sentence MSFT risk note.", produce exactly two sentences with no todo/process preamble; on the first attempt omit LOOP_DONE, and if loop feedback asks for completion then revise and append LOOP_DONE at the end;
- for other file requests, use the harness file tools and stay inside the scoped file root;
- use hosted web search for current news or other current-information requests;
- use the portfolio-risk-review Agent Skill for portfolio or watchlist risk reviews when it is available;
- use background agents for independent work that can run in parallel;
- use the shell tool for shell command execution and OS, shell, or working-directory checks;
- use the loop completion feedback to refine unfinished answers until the completion condition is met;
- do not call skill resources or scripts unless the loaded skill explicitly lists them;
- if hosted web search is unavailable, say that search could not be executed instead of printing tool-call JSON;
- use get_stock_price for other stock prices;
""";
public static ScenarioDefinition Create()
=> new(
Name: "finance",
Description: "Personal finance assistant harness sample.",
Samples: Samples,
HarnessInstructions: HarnessInstructions,
ChatInstructions: ChatInstructions,
SeedWorkspace: SeedWorkspace,
CreateTools: _ => CreateTools(),
CreateBackgroundAgents: CreateBackgroundAgents,
AutoApprovalRules: [AutoApproveFileAccessSample, HarnessApprovalRules.AutoApproveShellEnvironmentSample],
IsCompactionSamplePrompt: IsCompactionSamplePrompt,
CreatePromptMessages: CreatePromptMessages,
IsLoopingSamplePrompt: IsLoopingSamplePrompt,
GetLoopingSampleSubject: GetLoopingSampleSubject,
MentionsLoopingSampleSubject: MentionsLoopingSampleSubject,
CompleteAdvertisedSample: CompleteAdvertisedSample);
public static bool IsCompactionSamplePrompt(string prompt)
=> prompt.Equals("Look up MSFT, AAPL, and NVDA prices, then summarize the portfolio context briefly.", StringComparison.OrdinalIgnoreCase);
public static ChatMessage[] CreatePromptMessages(string prompt)
{
if (!IsCompactionSamplePrompt(prompt))
{
return [new ChatMessage(ChatRole.User, prompt)];
}
return
[
new(ChatRole.User, BuildCompactionStressContext("MSFT")),
new(ChatRole.Assistant, "Stored the MSFT portfolio context for later summarization."),
new(ChatRole.User, BuildCompactionStressContext("AAPL")),
new(ChatRole.Assistant, "Stored the AAPL portfolio context for later summarization."),
new(ChatRole.User, BuildCompactionStressContext("NVDA")),
new(ChatRole.Assistant, "Stored the NVDA portfolio context for later summarization."),
new(ChatRole.User, prompt),
];
}
public static bool IsLoopingSamplePrompt(string prompt)
=> prompt.Equals("Use looping to produce a two-sentence MSFT risk note.", StringComparison.OrdinalIgnoreCase);
public static string GetLoopingSampleSubject(string prompt) => "MSFT";
public static bool MentionsLoopingSampleSubject(string prompt, string responseText)
=> responseText.Contains("MSFT", StringComparison.OrdinalIgnoreCase)
|| responseText.Contains("Microsoft", StringComparison.OrdinalIgnoreCase);
private static void SeedWorkspace(HarnessPaths paths)
{
var skillDirectory = Path.Combine(paths.SkillsRoot, "portfolio-risk-review");
Directory.CreateDirectory(skillDirectory);
File.WriteAllText(Path.Combine(skillDirectory, "SKILL.md"), """
---
name: portfolio-risk-review
description: Review a small stock watchlist using a concise risk-first structure.
---
# Portfolio Risk Review
Use this skill when the user asks for a portfolio or watchlist risk review.
Structure the response with exactly three short sections:
1. Concentration
2. Catalysts
3. Watch Items
Keep the language concise. Mention that the review is informational and not financial advice.
""");
}
private static AIFunction[] CreateTools()
=>
[
AIFunctionFactory.Create(
GetStockPrice,
name: "get_stock_price",
description: "Get a mocked latest stock price for a ticker symbol."),
AIFunctionFactory.Create(
GetMsftPrice,
name: "get_msft_price",
description: "Get the mocked MSFT stock price. Use this for the sample request 'What's the price of MSFT?'"),
];
private static AIAgent[] CreateBackgroundAgents(IChatClient chatClient)
=>
[
chatClient.AsAIAgent(
instructions: """
You are msft-researcher, a compact background finance research agent.
Given a delegated task about Microsoft, return exactly two concise bullets:
- one catalyst
- one risk
Use only the prompt context and deterministic harness knowledge. Do not claim to have searched the web.
""",
name: "msft-researcher",
description: "Background agent for concise Microsoft catalyst and risk checks."),
chatClient.AsAIAgent(
instructions: """
You are nvda-researcher, a compact background finance research agent.
Given a delegated task about NVIDIA, return exactly two concise bullets:
- one catalyst
- one risk
Use only the prompt context and deterministic harness knowledge. Do not claim to have searched the web.
""",
name: "nvda-researcher",
description: "Background agent for concise NVIDIA catalyst and risk checks."),
];
private static ValueTask<bool> AutoApproveFileAccessSample(FunctionCallContent functionCall)
{
if (!string.Equals(functionCall.Name, FileAccessProvider.SaveFileToolName, StringComparison.Ordinal)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "fileName", out var fileName)
|| !fileName.Equals("auto-watchlist.md", StringComparison.OrdinalIgnoreCase)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "content", out var content))
{
return ValueTask.FromResult(false);
}
return ValueTask.FromResult(
content.Contains("MSFT", StringComparison.OrdinalIgnoreCase)
&& content.Contains("NVDA", StringComparison.OrdinalIgnoreCase));
}
private static CompletionResult CompleteAdvertisedSample(string prompt, string responseText, string? workspace)
{
if (prompt.Equals("What's the price of MSFT?", StringComparison.OrdinalIgnoreCase))
{
var quote = GetMsftPrice();
return CompletionResult.Replace(
$"The current price of MSFT is ${quote.Price:0.00} {quote.Currency}.",
"finance.advertised_sample",
"exact_sample_uses_stable_mocked_quote");
}
return CompletionResult.Preserve(responseText);
}
private static string BuildCompactionStressContext(string ticker)
{
var quote = GetStockPrice(ticker);
var symbol = quote.Symbol;
var priceText = quote.Found ? $"{quote.Price:0.00} {quote.Currency}" : "unknown";
var repeatedContext = string.Join(
Environment.NewLine,
Enumerable.Range(1, 80).Select(index =>
$"{symbol} prior research note {index:00}: mocked portfolio context, valuation sensitivity, risk commentary, sector comparison, and price anchor {priceText}."));
return $"""
Prior portfolio context for {symbol}
Current mocked price anchor: {priceText}
{repeatedContext}
""";
}
[Description("Get a mocked latest stock price for a ticker symbol.")]
private static StockQuote GetStockPrice([Description("The stock ticker symbol, such as MSFT or AAPL.")] string ticker)
{
Dictionary<string, decimal> prices = new(StringComparer.OrdinalIgnoreCase)
{
["AAPL"] = 214.10m,
["MSFT"] = 497.45m,
["NVDA"] = 158.24m,
};
var symbol = ticker.ToUpperInvariant();
var price = prices.GetValueOrDefault(symbol, 0m);
return new StockQuote(symbol, price, "USD", DateTimeOffset.UtcNow, price > 0);
}
private static StockQuote GetMsftPrice()
=> GetStockPrice("MSFT");
private sealed record StockQuote(
string Symbol,
decimal Price,
string Currency,
DateTimeOffset AsOf,
bool Found);
}Scenarios\Incident\Incident.cs
using System.ComponentModel;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
public static class IncidentScenario
{
public static readonly SamplePrompt[] Samples =
[
new("What is the status of incident INC-1042?", "Function invocation"),
new("Remember that checkout is the primary impacted service. Then ask: What is the primary impacted service?", "Per-service-call history persistence"),
new("Review incident INC-1042, checkout health, and payments health, then summarize response status briefly.", "Compaction"),
new("Create an incident checklist with acknowledge page, verify checkout health, and draft customer update.", "Todo provider"),
new("Remember that customer updates are due every 30 minutes.", "File memory provider"),
new("Create a file named incident-summary.md for INC-1042 with impact and next steps.", "File access provider"),
new("Create a file named handoff-notes.md for INC-1042.", "Tool approval"),
new("Search the web for the latest Azure status guidance and summarize anything relevant to incident response.", "Web search"),
new("Use the incident-review skill to review INC-1042.", "Skills provider"),
new("Ask the operations and communications background agents for one risk and one next step each, then compare the results.", "Background agents"),
new("Use the shell to print the current working directory and OS name.", "Shell environment"),
new("Use looping to produce a two-sentence incident risk note.", "Looping"),
];
public const string HarnessInstructions = """
You are a compact end-to-end harness demonstration agent.
You are also an incident response coordinator.
Keep responses short. Use tools only when useful.
Use get_incident_status for the exact sample request "What is the status of incident INC-1042?"
For incident response requests, call get_incident_status, get_service_health, and get_oncall_summary for relevant areas before summarizing.
Use the active AgentSession to answer follow-up questions such as "What is the primary impacted service?"
For multi-step planning requests, use the todo tools to create, inspect, and complete plan items.
For file creation, reading, and editing requests, call the provided file access tools.
For current service status or public cloud guidance, use the hosted web search tool.
If hosted web search is unavailable, say that search could not be executed instead of printing tool-call JSON.
For incident reviews, use the incident-review Agent Skill when it is available.
Do not call skill resources or scripts unless the loaded skill explicitly lists them.
For parallel incident response, delegate independent operations and communications checks to the available background agents.
For shell environment requests, use the shell tool and rely on the injected OS, shell, working-directory, and CLI probe context.
For the exact shell environment sample, run exactly: printf 'cwd=%s\n' "$PWD"; if command -v sw_vers >/dev/null 2>&1; then v="$(sw_vers -productVersion)"; case "$v" in 26.*) release='Tahoe' ;; *) release='' ;; esac; if [ -n "$release" ]; then printf 'os=macOS - %s %s\n' "$release" "$v"; else printf 'os=%s %s\n' "$(sw_vers -productName)" "$v"; fi; else printf 'os=%s\n' "$(uname -s)"; fi
On macOS, do not report uname -s as the user-facing OS name because it returns the Darwin kernel family.
For the exact looping sample, produce exactly two sentences about incident risk. Do not mention todos, todo lists, the loop, attempts, process, or instructions. On the first attempt, omit LOOP_DONE. If loop feedback asks for completion, revise the answer and append LOOP_DONE at the end.
Keep file reads and writes scoped to the configured harness file access root.
After creating or editing a file, respond with the relative path and a brief summary.
""";
public const string ChatInstructions = """
Demonstrate the harness in a simple way:
- for "What is the status of incident INC-1042?", call get_incident_status;
- for "Review incident INC-1042, checkout health, and payments health, then summarize response status briefly.", call get_incident_status, get_service_health for checkout, get_service_health for payments, and get_oncall_summary, then summarize response status briefly;
- for "Remember that checkout is the primary impacted service.", acknowledge checkout briefly;
- for "What is the primary impacted service?", use the chat history and answer checkout;
- for "Create an incident checklist with acknowledge page, verify checkout health, and draft customer update.", use the todo tools to add those three todos, then summarize the remaining todo list;
- for "Create a file named incident-summary.md for INC-1042 with impact and next steps.", write incident-summary.md using the file access tools, include INC-1042, impact, and next steps, then say which relative file path was created;
- for "Create a file named handoff-notes.md for INC-1042.", write handoff-notes.md using the file access tools, then say which relative file path was created;
- for "Search the web for the latest Azure status guidance and summarize anything relevant to incident response.", use the hosted web search tool and summarize current themes briefly;
- for "Use the incident-review skill to review INC-1042.", follow the incident-review Agent Skill structure;
- for "Ask the operations and communications background agents for one risk and one next step each, then compare the results.", start one task on operations-reviewer and one task on communications-reviewer, wait for both results, then compare them briefly;
- for "Use the shell to print the current working directory and OS name.", call the shell tool with exactly: printf 'cwd=%s\n' "$PWD"; if command -v sw_vers >/dev/null 2>&1; then v="$(sw_vers -productVersion)"; case "$v" in 26.*) release='Tahoe' ;; *) release='' ;; esac; if [ -n "$release" ]; then printf 'os=macOS - %s %s\n' "$release" "$v"; else printf 'os=%s %s\n' "$(sw_vers -productName)" "$v"; fi; else printf 'os=%s\n' "$(uname -s)"; fi
- when interpreting shell OS output on macOS, report the macOS product/version, not Darwin;
- for "Use looping to produce a two-sentence incident risk note.", produce exactly two sentences with no todo/process preamble; on the first attempt omit LOOP_DONE, and if loop feedback asks for completion then revise and append LOOP_DONE at the end;
- for other file requests, use the harness file tools and stay inside the scoped file root;
- use hosted web search for current service status or public cloud guidance;
- use the incident-review Agent Skill for incident reviews when it is available;
- use background agents for independent work that can run in parallel;
- use the shell tool for shell command execution and OS, shell, or working-directory checks;
- use the loop completion feedback to refine unfinished answers until the completion condition is met;
- do not call skill resources or scripts unless the loaded skill explicitly lists them;
- if hosted web search is unavailable, say that search could not be executed instead of printing tool-call JSON;
""";
public static ScenarioDefinition Create()
=> new(
Name: "incident",
Description: "Incident response coordinator harness sample.",
Samples: Samples,
HarnessInstructions: HarnessInstructions,
ChatInstructions: ChatInstructions,
SeedWorkspace: SeedWorkspace,
CreateTools: _ => CreateTools(),
CreateBackgroundAgents: CreateBackgroundAgents,
AutoApprovalRules: [AutoApproveIncidentSummarySample, HarnessApprovalRules.AutoApproveShellEnvironmentSample],
IsCompactionSamplePrompt: IsCompactionSamplePrompt,
CreatePromptMessages: CreatePromptMessages,
IsLoopingSamplePrompt: IsLoopingSamplePrompt,
GetLoopingSampleSubject: GetLoopingSampleSubject,
MentionsLoopingSampleSubject: MentionsLoopingSampleSubject,
CompleteAdvertisedSample: CompleteAdvertisedSample);
public static bool IsCompactionSamplePrompt(string prompt)
=> prompt.Equals("Review incident INC-1042, checkout health, and payments health, then summarize response status briefly.", StringComparison.OrdinalIgnoreCase);
public static ChatMessage[] CreatePromptMessages(string prompt)
{
if (!IsCompactionSamplePrompt(prompt))
{
return [new ChatMessage(ChatRole.User, prompt)];
}
return
[
new(ChatRole.User, BuildCompactionStressContext("checkout")),
new(ChatRole.Assistant, "Stored the checkout incident context for later summarization."),
new(ChatRole.User, BuildCompactionStressContext("payments")),
new(ChatRole.Assistant, "Stored the payments incident context for later summarization."),
new(ChatRole.User, BuildCompactionStressContext("communications")),
new(ChatRole.Assistant, "Stored the communications incident context for later summarization."),
new(ChatRole.User, prompt),
];
}
public static bool IsLoopingSamplePrompt(string prompt)
=> prompt.Equals("Use looping to produce a two-sentence incident risk note.", StringComparison.OrdinalIgnoreCase);
public static string GetLoopingSampleSubject(string prompt) => "incident";
public static bool MentionsLoopingSampleSubject(string prompt, string responseText)
=> responseText.Contains("incident", StringComparison.OrdinalIgnoreCase);
private static void SeedWorkspace(HarnessPaths paths)
{
var skillDirectory = Path.Combine(paths.SkillsRoot, "incident-review");
Directory.CreateDirectory(skillDirectory);
File.WriteAllText(Path.Combine(skillDirectory, "SKILL.md"), """
---
name: incident-review
description: Review incident response status using a concise risk-first structure.
---
# Incident Review
Use this skill when the user asks for incident response review, status, or handoff readiness.
Structure the response with exactly three short sections:
1. Impact
2. Mitigation
3. Next Update
Keep the language concise. Mention that the review is based on the harness sample data.
""");
}
private static AIFunction[] CreateTools()
=>
[
AIFunctionFactory.Create(
GetIncidentStatus,
name: "get_incident_status",
description: "Get a mocked incident status for an incident id."),
AIFunctionFactory.Create(
GetServiceHealth,
name: "get_service_health",
description: "Get mocked health for an impacted service."),
AIFunctionFactory.Create(
GetOncallSummary,
name: "get_oncall_summary",
description: "Get a deterministic on-call response summary for the current incident."),
];
private static AIAgent[] CreateBackgroundAgents(IChatClient chatClient)
=>
[
chatClient.AsAIAgent(
instructions: """
You are operations-reviewer, a compact background incident response agent.
Given a delegated incident task, return exactly two concise bullets:
- one operational risk
- one next step
Use only deterministic harness knowledge. Do not claim to have queried external systems.
""",
name: "operations-reviewer",
description: "Background agent for concise operations incident checks."),
chatClient.AsAIAgent(
instructions: """
You are communications-reviewer, a compact background incident response agent.
Given a delegated incident task, return exactly two concise bullets:
- one communication risk
- one next update
Use only deterministic harness knowledge. Do not claim to have queried external systems.
""",
name: "communications-reviewer",
description: "Background agent for concise incident communication checks."),
];
private static ValueTask<bool> AutoApproveIncidentSummarySample(FunctionCallContent functionCall)
{
if (!string.Equals(functionCall.Name, FileAccessProvider.SaveFileToolName, StringComparison.Ordinal)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "fileName", out var fileName)
|| !fileName.Equals("incident-summary.md", StringComparison.OrdinalIgnoreCase)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "content", out var content))
{
return ValueTask.FromResult(false);
}
return ValueTask.FromResult(
content.Contains("INC-1042", StringComparison.OrdinalIgnoreCase)
&& content.Contains("impact", StringComparison.OrdinalIgnoreCase));
}
private static CompletionResult CompleteAdvertisedSample(string prompt, string responseText, string? workspace)
{
if (prompt.Equals("What is the status of incident INC-1042?", StringComparison.OrdinalIgnoreCase))
{
var status = GetIncidentStatus("INC-1042");
return CompletionResult.Replace(
$"Incident {status.Id} is {status.State}. Severity is {status.Severity}; current mitigation is {status.Mitigation}.",
"incident.advertised_sample",
"exact_sample_uses_stable_incident_status");
}
return CompletionResult.Preserve(responseText);
}
private static string BuildCompactionStressContext(string area)
{
var health = GetServiceHealth(area);
var repeatedContext = string.Join(
Environment.NewLine,
Enumerable.Range(1, 80).Select(index =>
$"Incident INC-1042 {area} note {index:00}: mocked response context, health {health.Status}, error budget pressure, customer impact, mitigation status, and handoff observation."));
return $"""
Prior incident context for {area}
Current mocked health summary: {health.Summary}
{repeatedContext}
""";
}
[Description("Get a mocked incident status for an incident id.")]
private static IncidentStatus GetIncidentStatus([Description("The incident id, such as INC-1042.")] string id)
=> new(
string.IsNullOrWhiteSpace(id) ? "INC-1042" : id.Trim().ToUpperInvariant(),
"SEV2",
"mitigating",
"Checkout latency is elevated for a subset of users.",
"Traffic is shifted away from the degraded checkout shard.");
[Description("Get mocked health for an impacted service.")]
private static ServiceHealth GetServiceHealth([Description("The service name, such as checkout or payments.")] string service)
{
Dictionary<string, ServiceHealth> health = new(StringComparer.OrdinalIgnoreCase)
{
["checkout"] = new("checkout", "degraded", "p95 latency is above target while mitigation is in progress."),
["payments"] = new("payments", "healthy", "payment authorization success rate is within normal bounds."),
["communications"] = new("communications", "ready", "customer update draft is ready for review."),
};
return health.GetValueOrDefault(
service,
new ServiceHealth(service.ToLowerInvariant(), "unknown", "No mocked health data is available for the requested service."));
}
[Description("Get a deterministic on-call response summary.")]
private static OncallSummary GetOncallSummary()
=> new(
"primary-oncall",
"engaged",
"Next customer update due in 30 minutes; continue checkout mitigation and monitor payments.");
private sealed record IncidentStatus(
string Id,
string Severity,
string State,
string Impact,
string Mitigation);
private sealed record ServiceHealth(
string Service,
string Status,
string Summary);
private sealed record OncallSummary(
string Owner,
string State,
string Summary);
}Scenarios\Release\Release.cs
using System.ComponentModel;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
public static class ReleaseScenario
{
public static readonly SamplePrompt[] Samples =
[
new("What is the release status for version 2.4.0?", "Function invocation"),
new("Remember that our release freeze window starts Friday. Then ask: When does the release freeze window start?", "Per-service-call history persistence"),
new("Review build status, QA issues, and security issues for release 2.4.0, then summarize readiness briefly.", "Compaction"),
new("Create a release checklist with verify QA signoff, confirm security review, and publish release notes.", "Todo provider"),
new("Remember that rollback must complete within 15 minutes.", "File memory provider"),
new("Create a file named release-notes.md for version 2.4.0 with highlights and watch items.", "File access provider"),
new("Create a file named deployment-plan.md for version 2.4.0.", "Tool approval"),
new("Search the web for the latest .NET release notes and summarize anything relevant to release planning.", "Web search"),
new("Use the release-risk-review skill to review release 2.4.0.", "Skills provider"),
new("Ask the QA and security background agents for one risk and one validation step each, then compare the results.", "Background agents"),
new("Use the shell to print the current working directory and OS name.", "Shell environment"),
new("Use looping to produce a two-sentence release risk note.", "Looping"),
];
public const string HarnessInstructions = """
You are a compact end-to-end harness demonstration agent.
You are also a software release manager assistant.
Keep responses short. Use tools only when useful.
Use get_build_status for the exact sample request "What is the release status for version 2.4.0?"
For release readiness requests, call get_build_status, get_release_risk, and get_issue_summary for relevant areas before summarizing.
Use the active AgentSession to answer follow-up questions such as "When does the release freeze window start?"
For multi-step planning requests, use the todo tools to create, inspect, and complete plan items.
For file creation, reading, and editing requests, call the provided file access tools.
For current release notes or dependency information, use the hosted web search tool.
If hosted web search is unavailable, say that search could not be executed instead of printing tool-call JSON.
For release readiness or go/no-go reviews, use the release-risk-review Agent Skill when it is available.
Do not call skill resources or scripts unless the loaded skill explicitly lists them.
For parallel release review, delegate independent QA and security checks to the available background agents.
For shell environment requests, use the shell tool and rely on the injected OS, shell, working-directory, and CLI probe context.
For the exact shell environment sample, run exactly: printf 'cwd=%s\n' "$PWD"; if command -v sw_vers >/dev/null 2>&1; then v="$(sw_vers -productVersion)"; case "$v" in 26.*) release='Tahoe' ;; *) release='' ;; esac; if [ -n "$release" ]; then printf 'os=macOS - %s %s\n' "$release" "$v"; else printf 'os=%s %s\n' "$(sw_vers -productName)" "$v"; fi; else printf 'os=%s\n' "$(uname -s)"; fi
On macOS, do not report uname -s as the user-facing OS name because it returns the Darwin kernel family.
For the exact looping sample, produce exactly two sentences about release risk. Do not mention todos, todo lists, the loop, attempts, process, or instructions. On the first attempt, omit LOOP_DONE. If loop feedback asks for completion, revise the answer and append LOOP_DONE at the end.
Keep file reads and writes scoped to the configured harness file access root.
After creating or editing a file, respond with the relative path and a brief summary.
""";
public const string ChatInstructions = """
Demonstrate the harness in a simple way:
- for "What is the release status for version 2.4.0?", call get_build_status and get_release_risk;
- for "Review build status, QA issues, and security issues for release 2.4.0, then summarize readiness briefly.", call get_build_status, get_issue_summary for qa, get_issue_summary for security, and get_release_risk, then summarize readiness briefly;
- for "Remember that our release freeze window starts Friday.", acknowledge the freeze window briefly;
- for "When does the release freeze window start?", use the chat history and answer Friday;
- for "Create a release checklist with verify QA signoff, confirm security review, and publish release notes.", use the todo tools to add those three todos, then summarize the remaining todo list;
- for "Create a file named release-notes.md for version 2.4.0 with highlights and watch items.", write release-notes.md using the file access tools, include version 2.4.0, highlights, and watch items, then say which relative file path was created;
- for "Create a file named deployment-plan.md for version 2.4.0.", write deployment-plan.md using the file access tools, then say which relative file path was created;
- for "Search the web for the latest .NET release notes and summarize anything relevant to release planning.", use the hosted web search tool and summarize current themes briefly;
- for "Use the release-risk-review skill to review release 2.4.0.", follow the release-risk-review Agent Skill structure;
- for "Ask the QA and security background agents for one risk and one validation step each, then compare the results.", start one task on qa-reviewer and one task on security-reviewer, wait for both results, then compare them briefly;
- for "Use the shell to print the current working directory and OS name.", call the shell tool with exactly: printf 'cwd=%s\n' "$PWD"; if command -v sw_vers >/dev/null 2>&1; then v="$(sw_vers -productVersion)"; case "$v" in 26.*) release='Tahoe' ;; *) release='' ;; esac; if [ -n "$release" ]; then printf 'os=macOS - %s %s\n' "$release" "$v"; else printf 'os=%s %s\n' "$(sw_vers -productName)" "$v"; fi; else printf 'os=%s\n' "$(uname -s)"; fi
- when interpreting shell OS output on macOS, report the macOS product/version, not Darwin;
- for "Use looping to produce a two-sentence release risk note.", produce exactly two sentences with no todo/process preamble; on the first attempt omit LOOP_DONE, and if loop feedback asks for completion then revise and append LOOP_DONE at the end;
- for other file requests, use the harness file tools and stay inside the scoped file root;
- use hosted web search for current release notes or dependency information;
- use the release-risk-review Agent Skill for release readiness reviews when it is available;
- use background agents for independent work that can run in parallel;
- use the shell tool for shell command execution and OS, shell, or working-directory checks;
- use the loop completion feedback to refine unfinished answers until the completion condition is met;
- do not call skill resources or scripts unless the loaded skill explicitly lists them;
- if hosted web search is unavailable, say that search could not be executed instead of printing tool-call JSON;
""";
public static ScenarioDefinition Create()
=> new(
Name: "release",
Description: "Software release manager harness sample.",
Samples: Samples,
HarnessInstructions: HarnessInstructions,
ChatInstructions: ChatInstructions,
SeedWorkspace: SeedWorkspace,
CreateTools: _ => CreateTools(),
CreateBackgroundAgents: CreateBackgroundAgents,
AutoApprovalRules: [AutoApproveReleaseNotesSample, HarnessApprovalRules.AutoApproveShellEnvironmentSample],
IsCompactionSamplePrompt: IsCompactionSamplePrompt,
CreatePromptMessages: CreatePromptMessages,
IsLoopingSamplePrompt: IsLoopingSamplePrompt,
GetLoopingSampleSubject: GetLoopingSampleSubject,
MentionsLoopingSampleSubject: MentionsLoopingSampleSubject,
CompleteAdvertisedSample: CompleteAdvertisedSample);
public static bool IsCompactionSamplePrompt(string prompt)
=> prompt.Equals("Review build status, QA issues, and security issues for release 2.4.0, then summarize readiness briefly.", StringComparison.OrdinalIgnoreCase);
public static ChatMessage[] CreatePromptMessages(string prompt)
{
if (!IsCompactionSamplePrompt(prompt))
{
return [new ChatMessage(ChatRole.User, prompt)];
}
return
[
new(ChatRole.User, BuildCompactionStressContext("build")),
new(ChatRole.Assistant, "Stored the release build context for later summarization."),
new(ChatRole.User, BuildCompactionStressContext("qa")),
new(ChatRole.Assistant, "Stored the release QA context for later summarization."),
new(ChatRole.User, BuildCompactionStressContext("security")),
new(ChatRole.Assistant, "Stored the release security context for later summarization."),
new(ChatRole.User, prompt),
];
}
public static bool IsLoopingSamplePrompt(string prompt)
=> prompt.Equals("Use looping to produce a two-sentence release risk note.", StringComparison.OrdinalIgnoreCase);
public static string GetLoopingSampleSubject(string prompt) => "release";
public static bool MentionsLoopingSampleSubject(string prompt, string responseText)
=> responseText.Contains("release", StringComparison.OrdinalIgnoreCase);
private static void SeedWorkspace(HarnessPaths paths)
{
var skillDirectory = Path.Combine(paths.SkillsRoot, "release-risk-review");
Directory.CreateDirectory(skillDirectory);
File.WriteAllText(Path.Combine(skillDirectory, "SKILL.md"), """
---
name: release-risk-review
description: Review release readiness using a concise risk-first structure.
---
# Release Risk Review
Use this skill when the user asks for release readiness, release risk, or go/no-go review.
Structure the response with exactly three short sections:
1. Blockers
2. Validation
3. Go/No-Go
Keep the language concise. Mention that the review is based on the harness sample data.
""");
}
private static AIFunction[] CreateTools()
=>
[
AIFunctionFactory.Create(
GetBuildStatus,
name: "get_build_status",
description: "Get a mocked release build status for a release version."),
AIFunctionFactory.Create(
GetReleaseRisk,
name: "get_release_risk",
description: "Get a deterministic release risk summary for the current sample release."),
AIFunctionFactory.Create(
GetIssueSummary,
name: "get_issue_summary",
description: "Get a mocked issue summary for a release area such as qa, security, or docs."),
];
private static AIAgent[] CreateBackgroundAgents(IChatClient chatClient)
=>
[
chatClient.AsAIAgent(
instructions: """
You are qa-reviewer, a compact background release readiness agent.
Given a delegated release task, return exactly two concise bullets:
- one quality risk
- one validation step
Use only deterministic harness knowledge. Do not claim to have queried external systems.
""",
name: "qa-reviewer",
description: "Background agent for concise quality readiness checks."),
chatClient.AsAIAgent(
instructions: """
You are security-reviewer, a compact background release readiness agent.
Given a delegated release task, return exactly two concise bullets:
- one security risk
- one mitigation step
Use only deterministic harness knowledge. Do not claim to have queried external systems.
""",
name: "security-reviewer",
description: "Background agent for concise security readiness checks."),
];
private static ValueTask<bool> AutoApproveReleaseNotesSample(FunctionCallContent functionCall)
{
if (!string.Equals(functionCall.Name, FileAccessProvider.SaveFileToolName, StringComparison.Ordinal)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "fileName", out var fileName)
|| !fileName.Equals("release-notes.md", StringComparison.OrdinalIgnoreCase)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "content", out var content))
{
return ValueTask.FromResult(false);
}
return ValueTask.FromResult(
content.Contains("release", StringComparison.OrdinalIgnoreCase)
&& content.Contains("2.4.0", StringComparison.OrdinalIgnoreCase));
}
private static CompletionResult CompleteAdvertisedSample(string prompt, string responseText, string? workspace)
{
if (prompt.Equals("What is the release status for version 2.4.0?", StringComparison.OrdinalIgnoreCase))
{
var status = GetBuildStatus("2.4.0");
var risk = GetReleaseRisk();
return CompletionResult.Replace(
$"Release {status.Version} is {status.Status} on build {status.BuildNumber}. Risk is {risk.Level}: {risk.Recommendation}",
"release.advertised_sample",
"exact_sample_uses_stable_release_status");
}
return CompletionResult.Preserve(responseText);
}
private static string BuildCompactionStressContext(string area)
{
var summary = GetIssueSummary(area);
var repeatedContext = string.Join(
Environment.NewLine,
Enumerable.Range(1, 80).Select(index =>
$"Release 2.4.0 {area} note {index:00}: mocked readiness context, blocker count {summary.Blockers}, watch item count {summary.WatchItems}, validation commentary, rollback sensitivity, and release-gate observation."));
return $"""
Prior release context for {area}
Current mocked issue summary: {summary.Summary}
{repeatedContext}
""";
}
[Description("Get a mocked release build status for a release version.")]
private static BuildStatus GetBuildStatus([Description("The release version, such as 2.4.0.")] string version)
{
var normalizedVersion = string.IsNullOrWhiteSpace(version) ? "2.4.0" : version.Trim();
return new BuildStatus(
normalizedVersion,
"green",
"2026.07.14.3",
128,
0,
"All release gates passed in the harness sample.");
}
[Description("Get a deterministic release risk summary.")]
private static ReleaseRisk GetReleaseRisk()
=> new(
"medium",
"One documentation follow-up and one post-deploy monitoring item remain.",
"Proceed with staged rollout after release notes are finalized.");
[Description("Get a mocked issue summary for a release area.")]
private static IssueSummary GetIssueSummary([Description("The release area, such as qa, security, or docs.")] string area)
{
Dictionary<string, IssueSummary> summaries = new(StringComparer.OrdinalIgnoreCase)
{
["qa"] = new("qa", 0, 2, "Regression suite is green; two exploratory checks remain."),
["security"] = new("security", 0, 1, "Dependency review is clear; one advisory watch item remains."),
["docs"] = new("docs", 0, 1, "Release notes need final customer-impact wording."),
};
return summaries.GetValueOrDefault(
area,
new IssueSummary(area.ToLowerInvariant(), 0, 0, "No open issues in the requested release area."));
}
private sealed record BuildStatus(
string Version,
string Status,
string BuildNumber,
int TestsPassed,
int TestsFailed,
string Summary);
private sealed record ReleaseRisk(
string Level,
string Reason,
string Recommendation);
private sealed record IssueSummary(
string Area,
int Blockers,
int WatchItems,
string Summary);
}Secenario\HarnessApprovalRules.cs
using System.Text.RegularExpressions;
using Microsoft.Extensions.AI;
public static class HarnessApprovalRules
{
public static ValueTask<bool> AutoApproveShellEnvironmentSample(FunctionCallContent functionCall)
{
if (!string.Equals(functionCall.Name, "run_shell", StringComparison.Ordinal)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "command", out var command))
{
return ValueTask.FromResult(false);
}
return ValueTask.FromResult(IsSafeShellEnvironmentProbe(command));
}
public static bool IsSafeShellEnvironmentProbe(string command)
{
var normalized = Regex.Replace(command.Trim(), @"\s+", " ");
return normalized is "printf 'cwd=%s\\n' \"$PWD\"; if command -v sw_vers >/dev/null 2>&1; then v=\"$(sw_vers -productVersion)\"; case \"$v\" in 26.*) release='Tahoe' ;; *) release='' ;; esac; if [ -n \"$release\" ]; then printf 'os=macOS - %s %s\\n' \"$release\" \"$v\"; else printf 'os=%s %s\\n' \"$(sw_vers -productName)\" \"$v\"; fi; else printf 'os=%s\\n' \"$(uname -s)\"; fi"
or "printf \"cwd=%s\\n\" \"$PWD\"; if command -v sw_vers >/dev/null 2>&1; then v=\"$(sw_vers -productVersion)\"; case \"$v\" in 26.*) release=\"Tahoe\" ;; *) release=\"\" ;; esac; if [ -n \"$release\" ]; then printf \"os=macOS - %s %s\\n\" \"$release\" \"$v\"; else printf \"os=%s %s\\n\" \"$(sw_vers -productName)\" \"$v\"; fi; else printf \"os=%s\\n\" \"$(uname -s)\"; fi";
}
}Secenario\ScenarioCatalog.cs
public static class ScenarioCatalog
{
public static readonly string[] Names = ["finance", "incident", "release"];
public static ScenarioDefinition Create(string scenarioName)
{
if (TryCreate(scenarioName, out var scenario))
{
return scenario;
}
throw new ArgumentException($"Unknown scenario '{scenarioName}'. Available scenarios: {string.Join(", ", Names)}.");
}
public static bool TryCreate(string scenarioName, out ScenarioDefinition scenario)
{
switch (scenarioName.ToLowerInvariant())
{
case "finance":
scenario = FinanceScenario.Create();
return true;
case "incident":
scenario = IncidentScenario.Create();
return true;
case "release":
scenario = ReleaseScenario.Create();
return true;
default:
scenario = null!;
return false;
}
}
public static bool IsLoopingSamplePrompt(string prompt)
=> Names.Select(Create).Any(scenario => scenario.IsLoopingSamplePrompt(prompt));
public static string GetLoopingSampleSubject(string prompt)
=> Names.Select(Create).FirstOrDefault(scenario => scenario.IsLoopingSamplePrompt(prompt))?.GetLoopingSampleSubject(prompt)
?? "MSFT";
public static bool MentionsLoopingSampleSubject(string prompt, string responseText)
=> Names.Select(Create).FirstOrDefault(scenario => scenario.IsLoopingSamplePrompt(prompt))?.MentionsLoopingSampleSubject(prompt, responseText)
?? FinanceScenario.MentionsLoopingSampleSubject(prompt, responseText);
}Secenario\ScenarioDefinition.cs
using System.Text.Json;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
public sealed record ScenarioDefinition(
string Name,
string Description,
SamplePrompt[] Samples,
string HarnessInstructions,
string ChatInstructions,
Action<HarnessPaths> SeedWorkspace,
Func<HarnessPaths, AIFunction[]> CreateTools,
Func<IChatClient, AIAgent[]> CreateBackgroundAgents,
Func<FunctionCallContent, ValueTask<bool>>[] AutoApprovalRules,
Func<string, bool> IsCompactionSamplePrompt,
Func<string, ChatMessage[]> CreatePromptMessages,
Func<string, bool> IsLoopingSamplePrompt,
Func<string, string> GetLoopingSampleSubject,
Func<string, string, bool> MentionsLoopingSampleSubject,
Func<string, string, string?, CompletionResult> CompleteAdvertisedSample);
public sealed record SamplePrompt(string Prompt, string Capability);
public sealed record CompletionResult(string Text, bool ReplacedModelResponse, string Source, string Reason)
{
public static CompletionResult Preserve(string text)
=> new(text, false, "model", "model_response_used");
public static CompletionResult Replace(string text, string source, string reason)
=> new(text, true, source, reason);
}
internal static class ScenarioArguments
{
public static bool TryGetStringArgument(FunctionCallContent functionCall, string name, out string value)
{
value = string.Empty;
if (functionCall.Arguments is not { } arguments
|| !arguments.TryGetValue(name, out var rawValue)
|| rawValue is null)
{
return false;
}
value = rawValue switch
{
string stringValue => stringValue,
JsonElement { ValueKind: JsonValueKind.String } jsonValue => jsonValue.GetString() ?? string.Empty,
_ => rawValue.ToString() ?? string.Empty,
};
return value.Length > 0;
}
}HarnessRuntimes.cs
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Tools.Shell;
public sealed record HarnessPaths(string Workspace, string FilesRoot, string MemoryRoot, string SkillsRoot)
{
public static HarnessPaths Create(string sourceDirectory, string scenarioName)
{
var workspace = Environment.GetEnvironmentVariable("E2E_HARNESS_WORKSPACE")
?? Path.Combine(sourceDirectory, "scenario-workspace", scenarioName);
return new HarnessPaths(
workspace,
Path.Combine(workspace, "files"),
Path.Combine(workspace, "memory"),
workspace);
}
}
public sealed record HarnessRuntime(AIAgent Agent, ShellExecutor ShellExecutor, IDisposable? ChatClient = null) : IAsyncDisposable
{
public async ValueTask DisposeAsync()
{
await ShellExecutor.DisposeAsync();
ChatClient?.Dispose();
}
}HarnessShell.cs
using Microsoft.Agents.AI.Tools.Shell;
public static class HarnessShell
{
private static readonly string[] AllowedCommandPatterns =
[
@"^\s*(printf|echo|pwd|ls|cat|head|tail|wc|find|rg|grep|sed|awk|sort|uniq|cut|tr|date|uname|sw_vers|git|dotnet|ollama|command)\b",
];
private static readonly string[] BlockedCommandPatterns =
[
@"\brm\s+-rf\b",
@"\bsudo\b",
@"\bchmod\s+-R\b",
@"\bchown\s+-R\b",
@"\bmkfs\b",
@"\bdd\s+",
@"\bcurl\b.*\|\s*(sh|bash)\b",
@"\bwget\b.*\|\s*(sh|bash)\b",
];
public static ShellExecutor CreateExecutor(HarnessPaths paths)
=> new LocalShellExecutor(new LocalShellExecutorOptions
{
WorkingDirectory = paths.Workspace,
ConfineWorkingDirectory = true,
Timeout = TimeSpan.FromSeconds(10),
MaxOutputBytes = 16 * 1024,
CleanEnvironment = true,
Policy = new ShellPolicy(
denyList: BlockedCommandPatterns,
allowList: AllowedCommandPatterns),
});
}HarnessTelemetry.cs
using System.Diagnostics;
using System.Diagnostics.Metrics;
using Microsoft.Extensions.Logging;
using OpenTelemetry;
using OpenTelemetry.Logs;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
public static class HarnessTelemetry
{
public const string ActivitySourceName = "agent-harness.Console";
public const string HarnessActivitySourceName = "E2E.Harness.Sample";
public const string MeterName = "agent-harness.Console";
public static readonly ActivitySource ActivitySource = new(ActivitySourceName);
public static readonly Meter Meter = new(MeterName);
public static readonly Counter<long> PromptCounter = Meter.CreateCounter<long>("harness.prompts");
public static readonly Histogram<double> PromptDuration = Meter.CreateHistogram<double>("harness.prompt.duration.ms");
public static ILogger Logger { get; private set; } = NoopLogger.Instance;
public static OpenTelemetrySdk? Configure()
{
if (IsEnabledEnvironmentVariable("E2E_HARNESS_DISABLE_OTEL"))
{
return null;
}
var hasOtlpEndpoint =
!string.IsNullOrWhiteSpace(Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_ENDPOINT"))
|| !string.IsNullOrWhiteSpace(Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT"))
|| !string.IsNullOrWhiteSpace(Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_METRICS_ENDPOINT"))
|| !string.IsNullOrWhiteSpace(Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_LOGS_ENDPOINT"));
if (!hasOtlpEndpoint)
{
return null;
}
var sdk = OpenTelemetrySdk.Create(builder => builder
.ConfigureResource(resource => resource.AddService(
serviceName: Environment.GetEnvironmentVariable("OTEL_SERVICE_NAME") ?? "agent-harness-console"))
.WithTracing(tracing => tracing
.AddSource(ActivitySourceName)
.AddSource(HarnessActivitySourceName)
.AddOtlpExporter())
.WithMetrics(metrics => metrics
.AddMeter(MeterName)
.AddOtlpExporter())
.WithLogging(logging => logging.AddOtlpExporter()));
Logger = sdk.GetLoggerFactory().CreateLogger("agent-harness.Console");
return sdk;
}
private static bool IsEnabledEnvironmentVariable(string name)
{
var value = Environment.GetEnvironmentVariable(name);
return value is not null
&& value.Length > 0
&& !value.Equals("0", StringComparison.OrdinalIgnoreCase)
&& !value.Equals("false", StringComparison.OrdinalIgnoreCase)
&& !value.Equals("off", StringComparison.OrdinalIgnoreCase);
}
private sealed class NoopLogger : ILogger
{
public static readonly NoopLogger Instance = new();
public IDisposable? BeginScope<TState>(TState state)
where TState : notnull
=> null;
public bool IsEnabled(LogLevel logLevel)
=> false;
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter)
{
}
}
}OllamaSettings.cs
using System.Data.Common;
public sealed record OllamaSettings(string Endpoint, string ModelName)
{
public static OllamaSettings FromConnectionString(string connectionString)
{
var builder = new DbConnectionStringBuilder
{
ConnectionString = connectionString,
};
var endpoint = GetRequiredValue(builder, "Endpoint", "Uri", "Url");
var modelName = GetRequiredValue(builder, "Model", "ModelName");
return new OllamaSettings(endpoint, modelName);
}
private static string GetRequiredValue(DbConnectionStringBuilder builder, params string[] names)
{
foreach (var name in names)
{
if (builder.TryGetValue(name, out var value)
&& value?.ToString() is { Length: > 0 } stringValue)
{
return stringValue;
}
}
throw new InvalidOperationException(
$"ConnectionStrings__ollama is missing required value. Expected one of: {string.Join(", ", names)}.");
}
}Program.cs
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System.Text.Json;
using System.Text.RegularExpressions;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Tools.Shell;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Logging;
using OllamaSharp;
if (args.Any(arg => arg.Equals("--telemetry-smoke-test", StringComparison.OrdinalIgnoreCase)))
{
using var smokeTelemetry = HarnessTelemetry.Configure();
EmitTelemetrySmokeTest();
return;
}
var ollamaSettings = OllamaSettings.FromConnectionString(GetRequiredEnvironmentVariable("ConnectionStrings__ollama"));
var endpoint = ollamaSettings.Endpoint;
var modelName = ollamaSettings.ModelName;
using var telemetry = HarnessTelemetry.Configure();
const string LoopCompletionMarker = "LOOP_DONE";
var startupOptions = StartupOptions.Parse(args);
var scenario = ScenarioCatalog.Create(startupOptions.ScenarioName);
var capabilities = HarnessCapabilities.All;
var paths = HarnessPaths.Create(GetSourceDirectory(), scenario.Name);
PrepareWorkspace(paths, scenario);
if (startupOptions.PromptArgs.Length > 0)
{
if (!await ValidateModelAsync(endpoint, modelName))
{
Environment.ExitCode = 1;
return;
}
await using var runtime = CreateRuntime(endpoint, modelName, paths, scenario);
if (!await RunPromptAsync(runtime.Agent, string.Join(' ', startupOptions.PromptArgs), scenario, workspace: paths.Workspace, fileAccessRoot: paths.FilesRoot))
{
Environment.ExitCode = 1;
}
}
else
{
await RunCliAsync(scenario, capabilities, paths, endpoint, modelName);
}
static async Task<bool> ValidateModelAsync(string endpoint, string modelName)
{
if (!IsLikelyToolCallingModel(modelName))
{
Console.Error.WriteLine($"""
The configured Ollama model '{modelName}' is not known to support native tool calling.
Try a tool-capable local model:
ollama pull llama3.1
aspire start
""");
return false;
}
if (await IsModelInstalledAsync(endpoint, modelName))
{
return true;
}
Console.Error.WriteLine($"""
The Ollama model '{modelName}' is not installed locally.
Pull it first, then rerun this sample:
ollama pull {modelName}
aspire start
""");
return false;
}
static string GetRequiredEnvironmentVariable(string name)
=> Environment.GetEnvironmentVariable(name)
?? throw new InvalidOperationException($"{name} is required. Start the harness through the Aspire AppHost.");
static void PrepareWorkspace(HarnessPaths paths, ScenarioDefinition scenario)
{
Directory.CreateDirectory(paths.Workspace);
Directory.CreateDirectory(paths.FilesRoot);
Directory.CreateDirectory(paths.MemoryRoot);
Directory.CreateDirectory(paths.SkillsRoot);
scenario.SeedWorkspace(paths);
}
static void EmitTelemetrySmokeTest()
{
var tags = new KeyValuePair<string, object?>[]
{
new("harness.scenario", "telemetry-smoke"),
new("harness.mode", "smoke"),
new("harness.prompt.sample", false),
};
HarnessTelemetry.PromptCounter.Add(1, tags);
using var activity = HarnessTelemetry.ActivitySource.StartActivity("harness.telemetry.smoke");
activity?.SetTag("harness.scenario", "telemetry-smoke");
activity?.SetTag("harness.mode", "smoke");
HarnessTelemetry.PromptDuration.Record(1, tags);
HarnessTelemetry.Logger.LogInformation(
"Harness telemetry smoke test emitted for scenario {Scenario} in mode {Mode}",
"telemetry-smoke",
"smoke");
activity?.SetStatus(ActivityStatusCode.Ok);
}
HarnessRuntime CreateRuntime(string endpoint, string modelName, HarnessPaths paths, ScenarioDefinition scenario)
{
var httpClient = new HttpClient
{
BaseAddress = new Uri(endpoint),
Timeout = GetOllamaTimeout(),
};
IChatClient chatClient = new ChatClientBuilder(new OllamaApiClient(httpClient, modelName, jsonSerializerContext: null))
.UseOpenTelemetry(loggerFactory: null, sourceName: HarnessTelemetry.ActivitySourceName, configure: ConfigureAiTelemetry)
.Build();
var shellExecutor = HarnessShell.CreateExecutor(paths);
var agent = CreateHarnessAgent(chatClient, scenario, paths, shellExecutor);
return new HarnessRuntime(agent, shellExecutor, chatClient as IDisposable ?? httpClient);
}
static void ConfigureAiTelemetry(OpenTelemetryChatClient client)
{
client.EnableSensitiveData = IsEnabledEnvironmentVariable("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT");
}
static TimeSpan GetOllamaTimeout()
{
const int defaultTimeoutSeconds = 600;
var value = Environment.GetEnvironmentVariable("E2E_HARNESS_OLLAMA_TIMEOUT_SECONDS");
if (int.TryParse(value, out var timeoutSeconds) && timeoutSeconds > 0)
{
return TimeSpan.FromSeconds(timeoutSeconds);
}
return TimeSpan.FromSeconds(defaultTimeoutSeconds);
}
AIAgent CreateHarnessAgent(
IChatClient chatClient,
ScenarioDefinition scenario,
HarnessPaths paths,
ShellExecutor shellExecutor)
{
var options = CreateHarnessOptions(chatClient, scenario, paths, shellExecutor);
return chatClient.AsHarnessAgent(options);
}
HarnessAgentOptions CreateHarnessOptions(
IChatClient chatClient,
ScenarioDefinition scenario,
HarnessPaths paths,
ShellExecutor shellExecutor)
=> new()
{
Name = "E2E-Harness",
Description = "A focused harness sample for function invocation, session history, and context management.",
MaxContextWindowTokens = 8_192,
MaxOutputTokens = 768,
CompactionStrategy = ReportingCompactionStrategy.Instance,
DisableCompaction = false,
MaximumIterationsPerRequest = 6,
DisableTodoProvider = false,
DisableAgentModeProvider = true,
DisableFileMemory = false,
DisableFileAccess = false,
DisableAgentSkillsProvider = false,
DisableWebSearch = false,
AIContextProviders = [],
DisableToolAutoApproval = false,
DisableNonApprovalRequiredFunctionBypassing = false,
DisableOpenTelemetry = IsEnabledEnvironmentVariable("E2E_HARNESS_DISABLE_OTEL"),
OpenTelemetrySourceName = "E2E.Harness.Sample",
BackgroundAgents = scenario.CreateBackgroundAgents(chatClient),
ShellExecutor = shellExecutor,
ShellEnvironmentProviderOptions = CreateShellEnvironmentProviderOptions(),
LoopEvaluators = CreateLoopEvaluators(),
LoopAgentOptions = CreateLoopOptions(),
ToolApprovalAgentOptions = new ToolApprovalAgentOptions
{
AutoApprovalRules = scenario.AutoApprovalRules,
},
FileMemoryStore = new FileSystemAgentFileStore(paths.MemoryRoot),
FileAccessStore = new FileSystemAgentFileStore(paths.FilesRoot),
HarnessInstructions = scenario.HarnessInstructions,
ChatOptions = CreateHarnessChatOptions(scenario, paths),
};
ChatOptions CreateHarnessChatOptions(ScenarioDefinition scenario, HarnessPaths paths)
=> new()
{
Temperature = 0,
MaxOutputTokens = 768,
AllowMultipleToolCalls = true,
Instructions = scenario.ChatInstructions,
Tools = scenario.CreateTools(paths),
};
static ShellEnvironmentProviderOptions CreateShellEnvironmentProviderOptions()
=> new()
{
ProbeTools = ["dotnet", "git", "ollama"],
ProbeTimeout = TimeSpan.FromSeconds(3),
};
static LoopEvaluator[] CreateLoopEvaluators()
=>
[
new DelegateLoopEvaluator((context, cancellationToken) =>
{
if (!ScenarioCatalog.IsLoopingSamplePrompt(GetInitialUserPrompt(context)))
{
return ValueTask.FromResult(LoopEvaluation.Stop());
}
var responseText = context.LastResponse.Text ?? string.Empty;
var initialPrompt = GetInitialUserPrompt(context);
var evaluation = EvaluateLoopingSampleResponse(initialPrompt, context.Iteration, responseText);
WriteMuted($"[loop] iteration {context.Iteration} {evaluation.Status}: {evaluation.Reason}");
if (evaluation.Complete)
{
return ValueTask.FromResult(LoopEvaluation.Stop());
}
return ValueTask.FromResult(LoopEvaluation.Continue($"""
{evaluation.Feedback}
Revise the answer into exactly two final sentences about {ScenarioCatalog.GetLoopingSampleSubject(initialPrompt)} risk. Do not mention todos, todo lists, the loop, attempts, process, or instructions. Return only the two-sentence risk note, then append {LoopCompletionMarker} at the end when the answer is complete.
"""));
}),
];
static LoopAgentOptions CreateLoopOptions()
=> new()
{
MaxIterations = 3,
NonStreamingReturnsLastResponseOnly = true,
ExcludeOnBehalfOfMessages = true,
OnBehalfOfAuthorName = "loop",
};
async Task RunCliAsync(
ScenarioDefinition scenario,
string[] capabilities,
HarnessPaths paths,
string endpoint,
string modelName)
{
HarnessRuntime? runtime = null;
try
{
runtime = CreateRuntime(endpoint, modelName, paths, scenario);
var agent = runtime.Agent;
var state = new CliState(null, "build", null, null, new ToolApprovalState());
PrintWelcome(paths.Workspace, modelName, scenario, state.CurrentMode, capabilities.Length);
while (true)
{
Console.WriteLine();
WritePrompt(state.CurrentMode);
var input = Console.ReadLine();
if (input is null)
{
break;
}
input = input.Trim();
if (input.Length == 0)
{
continue;
}
if (input.StartsWith('/'))
{
if (TryParseScenarioSwitch(input, out var requestedScenarioName))
{
if (!ScenarioCatalog.TryCreate(requestedScenarioName, out var nextScenario))
{
WriteError($"Unknown scenario '{requestedScenarioName}'. Available scenarios: {string.Join(", ", ScenarioCatalog.Names)}.");
continue;
}
await runtime.DisposeAsync();
scenario = nextScenario;
paths = HarnessPaths.Create(GetSourceDirectory(), scenario.Name);
PrepareWorkspace(paths, scenario);
runtime = CreateRuntime(endpoint, modelName, paths, scenario);
agent = runtime.Agent;
state = new CliState(null, "build", null, null, new ToolApprovalState());
PrintWelcome(paths.Workspace, modelName, scenario, state.CurrentMode, capabilities.Length);
continue;
}
var result = HandleCommand(input, runtime, scenario, capabilities, paths, modelName, state);
state = result.State;
if (!result.Continue)
{
break;
}
continue;
}
if (!await ValidateModelAsync(endpoint, modelName))
{
continue;
}
var session = state.Session ?? await agent.CreateSessionAsync();
var knownUserName = TryExtractUserName(input, out var userName)
? userName
: state.KnownUserName;
var knownReleaseFreezeWindow = TryExtractReleaseFreezeWindow(input, out var releaseFreezeWindow)
? releaseFreezeWindow
: state.KnownReleaseFreezeWindow;
state = state with { Session = session, KnownUserName = knownUserName, KnownReleaseFreezeWindow = knownReleaseFreezeWindow };
await RunPromptAsync(agent, input, scenario, session, state.CurrentMode, paths.Workspace, state.KnownUserName, state.KnownReleaseFreezeWindow, paths.FilesRoot, state.ToolApprovals);
}
}
finally
{
if (runtime is not null)
{
await runtime.DisposeAsync();
}
}
}
static void WritePrompt(string currentMode)
{
WriteRuleWide();
Console.ForegroundColor = ConsoleColor.White;
Console.Write("> ");
Console.ResetColor();
}
CommandResult HandleCommand(
string input,
HarnessRuntime runtime,
ScenarioDefinition scenario,
string[] capabilities,
HarnessPaths paths,
string modelName,
CliState state)
{
var parts = input.Split(' ', 2, StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
var command = parts[0].ToLowerInvariant();
switch (command)
{
case "/exit":
case "/quit":
Console.WriteLine("bye");
return new CommandResult(false, state);
case "/help":
PrintHelp(paths.FilesRoot);
return new CommandResult(true, state);
case "/scenarios":
PrintScenarios();
return new CommandResult(true, state);
case "/scenario":
Console.WriteLine($"usage: /scenario <{string.Join("|", ScenarioCatalog.Names)}>");
return new CommandResult(true, state);
case "/init":
InitializeHarnessInstructions(paths, scenario);
return new CommandResult(true, state);
case "/capabilities":
PrintCapabilities(capabilities);
return new CommandResult(true, state);
case "/clear":
ClearScreen();
var clearedState = new CliState(null, "build", null, null, new ToolApprovalState());
PrintWelcome(paths.Workspace, modelName, scenario, clearedState.CurrentMode, capabilities.Length);
return new CommandResult(true, clearedState);
case "/sample":
PrintSamples(scenario.Samples);
return new CommandResult(true, state);
default:
Console.WriteLine($"unknown command: {command}. Try /help.");
return new CommandResult(true, state);
}
}
static bool TryParseScenarioSwitch(string input, out string scenarioName)
{
scenarioName = string.Empty;
var parts = input.Split(' ', 2, StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
if (parts.Length != 2 || !parts[0].Equals("/scenario", StringComparison.OrdinalIgnoreCase))
{
return false;
}
scenarioName = parts[1].ToLowerInvariant();
return true;
}
async Task<bool> RunPromptAsync(
AIAgent agent,
string prompt,
ScenarioDefinition scenario,
AgentSession? session = null,
string mode = "build",
string? workspace = null,
string? knownUserName = null,
string? knownReleaseFreezeWindow = null,
string? fileAccessRoot = null,
ToolApprovalState? toolApprovals = null)
{
var showRawResponse = IsEnabledEnvironmentVariable("E2E_HARNESS_SHOW_RAW_RESPONSE");
var isSamplePrompt = scenario.Samples.Any(sample => prompt.Equals(sample.Prompt, StringComparison.OrdinalIgnoreCase));
var promptTags = new KeyValuePair<string, object?>[]
{
new("harness.scenario", scenario.Name),
new("harness.mode", mode),
new("harness.prompt.sample", isSamplePrompt),
};
var startedAt = Stopwatch.GetTimestamp();
HarnessTelemetry.PromptCounter.Add(1, promptTags);
HarnessTelemetry.Logger.LogInformation(
"Running harness prompt for scenario {Scenario} in mode {Mode}. SamplePrompt={SamplePrompt}",
scenario.Name,
mode,
isSamplePrompt);
using var activity = HarnessTelemetry.ActivitySource.StartActivity("harness.prompt");
activity?.SetTag("harness.scenario", scenario.Name);
activity?.SetTag("harness.mode", mode);
activity?.SetTag("harness.workspace", workspace);
activity?.SetTag("harness.prompt.length", prompt.Length);
activity?.SetTag("harness.prompt.sample", isSamplePrompt);
try
{
session ??= await agent.CreateSessionAsync();
toolApprovals ??= new ToolApprovalState();
var compactionCountBeforeRun = ReportingCompactionStrategy.AppliedCount;
var response = await RunMessagesWithApprovalPromptsAsync(
agent,
scenario.CreatePromptMessages(prompt),
session,
fileAccessRoot,
toolApprovals,
"main");
var rawResponseText = response.Text?.Trim() ?? string.Empty;
if (showRawResponse)
{
WriteMuted(string.IsNullOrWhiteSpace(rawResponseText)
? "[raw llm response] <empty>"
: $"[raw llm response] {rawResponseText}");
}
activity?.SetTag("harness.response.empty", string.IsNullOrWhiteSpace(rawResponseText));
var responseText = string.IsNullOrWhiteSpace(rawResponseText)
? "The harness ran, but the local model returned an empty final message. Try a stronger tool-calling model or a more direct prompt."
: StripDoneMarker(rawResponseText);
var modelResponseClassification = ClassifyModelResponse(rawResponseText, responseText);
activity?.SetTag("harness.model.response.classification", modelResponseClassification);
if (LooksLikeTodoPseudoToolCall(responseText))
{
WriteError("""
The model returned a text-form todo tool call instead of a native tool call, so no todo item was created.
Use a model/provider combination that returns real tool calls for the todo provider.
""");
return false;
}
if (LooksLikeRecoverablePseudoToolCall(responseText))
{
response = await RunMessagesWithApprovalPromptsAsync(
agent,
[new ChatMessage(ChatRole.User, BuildPseudoToolRecoveryPrompt(prompt, responseText))],
session,
fileAccessRoot,
toolApprovals,
"pseudo_tool_recovery");
rawResponseText = response.Text?.Trim() ?? string.Empty;
if (showRawResponse)
{
WriteMuted(string.IsNullOrWhiteSpace(rawResponseText)
? "[raw llm response] <empty>"
: $"[raw llm response] {rawResponseText}");
}
responseText = string.IsNullOrWhiteSpace(rawResponseText)
? "The harness ran, but the local model returned an empty final message after pseudo tool-call recovery."
: StripDoneMarker(rawResponseText);
if (LooksLikeRecoverablePseudoToolCall(responseText))
{
response = await RunMessagesWithApprovalPromptsAsync(
agent,
[new ChatMessage(ChatRole.User, BuildRepeatedPseudoToolFinalAnswerPrompt(prompt, responseText))],
session,
fileAccessRoot,
toolApprovals,
"repeated_pseudo_tool_recovery");
rawResponseText = response.Text?.Trim() ?? string.Empty;
if (showRawResponse)
{
WriteMuted(string.IsNullOrWhiteSpace(rawResponseText)
? "[raw llm response] <empty>"
: $"[raw llm response] {rawResponseText}");
}
responseText = string.IsNullOrWhiteSpace(rawResponseText)
? "The harness ran, but the local model returned an empty final message after repeated pseudo tool-call recovery."
: StripDoneMarker(rawResponseText);
if (LooksLikeRecoverablePseudoToolCall(responseText))
{
WriteError("""
The model repeated a text-form tool call instead of producing a natural-language answer.
No native tool result was available from this run.
""");
return false;
}
}
}
if (IsShellEnvironmentSamplePrompt(prompt)
&& TryGetShellEnvironmentProbe(response, out var shellProbe)
&& !ShellEnvironmentResponseMatchesProbe(responseText, shellProbe))
{
response = await RunMessagesWithApprovalPromptsAsync(
agent,
[new ChatMessage(ChatRole.User, BuildShellEnvironmentCorrectionPrompt(prompt, responseText, shellProbe))],
session,
fileAccessRoot,
toolApprovals,
"shell_environment_correction");
rawResponseText = response.Text?.Trim() ?? string.Empty;
if (showRawResponse)
{
WriteMuted(string.IsNullOrWhiteSpace(rawResponseText)
? "[raw llm response] <empty>"
: $"[raw llm response] {rawResponseText}");
}
responseText = string.IsNullOrWhiteSpace(rawResponseText)
? $"The shell command returned cwd={shellProbe.Cwd} and os={shellProbe.Os}, but the local model returned an empty correction."
: StripDoneMarker(rawResponseText);
}
responseText = StripLoopCompletionMarker(responseText);
var completionEvents = new List<CompletionResult>();
responseText = ApplyCompletionResult(
scenario.CompleteAdvertisedSample(prompt, responseText, workspace),
completionEvents);
responseText = ApplyCompletionResult(
CompleteHistoryPersistenceSample(prompt, responseText, knownUserName, knownReleaseFreezeWindow),
completionEvents);
RecordCompletionDiagnostics(activity, completionEvents);
if (scenario.IsCompactionSamplePrompt(prompt) && ReportingCompactionStrategy.AppliedCount > compactionCountBeforeRun)
{
WriteMuted($"[compaction] applied {ReportingCompactionStrategy.AppliedCount - compactionCountBeforeRun} time(s) during this run");
}
WriteAssistant(responseText);
activity?.SetStatus(ActivityStatusCode.Ok);
RecordPromptDuration(startedAt, promptTags);
HarnessTelemetry.Logger.LogInformation(
"Harness prompt completed for scenario {Scenario} in mode {Mode}. EmptyResponse={EmptyResponse}",
scenario.Name,
mode,
string.IsNullOrWhiteSpace(rawResponseText));
return true;
}
catch (Exception exception) when (exception is InvalidOperationException or HttpRequestException or TaskCanceledException)
{
activity?.SetStatus(ActivityStatusCode.Error, exception.Message);
activity?.AddException(exception);
RecordPromptDuration(startedAt, promptTags);
HarnessTelemetry.Logger.LogError(
exception,
"Harness prompt failed for scenario {Scenario} in mode {Mode}",
scenario.Name,
mode);
WriteError(exception.Message);
return false;
}
catch (Exception exception)
{
activity?.SetStatus(ActivityStatusCode.Error, exception.Message);
activity?.AddException(exception);
RecordPromptDuration(startedAt, promptTags);
HarnessTelemetry.Logger.LogError(
exception,
"Harness prompt failed for scenario {Scenario} in mode {Mode}",
scenario.Name,
mode);
WriteError(exception.ToString());
return false;
}
}
static string ApplyCompletionResult(CompletionResult result, List<CompletionResult> completionEvents)
{
if (result.ReplacedModelResponse)
{
completionEvents.Add(result);
}
return result.Text;
}
static void RecordCompletionDiagnostics(Activity? activity, List<CompletionResult> completionEvents)
{
activity?.SetTag("harness.completion.replaced", completionEvents.Count > 0);
activity?.SetTag("harness.completion.count", completionEvents.Count);
if (completionEvents.Count == 0)
{
activity?.SetTag("harness.completion.source", "model");
activity?.SetTag("harness.completion.reason", "model_response_used");
return;
}
for (var index = 0; index < completionEvents.Count; index++)
{
var completion = completionEvents[index];
var prefix = $"harness.completion.{index}";
activity?.SetTag($"{prefix}.source", completion.Source);
activity?.SetTag($"{prefix}.reason", completion.Reason);
WriteMuted($"[harness completion] replaced model response: {completion.Source} ({completion.Reason})");
}
activity?.SetTag("harness.completion.source", completionEvents[^1].Source);
activity?.SetTag("harness.completion.reason", completionEvents[^1].Reason);
}
static string ClassifyModelResponse(string rawResponseText, string responseText)
{
if (string.IsNullOrWhiteSpace(rawResponseText))
{
return "empty";
}
if (LooksLikePseudoToolCall(responseText))
{
return "pseudo_tool_call_text";
}
return "natural_language";
}
static void RecordPromptDuration(long startedAt, KeyValuePair<string, object?>[] tags)
{
var elapsed = Stopwatch.GetElapsedTime(startedAt);
HarnessTelemetry.PromptDuration.Record(elapsed.TotalMilliseconds, tags);
}
async Task<AgentResponse> RunMessagesWithApprovalPromptsAsync(
AIAgent agent,
IEnumerable<ChatMessage> messages,
AgentSession session,
string? fileAccessRoot,
ToolApprovalState toolApprovals,
string initialTurnKind)
{
var initialMessages = messages as ChatMessage[] ?? messages.ToArray();
var response = await RunAgentTurnWithThinkingAsync(
() => agent.RunAsync(initialMessages, session),
initialTurnKind,
turnIndex: 0,
inputMessageCount: initialMessages.Length);
for (var approvalsHandled = 0; approvalsHandled < 10; approvalsHandled++)
{
var approvalRequest = FindApprovalRequest(response);
if (approvalRequest is null)
{
return response;
}
var decision = ResolveToolApproval(approvalRequest, fileAccessRoot, toolApprovals);
var approved = decision.Approved;
var approvalResponse = approvalRequest.CreateResponse(
approved,
decision.Message);
var approvalMessages = new[] { new ChatMessage(ChatRole.User, new AIContent[] { approvalResponse }) };
response = await RunAgentTurnWithThinkingAsync(
() => agent.RunAsync(
approvalMessages,
session),
"tool_approval_response",
approvalsHandled + 1,
approvalMessages.Length);
}
throw new InvalidOperationException("Too many tool approval requests in a single run.");
}
static async Task<AgentResponse> RunAgentTurnWithThinkingAsync(
Func<Task<AgentResponse>> runAsync,
string turnKind,
int turnIndex,
int inputMessageCount)
{
var startedAt = Stopwatch.GetTimestamp();
using var activity = HarnessTelemetry.ActivitySource.StartActivity("harness.agent_turn");
activity?.SetTag("harness.agent_turn.kind", turnKind);
activity?.SetTag("harness.agent_turn.index", turnIndex);
activity?.SetTag("harness.agent_turn.input_messages", inputMessageCount);
var thinking = ThinkingIndicator.Start();
try
{
var response = await runAsync();
activity?.SetTag("harness.agent_turn.response.empty", string.IsNullOrWhiteSpace(response.Text));
activity?.SetTag("harness.agent_turn.response.length", response.Text?.Length ?? 0);
activity?.SetTag("harness.agent_turn.response.messages", response.Messages.Count);
activity?.SetTag("harness.agent_turn.approval_requested", FindApprovalRequest(response) is not null);
activity?.SetStatus(ActivityStatusCode.Ok);
var elapsed = Stopwatch.GetElapsedTime(startedAt);
HarnessTelemetry.Logger.LogInformation(
"Harness agent turn completed. Kind={TurnKind} Index={TurnIndex} ElapsedMs={ElapsedMs}",
turnKind,
turnIndex,
elapsed.TotalMilliseconds);
return response;
}
catch (Exception exception)
{
activity?.SetStatus(ActivityStatusCode.Error, exception.Message);
activity?.AddException(exception);
var elapsed = Stopwatch.GetElapsedTime(startedAt);
HarnessTelemetry.Logger.LogError(
exception,
"Harness agent turn failed. Kind={TurnKind} Index={TurnIndex} ElapsedMs={ElapsedMs}",
turnKind,
turnIndex,
elapsed.TotalMilliseconds);
throw;
}
finally
{
await thinking.StopAsync();
}
}
static ToolApprovalRequestContent? FindApprovalRequest(AgentResponse response)
{
foreach (var message in response.Messages)
{
foreach (var content in message.Contents)
{
if (content is ToolApprovalRequestContent approvalRequest)
{
return approvalRequest;
}
}
}
return null;
}
static ToolApprovalDecision ResolveToolApproval(
ToolApprovalRequestContent approvalRequest,
string? fileAccessRoot,
ToolApprovalState toolApprovals)
{
if (approvalRequest.ToolCall is FunctionCallContent functionCall)
{
var matchingRule = toolApprovals.FindMatchingRule(functionCall);
if (matchingRule is not null)
{
WriteMuted($"[tool approval] auto-approved by standing rule: {matchingRule.Description}");
return new ToolApprovalDecision(true, $"Approved by standing rule: {matchingRule.Description}");
}
if (Console.IsInputRedirected
&& IsHeuristicallySafeAutoApproval(functionCall, fileAccessRoot, out var reason))
{
WriteMuted($"[tool approval] auto-approved: {reason}");
return new ToolApprovalDecision(true, $"Auto-approved by safe unattended heuristic: {reason}");
}
}
return PromptForToolApproval(approvalRequest, toolApprovals);
}
static ToolApprovalDecision PromptForToolApproval(
ToolApprovalRequestContent approvalRequest,
ToolApprovalState toolApprovals)
{
WriteSection("Tool Approval");
if (approvalRequest.ToolCall is FunctionCallContent functionCall)
{
Console.WriteLine($"Tool: {functionCall.Name}");
if (functionCall.Arguments is { Count: > 0 } arguments)
{
Console.WriteLine("Arguments:");
WriteFunctionArguments(arguments);
}
}
else
{
Console.WriteLine($"Tool call: {approvalRequest.ToolCall.CallId}");
}
while (true)
{
Console.Write("Approve this tool call? [y/N/d=don't ask again] ");
var input = Console.ReadLine()?.Trim();
if (string.IsNullOrEmpty(input) || input.Equals("n", StringComparison.OrdinalIgnoreCase) || input.Equals("no", StringComparison.OrdinalIgnoreCase))
{
return new ToolApprovalDecision(false, "Rejected by user.");
}
if (input.Equals("y", StringComparison.OrdinalIgnoreCase) || input.Equals("yes", StringComparison.OrdinalIgnoreCase))
{
return new ToolApprovalDecision(true, "Approved by user.");
}
if (input.Equals("d", StringComparison.OrdinalIgnoreCase)
|| input.Equals("da", StringComparison.OrdinalIgnoreCase)
|| input.Equals("always", StringComparison.OrdinalIgnoreCase)
|| input.Equals("don't ask again", StringComparison.OrdinalIgnoreCase)
|| input.Equals("dont ask again", StringComparison.OrdinalIgnoreCase))
{
if (approvalRequest.ToolCall is FunctionCallContent ruleFunctionCall)
{
var rule = ToolApprovalStandingRule.Create(ruleFunctionCall);
toolApprovals.AddRule(rule);
WriteMuted($"[tool approval] saved standing rule: {rule.Description}");
return new ToolApprovalDecision(true, $"Approved by user and saved standing rule: {rule.Description}");
}
return new ToolApprovalDecision(true, "Approved by user.");
}
Console.WriteLine("Please answer y, n, or d.");
}
}
static bool IsHeuristicallySafeAutoApproval(
FunctionCallContent functionCall,
string? fileAccessRoot,
out string reason)
{
reason = string.Empty;
if (string.Equals(functionCall.Name, "run_shell", StringComparison.Ordinal)
&& ScenarioArguments.TryGetStringArgument(functionCall, "command", out var command)
&& HarnessApprovalRules.IsSafeShellEnvironmentProbe(command))
{
reason = "safe shell environment probe";
return true;
}
if (!string.Equals(functionCall.Name, FileAccessProvider.SaveFileToolName, StringComparison.Ordinal)
|| string.IsNullOrWhiteSpace(fileAccessRoot)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "fileName", out var fileName)
|| !ScenarioArguments.TryGetStringArgument(functionCall, "content", out var content))
{
return false;
}
if (!IsSafeRelativeFileName(fileName)
|| content.Length > 64 * 1024)
{
return false;
}
var extension = Path.GetExtension(fileName);
string[] safeExtensions = [".csv", ".json", ".md", ".txt"];
if (!safeExtensions.Contains(extension, StringComparer.OrdinalIgnoreCase))
{
return false;
}
var fullPath = Path.GetFullPath(Path.Combine(fileAccessRoot, fileName));
var fullRoot = Path.GetFullPath(fileAccessRoot);
if (!fullPath.StartsWith(fullRoot.TrimEnd(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar) + Path.DirectorySeparatorChar, StringComparison.Ordinal))
{
return false;
}
reason = $"safe scoped file write to {fileName}";
return true;
}
static bool IsSafeRelativeFileName(string fileName)
{
if (string.IsNullOrWhiteSpace(fileName)
|| Path.IsPathRooted(fileName)
|| fileName.Contains('\0'))
{
return false;
}
var segments = fileName.Split(new[] { Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar }, StringSplitOptions.RemoveEmptyEntries);
return segments.Length > 0
&& segments.All(segment => segment is not "." and not ".." && !segment.StartsWith(".", StringComparison.Ordinal));
}
static void WriteFunctionArguments(IDictionary<string, object?> arguments)
{
foreach (KeyValuePair<string, object?> argument in arguments)
{
Console.WriteLine($" {argument.Key}: {FormatFunctionArgumentValue(argument.Value)}");
}
}
static string FormatFunctionArgumentValue(object? value)
=> value switch
{
null => "null",
JsonElement jsonValue => jsonValue.ValueKind == JsonValueKind.String
? jsonValue.GetString() ?? string.Empty
: jsonValue.GetRawText(),
string stringValue => stringValue,
_ => value.ToString() ?? string.Empty,
};
static void PrintWelcome(string workspace, string modelName, ScenarioDefinition scenario, string currentMode, int capabilityCount)
{
Console.WriteLine();
WriteWelcomeFrame(modelName, scenario.Name, currentMode);
Console.WriteLine();
var instructionsPath = GetHarnessInstructionsPath(workspace);
if (File.Exists(instructionsPath))
{
WriteStartupNotice(ConsoleColor.Blue, $"Harness instructions loaded: {instructionsPath}");
}
else
{
WriteStartupNotice(ConsoleColor.Blue, "No harness instructions found. Run /init to generate a harness-instructions.md file for this project.");
}
WriteStartupNotice(ConsoleColor.Blue, $"Environment loaded: {capabilityCount} focused capabilities");
WriteStartupNotice(ConsoleColor.Blue, $"Scenario active: {scenario.Name} - {scenario.Description}");
WriteStartupNotice(ConsoleColor.Blue, $"Workspace trusted: {workspace}");
WriteStartupNotice(ConsoleColor.Blue, $"File access root: {Path.Combine(workspace, "files")}");
WriteStartupNotice(ConsoleColor.Blue, $"File memory root: {Path.Combine(workspace, "memory")}");
WriteStartupNotice(ConsoleColor.Blue, $"Skills root: {workspace}");
Console.WriteLine();
WriteAccent("Please describe a task, or use /help to browse commands.");
}
static void WriteWelcomeFrame(string modelName, string scenarioName, string currentMode)
{
var width = GetFrameWidth();
WriteFrameBorder('╭', '─', '╮', ConsoleColor.Magenta, width);
WriteFrameBlank(width);
WriteFrameLine(width, (" ╭A╮ ", ConsoleColor.Cyan), ("Agent Harness Console", ConsoleColor.Magenta), (" v1.0", ConsoleColor.DarkGray));
WriteFrameLine(width, (" ╰H╯ ", ConsoleColor.Magenta), ("Aspire-managed local AI workbench", ConsoleColor.White));
WriteFrameLine(width,
(" ─▶ ", ConsoleColor.Magenta),
("model ", ConsoleColor.DarkGray),
(modelName, ConsoleColor.Cyan),
(" scenario ", ConsoleColor.DarkGray),
(scenarioName, ConsoleColor.Cyan),
(" mode ", ConsoleColor.DarkGray),
(currentMode, currentMode == "plan" ? ConsoleColor.Yellow : ConsoleColor.Green),
(" provider ", ConsoleColor.DarkGray),
("ollama", ConsoleColor.Cyan));
WriteFrameBlank(width);
WriteFrameLine(width,
(" Tip: ", ConsoleColor.DarkGray),
("/sample", ConsoleColor.Cyan),
(" shows starter prompts; ", ConsoleColor.DarkGray),
("/capabilities", ConsoleColor.Cyan),
(" lists coverage.", ConsoleColor.DarkGray));
WriteFrameLine(width, (" Review outputs and approve tool calls deliberately.", ConsoleColor.DarkGray));
WriteFrameBlank(width);
WriteFrameBorder('╰', '─', '╯', ConsoleColor.Magenta, width);
Console.ResetColor();
}
static void WriteFrameLine(int width, params (string Text, ConsoleColor Color)[] segments)
{
var innerWidth = Math.Max(0, width - 2);
Console.ForegroundColor = ConsoleColor.Magenta;
Console.Write("│");
var contentLength = 0;
foreach (var (text, color) in segments)
{
if (contentLength >= innerWidth)
{
break;
}
var remainingWidth = innerWidth - contentLength;
var textToWrite = text.Length > remainingWidth
? text[..remainingWidth]
: text;
contentLength += textToWrite.Length;
Console.ForegroundColor = color;
Console.Write(textToWrite);
}
var padding = Math.Max(0, innerWidth - contentLength);
Console.Write(new string(' ', padding));
Console.ForegroundColor = ConsoleColor.Magenta;
Console.WriteLine("│");
Console.ResetColor();
}
static void WriteFrameBlank(int width)
{
Console.ForegroundColor = ConsoleColor.Magenta;
Console.Write("│");
Console.WriteLine(new string(' ', Math.Max(0, width - 2)) + "│");
Console.ResetColor();
}
static void WriteFrameBorder(char left, char fill, char right, ConsoleColor color, int width)
{
Console.ForegroundColor = color;
Console.Write(left);
Console.Write(new string(fill, width - 2));
Console.WriteLine(right);
Console.ResetColor();
}
static void WriteStartupNotice(ConsoleColor bulletColor, string message)
{
Console.ForegroundColor = bulletColor;
Console.Write("● ");
Console.ForegroundColor = ConsoleColor.White;
Console.WriteLine(message);
Console.ResetColor();
}
static void WriteAccent(string message)
{
Console.ForegroundColor = ConsoleColor.Magenta;
Console.WriteLine(message);
Console.ResetColor();
}
static void ClearScreen()
{
if (Console.IsOutputRedirected)
{
Console.WriteLine();
WriteRuleWide();
Console.WriteLine();
return;
}
Console.Clear();
}
static void PrintHelp(string filesRoot)
{
WriteSection("Commands");
Console.WriteLine("""
/help Show this command list
/scenarios List available scenarios
/scenario <name> Switch scenario and start a fresh session
/init Generate harness-instructions.md in the workspace
/sample Show prompts to try
/capabilities List covered harness features
/clear Start a fresh session
/exit Quit
""");
WriteMuted($"file root {filesRoot}");
}
static void PrintScenarios()
{
WriteSection("Scenarios");
foreach (var scenarioName in ScenarioCatalog.Names)
{
var scenario = ScenarioCatalog.Create(scenarioName);
Console.ForegroundColor = ConsoleColor.White;
Console.Write(scenario.Name);
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.Write(" - ");
Console.ResetColor();
Console.WriteLine(scenario.Description);
}
}
static void InitializeHarnessInstructions(HarnessPaths paths, ScenarioDefinition scenario)
{
Directory.CreateDirectory(paths.Workspace);
var instructionsPath = GetHarnessInstructionsPath(paths.Workspace);
if (File.Exists(instructionsPath))
{
WriteMuted($"harness instructions already exist {instructionsPath}");
return;
}
File.WriteAllText(instructionsPath, BuildHarnessInstructionsFile(scenario));
WriteAccent($"created {instructionsPath}");
}
static string GetHarnessInstructionsPath(string workspace)
=> Path.Combine(workspace, "harness-instructions.md");
static string BuildHarnessInstructionsFile(ScenarioDefinition scenario)
=> $"""
# Harness Instructions
{scenario.HarnessInstructions.Trim()}
## Sample Prompts
{string.Join(Environment.NewLine, scenario.Samples.Select(sample => $"- {sample.Prompt} ({sample.Capability})"))}
""";
static void PrintSamples(SamplePrompt[] samples)
{
WriteSection("Try");
foreach (var sample in samples)
{
WritePromptSample(sample);
}
}
static void WritePromptSample(SamplePrompt sample)
{
Console.Write(" ");
Console.ForegroundColor = ConsoleColor.White;
Console.Write(sample.Prompt);
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.Write(" -> ");
Console.ForegroundColor = ConsoleColor.Cyan;
Console.WriteLine(sample.Capability);
Console.ResetColor();
}
static void PrintCapabilities(string[] capabilities)
{
WriteSection("Covered Capabilities");
foreach (var sample in HarnessCapabilitySamples.All)
{
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.Write("- ");
Console.ForegroundColor = ConsoleColor.White;
Console.Write(sample.Capability);
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.Write(" - ");
Console.ResetColor();
Console.WriteLine(sample.Description);
}
}
static void WriteAssistant(string text)
{
Console.ForegroundColor = ConsoleColor.Green;
Console.WriteLine(text);
Console.ResetColor();
}
static string StripDoneMarker(string text)
{
return !text.EndsWith(LoopCompletionMarker, StringComparison.Ordinal)
&& text.EndsWith("DONE", StringComparison.OrdinalIgnoreCase)
? text[..^4].TrimEnd()
: text;
}
static CompletionResult CompleteHistoryPersistenceSample(string prompt, string responseText, string? knownUserName, string? knownReleaseFreezeWindow)
{
if (TryExtractUserName(prompt, out var userName))
{
return CompletionResult.Replace(
$"Nice to meet you, {userName}.",
"history.name_acknowledgement",
"exact_name_setting_prompt_uses_stable_acknowledgement");
}
if (IsNameRecallPrompt(prompt)
&& !string.IsNullOrWhiteSpace(knownUserName)
&& (IsEmptyModelResponse(responseText) || !responseText.Contains(knownUserName, StringComparison.OrdinalIgnoreCase)))
{
return CompletionResult.Replace(
$"Your name is {knownUserName}.",
"history.name_recall",
GetMissingRequiredContentReason(responseText));
}
if (TryExtractReleaseFreezeWindow(prompt, out var releaseFreezeWindow)
&& (LooksLikePseudoToolCall(responseText) || !responseText.Contains(releaseFreezeWindow, StringComparison.OrdinalIgnoreCase)))
{
return CompletionResult.Replace(
$"Release freeze starts {releaseFreezeWindow}.",
"history.release_freeze_acknowledgement",
GetMissingRequiredContentReason(responseText));
}
if (IsReleaseFreezeRecallPrompt(prompt)
&& !string.IsNullOrWhiteSpace(knownReleaseFreezeWindow)
&& (IsEmptyModelResponse(responseText) || !responseText.Contains(knownReleaseFreezeWindow, StringComparison.OrdinalIgnoreCase)))
{
return CompletionResult.Replace(
$"The release freeze window starts {knownReleaseFreezeWindow}.",
"history.release_freeze_recall",
GetMissingRequiredContentReason(responseText));
}
return CompletionResult.Preserve(responseText);
}
static string GetMissingRequiredContentReason(string responseText)
{
if (IsEmptyModelResponse(responseText))
{
return "model_response_empty";
}
if (LooksLikePseudoToolCall(responseText))
{
return "model_returned_pseudo_tool_call_text";
}
return "model_response_missing_required_content";
}
static bool TryExtractUserName(string prompt, out string userName)
{
userName = string.Empty;
var match = Regex.Match(
prompt,
@"\bmy\s+name\s+(?:is\s+)?(?<name>[A-Za-z][A-Za-z'-]*)\b",
RegexOptions.IgnoreCase);
if (!match.Success)
{
return false;
}
userName = match.Groups["name"].Value;
return true;
}
static bool IsNameRecallPrompt(string prompt)
=> Regex.IsMatch(prompt, @"\bwhat\s+is\s+my\s+name\b", RegexOptions.IgnoreCase);
static bool TryExtractReleaseFreezeWindow(string prompt, out string releaseFreezeWindow)
{
releaseFreezeWindow = string.Empty;
var match = Regex.Match(
prompt,
@"\brelease\s+freeze\s+window\s+starts\s+(?<window>[A-Za-z][A-Za-z0-9' -]*)\b",
RegexOptions.IgnoreCase);
if (!match.Success)
{
return false;
}
releaseFreezeWindow = match.Groups["window"].Value.Trim().TrimEnd('.', ',', ';');
return releaseFreezeWindow.Length > 0;
}
static bool IsReleaseFreezeRecallPrompt(string prompt)
=> Regex.IsMatch(prompt, @"\bwhen\s+does\s+the\s+release\s+freeze\s+window\s+start\b", RegexOptions.IgnoreCase);
static bool IsShellEnvironmentSamplePrompt(string prompt)
=> prompt.Equals("Use the shell to print the current working directory and OS name.", StringComparison.OrdinalIgnoreCase);
static string GetInitialUserPrompt(LoopContext context)
=> context.InitialMessages
.Where(message => message.Role == ChatRole.User)
.Select(message => message.Text)
.FirstOrDefault(text => !string.IsNullOrWhiteSpace(text))
?.Trim()
?? string.Empty;
static LoopingSampleEvaluation EvaluateLoopingSampleResponse(string prompt, int iteration, string responseText)
{
if (iteration == 1)
{
return LoopingSampleEvaluation.Rejected(
"first pass rejected to demonstrate re-invocation",
"The first pass is intentionally not accepted so the LoopAgent demonstrates a re-invocation.");
}
if (!responseText.Contains(LoopCompletionMarker, StringComparison.Ordinal))
{
return LoopingSampleEvaluation.Rejected(
$"missing {LoopCompletionMarker}",
$"The latest response did not include {LoopCompletionMarker}.");
}
var finalText = StripLoopCompletionMarker(responseText);
if (ContainsLoopingSamplePreamble(finalText))
{
return LoopingSampleEvaluation.Rejected(
"todo/process preamble found",
"The latest response included todo/process preamble.");
}
var sentenceCount = CountSentences(finalText);
if (sentenceCount != 2)
{
return LoopingSampleEvaluation.Rejected(
$"expected 2 sentences, found {sentenceCount}",
"The latest response was not exactly two sentences.");
}
if (!ScenarioCatalog.MentionsLoopingSampleSubject(prompt, finalText)
|| !finalText.Contains("risk", StringComparison.OrdinalIgnoreCase))
{
return LoopingSampleEvaluation.Rejected(
"required subject/risk terms missing",
$"The latest response did not mention both {ScenarioCatalog.GetLoopingSampleSubject(prompt)} and risk.");
}
return new LoopingSampleEvaluation(true, "accepted", "completion condition satisfied", string.Empty);
}
static bool ContainsLoopingSamplePreamble(string responseText)
=> Regex.IsMatch(responseText, @"\b(todo|loop|attempt|process|instruction|current todo list)\b", RegexOptions.IgnoreCase);
static int CountSentences(string responseText)
=> Regex.Matches(responseText, @"[.!?](?:\s|$)").Count;
static string StripLoopCompletionMarker(string responseText)
=> responseText.Replace(LoopCompletionMarker, string.Empty, StringComparison.Ordinal).Trim();
static string BuildPseudoToolRecoveryPrompt(string originalPrompt, string pseudoToolCall)
=> $"""
The previous assistant message was a JSON-shaped tool request, but it was emitted as plain text and no native tool result was returned to the harness.
Original user request:
{originalPrompt}
Plain-text pseudo tool call that did not execute:
{pseudoToolCall}
Respond in natural language. Do not invent current news, prices, citations, resources, shell command output, or tool results. Do not emit JSON or another tool call. If a needed hosted search result, skill resource, skill script, shell command, or background task result is unavailable, say what could not be executed and continue only with information already available in the conversation.
""";
static string BuildRepeatedPseudoToolFinalAnswerPrompt(string originalPrompt, string repeatedPseudoToolCall)
=> $"""
You again emitted a JSON-shaped tool request as plain text. It still did not execute.
Original user request:
{originalPrompt}
Repeated plain-text pseudo tool call:
{repeatedPseudoToolCall}
Now provide the final answer as plain English only. Do not output JSON, code fences, tool names with underscores, or another tool call. Do not invent background-agent results, search results, shell command output, resources, citations, or prices. Say that the requested delegation/tool action could not be executed in this run, then briefly state what would be needed for a valid result.
""";
static string BuildShellEnvironmentCorrectionPrompt(string originalPrompt, string previousAnswer, ShellEnvironmentProbe shellProbe)
=> $"""
The previous answer did not match the actual shell tool output.
Original user request:
{originalPrompt}
Actual shell stdout:
{shellProbe.Output}
Previous answer:
{previousAnswer}
Provide a corrected natural-language answer using the actual shell stdout only. Report the current working directory exactly as "{shellProbe.Cwd}" and the OS exactly as "{shellProbe.Os}". Do not call another tool and do not infer a different OS version.
""";
static bool TryGetShellEnvironmentProbe(AgentResponse response, out ShellEnvironmentProbe probe)
{
foreach (var message in response.Messages)
{
foreach (var content in message.Contents)
{
if (content is not FunctionResultContent functionResult || functionResult.Result is null)
{
continue;
}
var output = functionResult.Result switch
{
ShellResult shellResult => shellResult.Stdout,
string text => text,
JsonElement { ValueKind: JsonValueKind.String } json => json.GetString() ?? string.Empty,
JsonElement json => json.ToString(),
_ => JsonSerializer.Serialize(functionResult.Result),
};
if (TryParseShellEnvironmentProbeOutput(output, out probe))
{
return true;
}
}
}
probe = default;
return false;
}
static bool TryParseShellEnvironmentProbeOutput(string output, out ShellEnvironmentProbe probe)
{
string? cwd = null;
string? os = null;
foreach (var line in output.Split(['\r', '\n'], StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
{
if (line.StartsWith("cwd=", StringComparison.Ordinal))
{
cwd = line["cwd=".Length..];
}
else if (line.StartsWith("os=", StringComparison.Ordinal))
{
os = line["os=".Length..];
}
}
if (!string.IsNullOrWhiteSpace(cwd) && !string.IsNullOrWhiteSpace(os))
{
probe = new ShellEnvironmentProbe(cwd, os, output.Trim());
return true;
}
probe = default;
return false;
}
static bool ShellEnvironmentResponseMatchesProbe(string responseText, ShellEnvironmentProbe shellProbe)
=> responseText.Contains(shellProbe.Cwd, StringComparison.Ordinal)
&& responseText.Contains(shellProbe.Os, StringComparison.Ordinal);
static bool LooksLikePseudoToolCall(string responseText)
{
var text = responseText.TrimStart();
return text.StartsWith("{\"name\":", StringComparison.OrdinalIgnoreCase)
|| text.Contains("{\"name\":", StringComparison.OrdinalIgnoreCase);
}
static bool LooksLikeTodoPseudoToolCall(string responseText)
=> LooksLikePseudoToolCall(responseText)
&& responseText.Contains("\"todos_", StringComparison.OrdinalIgnoreCase);
static bool LooksLikeHostedWebSearchPseudoToolCall(string responseText)
=> LooksLikePseudoToolCall(responseText)
&& responseText.Contains("\"hosted_web_search\"", StringComparison.OrdinalIgnoreCase);
static bool LooksLikeRecoverablePseudoToolCall(string responseText)
=> LooksLikeHostedWebSearchPseudoToolCall(responseText)
|| LooksLikeSkillProviderPseudoToolCall(responseText)
|| LooksLikeBackgroundAgentsPseudoToolCall(responseText)
|| LooksLikeShellEnvironmentPseudoToolCall(responseText);
static bool LooksLikeSkillProviderPseudoToolCall(string responseText)
=> LooksLikePseudoToolCall(responseText)
&& (responseText.Contains("\"load_skill\"", StringComparison.OrdinalIgnoreCase)
|| responseText.Contains("\"read_skill_resource\"", StringComparison.OrdinalIgnoreCase)
|| responseText.Contains("\"run_skill_script\"", StringComparison.OrdinalIgnoreCase));
static bool LooksLikeBackgroundAgentsPseudoToolCall(string responseText)
=> LooksLikePseudoToolCall(responseText)
&& responseText.Contains("\"background_agents_", StringComparison.OrdinalIgnoreCase);
static bool LooksLikeShellEnvironmentPseudoToolCall(string responseText)
=> LooksLikePseudoToolCall(responseText)
&& responseText.Contains("\"run_shell\"", StringComparison.OrdinalIgnoreCase);
static bool IsEmptyModelResponse(string responseText)
=> responseText.StartsWith("The harness ran, but the local model returned an empty final message.", StringComparison.OrdinalIgnoreCase);
static void WriteError(string message)
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine("error");
Console.ResetColor();
Console.WriteLine(message);
}
static void WriteMuted(string message)
{
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.WriteLine(message);
Console.ResetColor();
}
static void WriteSection(string title)
{
Console.WriteLine();
Console.ForegroundColor = ConsoleColor.Cyan;
Console.WriteLine(title);
Console.ResetColor();
WriteRule();
}
static void WriteRule()
{
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.WriteLine(new string('-', 56));
Console.ResetColor();
}
static void WriteRuleWide()
{
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.WriteLine(new string('─', GetFrameWidth()));
Console.ResetColor();
}
static int GetFrameWidth()
{
var width = Console.IsOutputRedirected ? 104 : Console.WindowWidth - 2;
return Math.Clamp(width, 64, 132);
}
static bool IsLikelyToolCallingModel(string modelName)
{
var normalizedName = modelName.Split(':', 2)[0].ToLowerInvariant();
return normalizedName is "llama3.1"
or "mistral-nemo"
or "firefunction-v2"
or "command-r"
or "command-r-plus"
or "qwen3"
or "qwen3.5"
or "qwen3.5-plus";
}
static async Task<bool> IsModelInstalledAsync(string endpoint, string modelName)
{
try
{
using HttpClient httpClient = new();
using var response = await httpClient.GetAsync(new Uri(new Uri(endpoint), "/api/tags"));
response.EnsureSuccessStatusCode();
await using var stream = await response.Content.ReadAsStreamAsync();
using var document = await JsonDocument.ParseAsync(stream);
if (!document.RootElement.TryGetProperty("models", out var models))
{
return false;
}
foreach (var model in models.EnumerateArray())
{
if (model.TryGetProperty("name", out var name)
&& IsSameOllamaModel(name.GetString(), modelName))
{
return true;
}
}
}
catch (Exception exception) when (exception is HttpRequestException or TaskCanceledException or JsonException)
{
Console.Error.WriteLine($"Unable to query Ollama at {endpoint}: {exception.Message}");
}
return false;
}
static bool IsSameOllamaModel(string? installedModelName, string requestedModelName)
{
if (string.IsNullOrWhiteSpace(installedModelName))
{
return false;
}
return string.Equals(installedModelName, requestedModelName, StringComparison.OrdinalIgnoreCase)
|| !requestedModelName.Contains(':', StringComparison.Ordinal)
&& string.Equals(installedModelName, $"{requestedModelName}:latest", StringComparison.OrdinalIgnoreCase);
}
static bool IsEnabledEnvironmentVariable(string name)
{
var value = Environment.GetEnvironmentVariable(name);
return value is not null
&& value.Length > 0
&& !value.Equals("0", StringComparison.OrdinalIgnoreCase)
&& !value.Equals("false", StringComparison.OrdinalIgnoreCase)
&& !value.Equals("off", StringComparison.OrdinalIgnoreCase);
}
static string GetSourceDirectory([CallerFilePath] string sourceFilePath = "")
=> Path.GetDirectoryName(sourceFilePath) ?? Environment.CurrentDirectory;
public sealed record LoopingSampleEvaluation(bool Complete, string Status, string Reason, string Feedback)
{
public static LoopingSampleEvaluation Rejected(string reason, string feedback)
=> new(false, "rejected", reason, feedback);
}
public readonly record struct ShellEnvironmentProbe(string Cwd, string Os, string Output);
public sealed record CliState(AgentSession? Session, string CurrentMode, string? KnownUserName, string? KnownReleaseFreezeWindow, ToolApprovalState ToolApprovals);
public sealed record CommandResult(bool Continue, CliState State);
public sealed record ToolApprovalDecision(bool Approved, string Message);
public sealed record StartupOptions(string ScenarioName, string[] PromptArgs)
{
public static StartupOptions Parse(string[] args)
{
var scenarioName = Environment.GetEnvironmentVariable("E2E_HARNESS_SCENARIO") ?? "finance";
var promptArgs = new List<string>();
for (var index = 0; index < args.Length; index++)
{
var arg = args[index];
if (arg.Equals("--scenario", StringComparison.OrdinalIgnoreCase))
{
if (index + 1 >= args.Length)
{
throw new ArgumentException("--scenario requires a scenario name.");
}
scenarioName = args[++index];
}
else if (arg.StartsWith("--scenario=", StringComparison.OrdinalIgnoreCase))
{
scenarioName = arg.Split('=', 2)[1];
}
else
{
promptArgs.Add(arg);
}
}
return new StartupOptions(scenarioName.ToLowerInvariant(), [.. promptArgs]);
}
}
public sealed class ToolApprovalState
{
private readonly List<ToolApprovalStandingRule> rules = [];
public ToolApprovalStandingRule? FindMatchingRule(FunctionCallContent functionCall)
=> rules.FirstOrDefault(rule => rule.Matches(functionCall));
public void AddRule(ToolApprovalStandingRule rule)
{
if (!rules.Any(existing => existing.Equals(rule)))
{
rules.Add(rule);
}
}
}
public sealed record ToolApprovalStandingRule(string ToolName, string? FileName, string Description)
{
public static ToolApprovalStandingRule Create(FunctionCallContent functionCall)
{
var fileName = string.Equals(functionCall.Name, FileAccessProvider.SaveFileToolName, StringComparison.Ordinal)
&& TryGetStringArgumentValue(functionCall, "fileName", out var value)
? value
: null;
var description = fileName is null
? functionCall.Name
: $"{functionCall.Name} for {fileName}";
return new ToolApprovalStandingRule(functionCall.Name, fileName, description);
}
public bool Matches(FunctionCallContent functionCall)
{
if (!string.Equals(functionCall.Name, ToolName, StringComparison.Ordinal))
{
return false;
}
return FileName is null
|| TryGetStringArgumentValue(functionCall, "fileName", out var fileName)
&& string.Equals(fileName, FileName, StringComparison.OrdinalIgnoreCase);
}
private static bool TryGetStringArgumentValue(FunctionCallContent functionCall, string name, out string value)
{
value = string.Empty;
if (functionCall.Arguments is not { } arguments
|| !arguments.TryGetValue(name, out var rawValue)
|| rawValue is null)
{
return false;
}
value = rawValue switch
{
string stringValue => stringValue,
JsonElement { ValueKind: JsonValueKind.String } jsonValue => jsonValue.GetString() ?? string.Empty,
_ => rawValue.ToString() ?? string.Empty,
};
return value.Length > 0;
}
}
public static class HarnessCapabilities
{
public static readonly string[] All =
HarnessCapabilitySamples.All
.Select(sample => sample.Capability)
.ToArray();
}
public static class HarnessCapabilitySamples
{
public static readonly CapabilitySample[] All =
[
new(
"Function invocation",
"Automatic tool-calling loop with a configurable iteration limit.",
"What's the price of MSFT?",
"The agent calls get_msft_price and returns the deterministic mocked quote, 497.45 USD."),
new(
"Per-service-call history persistence",
"Chat history is persisted after every individual model call, enabling crash recovery and inspection mid-run.",
"My name is Ajay. Then ask: What is my name?",
"The second turn uses the same AgentSession and recalls Ajay from the previous turn."),
new(
"Compaction",
"Context-window compaction keeps long tool-calling loops from overflowing the context window. Active when a token budget or custom strategy is provided.",
"Look up MSFT, AAPL, and NVDA prices, then summarize the portfolio context briefly.",
"The harness injects large prior context and uses a custom context-window compaction strategy so this sample can report when compaction is applied."),
new(
"Todo provider",
"A persistent todo list the agent uses to track multi-step plans.",
"Create a todo list with research MSFT, compare NVDA, and summarize next steps.",
"The enabled todo provider exposes native todo tools that persist items in the active AgentSession."),
new(
"File memory provider",
"File-based session memory for notes and artifacts that persist across turns.",
"Remember that my risk limit is 2%.",
"The harness writes durable memory files under scenario-workspace/<scenario>/memory."),
new(
"File access provider",
"Read/write file tools scoped to the harness working directory.",
"Create a file named auto-watchlist.md with MSFT and NVDA.",
"The harness scopes an auto-approved file write to scenario-workspace/<scenario>/files."),
new(
"Tool approval",
"Approval-required tools are surfaced to the user before execution, with in-session standing approvals and safe unattended auto-approval.",
"Create a file named watchlist.md with MSFT and NVDA.",
"The harness prompts for file_access_save_file approval in an interactive run, supports don't-ask-again standing approval rules, and auto-approves a safe scoped file write during redirected unattended runs."),
new(
"Web search",
"A hosted web search tool is added by default for current information.",
"Search the web for the latest Microsoft stock news and summarize the top themes.",
"The harness leaves web search enabled so the hosted web search tool is available by default for current-information requests."),
new(
"Skills provider",
"Discovers and progressively loads Agent Skills from the file system.",
"Use the portfolio-risk-review skill to review MSFT and NVDA.",
"The harness seeds scenario-specific SKILL.md files under scenario-workspace/<scenario> and enables the default Agent Skills provider for file-system discovery."),
new(
"Background agents",
"Delegate parallel work to background sub-agents.",
"Ask the MSFT and NVDA background agents for one catalyst and one risk each, then compare the results.",
"The harness registers msft-researcher and nvda-researcher background agents so the parent can start parallel tasks, wait for completion, and synthesize their results."),
new(
"Shell environment",
"Shell command execution plus OS/shell/working-directory probing.",
"Use the shell to print the current working directory and OS name.",
"The harness configures a LocalShellExecutor rooted at scenario-workspace/<scenario> and the ShellEnvironmentProvider injects OS, shell, working-directory, and CLI probe context."),
new(
"Looping",
"Re-invoke the agent until a completion condition is satisfied.",
"Use looping to produce a two-sentence MSFT risk note.",
"The harness wraps the agent with LoopAgent, forces at least one re-invocation, requires a two-sentence Microsoft/MSFT risk note plus LOOP_DONE, rejects todo/process preamble, and strips the marker before displaying the final answer."),
// Additional capabilities are intentionally commented out while focusing on the active capability set.
];
}
public sealed record CapabilitySample(string Capability, string Description, string Prompt, string Validation);ReportingCompactionStrategy.cs
using Microsoft.Agents.AI.Compaction;
using Microsoft.Extensions.Logging;
public sealed class ReportingCompactionStrategy : CompactionStrategy
{
public static readonly ReportingCompactionStrategy Instance = new();
private readonly ContextWindowCompactionStrategy inner = new(1_024, 256);
private int appliedCount;
private ReportingCompactionStrategy()
: base(CompactionTriggers.Always, CompactionTriggers.Never)
{
}
public static int AppliedCount => Instance.appliedCount;
protected override async ValueTask<bool> CompactCoreAsync(
CompactionMessageIndex index,
ILogger logger,
CancellationToken cancellationToken)
{
var compacted = await inner.CompactAsync(index, logger, cancellationToken);
if (compacted)
{
Interlocked.Increment(ref appliedCount);
}
return compacted;
}
}ThinkingIndicator.cs
public sealed class ThinkingIndicator
{
private readonly CancellationTokenSource? cancellation;
private readonly Task? animation;
private ThinkingIndicator(CancellationTokenSource? cancellation, Task? animation)
{
this.cancellation = cancellation;
this.animation = animation;
}
public static ThinkingIndicator Start()
{
if (Console.IsOutputRedirected)
{
return new ThinkingIndicator(null, null);
}
CancellationTokenSource cancellation = new();
var animation = Task.Run(async () =>
{
var frames = new[] { "thinking", "thinking.", "thinking..", "thinking..." };
var index = 0;
while (!cancellation.Token.IsCancellationRequested)
{
Console.Write("\r");
Console.ForegroundColor = ConsoleColor.Yellow;
Console.Write(frames[index++ % frames.Length]);
Console.ResetColor();
await Task.Delay(350, cancellation.Token);
}
}, cancellation.Token);
return new ThinkingIndicator(cancellation, animation);
}
public async Task StopAsync()
{
if (cancellation is null || animation is null)
{
return;
}
await cancellation.CancelAsync();
try
{
await animation;
}
catch (OperationCanceledException)
{
}
cancellation.Dispose();
Console.Write("\r");
var width = Console.IsOutputRedirected ? 96 : Math.Clamp(Console.WindowWidth - 4, 56, 116);
Console.Write(new string(' ', width));
Console.Write("\r");
}
}Let's run the app
Start with aspire run to launch the Aspire Dashboard.
aspire run
As you can see from the images above, the Aspire Dashboard includes the agent harness as a console application and ensures that Ollama is ready before you can launch the console app.
Click on Start resource option to run the agent harness and play with it.
/help
/capabilities
Result of /sample command when scenario = finance.
Result of /sample command when scenario = incident.
Result of /sample command when scenario = release.
The image below illustrates the steps to change a scenario.
Explore Aspire's various options to understand what happens behind the scenes when you submit a prompt. The screenshots below showcase the range of details available through the Aspire Dashboard.
Using the Arize Phoenix dashboard, you can see more details.
Conclusion
Building an agent is not just about connecting a language model to a prompt. Once it can call tools, create files, run shell commands, remember context, delegate work, or act without constant supervision, it needs a harness, bringing together planning, approvals, memory, compaction, tool boundaries, background agents, and observability.
This is where Microsoft Agent Framework becomes useful. Rather than hand-rolling every part of the agent loop, you can compose its built-in providers and focus on the behaviour you want to demonstrate or ship. In this sample, a single harness supports finance, incident, and release scenarios with a common runtime model, making it easy to compare capabilities across domains.
Pairing the harness with Aspire and Arize Phoenix makes it easier to operate locally. Aspire orchestrates Ollama, the model, the console agent, and telemetry; Phoenix adds deeper visibility into LLM calls and traces. Together, they turn the demo from a black-box into something you can inspect, debug, and improve.
The takeaway: a production-style agent needs more than a clever prompt. It needs a runtime that controls what the agent can do, preserves state, recovers from long conversations, shows what happened, and keeps humans in the loop for sensitive actions. Microsoft Agent Framework's harness gives .NET developers a practical starting point for building one.