Choosing a model and an SDK in .NET

You now know what the two calls do. This lesson is the decision that comes before either of them on a real project: which model, and which library. It is usually made in the first hour of a spike, by whoever pasted the first sample, and then lived with for a year.

It does not have to be. Made deliberately, it is two decisions rather than one, and only one of them is expensive to change.

Two decisions, not one

Which model is a question about capability, cost, latency and where the data is allowed to go. Which library is a question about how your C# reaches whichever model you picked. Conflating them is what produces a codebase where changing model means changing two hundred files.

The provider sits between the two: the same gpt-4o-mini weights are reachable through OpenAI directly, through an Azure OpenAI deployment in your own subscription, or through a gateway your platform team runs — with different auth, different quota, different residency and identical prompts.

The three layers of a .NET AI stack

The rule that follows is short: your application code depends on the abstraction, exactly one file references a provider client, and you add the orchestration layer only when you can name the problem it solves. The lessons so far have all obeyed it — every sample took an IChatClient and none of them knew who was serving it.

Skipping the abstraction is a real option and occasionally the right one: a single integration against a single provider you own, with no plan to compare anything, does not need a seam. Know that you are choosing it, and know that the cost of adding it later is proportional to how much code has learned the provider's types.

Choosing a model

There is no ranking, only a fit against constraints. In rough order of how often each one decides it:

Two habits make the choice cheap to revisit. Start at the bottom of the range — pick the smallest model, run it against a set of examples you have graded by hand, and move up only where it fails. Doing it the other way round leaves you with a system that works and a bill nobody can explain. And route by task: a small model classifying an incoming message, a larger one drafting the reply, is a standard shape and usually cheaper than either model alone.

Whatever you pick, pin the dated version rather than the family alias. gpt-4o is a pointer that moves under you; gpt-4o-2024-11-20 is a dependency you upgraded deliberately.

Choosing the SDK

The realistic options for a .NET service, and when each is the answer:

• Microsoft.Extensions.AI over a provider client — the default, and what this course assumes. You get IChatClient, IEmbeddingGenerator, and caching, logging, telemetry and automatic function invocation as pipeline stages rather than as code you wrote.
• Azure.AI.OpenAI — when the model runs as an Azure OpenAI deployment. Entra ID auth instead of a key, your subscription's quota, your region. Note that you address a deployment name you chose, not a model name, so the string in your config means something different to every environment.
• OpenAI — the same models direct from OpenAI, with an API key. Simplest to start, and the one to reach for in a spike.
• A vendor SDK — Anthropic, Mistral and the rest, where you want the vendor's own surface and are willing to write the IChatClient adapter if one is not published.
• A local runtime (OllamaSharp, ONNX Runtime GenAI) — when the data cannot leave the machine, when you want a test fixture that costs nothing, or when the task is small enough for a small model. Also the cheapest way to run your evaluation set fifty times.

One composition root, three branches, and nothing downstream can tell which ran:

using System.ClientModel;
using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using OllamaSharp;
using OpenAI;
 
var builder = Host.CreateApplicationBuilder(args);
AiOptions ai = builder.Configuration.GetSection("Ai").Get<AiOptions>()!;
 
// The only place in the application that knows who serves the model.
IChatClient client = ai.Provider switch
{
    // Managed identity in Azure, your developer identity locally. No key anywhere.
    "azure" => new AzureOpenAIClient(new Uri(ai.Endpoint!), new DefaultAzureCredential())
        .GetChatClient(ai.Deployment!)
        .AsIChatClient(),
 
    "openai" => new OpenAIClient(new ApiKeyCredential(ai.ApiKey!))
        .GetChatClient(ai.Model!)
        .AsIChatClient(),
 
    // A local model, for tests and for data that must not leave the box.
    "ollama" => new OllamaApiClient(new Uri(ai.Endpoint!), ai.Model!),
 
    _ => throw new InvalidOperationException($"Unknown Ai:Provider '{ai.Provider}'."),
};
 
builder.Services
    .AddChatClient(client)
    .UseLogging()
    .UseOpenTelemetry();

The options class is worth as much as the switch. Configuration is where a model choice becomes visible to an operator instead of buried in a constructor:

/// <summary>Everything about the model choice that differs between environments.</summary>
public sealed class AiOptions
{
    /// <summary>azure | openai | ollama — the only provider-specific string in config.</summary>
    public required string Provider { get; init; }
 
    /// <summary>Dated model version, never a family alias: 'gpt-4o-mini-2024-07-18'.</summary>
    public string? Model { get; init; }
 
    /// <summary>Azure addresses a deployment you named, which may not match the model.</summary>
    public string? Deployment { get; init; }
 
    public string? Endpoint { get; init; }
 
    /// <summary>Absent on Azure, where the credential replaces it.</summary>
    public string? ApiKey { get; init; }
}

When you do need something only one provider offers, you do not have to abandon the abstraction to get it. GetService reaches the underlying client, and RawRepresentation on a response reaches the provider's own object — an escape hatch in one place beats a provider type in every signature.

Keeping the choice reversible

Reversible does not mean free. Be precise about what a swap actually costs.

Transfers as-is. Anything typed against IChatClient or IEmbeddingGenerator: your budgeting, your parsing, your validation, your retries, your tests. That is the point of the seam, and in a codebase that respects it a provider change really is one file plus configuration.

Does not transfer. More than people expect:

• Prompts. They are tuned to a model's habits. Expect to re-tune, and expect the difference to show up in your evaluation set rather than in a demo.
• Token counts. A different family means a different vocabulary, so your budget arithmetic and every cached count are wrong until recomputed.
• Embeddings. Vectors from two models are not comparable at all. Changing an embedding model means re-embedding the entire corpus — the most expensive swap in this list, and the reason to choose that one with more care than the rest.
• Tool-calling and structured-output reliability. Both are supported almost everywhere and are not equally good anywhere.
• Cost and latency. Which is usually why you are swapping.

Next, and last in this course: what all of this reliably does well, how it fails, and what you put around it before it goes anywhere near a user.