一、软件介绍

文末提供程序和源码下载

      LLM Tornado开源程序 - 使用 100+ API 的 .NET 库，包括 OpenAI、Anthropic、Google、DeepSeek、Cohere、Mistral、Azure、xAI、Perplexity、Groq 和自托管 API。每个月至少发布一个新的大型语言模型。如果使用最新、闪亮的模型就像切换一个参数一样简单，那不是很棒吗？LLMTornado 是一个用于构建支持 RAG/Agentic 的 AI、应用程序的框架，允许您做到这一点。

二、软件特征

• 100+ 支持的提供商：OpenAI、Anthropic、Google、DeepSeek、Cohere、Mistral、Azure、xAI、Perplexity、Groq 以及任何（自托管的）OpenAI 兼容推理服务器，例如 Ollama。 在此处查看完整的功能矩阵。
• 许多提供商的 API 协调 。例如，假设一个请求不小心将温度传递给推理模型，而该模型不支持这样的参数。我们会处理这个问题，以最大限度地提高调用成功的可能性。这适用于提供商的各种奇思妙想，例如 developer_message 与 system_prompt（在 Tornado 中，消息只有一个系统角色），Google 具有完全不同的端点来一次嵌入多个文本，以及许多其他烦恼。
• 为每个提供商提供强大、强类型的供应商扩展 ，提供独特的内容。支持丰富、有弹性的应用程序，同时最大限度地减少供应商锁定。
• 易于掌握的原语，用于构建代理系统、聊天机器人和基于 RAG 的应用程序。比 Semantic Kernel 更简单，但比原始 API 更强大。易于扩展。
• 专注于最大限度地减少重大更改。我们认真对待这些并提前考虑。更新 Tornado 通常不需要您执行任何作，即使发布了新的主要版本也是如此。

您可以使用 Tornado 做的很棒的事情：

• Chat with your documents  与您的文档聊天
• Voice call with AI using your microphone
• 使用麦克风通过 AI 进行语音通话
• Orchestrate Assistants  编排助手
• Generate images  生成图像
• Summarize a video (local file / YouTube)
• 总结视频（本地文件 / YouTube）
• Turn text & images into high quality embeddings
• 将文本和图像转换为高质量的嵌入
• Transcribe audio in real time
• 实时转录音频
• Create Chatbots utilizing multiple Agents:
• 使用多个代理创建聊天机器人：

三、开始

通过 NuGet 安装 LLM Tornado：

dotnet add package LlmTornado

Optional: extra features and quality of life extension methods are distributed in Contrib addon:

dotnet add package LlmTornado LlmTornado.Contrib

在多个提供者之间进行推理就像更改ChatModel参数一样简单。Tornado实例可以使用多个API密钥构建，然后根据模型自动使用正确的密钥：

TornadoApi api = new TornadoApi([
    new (LLmProviders.OpenAi, "OPEN_AI_KEY"),
    new (LLmProviders.Anthropic, "ANTHROPIC_KEY"),
    new (LLmProviders.Cohere, "COHERE_KEY"),
    new (LLmProviders.Google, "GOOGLE_KEY"),
    new (LLmProviders.Groq, "GROQ_KEY"),
    new (LLmProviders.DeepSeek, "DEEP_SEEK_KEY"),
    new (LLmProviders.Mistral, "MISTRAL_KEY"),
    new (LLmProviders.XAi, "XAI_KEY"),
    new (LLmProviders.Perplexity, "PERPLEXITY_KEY")
]);

List<ChatModel> models = [
    ChatModel.OpenAi.O3.Mini, ChatModel.Anthropic.Claude37.Sonnet,
    ChatModel.Cohere.Command.RPlus, ChatModel.Google.Gemini.Gemini2Flash001,
    ChatModel.Groq.Meta.Llama370B, ChatModel.DeepSeek.Models.Chat,
    ChatModel.Mistral.Premier.MistralLarge, ChatModel.XAi.Grok.Grok2241212,
    ChatModel.Perplexity.Sonar.Default
];

foreach (ChatModel model in models)
{
    string? response = await api.Chat.CreateConversation(model)
        .AppendSystemMessage("You are a fortune teller.")
        .AppendUserInput("What will my future bring?")
        .GetResponse();

    Console.WriteLine(response);
}
💡 Instead of passing in a strongly typed model, you can pass a string instead: await api.Chat.CreateConversation("gpt-4o"), Tornado will automatically resolve the provider.

Vendor Extensions
Tornado has a powerful concept of VendorExtensions which can be applied to various endpoints and are strongly typed. Many Providers offer unique/niche APIs, often enabling use cases otherwise unavailable. For example, let's set a reasoning budget for Anthropic's Claude 3.7:

public static async Task AnthropicSonnet37Thinking()
{
    Conversation chat = Program.Connect(LLmProviders.Anthropic).Chat.CreateConversation(new ChatRequest
    {
        Model = ChatModel.Anthropic.Claude37.Sonnet,
        VendorExtensions = new ChatRequestVendorExtensions(new ChatRequestVendorAnthropicExtensions
        {
            Thinking = new AnthropicThinkingSettings
            {
                BudgetTokens = 2_000,
                Enabled = true
            }
        })
    });
    
    chat.AppendUserInput("Explain how to solve differential equations.");

    ChatRichResponse blocks = await chat.GetResponseRich();

    if (blocks.Blocks is not null)
    {
        foreach (ChatRichResponseBlock reasoning in blocks.Blocks.Where(x => x.Type is ChatRichResponseBlockTypes.Reasoning))
        {
            Console.ForegroundColor = ConsoleColor.DarkGray;
            Console.WriteLine(reasoning.Reasoning?.Content);
            Console.ResetColor();
        }

        foreach (ChatRichResponseBlock reasoning in blocks.Blocks.Where(x => x.Type is ChatRichResponseBlockTypes.Message))
        {
            Console.WriteLine(reasoning.Message);
        }
    }
}
🔮 Custom Providers
Instead of consuming commercial APIs, one can roll their own inference servers easily with a myriad of tools available. Here is a simple demo for streaming response with Ollama, but the same approach can be used for any custom provider:

public static async Task OllamaStreaming()
{
    TornadoApi api = new TornadoApi(new Uri("http://localhost:11434")); // default Ollama port
    
    await api.Chat.CreateConversation(new ChatModel("falcon3:1b")) // <-- replace with your model
        .AppendUserInput("Why is the sky blue?")
        .StreamResponse(Console.Write);
}

Advanced Inference
Streaming
Tornado offers several levels of abstraction, trading more details for more complexity. The simple use cases where only plaintext is needed can be represented in a terse format:

await api.Chat.CreateConversation(ChatModel.Anthropic.Claude3.Sonnet)
    .AppendSystemMessage("You are a fortune teller.")
    .AppendUserInput("What will my future bring?")
    .StreamResponse(Console.Write);
Streaming with Rich content
When plaintext is insufficient, switch to StreamResponseRich or GetResponseRich() APIs. Tools requested by the model can be resolved later and never returned to the model. This is useful in scenarios where we use the tools without intending to continue the conversation:

//Ask the model to generate two images, and stream the result:
public static async Task GoogleStreamImages()
{
    Conversation chat = api.Chat.CreateConversation(new ChatRequest
    {
        Model = ChatModel.Google.GeminiExperimental.Gemini2FlashImageGeneration,
        Modalities = [ ChatModelModalities.Text, ChatModelModalities.Image ]
    });
    
    chat.AppendUserInput([
        new ChatMessagePart("Generate two images: a lion and a squirrel")
    ]);
    
    await chat.StreamResponseRich(new ChatStreamEventHandler
    {
        MessagePartHandler = async (part) =>
        {
            if (part.Text is not null)
            {
                Console.Write(part.Text);
                return;
            }

            if (part.Image is not null)
            {
                // In our tests this executes Chafa to turn the raw base64 data into Sixels
                await DisplayImage(part.Image.Url);
            }
        },
        BlockFinishedHandler = (block) =>
        {
            Console.WriteLine();
            return ValueTask.CompletedTask;
        },
        OnUsageReceived = (usage) =>
        {
            Console.WriteLine();
            Console.WriteLine(usage);
            return ValueTask.CompletedTask;
        }
    });
}
Tools with immediate resolve
Tools requested by the model can be resolved and the results returned immediately. This has the benefit of automatically continuing the conversation:

Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O,
    Tools =
    [
        new Tool(new ToolFunction("get_weather", "gets the current weather", new
        {
            type = "object",
            properties = new
            {
                location = new
                {
                    type = "string",
                    description = "The location for which the weather information is required."
                }
            },
            required = new List<string> { "location" }
        }))
    ]
})
.AppendSystemMessage("You are a helpful assistant")
.AppendUserInput("What is the weather like today in Prague?");

ChatStreamEventHandler handler = new ChatStreamEventHandler
{
  MessageTokenHandler = (x) =>
  {
      Console.Write(x);
      return Task.CompletedTask;
  },
  FunctionCallHandler = (calls) =>
  {
      calls.ForEach(x => x.Result = new FunctionResult(x, "A mild rain is expected around noon.", null));
      return Task.CompletedTask;
  },
  AfterFunctionCallsResolvedHandler = async (results, handler) => { await chat.StreamResponseRich(handler); }
};

await chat.StreamResponseRich(handler);
Tools with deferred resolve
Instead of resolving the tool call, we can postpone/quit the conversation. This is useful for extractive tasks, where we care only for the tool call:

Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.Turbo,
    Tools = new List<Tool>
    {
        new Tool
        {
            Function = new ToolFunction("get_weather", "gets the current weather")
        }
    },
    ToolChoice = new OutboundToolChoice(OutboundToolChoiceModes.Required)
});

chat.AppendUserInput("Who are you?"); // user asks something unrelated, but we force the model to use the tool
ChatRichResponse response = await chat.GetResponseRich(); // the response contains one block of type Function
GetResponseRichSafe() API is also available, which is guaranteed not to throw on the network level. The response is wrapped in a network-level wrapper, containing additional information. For production use cases, either use try {} catch {} on all the HTTP request-producing Tornado APIs, or use the safe APIs.

Simple frontend example - REPL
This interactive demo can be expanded into an end-user-facing interface in the style of ChatGPT. We show how to use strongly typed tools together with streaming and resolving parallel tool calls. ChatStreamEventHandler is a convenient class with a subscription interface for listening to the various streaming events:

public static async Task OpenAiFunctionsStreamingInteractive()
{
    // 1. set up a sample tool using a strongly typed model
    ChatPluginCompiler compiler = new ChatPluginCompiler();
    compiler.SetFunctions([
        new ChatPluginFunction("get_weather", "gets the current weather in a given city", [
            new ChatFunctionParam("city_name", "name of the city", ChatPluginFunctionAtomicParamTypes.String)
        ])
    ]);
    
    // 2. in this scenario, the conversation starts with the user asking for the current weather in two of the supported cities.
    // we can try asking for the weather in the third supported city (Paris) later.
    Conversation chat = api.Chat.CreateConversation(new ChatRequest
    {
        Model = ChatModel.OpenAi.Gpt4.Turbo,
        Tools = compiler.GetFunctions()
    }).AppendUserInput("Please call functions get_weather for Prague and Bratislava (two function calls).");

    // 3. repl
    while (true)
    {
        // 3.1 stream the response from llm
        await StreamResponse();

        // 3.2 read input
        while (true)
        {
            Console.WriteLine();
            Console.Write("> ");
            string? input = Console.ReadLine();

            if (input?.ToLowerInvariant() is "q" or "quit")
            {
                return;
            }
            
            if (!string.IsNullOrWhiteSpace(input))
            {
                chat.AppendUserInput(input);
                break;
            }
        }
    }

    async Task StreamResponse()
    {
        await chat.StreamResponseRich(new ChatStreamEventHandler
        {
            MessageTokenHandler = async (token) =>
            {
                Console.Write(token);
            },
            FunctionCallHandler = async (fnCalls) =>
            {
                foreach (FunctionCall x in fnCalls)
                {
                    if (!x.TryGetArgument("city_name", out string? cityName))
                    {
                        x.Result = new FunctionResult(x, new
                        {
                            result = "error",
                            message = "expected city_name argument"
                        }, null, true);
                        continue;
                    }

                    x.Result = new FunctionResult(x, new
                    {
                        result = "ok",
                        weather = cityName.ToLowerInvariant() is "prague" ? "A mild rain" : cityName.ToLowerInvariant() is "paris" ? "Foggy, cloudy" : "A sunny day"
                    }, null, true);
                }
            },
            AfterFunctionCallsResolvedHandler = async (fnResults, handler) =>
            {
                await chat.StreamResponseRich(handler);
            }
        });
    }
}
Other endpoints such as Images, Embedding, Speech, Assistants, Threads and Vision are also supported!
Check the links for simple-to-understand examples!

软件下载

夸克网盘分享

本文信息来源于GitHub作者地址：https://github.com/lofcz/LlmTornado