Skip to content

Commit

Permalink
.Net: Example of retry logic using Filters (#6152)
Browse files Browse the repository at this point in the history
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Based on: #6105

This example shows how to perform retry with filter and switch to
another model as a fallback.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
  • Loading branch information
dmytrostruk committed May 8, 2024
1 parent 26ad632 commit 8c82204
Show file tree
Hide file tree
Showing 2 changed files with 73 additions and 0 deletions.
72 changes: 72 additions & 0 deletions dotnet/samples/Concepts/Filtering/RetryWithFilters.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
// Copyright (c) Microsoft. All rights reserved.

using System.Net;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;

namespace Filtering;

/// <summary>
/// This example shows how to perform retry with filter and switch to another model as a fallback.
/// </summary>
public class RetryWithFilters(ITestOutputHelper output) : BaseTest(output)
{
[Fact]
public async Task ChangeModelAndRetryAsync()
{
// Default and fallback models for demonstration purposes
const string DefaultModelId = "gpt-4";
const string FallbackModelId = "gpt-3.5-turbo-1106";

var builder = Kernel.CreateBuilder();

// Add OpenAI chat completion service with invalid API key to force a 401 Unauthorized response
builder.AddOpenAIChatCompletion(modelId: DefaultModelId, apiKey: "invalid_key");

// Add OpenAI chat completion service with valid configuration as a fallback
builder.AddOpenAIChatCompletion(modelId: FallbackModelId, apiKey: TestConfiguration.OpenAI.ApiKey);

// Add retry filter
builder.Services.AddSingleton<IFunctionInvocationFilter>(new RetryFilter(FallbackModelId));

// Build kernel
var kernel = builder.Build();

// Initially, use "gpt-4" with invalid API key to simulate exception
var executionSettings = new OpenAIPromptExecutionSettings { ModelId = DefaultModelId, MaxTokens = 20 };

var result = await kernel.InvokePromptAsync("Hi, can you help me today?", new(executionSettings));

Console.WriteLine(result);

// Output: Of course! I'll do my best to help you. What do you need assistance with?
}

/// <summary>
/// Filter to change the model and perform retry in case of exception.
/// </summary>
private sealed class RetryFilter(string fallbackModelId) : IFunctionInvocationFilter
{
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
{
try
{
// Try to invoke function
await next(context);
}
// Catch specific exception
catch (HttpOperationException exception) when (exception.StatusCode == HttpStatusCode.Unauthorized)
{
// Get current execution settings
PromptExecutionSettings executionSettings = context.Arguments.ExecutionSettings![PromptExecutionSettings.DefaultServiceId];

// Override settings with fallback model id
executionSettings.ModelId = fallbackModelId;

// Try to invoke function again
await next(context);
}
}
}
}
1 change: 1 addition & 0 deletions dotnet/samples/Concepts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ Down below you can find the code snippets that demonstrate the usage of many Sem
- [FunctionInvocationFiltering](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/FunctionInvocationFiltering.cs)
- [Legacy_KernelHooks](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/Legacy_KernelHooks.cs)
- [PromptRenderFiltering](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/PromptRenderFiltering.cs)
- [RetryWithFilters](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/RetryWithFilters.cs)

## Functions - Invoking [`Method`](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromMethod.cs) or [`Prompt`](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromPrompt.cs) functions with [`Kernel`](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Abstractions/Kernel.cs)

Expand Down

0 comments on commit 8c82204

Please sign in to comment.