Performance of an Api that mocks any Api

[Leonardo-Ferreira]

One of the challenges I face when I'm "battle-proofing" my Apis is determining how they behave when their dependencies are not behaving as expected.

It's easy to setup a test environment that performs "perfectly" and find out that your code can support 1000 req/s per sec. But the question im trying to answer is "How does my application behave when that Api that I have as a dependency stops responding at 15ms and starts fluctuating between 500 and 2500ms?"

In order to do that, I thought: what if I create a api, that you can dynamically setup a test, that accepts anything and replies a random-within-a-set-of-pre-defined-responses BUT delays the requests by a random value between 500 and 2500?

So, first I created a controller and

[HttpPost("/{*path}")]
public async Task GenericPost(string? path)
public async Task GenericGet(string? path)
public async Task GenericPut(string? path)
and so on

Then based on a lot of caching and optimizations, I got my part of the code working consistently under 5ms on my local machine, in debug mode... but that's only my part of the code. On the client side (fiddler everywhere) a call to my app running http on localhost takes no less than 10ms (for this case, my code took 1ms).

What can I do to cut that "infra" time?

Ideias i had so far but don't know how to implement:

• Make the controllers singleton
• Don't do routing via asp.net (whatever you get, whatever the url, whatever the method, throw it to this GenericHandler)
• Don't do parameter mapping
• Use slim/empty builders (but ill need HTTPS and HTTP/2)

Currently my "Program.cs" is:

public class Program
{
    public static void Main(string[] args)
    {
        var builder = WebApplication.CreateBuilder(args);

        // Add services to the container.
        builder.Services.AddMemoryCache();
        builder.Services.AddControllers();
        builder.Services.AddSingleton<Random>();

        var app = builder.Build();

        app.MapControllers();

        app.Run();
    }
}

Edit 1

• Setting the header Content-Length helped a bit... somehow Transfer-Encoding: chunked was adding-up

[martincostello]

You could consider instead using chaos engineering in your application that calls remote services and inject faults and delays into its HttpClient calls directly. Then you don't need to create a "fake" API and optimise it and integrate it into your code, you instead inject runtime behaviours into your real application and observe the effects.

[Leonardo-Ferreira]

given the main question How does my application behave when that Api that I have as a dependency stops responding at 15ms and starts fluctuating between 500 and 2500ms?, I believe that this API is a better option than "inject faults and delays into the httpclient" because there might be a lot of ripple effects. Your code requests an api that calls another api and calls another database, and generates logs and whole lot of other possible side effects, and you need to know is how your api behaves... before you know you're crashing other unrelated APIs and a bunch of squads are yelling at you...

don't get me wrong, chaos eng is valuable... but the scope here is other