[Enhancement]: Ollama module

[frankhaugen]

Problem

Its not having an Ollama module.

I realize that my POC here needs to have a discussion of design and features, but its a fun litte thing

Solution

Make a module, here's a prototype of Ollama working with Testcontainers in LinqPad 8:

Deps the nugets Testcontainers and OllamaSharp

var container = new ContainerBuilder()
	.WithName("ollama-container")
	.WithImage("ollama/ollama")
	.WithPortBinding(11434, 11434)
	.WithVolumeMount("ollama", "/root/.ollama")
	.WithExposedPort(11434)
	.Build();
	
var keepAlive = true;

container.Started += async (sender, args) => 
{
	Console.WriteLine("Container started");
	try
	{
		Console.WriteLine("Start ollama service");
		await container.ExecAsync(new List<string>() {
			"ollama run llama2"
		});
		var result = await RunAsync();
		result.Dump();
	}
	catch (Exception ex)
	{
		Console.WriteLine(ex);
	}
	
	keepAlive = false;
};

await container.StartAsync();

while (!QueryCancelToken.IsCancellationRequested && keepAlive)
{
	await Task.Delay(1000);
}

var logs = await container.GetLogsAsync();

await container.StopAsync();
await container.DisposeAsync();

async Task<string> RunAsync()
{
	try
	{
		var client = new OllamaApiClient("http://localhost:11434/api/generate", "llama2");
		ChatRequest request = new ChatRequest();
		request.Messages ??= new List<Message>();
		request.Messages.Add(new Message(ChatRole.User, "Write me a dad-joke about cats please?", Enumerable.Empty<string>().ToArray()));
		request.Model = "llama2";
		request.Stream = false;
		var messages = new List<ChatResponseStream>();
		var responses = await client.SendChat(request, x => messages.Add(x));
		return string.Join("\n\n", responses.Select(r => r.Content));
	}
	catch (Exception ex)
	{
		return ex.ToString();
	}
}

Benefit

Its really cool

Alternatives

To be less cool

Would you like to help contributing this enhancement?

Yes

[frankhaugen]

The output from a non-initial run:

[testcontainers.org 00:00:00.07] Connected to Docker:
Host: npipe://./pipe/docker_engine
Server Version: 24.0.7
Kernel Version: 5.10.102.1-microsoft-standard-WSL2
API Version: 1.43
Operating System: Docker Desktop
Total Memory: 62.39 GB
[testcontainers.org 00:00:00.30] Docker container 042f3c377795 created
[testcontainers.org 00:00:00.35] Start Docker container 042f3c377795
[testcontainers.org 00:00:01.67] Wait for Docker container 042f3c377795 to complete readiness checks
[testcontainers.org 00:00:01.69] Docker container 042f3c377795 ready
[testcontainers.org 00:00:01.73] Docker container a9d04460af00 created
[testcontainers.org 00:00:01.76] Start Docker container a9d04460af00
[testcontainers.org 00:00:02.01] Wait for Docker container a9d04460af00 to complete readiness checks
[testcontainers.org 00:00:02.02] Docker container a9d04460af00 ready
Container started
Start ollama service
[testcontainers.org 00:00:02.03] Execute "ollama run llama2" at Docker container a9d04460af00
Write me a dad-joke about cats please?
Why did the cat join a band? Because he wanted to be the purr-cussionist! 😹
[testcontainers.org 00:00:08.85] Stop Docker container a9d04460af00
[testcontainers.org 00:00:09.24] Delete Docker container a9d04460af00

[HofmeisterAn]

Hi 👋, thanks for creating the request.

Total Memory: 62.39 GB

This sounds like you are running a pretty nice machine 😄.

Since you are interested in contributing the Ollama module, which is great, I would like to share some additional information. You will find a basic guide about implementing a module here. All of our existing modules can be used as a blueprint too. All of them follow our best practices, which are necessary for a smoothly working module across various systems, etc.

If you have any further questions, do not hesitate to reach out to me or the other developers in our Testcontainers community on Slack.

[frankhaugen]

If you have any further questions, do not hesitate to reach out to me or the other developers in our Testcontainers community on Slack.

So i spent the day, (in the few moments between my dayjob's demands for attention), got it pretty nice, then when I was going to go through the best practices, and there's a stumbling block for. I might not need Volume to be statically named, but a default and configurable one would save on time as the there are resources downloaded.

From the best practices:

Avoid assigning static names to Docker resources such as containers, networks, and volumes. Static names may clash with existing resources, causing tests to fail. Instead, use random assigned names. By default, Docker assigns random names.

I will make the PR, and link this issue and we can discuss when actually looking at the