Skip to content
This repository has been archived by the owner on Jan 8, 2024. It is now read-only.

Allow setting nomad resource limits for waypoint server pre_task #4730

Open
Zarickan opened this issue May 18, 2023 · 1 comment
Open

Allow setting nomad resource limits for waypoint server pre_task #4730

Zarickan opened this issue May 18, 2023 · 1 comment
Labels
core/install jira Will add an Issue to Jira plugin/nomad

Comments

@Zarickan
Copy link

Zarickan commented May 18, 2023

Is your feature request related to a problem? Please describe.
The pre_task for the waypoint server and runner jobs default to 200 MHz CPU and 600 MB memory, without option of specifying their values. This means that I am unable to run waypoint on my small/resource constrained nomad cluster as waypoint would required 2*600 MB for the pre_task parts of both the runner and server jobs.

It is unexpected that the resources required for the server and runner respectively are actually 600 MB + x and 200 MHz + y when specifying x and y as the memory and CPU values.

Describe the solution you'd like
Ability to configure the resources for the pre_task part of the waypoint server job when running waypoint install the same way it can be configured for the server and runner jobs with e.g. -nomad-server-cpu and -nomad-server-memory

Describe alternatives you've considered
Increasing capacity of workers in the nomad cluster.

Explain any additional use-cases
n/a

Additional context
As far as I can tell from the code all the pre_task does is run chown on a folder:

func SetupPretask(volumeMounts []*api.VolumeMount) *api.Task {
preTask := api.NewTask("pre_task", "docker")
// Observed WP user and group IDs in the published container, update if those ever change
waypointUserID := 100
waypointGroupID := 1000
cpu := DefaultResourcesCPU
mem := DefaultResourcesMemory
preTask.Config = map[string]interface{}{
// Doing this because this is the only way https://github.com/hashicorp/nomad/issues/8892
"image": "busybox:latest",
"command": "sh",
"args": []string{"-c", fmt.Sprintf("chown -R %d:%d /data/", waypointUserID, waypointGroupID)},
}
preTask.VolumeMounts = volumeMounts
preTask.Resources = &api.Resources{
CPU: &cpu,
MemoryMB: &mem,
}
preTask.Lifecycle = &api.TaskLifecycle{
Hook: "prestart",
Sidecar: false,
}
return preTask
}

const (
// bytes
DefaultCSIVolumeCapacityMin = int64(1073741824)
DefaultCSIVolumeCapacityMax = int64(2147483648)
DefaultCSIVolumeMountFS = "xfs"
DefaultResourcesCPU = 200
DefaultResourcesMemory = 600
)

This seems like a lot of resources to allocate in the pre_task to something so simple.

@Zarickan Zarickan added the new label May 18, 2023
@evanphx evanphx added plugin/nomad core/install jira Will add an Issue to Jira and removed new labels May 24, 2023
@evanphx
Copy link
Contributor

evanphx commented May 24, 2023

Hi @Zarickan! Great find, totally agree those limits are wrong for the pretask. We'll update the pretask to use far, far smaller limits because it just needs to run a couple of unix commands anyway.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
core/install jira Will add an Issue to Jira plugin/nomad
Projects
None yet
Development

No branches or pull requests

2 participants