From 93814959f1dbb963128c1e9273db2ac492995862 Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 14:07:04 +0200 Subject: [PATCH 01/11] Add section about capacity planning --- docs/Guides/Recommendations.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index f0ee7b1985..97e8e96158 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -8,7 +8,8 @@ This document contains a set of recommendations when using Fastify. - [HAProxy](#haproxy) - [Nginx](#nginx) - [Kubernetes](#kubernetes) - +- [Kubernetes](#kubernetes) +- [Capacity planning for production](#capacity) ## Use A Reverse Proxy @@ -298,3 +299,17 @@ readinessProbe: timeoutSeconds: 3 successThreshold: 1 failureThreshold: 5 +``` + +## Capacity planning for production + + +In order to rightsize your fastify application deployment in production, the following considerations need to be taken into account: + +* In order to have the lowest possible latency, 2 vCPU are recommended per app instance (k8s pod). The second vCPU will mostly be used by the GC and libuv threadpool. This will minimize the latency for your users, as well as the memory usage, as the GC will be run more frequently. The main thread won't have to stop to let the GC run. + +* In order to optimize for throughput (handling the largest possible amount of requests per second per vCPU), consider using smaller amount of vCPUs per app instance. It is totally fine to run Node.js application with 1 vCPU. + +* You may experiment with even smaller amount of vCPU, which may provide even better throughput in certain use-cases. There are reports of e. g. API gateway solutions working well with 100m-200m vCPU in Kubernetes. + +See [Node's Event Loop From the Inside Out ](https://www.youtube.com/watch?v=P9csgxBgaZ8) in order to understand the workings of Node.js in greater detail and make a better determination about what your specific application needs. From e9bd231edc0a73ce7043d2416b5db9d0194ed8b7 Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 14:13:00 +0200 Subject: [PATCH 02/11] Fix linter errors --- docs/Guides/Recommendations.md | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 97e8e96158..40d2fa8f4e 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -304,12 +304,23 @@ readinessProbe: ## Capacity planning for production -In order to rightsize your fastify application deployment in production, the following considerations need to be taken into account: - -* In order to have the lowest possible latency, 2 vCPU are recommended per app instance (k8s pod). The second vCPU will mostly be used by the GC and libuv threadpool. This will minimize the latency for your users, as well as the memory usage, as the GC will be run more frequently. The main thread won't have to stop to let the GC run. - -* In order to optimize for throughput (handling the largest possible amount of requests per second per vCPU), consider using smaller amount of vCPUs per app instance. It is totally fine to run Node.js application with 1 vCPU. - -* You may experiment with even smaller amount of vCPU, which may provide even better throughput in certain use-cases. There are reports of e. g. API gateway solutions working well with 100m-200m vCPU in Kubernetes. - -See [Node's Event Loop From the Inside Out ](https://www.youtube.com/watch?v=P9csgxBgaZ8) in order to understand the workings of Node.js in greater detail and make a better determination about what your specific application needs. +In order to rightsize your fastify application deployment in production, the +following considerations need to be taken into account: + +* In order to have the lowest possible latency, 2 vCPU are recommended per app +instance (k8s pod). The second vCPU will mostly be used by the GC and libuv +threadpool. This will minimize the latency for your users, as well as the +memory usage, as the GC will be run more frequently. The main thread won't +have to stop to let the GC run. + +* In order to optimize for throughput (handling the largest possible amount of +requests per second per vCPU), consider using smaller amount of vCPUs per app +instance. It is totally fine to run Node.js application with 1 vCPU. + +* You may experiment with even smaller amount of vCPU, which may provide even +better throughput in certain use-cases. There are reports of e. g. API gateway +solutions working well with 100m-200m vCPU in Kubernetes. + +See [Node's Event Loop From the Inside Out ](https://www.youtube.com/watch?v=P9csgxBgaZ8) +in order to understand the workings of Node.js in greater detail and make a +better determination about what your specific application needs. From 9f2bc1beba9f39adb69b34d83ba70f8e5828f574 Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 16:52:40 +0200 Subject: [PATCH 03/11] Address code review comments --- docs/Guides/Recommendations.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 40d2fa8f4e..22cae20b48 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -304,8 +304,16 @@ readinessProbe: ## Capacity planning for production -In order to rightsize your fastify application deployment in production, the -following considerations need to be taken into account: +In order to rightsize your fastify application deployment in production, +you are highly recommended to perform your own measurements against +different configurations of the production environment +(2 vCPU, 1 vCPU, fractional parts of vCPU). + +You can use such tools as [k6](https://github.com/grafana/k6) +or [autocannon](https://github.com/mcollina/autocannon) for conducting +the necessary performance tests. + +That said, you may also consider the following as a rule of a thumb: * In order to have the lowest possible latency, 2 vCPU are recommended per app instance (k8s pod). The second vCPU will mostly be used by the GC and libuv From b1611d8e1e691a5b682130272204668ad98eb9d5 Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 18:19:46 +0200 Subject: [PATCH 04/11] Update docs/Guides/Recommendations.md Co-authored-by: James Sumners --- docs/Guides/Recommendations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 22cae20b48..42533e724d 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -301,7 +301,7 @@ readinessProbe: failureThreshold: 5 ``` -## Capacity planning for production +## Capacity Planning For Production In order to rightsize your fastify application deployment in production, From 550f658c763a6588ce17cfcc385a045f62125579 Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 18:20:18 +0200 Subject: [PATCH 05/11] Update docs/Guides/Recommendations.md Co-authored-by: James Sumners --- docs/Guides/Recommendations.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 42533e724d..38a0d09538 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -304,10 +304,12 @@ readinessProbe: ## Capacity Planning For Production -In order to rightsize your fastify application deployment in production, +In order to rightsize your Fastify application deployment in production, you are highly recommended to perform your own measurements against -different configurations of the production environment -(2 vCPU, 1 vCPU, fractional parts of vCPU). +different configurations of the production environment, which may +use real CPU cores, virtual CPU cores (vCPU), or even fractional +vCPU cores. We will use the term vCPU throughout this +recommendation to represent any CPU type. You can use such tools as [k6](https://github.com/grafana/k6) or [autocannon](https://github.com/mcollina/autocannon) for conducting From 1fc27570ffb08f2a7a31d8b85b0816d1ccdd171f Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 18:22:13 +0200 Subject: [PATCH 06/11] Update docs/Guides/Recommendations.md Co-authored-by: James Sumners --- docs/Guides/Recommendations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 38a0d09538..0b767ae9c7 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -318,9 +318,9 @@ the necessary performance tests. That said, you may also consider the following as a rule of a thumb: * In order to have the lowest possible latency, 2 vCPU are recommended per app -instance (k8s pod). The second vCPU will mostly be used by the GC and libuv +instance (e.g., a k8s pod). The second vCPU will mostly be used by the garbage collector (GC) and libuv threadpool. This will minimize the latency for your users, as well as the -memory usage, as the GC will be run more frequently. The main thread won't +memory usage, as the GC will be run more frequently. Also, the main thread won't have to stop to let the GC run. * In order to optimize for throughput (handling the largest possible amount of From baf6b186c2e852f92ab61b17522a1d8c30975efe Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 18:23:04 +0200 Subject: [PATCH 07/11] Rewrap --- docs/Guides/Recommendations.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 0b767ae9c7..8a017251d9 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -318,10 +318,10 @@ the necessary performance tests. That said, you may also consider the following as a rule of a thumb: * In order to have the lowest possible latency, 2 vCPU are recommended per app -instance (e.g., a k8s pod). The second vCPU will mostly be used by the garbage collector (GC) and libuv -threadpool. This will minimize the latency for your users, as well as the -memory usage, as the GC will be run more frequently. Also, the main thread won't -have to stop to let the GC run. +instance (e.g., a k8s pod). The second vCPU will mostly be used by the +garbage collector (GC) and libuv threadpool. This will minimize the latency +for your users, as well as the memory usage, as the GC will be run more +frequently. Also, the main thread won't have to stop to let the GC run. * In order to optimize for throughput (handling the largest possible amount of requests per second per vCPU), consider using smaller amount of vCPUs per app From 201ffb85a7174a6b60d29eb6e5fb885a6096aa7a Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 18:25:34 +0200 Subject: [PATCH 08/11] Consistent casing --- docs/Guides/Recommendations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 8a017251d9..9361642b80 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -9,7 +9,7 @@ This document contains a set of recommendations when using Fastify. - [Nginx](#nginx) - [Kubernetes](#kubernetes) - [Kubernetes](#kubernetes) -- [Capacity planning for production](#capacity) +- [Capacity Planning For Production](#capacity) ## Use A Reverse Proxy From 7653eaa0f5076c2bc6fcbb56589747f5a73714a2 Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Sun, 30 Oct 2022 18:27:33 +0200 Subject: [PATCH 09/11] Clarify environment terminology --- docs/Guides/Recommendations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 9361642b80..b3d1d9bd60 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -304,9 +304,9 @@ readinessProbe: ## Capacity Planning For Production -In order to rightsize your Fastify application deployment in production, +In order to rightsize the production environment for your Fastify application, you are highly recommended to perform your own measurements against -different configurations of the production environment, which may +different configurations of the environment, which may use real CPU cores, virtual CPU cores (vCPU), or even fractional vCPU cores. We will use the term vCPU throughout this recommendation to represent any CPU type. From 1de62e2ec977ec1358bff5dff9923f55a6c0ce9f Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Tue, 1 Nov 2022 10:27:35 +0200 Subject: [PATCH 10/11] Update docs/Guides/Recommendations.md Co-authored-by: Matteo Collina --- docs/Guides/Recommendations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index b3d1d9bd60..753d961047 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -324,7 +324,7 @@ for your users, as well as the memory usage, as the GC will be run more frequently. Also, the main thread won't have to stop to let the GC run. * In order to optimize for throughput (handling the largest possible amount of -requests per second per vCPU), consider using smaller amount of vCPUs per app +requests per second per vCPU available), consider using smaller amount of vCPUs per app instance. It is totally fine to run Node.js application with 1 vCPU. * You may experiment with even smaller amount of vCPU, which may provide even From fec796aa1ad4a560fcb0ca8c65be72adadfa7d1d Mon Sep 17 00:00:00 2001 From: Igor Savin Date: Tue, 1 Nov 2022 11:41:11 +0200 Subject: [PATCH 11/11] Fix line size --- docs/Guides/Recommendations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/Guides/Recommendations.md b/docs/Guides/Recommendations.md index 753d961047..ad40fb8458 100644 --- a/docs/Guides/Recommendations.md +++ b/docs/Guides/Recommendations.md @@ -324,8 +324,8 @@ for your users, as well as the memory usage, as the GC will be run more frequently. Also, the main thread won't have to stop to let the GC run. * In order to optimize for throughput (handling the largest possible amount of -requests per second per vCPU available), consider using smaller amount of vCPUs per app -instance. It is totally fine to run Node.js application with 1 vCPU. +requests per second per vCPU available), consider using smaller amount of vCPUs +per app instance. It is totally fine to run Node.js application with 1 vCPU. * You may experiment with even smaller amount of vCPU, which may provide even better throughput in certain use-cases. There are reports of e. g. API gateway