From a6b580b17fff7ad12a5c97c044f6799b642c8484 Mon Sep 17 00:00:00 2001 From: Ivan De Marino Date: Wed, 27 Jul 2022 09:21:58 +0100 Subject: [PATCH 1/5] New SDKv2/logging documentation section, including new Logging HTTP Transport page --- website/data/plugin-sdk-nav-data.json | 13 ++ .../plugin/sdkv2/logging/http-transport.mdx | 211 ++++++++++++++++++ website/docs/plugin/sdkv2/logging/index.mdx | 26 +++ 3 files changed, 250 insertions(+) create mode 100644 website/docs/plugin/sdkv2/logging/http-transport.mdx create mode 100644 website/docs/plugin/sdkv2/logging/index.mdx diff --git a/website/data/plugin-sdk-nav-data.json b/website/data/plugin-sdk-nav-data.json index aa1ca1695e..d14ca45b88 100644 --- a/website/data/plugin-sdk-nav-data.json +++ b/website/data/plugin-sdk-nav-data.json @@ -96,6 +96,19 @@ } ] }, + { + "title": "Logging", + "routes": [ + { + "title": "Overview", + "path": "logging" + }, + { + "title": "HTTP Transport", + "path": "logging/http-transport" + } + ] + }, { "title": "Testing", "routes": [ diff --git a/website/docs/plugin/sdkv2/logging/http-transport.mdx b/website/docs/plugin/sdkv2/logging/http-transport.mdx new file mode 100644 index 0000000000..4041177d32 --- /dev/null +++ b/website/docs/plugin/sdkv2/logging/http-transport.mdx @@ -0,0 +1,211 @@ +--- +page_title: Plugin Development - Logging HTTP Transport +description: |- + SDKv2 provides a helper to easily send all the HTTP Transactions to structured logging. +--- + +# Background + +Since Terraform `v0.9.5`, the public interface used by Provider Developers included an _helper_ inside the package `helper/logging`: [`NewTransport()`](https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/logging/transport.go), an implementation of the Golang standard library [`http.RoundTripper`](https://pkg.go.dev/net/http#RoundTripper). This was provided as an easy way for Terraform provider developers to add _DEBUG-level_ logging to the HTTP transactions of their provider. + +Unfortunately this helper is designed to log the entirety of each request/response it handles, including every header and the entire body, regardless of how _sensitive_ the content is. Over time security concerns have been [raised](https://github.com/hashicorp/terraform-plugin-sdk/issues/2), and required provider developers to build [workarounds](https://github.com/hashicorp/terraform-provider-tfe/pull/479). + +The recommended way to produce logs from your provider is using **[terraform-plugin-log](https://www.terraform.io/plugin/log)** library, and that provides [log filtering](https://www.terraform.io/plugin/log/filtering) functionality. So, we built a new _RoundTripper_ to deprecate the existing one. + + +# Setting up logging for HTTP Transactions + +SDK offers helpers to put in place logging of HTTP Transactions: it’s built on top of **[terraform-plugin-log](https://www.terraform.io/plugin/log)**, so it allows to leverage its features, without having to write the whole implementation of `http.RoundTripper`. + +There are 2 functions inside the package `helper/logging`, each targeting a specific logging setup that the provider developer might have chosen (see [“Writing Log Output”](https://www.terraform.io/plugin/log/writing) for details): + +* `NewLoggingHTTPTransport(transport http.RoundTripper)` +* `NewSubsystemLoggingHTTPTransport(subsystem string, transport http.RoundTripper)` + +Using one or the other depends on the developer, and if they want their logging to happen against the `tflog` Provider root logger, or against a `tflog` Provider [Subsystem logger](https://www.terraform.io/plugin/log/writing#subsystems). For the latter, the only thing to remember is that the `subsystem` string used with `NewSubsystemLoggingHTTPTransport()`, has to match the [pre-created subsystem logger name](https://www.terraform.io/plugin/log/writing#create-subsystems). + +To make use of our logging transport, there are 2 things that a provider developer should do: + +* Setting up the HTTP Client to use this new Transport +* Ensuring that each HTTP Request, before being submitted to the Client, carries the configured `context.Context`, storing the desired logging configuration + + +## Creating HTTP Client that uses the new logging transport + +Once the correct _Transport _has been created, it should be used when setting up the `http.Client` that the provider is going to use. For example, a good place to set up the client could be the `schema.Provider` `ConfigureContextFunc`: + +```go +func New() (*schema.Provider, error) { + return &schema.Provider{ + // omitting the rest of the schema definition + + ConfigureContextFunc: func (ctx context.Context, rsc *schema.ResourceData) (interface{}, diag.Diagnostics) { + + // omitting provider-specific configuration logic + + transport := logging.NewLoggingHTTPTransport(http.DefaultTransport) + client := http.Client{ + Transport: transport, + } + + return client, diag.Diagnostics{} + } + } +} +``` + +This will set up a client that is identical to the default Golang `http.Client`, except it uses the new logging transport. + + +## Sending HTTP Requests with the appropriate Context + +All calls to `tflog` package functionality must use an SDK provided `context.Context`, which stores the logging implementation. Providers written with `terraform-plugin-sdk` must use context-aware functionality, such as the [`helper/schema.Resource` type `ReadContext` field](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema#Resource.ReadContext). + +Use [`http.NewRequestWithContext()` function](https://pkg.go.dev/net/http#NewRequestWithContext) to create an HTTP request that includes the logging configuration from the `context.Context`. For example: + +```go +// inside a context-aware Resource function +req, err := http.NewRequestWithContext(ctx, "GET", "https://www.terraform.io", nil) +if err != nil { + return fmt.Errorf("Failed to create a new request: %w", err) +} + +res, err := client.Do(req) +if err != nil { + return fmt.Errorf("Request failed: %w", err) +} +defer res.Body.Close() +``` + +Use the [`(http.Request).WithContext()` method](https://pkg.go.dev/net/http#Request.WithContext) to set the context for the `http.Request` if the request is generated separate from where the `context.Context` is available. + + +## Log format for an HTTP Transaction + +Each HTTP Transaction processed by the new logging transport, will result in 2 log entries. Here is how the log for the HTTP Request towards [https://terraform.io](https://terraform.io) looks like: + +```text +2022-07-26T18:54:08.880+0100 [DEBUG] provider: Sending HTTP Request: Accept-Encoding=gzip Content-Length=0 \ + Host=www.terraform.io User-Agent=Go-http-client/1.1 \ + tf_http_op_type=request tf_http_req_method=GET \ + tf_http_req_uri=/ tf_http_req_version=HTTP/1.1 tf_http_trans_id=7e80e48d-8f32-f527-1412-52a8c84359e7 +``` + +And, if [logging in JSON format](https://www.terraform.io/internals/debugging) is enabled: + +```json +{ + "@level": "debug", + "@message": "Sending HTTP Request", + "@module": "provider", + "@timestamp": "2022-07-26T18:54:08.880+0100", + + "Accept-Encoding": "gzip", + "Content-Length": "0", + "Host": "www.terraform.io", + "User-Agent": "Go-http-client/1.1", + + "tf_http_op_type": "request", + "tf_http_req_body": "", + "tf_http_req_method": "GET", + "tf_http_req_uri": "/", + "tf_http_req_version": "HTTP/1.1", + "tf_http_trans_id": "7e80e48d-8f32-f527-1412-52a8c84359e7" +} +``` + +And the corresponding HTTP Response: + +```text +2022-07-26T18:54:10.734+0100 [DEBUG] provider: Received HTTP Response: Age=9 \ + Cache-Control="public, max-age=0, must-revalidate" Content-Type=text/html \ + Date="Tue, 26 Jul 2022 13:16:46 GMT" Etag="... ABCDEFGH..." Server=Vercel \ + Strict-Transport-Security="max-age=63072000" X-Frame-Options=SAMEORIGIN X-Matched-Path=/ \ + X-Nextjs-Cache=HIT X-Powered-By=Next.js X-Vercel-Cache=STALE \ + X-Vercel-Id=lhr1::iad1::lx2h8-99999999999999-fffffffffff \ + tf_http_op_type=response tf_http_res_body="... LOTS OF HTML ..." tf_http_res_status_code=200 \ + tf_http_res_status_reason="200 OK" tf_http_res_version=HTTP/2.0 \ + tf_http_trans_id=7e80e48d-8f32-f527-1412-52a8c84359e7 +``` + +And in JSON format: + +```json +{ + "@level": "debug", + "@message": "Received HTTP Response", + "@module": "provider", + "@timestamp": "2022-07-26T18:54:10.734+0100", + + "Age": "9", + "Cache-Control": "public, max-age=0, must-revalidate", + "Content-Type": "text/html", + "Date": "Tue, 26 Jul 2022 13:16:46 GMT", + "Etag": "... ABCDEFGH...", + "Server": "Vercel", + "Strict-Transport-Security": "max-age=63072000", + "X-Frame-Options": "SAMEORIGIN", + "X-Matched-Path": "/", + "X-Nextjs-Cache": "HIT", + "X-Powered-By": "Next.js", + "X-Vercel-Cache": "STALE", + "X-Vercel-Id": "lhr1::iad1::lx2h8-99999999999999-fffffffffff", + + "tf_http_op_type": "response", + "tf_http_res_body": "... LOTS OF HTML ...", + "tf_http_res_status_code": 200, + "tf_http_res_status_reason": "200 OK", + "tf_http_res_version": "HTTP/2.0", + "tf_http_trans_id": "7e80e48d-8f32-f527-1412-52a8c84359e7" +} +``` + +The log contains various parts, represented as [fields](https://www.terraform.io/plugin/log/writing#fields) in the JSON format above. It’s important to familiarize yourself with those: + +| Log field name | Description | Possible values | Applies to | +|---------------------------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|:------------------:| +| `tf_http_op_type` | Which HTTP operation log refers to | [`request`, `response`] | Request / Response | +| `tf_http_trans_id` | Unique identifier used by Request and Response that belong to the same HTTP Transaction. | A universally unique identifier ([UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier)) | Request / Response | +| `tf_http_req_body` | Request body | | Request | +| `tf_http_req_method` | Request method | A canonical [HTTP methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Method) | Request | +| `tf_http_req_uri` | Request URI | Ex. `"/path"` | Request | +| `tf_http_req_version` | Request HTTP version | Ex. `"HTTP/1.1"` | Request | +| `tf_http_res_body` | Response body | | Response | +| `tf_http_res_status_code` | Response status code | A canonical [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) | Response | +| `tf_http_res_status_reason | Response status reason | Canonical textual description of the corresponding `tf_http_res_status_code` | Response | +| `tf_http_res_version` | Response HTTP version | Ex. `"HTTP/2.0"` | Response | +| (Other fields) | Request / Response headers. One field per header. If header contains a single value, field value will be set to that. Otherwise, field value will be a slice of strings. | | Request / Response | + + +## Filtering sensitive data + +To configure [log filtering](https://www.terraform.io/plugin/log/filtering) in the HTTP Transport to the provider needs, all it is required is to configure the `context.Context` before it's set in the `http.Request`. + +For example, when using `NewSubsystemLoggingHTTPTransport()` to log our HTTP Transactions, to mask all the header values of the HTTP Requests containing an `Authorization` and `Proxy-Authorization` credentials: + +```go +// inside a context-aware Resource function +ctx := tflog.SubsystemMaskFieldValuesWithFieldKeys(ctx, "my-subsystem", "Authorization") +ctx = tflog.SubsystemMaskFieldValuesWithFieldKeys(ctx, "my-subsystem", "Proxy-Authorization") + +req, err := http.NewRequestWithContext(ctx, "GET", "https://www.terraform.io", nil) +if err != nil { + return fmt.Errorf("Failed to create a new request: %w", err) +} + +res, err := client.Do(req) +if err != nil { + return fmt.Errorf("Request failed: %w", err) +} +defer res.Body.Close() +``` + + +# Resources + +* [Plugin Development](https://www.terraform.io/plugin) +* [Plugin Development - Logging](https://www.terraform.io/plugin/log) +* [terraform-plugin-log - tflog](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log/tflog) package documentation +* [terraform-plugin-log](https://github.com/hashicorp/terraform-plugin-log) GitHub repository +* [terraform-plugin-sdk](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2) [`NewLoggingHTTPTransport()`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging#NewLoggingHTTPTransport) and [`NewSubsystemLoggingHTTPTransport()`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging#NewSubsystemLoggingHTTPTransport) +* [New Logging Transport](https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/logging/logging_http_transport.go) source code diff --git a/website/docs/plugin/sdkv2/logging/index.mdx b/website/docs/plugin/sdkv2/logging/index.mdx new file mode 100644 index 0000000000..a66d2e0def --- /dev/null +++ b/website/docs/plugin/sdkv2/logging/index.mdx @@ -0,0 +1,26 @@ +--- +page_title: Plugin Development - Logging +description: |- + Producing high quality logs can make a huge difference when trying to debug an issue. + Learn more about how to set-up logging and write meaningful logs. +--- + +# Logging + +Terraform Plugin SDKv2 integrates with a [structured logging framework](https://www.terraform.io/plugin/log): [terraform-plugin-log](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log). This plays a crucial role in building providers that facilitate timely [debugging](https://www.terraform.io/plugin/debugging) of issues. + +## Managing Log Output + +Learn how to use [environment variables and other methods](https://www.terraform.io/plugin/log/managing) to enable and filter logs. + +## Writing Log Output + +Learn how to [implement code in provider logic](https://www.terraform.io/plugin/log/writing) to output logs. + +## Filtering Log Output + +Learn how to [implement code in provider logic](https://www.terraform.io/plugin/log/filtering) to mask certain log messages and structured log fields or omit logs entirely. + +## Log HTTP Transactions + +Learn how to [set up the Logging HTTP Transport](./logging/http-transport) to log HTTP Transactions via the [structured logging framework](https://www.terraform.io/plugin/log). From 6872b301c9970ead1da1d163192e5ef6d9aaa0c6 Mon Sep 17 00:00:00 2001 From: Ivan De Marino Date: Wed, 27 Jul 2022 09:34:39 +0100 Subject: [PATCH 2/5] Tweak the release notes for PR 1006 --- .changelog/1006.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.changelog/1006.txt b/.changelog/1006.txt index 9dda15146a..a988237e8d 100644 --- a/.changelog/1006.txt +++ b/.changelog/1006.txt @@ -1,5 +1,5 @@ ```release-note:feature -helper/logging: New `NewLoggingHTTPTransport()` and `NewSubsystemLoggingHTTPTransport()` functions, providing `http.RoundTripper` Transport implementations that log request/response using [terraform-plugin-log](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log) +helper/logging: New `NewLoggingHTTPTransport()` and `NewSubsystemLoggingHTTPTransport()` functions, providing `http.RoundTripper` Transport implementations that log request/response using [terraform-plugin-log](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log) ([#546](https://github.com/hashicorp/terraform-plugin-sdk/issues/546)) ``` ```release-note:note From 6cc3b72511bd88ec236cfcdea3f648bd6f5eafbe Mon Sep 17 00:00:00 2001 From: Ivan De Marino Date: Thu, 28 Jul 2022 18:37:01 +0100 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> --- .../plugin/sdkv2/logging/http-transport.mdx | 67 ++++++++++--------- website/docs/plugin/sdkv2/logging/index.mdx | 9 ++- 2 files changed, 39 insertions(+), 37 deletions(-) diff --git a/website/docs/plugin/sdkv2/logging/http-transport.mdx b/website/docs/plugin/sdkv2/logging/http-transport.mdx index 4041177d32..d5ab726198 100644 --- a/website/docs/plugin/sdkv2/logging/http-transport.mdx +++ b/website/docs/plugin/sdkv2/logging/http-transport.mdx @@ -1,36 +1,31 @@ --- page_title: Plugin Development - Logging HTTP Transport description: |- - SDKv2 provides a helper to easily send all the HTTP Transactions to structured logging. + SDKv2 provides a helper to send all the HTTP transactions to structured logging. --- -# Background +# HTTP Transport -Since Terraform `v0.9.5`, the public interface used by Provider Developers included an _helper_ inside the package `helper/logging`: [`NewTransport()`](https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/logging/transport.go), an implementation of the Golang standard library [`http.RoundTripper`](https://pkg.go.dev/net/http#RoundTripper). This was provided as an easy way for Terraform provider developers to add _DEBUG-level_ logging to the HTTP transactions of their provider. +Terraform's public interface has included `helper/logging`: [`NewTransport()`](https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/logging/transport.go) since v0.9.5. This helper is an implementation of the Golang standard library [`http.RoundTripper`](https://pkg.go.dev/net/http#RoundTripper) that lets you add logging at the `DEBUG` level to your provider's HTTP transactions. -Unfortunately this helper is designed to log the entirety of each request/response it handles, including every header and the entire body, regardless of how _sensitive_ the content is. Over time security concerns have been [raised](https://github.com/hashicorp/terraform-plugin-sdk/issues/2), and required provider developers to build [workarounds](https://github.com/hashicorp/terraform-provider-tfe/pull/479). +We do not recommend using this original helper because it is designed to log the entirety of each request and response. This includes any sensitive content that may be present in the message header or body, presenting security concerns. -The recommended way to produce logs from your provider is using **[terraform-plugin-log](https://www.terraform.io/plugin/log)** library, and that provides [log filtering](https://www.terraform.io/plugin/log/filtering) functionality. So, we built a new _RoundTripper_ to deprecate the existing one. +Instead, we recommend using the [terraform-plugin-log](https://www.terraform.io/plugin/log) library to produce logs for your provider. This library does not present the same security concerns and provides [log filtering](https://www.terraform.io/plugin/log/filtering) functionality. This page explains how to set up the new `RoundTripper()` helper to log HTTP Transactions with `terraform-plugin-log`. -# Setting up logging for HTTP Transactions +# Setting Up Logging for HTTP Transactions -SDK offers helpers to put in place logging of HTTP Transactions: it’s built on top of **[terraform-plugin-log](https://www.terraform.io/plugin/log)**, so it allows to leverage its features, without having to write the whole implementation of `http.RoundTripper`. +The recommended logging helper for SDK is built on top of [terraform-plugin-log](https://www.terraform.io/plugin/log). This lets you leverage the features from our structured logging framework without having to write an entire implementation of `http.RoundTripper`. -There are 2 functions inside the package `helper/logging`, each targeting a specific logging setup that the provider developer might have chosen (see [“Writing Log Output”](https://www.terraform.io/plugin/log/writing) for details): +There are two functions inside `helper/logging` that target a specific logging setup for your provider. Refer to [“Writing Log Output”](https://www.terraform.io/plugin/log/writing) for details. -* `NewLoggingHTTPTransport(transport http.RoundTripper)` -* `NewSubsystemLoggingHTTPTransport(subsystem string, transport http.RoundTripper)` +* `NewLoggingHTTPTransport(transport http.RoundTripper)`: Use this method when you want logging against the `tflog` Provider root logger. +* `NewSubsystemLoggingHTTPTransport(subsystem string, transport http.RoundTripper)`: Use this method when you want logging against a `tflog` Provider [Subsystem logger](https://www.terraform.io/plugin/log/writing#subsystems). The `subsystem` string you use with `NewSubsystemLoggingHTTPTransport()` must match the [pre-created subsystem logger name](https://www.terraform.io/plugin/log/writing#create-subsystems). -Using one or the other depends on the developer, and if they want their logging to happen against the `tflog` Provider root logger, or against a `tflog` Provider [Subsystem logger](https://www.terraform.io/plugin/log/writing#subsystems). For the latter, the only thing to remember is that the `subsystem` string used with `NewSubsystemLoggingHTTPTransport()`, has to match the [pre-created subsystem logger name](https://www.terraform.io/plugin/log/writing#create-subsystems). +To set up HTTP transport, you must create the HTTP Client to use the new transport and then add logging configuration to the HTTP request context. -To make use of our logging transport, there are 2 things that a provider developer should do: -* Setting up the HTTP Client to use this new Transport -* Ensuring that each HTTP Request, before being submitted to the Client, carries the configured `context.Context`, storing the desired logging configuration - - -## Creating HTTP Client that uses the new logging transport +### Creating the HTTP Client Once the correct _Transport _has been created, it should be used when setting up the `http.Client` that the provider is going to use. For example, a good place to set up the client could be the `schema.Provider` `ConfigureContextFunc`: @@ -57,11 +52,11 @@ func New() (*schema.Provider, error) { This will set up a client that is identical to the default Golang `http.Client`, except it uses the new logging transport. -## Sending HTTP Requests with the appropriate Context +## Adding Context to HTTP Requests -All calls to `tflog` package functionality must use an SDK provided `context.Context`, which stores the logging implementation. Providers written with `terraform-plugin-sdk` must use context-aware functionality, such as the [`helper/schema.Resource` type `ReadContext` field](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema#Resource.ReadContext). +All calls to the `tflog` package must contain an SDK provided `context.Context` that stores the logging implementation. Providers written with `terraform-plugin-sdk` must use context-aware functionality, such as the [`helper/schema.Resource` type `ReadContext` field](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema#Resource.ReadContext). -Use [`http.NewRequestWithContext()` function](https://pkg.go.dev/net/http#NewRequestWithContext) to create an HTTP request that includes the logging configuration from the `context.Context`. For example: +The following example uses [`http.NewRequestWithContext()` function](https://pkg.go.dev/net/http#NewRequestWithContext) to create an HTTP request that includes the logging configuration from the `context.Context`. ```go // inside a context-aware Resource function @@ -77,12 +72,16 @@ if err != nil { defer res.Body.Close() ``` -Use the [`(http.Request).WithContext()` method](https://pkg.go.dev/net/http#Request.WithContext) to set the context for the `http.Request` if the request is generated separate from where the `context.Context` is available. +Use the [`(http.Request).WithContext()` method](https://pkg.go.dev/net/http#Request.WithContext) to set the context for the `http.Request` if the request is generated separately from where the `context.Context` is available. + +## HTTP Transaction Log Format -## Log format for an HTTP Transaction +The logging transport produces two log entries for each HTTP transaction: one for the request and one for the response. -Each HTTP Transaction processed by the new logging transport, will result in 2 log entries. Here is how the log for the HTTP Request towards [https://terraform.io](https://terraform.io) looks like: +### Request Example + +The following example shows a log generated from an HTTP Request to [https://terraform.io](https://terraform.io). ```text 2022-07-26T18:54:08.880+0100 [DEBUG] provider: Sending HTTP Request: Accept-Encoding=gzip Content-Length=0 \ @@ -91,7 +90,7 @@ Each HTTP Transaction processed by the new logging transport, will result in 2 l tf_http_req_uri=/ tf_http_req_version=HTTP/1.1 tf_http_trans_id=7e80e48d-8f32-f527-1412-52a8c84359e7 ``` -And, if [logging in JSON format](https://www.terraform.io/internals/debugging) is enabled: +The following example shows the same logs after you enable [logging in JSON format](https://www.terraform.io/internals/debugging). ```json { @@ -114,7 +113,9 @@ And, if [logging in JSON format](https://www.terraform.io/internals/debugging) i } ``` -And the corresponding HTTP Response: +### Response Example + +The following example shows logs from a [https://terraform.io](https://terraform.io) HTTP response. ```text 2022-07-26T18:54:10.734+0100 [DEBUG] provider: Received HTTP Response: Age=9 \ @@ -128,7 +129,7 @@ And the corresponding HTTP Response: tf_http_trans_id=7e80e48d-8f32-f527-1412-52a8c84359e7 ``` -And in JSON format: +the following example shows the same logs in JSON format. ```json { @@ -160,12 +161,14 @@ And in JSON format: } ``` -The log contains various parts, represented as [fields](https://www.terraform.io/plugin/log/writing#fields) in the JSON format above. It’s important to familiarize yourself with those: +### Log Information + +Each log contains the following information, which is represented as [fields](https://www.terraform.io/plugin/log/writing#fields) in the JSON format. | Log field name | Description | Possible values | Applies to | |---------------------------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|:------------------:| | `tf_http_op_type` | Which HTTP operation log refers to | [`request`, `response`] | Request / Response | -| `tf_http_trans_id` | Unique identifier used by Request and Response that belong to the same HTTP Transaction. | A universally unique identifier ([UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier)) | Request / Response | +| `tf_http_trans_id` | Unique identifier used by Request and Response that belong to the same HTTP Transaction | A universally unique identifier ([UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier)) | Request / Response | | `tf_http_req_body` | Request body | | Request | | `tf_http_req_method` | Request method | A canonical [HTTP methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Method) | Request | | `tf_http_req_uri` | Request URI | Ex. `"/path"` | Request | @@ -174,14 +177,14 @@ The log contains various parts, represented as [fields](https://www.terraform.io | `tf_http_res_status_code` | Response status code | A canonical [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) | Response | | `tf_http_res_status_reason | Response status reason | Canonical textual description of the corresponding `tf_http_res_status_code` | Response | | `tf_http_res_version` | Response HTTP version | Ex. `"HTTP/2.0"` | Response | -| (Other fields) | Request / Response headers. One field per header. If header contains a single value, field value will be set to that. Otherwise, field value will be a slice of strings. | | Request / Response | +| (Other fields) | Request / Response headers. One field per header. If the header contains a single value, the log field value is set to that value. Otherwise, the field value is a slice of strings. | | Request / Response | -## Filtering sensitive data +## Filtering Sensitive Data -To configure [log filtering](https://www.terraform.io/plugin/log/filtering) in the HTTP Transport to the provider needs, all it is required is to configure the `context.Context` before it's set in the `http.Request`. +To [filter logs](https://www.terraform.io/plugin/log/filtering), you must configure the `context.Context` before before it is added to the `http.Request`. -For example, when using `NewSubsystemLoggingHTTPTransport()` to log our HTTP Transactions, to mask all the header values of the HTTP Requests containing an `Authorization` and `Proxy-Authorization` credentials: +The following example masks all the header values of HTTP Requests containing an `Authorization` and `Proxy-Authorization` credentials. ```go // inside a context-aware Resource function diff --git a/website/docs/plugin/sdkv2/logging/index.mdx b/website/docs/plugin/sdkv2/logging/index.mdx index a66d2e0def..b5f6fe4b7c 100644 --- a/website/docs/plugin/sdkv2/logging/index.mdx +++ b/website/docs/plugin/sdkv2/logging/index.mdx @@ -1,13 +1,12 @@ --- page_title: Plugin Development - Logging description: |- - Producing high quality logs can make a huge difference when trying to debug an issue. - Learn more about how to set-up logging and write meaningful logs. + High-quality logs are important when debugging your provider. Learn to set-up logging and write meaningful logs. --- # Logging -Terraform Plugin SDKv2 integrates with a [structured logging framework](https://www.terraform.io/plugin/log): [terraform-plugin-log](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log). This plays a crucial role in building providers that facilitate timely [debugging](https://www.terraform.io/plugin/debugging) of issues. +Terraform Plugin SDKv2 integrates with the structured logging framework [terraform-plugin-log](https://www.terraform.io/plugin/log). High-quality logs are critical to quickly [debugging your provider](https://www.terraform.io/plugin/debugging). ## Managing Log Output @@ -19,8 +18,8 @@ Learn how to [implement code in provider logic](https://www.terraform.io/plugin/ ## Filtering Log Output -Learn how to [implement code in provider logic](https://www.terraform.io/plugin/log/filtering) to mask certain log messages and structured log fields or omit logs entirely. +Learn how to [implement code in provider logic](https://www.terraform.io/plugin/log/filtering) to omit logs or mask specific log messages and structured log fields. ## Log HTTP Transactions -Learn how to [set up the Logging HTTP Transport](./logging/http-transport) to log HTTP Transactions via the [structured logging framework](https://www.terraform.io/plugin/log). +Learn how to [set up the Logging HTTP Transport](./logging/http-transport) to log HTTP Transactions with the [structured logging framework](https://www.terraform.io/plugin/log). From a54afb1a1bbc9706c4bcade49db50dc0c7873da3 Mon Sep 17 00:00:00 2001 From: Ivan De Marino Date: Thu, 28 Jul 2022 18:37:23 +0100 Subject: [PATCH 4/5] Apply suggestions from code review Co-authored-by: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> --- website/docs/plugin/sdkv2/logging/http-transport.mdx | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/website/docs/plugin/sdkv2/logging/http-transport.mdx b/website/docs/plugin/sdkv2/logging/http-transport.mdx index d5ab726198..f5f47bb9ee 100644 --- a/website/docs/plugin/sdkv2/logging/http-transport.mdx +++ b/website/docs/plugin/sdkv2/logging/http-transport.mdx @@ -27,7 +27,7 @@ To set up HTTP transport, you must create the HTTP Client to use the new transpo ### Creating the HTTP Client -Once the correct _Transport _has been created, it should be used when setting up the `http.Client` that the provider is going to use. For example, a good place to set up the client could be the `schema.Provider` `ConfigureContextFunc`: +After you create the transport , you must use it to set up the `http.Client` for the provider. The following example sets up the client in `schema.Provider` `ConfigureContextFunc`. The client is identical to the default Golang `http.Client`, except it uses the new logging transport. ```go func New() (*schema.Provider, error) { @@ -47,9 +47,6 @@ func New() (*schema.Provider, error) { } } } -``` - -This will set up a client that is identical to the default Golang `http.Client`, except it uses the new logging transport. ## Adding Context to HTTP Requests From afe1761e9faf6b5614d8b64c613589319eb76d2c Mon Sep 17 00:00:00 2001 From: Ivan De Marino Date: Thu, 28 Jul 2022 18:45:19 +0100 Subject: [PATCH 5/5] Streamline the links at the end of the website doc for HTTP Transport --- .../plugin/sdkv2/logging/http-transport.mdx | 35 +++++++++---------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/website/docs/plugin/sdkv2/logging/http-transport.mdx b/website/docs/plugin/sdkv2/logging/http-transport.mdx index f5f47bb9ee..82db67eb93 100644 --- a/website/docs/plugin/sdkv2/logging/http-transport.mdx +++ b/website/docs/plugin/sdkv2/logging/http-transport.mdx @@ -162,18 +162,18 @@ the following example shows the same logs in JSON format. Each log contains the following information, which is represented as [fields](https://www.terraform.io/plugin/log/writing#fields) in the JSON format. -| Log field name | Description | Possible values | Applies to | -|---------------------------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|:------------------:| -| `tf_http_op_type` | Which HTTP operation log refers to | [`request`, `response`] | Request / Response | -| `tf_http_trans_id` | Unique identifier used by Request and Response that belong to the same HTTP Transaction | A universally unique identifier ([UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier)) | Request / Response | -| `tf_http_req_body` | Request body | | Request | -| `tf_http_req_method` | Request method | A canonical [HTTP methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Method) | Request | -| `tf_http_req_uri` | Request URI | Ex. `"/path"` | Request | -| `tf_http_req_version` | Request HTTP version | Ex. `"HTTP/1.1"` | Request | -| `tf_http_res_body` | Response body | | Response | -| `tf_http_res_status_code` | Response status code | A canonical [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) | Response | -| `tf_http_res_status_reason | Response status reason | Canonical textual description of the corresponding `tf_http_res_status_code` | Response | -| `tf_http_res_version` | Response HTTP version | Ex. `"HTTP/2.0"` | Response | +| Log field name | Description | Possible values | Applies to | +|---------------------------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|:------------------:| +| `tf_http_op_type` | Which HTTP operation log refers to | [`request`, `response`] | Request / Response | +| `tf_http_trans_id` | Unique identifier used by Request and Response that belong to the same HTTP Transaction | A universally unique identifier ([UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier)) | Request / Response | +| `tf_http_req_body` | Request body | | Request | +| `tf_http_req_method` | Request method | A canonical [HTTP methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Method) | Request | +| `tf_http_req_uri` | Request URI | Ex. `"/path"` | Request | +| `tf_http_req_version` | Request HTTP version | Ex. `"HTTP/1.1"` | Request | +| `tf_http_res_body` | Response body | | Response | +| `tf_http_res_status_code` | Response status code | A canonical [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) | Response | +| `tf_http_res_status_reason | Response status reason | Canonical textual description of the corresponding `tf_http_res_status_code` | Response | +| `tf_http_res_version` | Response HTTP version | Ex. `"HTTP/2.0"` | Response | | (Other fields) | Request / Response headers. One field per header. If the header contains a single value, the log field value is set to that value. Otherwise, the field value is a slice of strings. | | Request / Response | @@ -201,11 +201,8 @@ defer res.Body.Close() ``` -# Resources +# Links -* [Plugin Development](https://www.terraform.io/plugin) -* [Plugin Development - Logging](https://www.terraform.io/plugin/log) -* [terraform-plugin-log - tflog](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log/tflog) package documentation -* [terraform-plugin-log](https://github.com/hashicorp/terraform-plugin-log) GitHub repository -* [terraform-plugin-sdk](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2) [`NewLoggingHTTPTransport()`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging#NewLoggingHTTPTransport) and [`NewSubsystemLoggingHTTPTransport()`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging#NewSubsystemLoggingHTTPTransport) -* [New Logging Transport](https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/logging/logging_http_transport.go) source code +* [Plugin Development - Logging](https://www.terraform.io/plugin/log) - Learn more about the logging framework +* [terraform-plugin-log - tflog](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log/tflog) - Read the Golang documentation for the logging framework +* Read the Golang documentation for [`NewLoggingHTTPTransport()`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging#NewLoggingHTTPTransport) and [`NewSubsystemLoggingHTTPTransport()`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging#NewSubsystemLoggingHTTPTransport)