Allow to configure allowed levels by string value #22

mcosta74 · 2022-03-27T11:02:23Z

This PR allows to configure the allowed log levels from a string.

Closes: #18

Use Case:

Read the allowed log levels reading environment variable

logger = level.NewFilter(logger, level.AllowByString(os.Getenv("LOG_LEVEL"), level.AllowInfo)) // <--

ChrisHines · 2022-03-27T16:11:52Z

@mcosta74 Thanks for the well written PR. You added docs, examples, and all the nice touches. Much appreciated.

I'm still not sure if we want to add this API, but if we do I think your approach takes the right approach. So, before I add any comments on the details I want to poll the other maintainers for their opinion.

@peterbourgon, @sagikazarmark what do you think of adding this option?

sagikazarmark · 2022-03-27T16:40:03Z

I'm not 100% sure why a fallback option is necessary. As far as I can tell, it's equivalent to the following:

logger = level.NewFilter(logger, level.AllowInfo, level.AllowByString(os.Getenv("LOG_LEVEL")))

If the string doesn't match any of the log levels, it should just be noop.

Personally, I'm not a huge fan of fallbacks in this case though. Consider the following: let's say you misconfigure the production instance and as a result debug logs will start flowing in, potentially containing PII without you even noticing.

I usually log a warning or an error whenever the log config is invalid (which is also why I manually parse log levels and match them to the appropriate log levels).

The name AllowByString feels a bit weird. I'd go with something like Parse.

Other than these, I don't mind adding a few utility functions if they really help.

mcosta74 · 2022-03-27T16:57:07Z

If the string doesn't match any of the log levels, it should just be noop.

Personally, I'm not a huge fan of fallbacks in this case though. Consider the following: let's say you misconfigure the production instance and as a result debug logs will start flowing in, potentially containing PII without you even noticing.

@sagikazarmark Agree, the fallback mechanism may obfuscate a misconfiguration.

Should I apply the changes or better to wait a final decision about adding or not that function ?

The name AllowByString feels a bit weird. I'd go with something like Parse.

I'm open to change the name if the approach is accepted

mcosta74 · 2022-03-28T07:46:34Z

I've applied the changes suggested by @sagikazarmark.
Let me know about the final decision.

level/doc.go

level/level.go

mcosta74 · 2022-04-05T06:05:59Z

hello @peterbourgon, I've applied all changes requested.

Proper grammar and spelling, please :) Let me know if you'd like suggestions.

Please let me know if you find better wording for the doc strings. I'm open to change.

peterbourgon

This isn't totally complete but I hope my intent is clear :)

level/doc.go

level/level.go

peterbourgon · 2022-04-05T12:55:07Z

level/example_test.go

+func Example_parse() {
+	// Set up logger with level filter.
+	allow, err := level.ParseOption("info")
+	if err != nil {
+		// fallback to default value in case parse failure
+		allow = level.AllowInfo()
+	}
+	logger := log.NewLogfmtLogger(os.Stdout)
+	logger = level.NewFilter(logger, allow)
+	logger = log.With(logger, "caller", log.DefaultCaller)
+
+	// Use level helpers to log at different levels.
+	level.Error(logger).Log("err", errors.New("bad data"))
+	level.Info(logger).Log("event", "data saved")
+	level.Debug(logger).Log("next item", 17) // filtered
+
+	// Output:
+	// level=error caller=example_test.go:53 err="bad data"
+	// level=info caller=example_test.go:54 event="data saved"
+}


Suggested change

func Example_parse() {

// Set up logger with level filter.

allow, err := level.ParseOption("info")

if err != nil {

// fallback to default value in case parse failure

allow = level.AllowInfo()

}

logger := log.NewLogfmtLogger(os.Stdout)

logger = level.NewFilter(logger, allow)

logger = log.With(logger, "caller", log.DefaultCaller)

// Use level helpers to log at different levels.

level.Error(logger).Log("err", errors.New("bad data"))

level.Info(logger).Log("event", "data saved")

level.Debug(logger).Log("next item", 17) // filtered

// Output:

// level=error caller=example_test.go:53 err="bad data"

// level=info caller=example_test.go:54 event="data saved"

}

func Example_parse() {

fs := flag.NewFlagSet("myprogram", flag.ExitOnError)

lv := fs.String("log", "info", "debug, info, warn, or error")

fs.Parse([]string{"-log", "warn"})

logger := log.NewLogfmtLogger(os.Stdout)

logger = level.NewFilter(logger, level.MustParseLevel(*lv))

level.Error(logger).Log("msg", "alpha")

level.Warn(logger).Log("msg", "beta")

level.Info(logger).Log("msg", "delta")

level.Debug(logger).Log("msg", "kappa")

// Output:

// level=error msg="alpha"

// level=warn msg="beta"

}

mcosta74 · 2022-04-09T16:48:42Z

Looks like we have an agreement. Let me implement these API ============== Massimo Costa Il ven 8 apr 2022, 20:16 Chris Hines ***@***.***> ha scritto:

…

***@***.**** commented on this pull request. ------------------------------ In level/level.go <#22 (comment)>: > +// ParseOption creates an Option from a string value. It might be useful in case we need to configure +// the log level using a flag, an environment variable, a configuration file, ... +// +// Leading and trailing spaces are ignored and the comparison is Case Insensitive: +// "ERROR", "Error", " Error " are accepted +// +// If the level parameter does not match any of the available values, the function returns a not nil error. +func ParseOption(level string) (Option, error) { + level = strings.TrimSpace(strings.ToLower(level)) + + switch level { + case debugValue.name: + return AllowDebug(), nil + + case infoValue.name: + return AllowInfo(), nil + + case warnValue.name: + return AllowWarn(), nil + + case errorValue.name: + return AllowError(), nil + + default: + // return a noop Option in case of unknown level string + return nil, fmt.Errorf("invalid level: %s", level) + } +} 🎉 Great. 🎉 I'm on board too. — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4I7OKBLXWIID3Q36AU3CDVEBZYVANCNFSM5RYT3RZA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

coveralls · 2022-04-11T02:08:53Z

Pull Request Test Coverage Report for Build 2145714702

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

25 of 25 (100.0%) changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+2.5%) to 74.597%

Files with Coverage Reduction	New Missed Lines	%
level/level.go	1	99.11%

Totals
Change from base Build 2065105133:	2.5%
Covered Lines:	555
Relevant Lines:	744

💛 - Coveralls

mcosta74 · 2022-04-11T12:48:25Z

I think I implemented the agreed API, if the code is code we can also squash the commits

ChrisHines · 2022-04-11T14:49:10Z

I think I implemented the agreed API, if the code is code we can also squash the commits

Thanks, @mcosta74. I will review the code sometime in the next few days. Also, thanks for you patience during the API debate.

mcosta74 · 2022-04-11T20:03:08Z

I think I implemented the agreed API, if the code is code we can also squash the commits

Thanks, @mcosta74. I will review the code sometime in the next few days. Also, thanks for you patience during the API debate.

no problem at all, I think it was a very interesting and productive discussion. I would only suggest to create a CONTRIBUTION page (maybe with a discussion template) to clarify what are the principles to follow in case someone else wants to propose new API.

peterbourgon

A few things.

peterbourgon · 2022-04-12T19:24:55Z

level/level.go

@@ -66,6 +75,26 @@ func (l *logger) Log(keyvals ...interface{}) error {
 // Option sets a parameter for the leveled logger.
 type Option func(*logger)

+// Allow allows to configure the log level with a Value.


Suggested change

// Allow allows to configure the log level with a Value.

// Allow the provided log level to pass.

peterbourgon · 2022-04-12T19:27:26Z

level/level_test.go

@@ -12,11 +12,51 @@ import (
 )

 func TestVariousLevels(t *testing.T) {
+


Suggested change

level/level_test.go

level/level.go

peterbourgon · 2022-04-12T23:27:52Z

level/level.go

+// ParseDefault returns a Value from a level string or the default if the level is not valid.
+func ParseDefault(level string, def Value) Value {
+	if v, err := Parse(level); err != nil {
+		return def
+	} else {
+		return v
+	}
+}


Suggested change

// ParseDefault returns a Value from a level string or the default if the level is not valid.

func ParseDefault(level string, def Value) Value {

if v, err := Parse(level); err != nil {

return def

} else {

return v

}

}

// ParseDefault calls Parse, and returns the default Value on error.

func ParseDefault(level string, def Value) Value {

if v, err := Parse(level); err == nil {

return v

}

return def

}

peterbourgon · 2022-04-12T23:41:56Z

level/level.go

+// Parse returns a Value from a level string. Allowed values are "debug", "info", "warn" and "error".
+// Levels are normalized via strings.TrimSpace and strings.ToLower
+func Parse(level string) (Value, error) {


Suggested change

// Parse returns a Value from a level string. Allowed values are "debug", "info", "warn" and "error".

// Levels are normalized via strings.TrimSpace and strings.ToLower

func Parse(level string) (Value, error) {

// Parse a string to it's corresponding level value. Valid strings are "debug",

// "info", "warn", and "error". Strings are normalized via strings.TrimSpace and

// strings.ToLower.

func Parse(level string) (Value, error) {

peterbourgon · 2022-04-12T23:50:10Z

level/doc.go

+// It's also possible to configure log level from a string (for instance from
+// a flag, environment variable or configuration file).
+//
+//    fs := flag.NewFlagSet("example")
+//    lvl := fs.String("log-level", "", `"debug", "info", "warn" or "error"`)
+//
+//    var logger log.Logger
+//    logger = log.NewLogfmtLogger(os.Stderr)
+//    logger = level.NewFilter(logger, level.Allow(level.ParseDefault(*lvl, level.InfoValue()))) // <--
+//    logger = log.With(logger, "ts", log.DefaultTimestampUTC)
+//


Suggested change

// It's also possible to configure log level from a string (for instance from

// a flag, environment variable or configuration file).

//

// fs := flag.NewFlagSet("example")

// lvl := fs.String("log-level", "", `"debug", "info", "warn" or "error"`)

//

// var logger log.Logger

// logger = log.NewLogfmtLogger(os.Stderr)

// logger = level.NewFilter(logger, level.Allow(level.ParseDefault(*lvl, level.InfoValue()))) // <--

// logger = log.With(logger, "ts", log.DefaultTimestampUTC)

//

// It's also possible to configure log level from a string. For instance, from

// a flag, environment variable or configuration file.

//

// fs := flag.NewFlagSet("myprogram")

// lvl := fs.String("log", "info", "debug, info, warn, error")

//

// var logger log.Logger

// logger = log.NewLogfmtLogger(os.Stderr)

// logger = level.NewFilter(logger, level.Allow(level.ParseDefault(*lvl, level.InfoValue()))) // <--

// logger = log.With(logger, "ts", log.DefaultTimestampUTC)

//

ChrisHines

This is a nice addition to the package. Thanks for the contribution. I have a few nits, mostly style tweaks.

level/level.go

level/level_test.go

sagikazarmark · 2022-04-19T02:44:16Z

level/level.go

+// Parse a string to its corresponding level value. Valid strings are "debug",
+// "info", "warn", and "error". Strings are normalized via strings.TrimSpace and
+// strings.ToLower.
+func Parse(level string) (Value, error) {


If the only returned error is invalid level, then maybe the second argument could simply be a boolran?

isn't more idiomatic to return an error? maybe in the future we might return different types of errors and we'll not need to introduce breaking changes in the API

I agree, error is a bit more future proof and IMO not meaningfully different for callers.

ChrisHines

LGTM. 🎉

ChrisHines · 2022-04-20T16:22:43Z

@peterbourgon I think we're just waiting for you to sign off on this since you previously requested changes.

sagikazarmark

LGTM

level/level.go

Co-authored-by: Márk Sági-Kazár <sagikazarmark@users.noreply.github.com>

mcosta74 · 2022-04-26T06:46:54Z

Any update on this ?

ChrisHines · 2022-04-27T00:04:40Z

@peterbourgon Everyone involved has approved this PR except you. I think @mcosta74 has addressed all of your feedback. Please verify and approve if you agree. Thanks.

peterbourgon · 2022-04-27T07:24:09Z

Oops, sorry! I didn't know I was blocking.

ChrisHines · 2022-04-27T13:56:08Z

Thanks for the contribution @mcosta74 🎉 .

mcosta74 · 2022-04-27T14:00:08Z

Thanks for the contribution @mcosta74 🎉 .

you're welcome

mcosta74 · 2022-04-29T06:59:57Z

Hello, are you planning to release a new version of the package ?

peterbourgon · 2022-04-29T08:28:35Z

Sure, we can do that — v0.2.1 or v0.3.0? I don't think we made any breaking changes, so my instinct is v0.2.1?

ChrisHines · 2022-04-29T12:29:28Z

My vote is v0.2.1 as well. No breaking changes, just a new feature.

peterbourgon · 2022-04-29T13:12:58Z

Great. Assuming no objections, I'll tag a release in a day or so.

mcosta74 · 2022-04-29T14:35:27Z

not in my hands to make decisions but, since we added a feature following the SemVer guidelines I'd go for a 0.3.0. I'll keep the last number for bug-fixes

ChrisHines · 2022-04-29T14:46:50Z

@mcosta74 That is a perfectly correct reading of the SemVer guidelines. But my interpretation is that only strictly applies for major version >= 1.

From semver.org:

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

Version 1.0.0 defines the public API. The way in which the version number is incremented after this release is dependent on this public API and how it changes.

So I think it is also correct for us to do whatever we want. I don't want to suggest we should start making lots of breaking changes. I consider this module highly stable, and the API of the core log package extremely stable. We haven't made any breaking changes that I recall in ~5 years and I don't want to start doing so.

But if we stay on major version 0 per #24 (comment), then I think it is fair and appropriate to adapt the y semver number for "biggish" changes and the z number for "smallish" changes. I think it is more about signaling to consumers how much they should pay attention before upgrading.

mcosta74 · 2022-04-29T15:14:03Z

@ChrisHines Thanks for the explanation. Whit this in mind I agree that 0.2.1 is the right choice

mcosta74 · 2022-05-14T10:26:12Z

Hi @peterbourgon any ETA about the release? I'm using the "unreleased" version in a project we're building but I'd like to use a "released" one

peterbourgon · 2022-05-14T16:43:19Z

My apologies for the delay. v0.2.1 is now released. Please let me know if there are any issues.

mcosta74 · 2022-05-14T17:16:05Z

@peterbourgon, no problem. Thank you again for the products you give to the community.

peterbourgon requested changes Mar 28, 2022

View reviewed changes

level/doc.go Outdated Show resolved Hide resolved

level/level.go Outdated Show resolved Hide resolved

level/level.go Outdated Show resolved Hide resolved

peterbourgon requested changes Apr 5, 2022

View reviewed changes

peterbourgon requested changes Apr 13, 2022

View reviewed changes

mcosta74 added 4 commits April 13, 2022 08:45

feat: allow to configure level from string

02c0ef4

fix staticcheck violation

ad6e4ef

fix UTs

197b881

fix: apply suggested changes

9a046be

ChrisHines requested changes Apr 17, 2022

View reviewed changes

fix: apply some of the suggested changes

3ce8b18

sagikazarmark reviewed Apr 19, 2022

View reviewed changes

fix: more idiomatic code

054363c

ChrisHines approved these changes Apr 19, 2022

View reviewed changes

sagikazarmark approved these changes Apr 20, 2022

View reviewed changes

level/level.go Outdated Show resolved Hide resolved

Update level/level.go

fbee0cc

Co-authored-by: Márk Sági-Kazár <sagikazarmark@users.noreply.github.com>

peterbourgon approved these changes Apr 27, 2022

View reviewed changes

ChrisHines merged commit 0b69c70 into go-kit:main Apr 27, 2022

mcosta74 deleted the feat/configure-level-with-string branch April 27, 2022 13:59

	// Allow allows to configure the log level with a Value.
	// Allow the provided log level to pass.

		@@ -12,11 +12,51 @@ import (
		)

		func TestVariousLevels(t *testing.T) {

Allow to configure allowed levels by string value #22

Allow to configure allowed levels by string value #22

Conversation

mcosta74 commented Mar 27, 2022 • edited

ChrisHines commented Mar 27, 2022

sagikazarmark commented Mar 27, 2022

mcosta74 commented Mar 27, 2022 • edited

mcosta74 commented Mar 28, 2022

mcosta74 commented Apr 5, 2022

peterbourgon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcosta74 commented Apr 9, 2022 via email

coveralls commented Apr 11, 2022 • edited

Pull Request Test Coverage Report for Build 2145714702

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

mcosta74 commented Apr 11, 2022

ChrisHines commented Apr 11, 2022

mcosta74 commented Apr 11, 2022

peterbourgon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChrisHines left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChrisHines left a comment

Choose a reason for hiding this comment

ChrisHines commented Apr 20, 2022

sagikazarmark left a comment

Choose a reason for hiding this comment

mcosta74 commented Apr 26, 2022

ChrisHines commented Apr 27, 2022

peterbourgon commented Apr 27, 2022

ChrisHines commented Apr 27, 2022

mcosta74 commented Apr 27, 2022

mcosta74 commented Apr 29, 2022

peterbourgon commented Apr 29, 2022

ChrisHines commented Apr 29, 2022

peterbourgon commented Apr 29, 2022

mcosta74 commented Apr 29, 2022

ChrisHines commented Apr 29, 2022

mcosta74 commented Apr 29, 2022

mcosta74 commented May 14, 2022

peterbourgon commented May 14, 2022

mcosta74 commented May 14, 2022

mcosta74 commented Mar 27, 2022 •

edited

mcosta74 commented Mar 27, 2022 •

edited

coveralls commented Apr 11, 2022 •

edited