Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

zapcore: Add Buffered Writer #961

Merged
merged 74 commits into from Jun 8, 2021
Merged

zapcore: Add Buffered Writer #961

merged 74 commits into from Jun 8, 2021
+408 −4

Commits on May 25, 2021

  1. add buffer sync

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    c1e68d7 View commit details
    Browse the repository at this point in the history

  2. support config bufferSize and flushInterval, improve logic

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    df42521 View commit details
    Browse the repository at this point in the history

  3. improve

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    7f5e097 View commit details
    Browse the repository at this point in the history

  4. update comment

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    7eec047 View commit details
    Browse the repository at this point in the history

  5. WriterSyncer support Close method

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    3c8ec7b View commit details
    Browse the repository at this point in the history

  6. improve

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    046355f View commit details
    Browse the repository at this point in the history

  7. fix spell

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    afe872f View commit details
    Browse the repository at this point in the history

  8. improve cancel logic

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    cce50b1 View commit details
    Browse the repository at this point in the history

  9. improve

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    ef808a2 View commit details
    Browse the repository at this point in the history

  10. remove close

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    3af046a View commit details
    Browse the repository at this point in the history

  11. rename cancel to close and keep syncing

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    e0a440d View commit details
    Browse the repository at this point in the history

  12. fix lint

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    e571e25 View commit details
    Browse the repository at this point in the history

  13. 100% test coverage

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    4a25c9f View commit details
    Browse the repository at this point in the history

  14. improve comment

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    99f4ea8 View commit details
    Browse the repository at this point in the history

  15. improve comment

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    223dd97 View commit details
    Browse the repository at this point in the history

  16. fix test error

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    dd3698e View commit details
    Browse the repository at this point in the history

  17. fix race condition in test case

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    f3079de View commit details
    Browse the repository at this point in the history

  18. 100% test coverage

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    2c09dc4 View commit details
    Browse the repository at this point in the history

  19. add loop and fix typo

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    f0f2a28 View commit details
    Browse the repository at this point in the history

  20. Update zapcore/write_syncer.go 

    Co-authored-by: Prashant Varanasi <github@prashantv.com>
    @sysulq @prashantv @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    fb6efc3 View commit details
    Browse the repository at this point in the history

  21. improve test

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    0977e02 View commit details
    Browse the repository at this point in the history

  22. validate loop logic in test case

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    65775e1 View commit details
    Browse the repository at this point in the history

  23. group default config

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    470b7fa View commit details
    Browse the repository at this point in the history

  24. improve close logic

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    160e84d View commit details
    Browse the repository at this point in the history

  25. Update zapcore/write_syncer.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @sysulq @abhinav @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    ee04403 View commit details
    Browse the repository at this point in the history

  26. Update zapcore/write_syncer_test.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @sysulq @abhinav @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    80433c2 View commit details
    Browse the repository at this point in the history

  27. Update zapcore/write_syncer_test.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @sysulq @abhinav @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    9bda819 View commit details
    Browse the repository at this point in the history

  28. Update zapcore/write_syncer_test.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @sysulq @abhinav @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    dc370da View commit details
    Browse the repository at this point in the history

  29. Update zapcore/write_syncer_test.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @sysulq @abhinav @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    2ecf1f8 View commit details
    Browse the repository at this point in the history

  30. Update zapcore/write_syncer_test.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @sysulq @abhinav @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    a522baf View commit details
    Browse the repository at this point in the history

  31. Update zapcore/write_syncer_test.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @sysulq @abhinav @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    4273b8e View commit details
    Browse the repository at this point in the history

  32. improve close logic and use goleak

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    d884534 View commit details
    Browse the repository at this point in the history

  33. Drop type CloseFunc 

    We'd like to avoid introducing a new top-level type if we can. Delete
the CloseFunc type in favor of returning a naked `func() error`.
    @abhinav @moisesvega
    abhinav authored and moisesvega committed May 25, 2021
    Copy the full SHA
    0392f30 View commit details
    Browse the repository at this point in the history

  34. doc: Rewrite Buffer documentation

    @abhinav @moisesvega
    abhinav authored and moisesvega committed May 25, 2021
    Copy the full SHA
    ddace4d View commit details
    Browse the repository at this point in the history

  35. Prefix default{BufferSize, FlushInterval} with _ 

    Per https://github.com/uber-go/guide/blob/master/style.md#prefix-unexported-globals-with-_
    @abhinav @moisesvega
    abhinav authored and moisesvega committed May 25, 2021
    Copy the full SHA
    e96c0ee View commit details
    Browse the repository at this point in the history

  36. Buffer/close: return a bound method 

    Instead of returning an anonmyous function, return a bound method. The
bound method is named `close`, not `Close` to avoid accidentally
exposing an `io.Closer` interface.
    @abhinav @moisesvega
    abhinav authored and moisesvega committed May 25, 2021
    Copy the full SHA
    570d2ff View commit details
    Browse the repository at this point in the history

  37. Buffer/close: Close the channel instead of posting 

    For the channel that signals end of the goroutine, we should close it
instead of posting a single value to it.
    @abhinav @moisesvega
    abhinav authored and moisesvega committed May 25, 2021
    Copy the full SHA
    d1e244a View commit details
    Browse the repository at this point in the history

  38. _default{BufferSize, FlushInterval}: docs

    @abhinav @moisesvega
    abhinav authored and moisesvega committed May 25, 2021
    Copy the full SHA
    83f6331 View commit details
    Browse the repository at this point in the history

  39. buffer: Move loop into a method 

    Move the flush loop into its own method instead of spawning it in a
closure.
    @abhinav @moisesvega
    abhinav authored and moisesvega committed May 25, 2021
    Copy the full SHA
    ba0b65e View commit details
    Browse the repository at this point in the history

  40. buffer/close: stop the ticker 

    On close, stop the ticker from posting ticks to the channel.
    @abhinav @moisesvega
    abhinav authored and moisesvega committed May 25, 2021
    Copy the full SHA
    b8a0b28 View commit details
    Browse the repository at this point in the history

  41. Update zapcore/write_syncer_test.go 

    Co-authored-by: Prashant Varanasi <github@prashantv.com>
    @sysulq @prashantv @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    1c5daa9 View commit details
    Browse the repository at this point in the history

  42. improve code style

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    f74c558 View commit details
    Browse the repository at this point in the history

  43. Update zapcore/write_syncer.go 

    Co-authored-by: Prashant Varanasi <github@prashantv.com>
    @sysulq @prashantv @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    6f88240 View commit details
    Browse the repository at this point in the history

  44. Update zapcore/write_syncer.go 

    Co-authored-by: Prashant Varanasi <github@prashantv.com>
    @sysulq @prashantv @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    963142b View commit details
    Browse the repository at this point in the history

  45. Update zapcore/write_syncer.go 

    Co-authored-by: Prashant Varanasi <github@prashantv.com>
    @sysulq @prashantv @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    f753926 View commit details
    Browse the repository at this point in the history

  46. add comment

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    1db4692 View commit details
    Browse the repository at this point in the history

  47. Update zapcore/write_syncer.go 

    Co-authored-by: Prashant Varanasi <github@prashantv.com>
    @sysulq @prashantv @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    545d164 View commit details
    Browse the repository at this point in the history

  48. Update zapcore/write_syncer.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @sysulq @abhinav @moisesvega
    2 people authored and moisesvega committed May 25, 2021
    Copy the full SHA
    c777047 View commit details
    Browse the repository at this point in the history

  49. update

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    1b0fcfd View commit details
    Browse the repository at this point in the history

  50. Remove Lock/Unlock from tests

    @prashantv @moisesvega
    prashantv authored and moisesvega committed May 25, 2021
    Copy the full SHA
    02c9d3f View commit details
    Browse the repository at this point in the history

  51. remove double buffer check

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    4804188 View commit details
    Browse the repository at this point in the history

  52. set timer to zero

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    4ca46e9 View commit details
    Browse the repository at this point in the history

  53. remove errorWriter

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    b1a95ff View commit details
    Browse the repository at this point in the history

  54. add SyncBuffer

    @sysulq @moisesvega
    sysulq authored and moisesvega committed May 25, 2021
    Copy the full SHA
    6d58a7d View commit details
    Browse the repository at this point in the history

  55. Add buffered write syncer

    @moisesvega
    moisesvega committed May 25, 2021
    Copy the full SHA
    2312b39 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2021

  1. Set default Clock zapcore/buffered_write_syncer.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @moisesvega @abhinav
    moisesvega and abhinav committed Jun 1, 2021
    Copy the full SHA
    7c32a14 View commit details
    Browse the repository at this point in the history

  2. Use require instead assert zapcore/buffered_write_syncer_bench_test.go 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @moisesvega @abhinav
    moisesvega and abhinav committed Jun 1, 2021
    Copy the full SHA
    ffba68b View commit details
    Browse the repository at this point in the history

  3. Update Close() comment, use filepath to create tmpDir and mutate Cloc… 

    …k on loadConfig if needed
    @moisesvega
    moisesvega committed Jun 1, 2021
    Copy the full SHA
    a3fbb8e View commit details
    Browse the repository at this point in the history

  4. Use ioutil for creating tmp file

    @moisesvega
    moisesvega committed Jun 1, 2021
    Copy the full SHA
    39f7316 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2021

  1. Sync underlying writeSyncer

    @moisesvega
    moisesvega committed Jun 4, 2021
    Copy the full SHA
    10b45db View commit details
    Browse the repository at this point in the history

  2. Update default documentation and don't double lock

    @moisesvega
    moisesvega committed Jun 4, 2021
    Copy the full SHA
    0962b0f View commit details
    Browse the repository at this point in the history

  3. Add test case with lockedWriteSyncer

    @moisesvega
    moisesvega committed Jun 4, 2021
    Copy the full SHA
    7dedbc0 View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2021

  1. Use empty assert.Empty 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @moisesvega @abhinav
    moisesvega and abhinav committed Jun 8, 2021
    Copy the full SHA
    f8f3a3d View commit details
    Browse the repository at this point in the history

  2. Use zapcore.Lock for BufferedWriteSyncer tests 

    Co-authored-by: Abhinav Gupta <mail@abhinavg.net>
    @moisesvega @abhinav
    moisesvega and abhinav committed Jun 8, 2021
    Copy the full SHA
    acfd294 View commit details
    Browse the repository at this point in the history

  3. Add asserts on close/remove in benchmark buffered write and add test … 

    …case for wrap twice sync
    @moisesvega
    moisesvega committed Jun 8, 2021
    Copy the full SHA
    f32b796 View commit details
    Browse the repository at this point in the history

  4. Take a pass over the BufferedWriteSyncer docs

    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    17cdba5 View commit details
    Browse the repository at this point in the history

  5. Move consts to the top of the file

    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    97ae2a6 View commit details
    Browse the repository at this point in the history

  6. Rename Close to Stop

    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    b58819e View commit details
    Browse the repository at this point in the history

  7. Stop: Wait until flushLoop stops

    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    afe558a View commit details
    Browse the repository at this point in the history

  8. fixup! Rename Close to Stop

    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    49444fc View commit details
    Browse the repository at this point in the history

  9. Stop/Sync: handle uninitialized calls 

    These methods should not panic if called without `Write` first (which is
what starts the background goroutine).
    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    43d9b54 View commit details
    Browse the repository at this point in the history

  10. Rename WriteSyncer field to WS 

    Writing `BufferedWriteSyncer{WriteSyncer: ...}` is a mouthful. Let's
go for something short and obvious like:

```
package io

type LimitedReader struct {
    R Reader // underlying reader
    N int64  // max bytes remaining
}
```
    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    7c7fb4d View commit details
    Browse the repository at this point in the history

  11. Overwrite WS instead of maintaining to WriteSyncer fields

    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    f30d68b View commit details
    Browse the repository at this point in the history

  12. Drop Lock unpacking, check initialized, fix docs

    @abhinav
    abhinav committed Jun 8, 2021
    Copy the full SHA
    ac5b8ad View commit details
    Browse the repository at this point in the history