Why does `gbytes.Buffer.Read` return an error when the buffer is closed? #573

davidhsingyuchen · 2022-08-24T06:33:34Z

I'm trying to pass session.Wait().Out to bufio.NewScanner to parse the stdout and use the result in subsequent commands, but I guess that Wait() closes the buffer (i.e., Out), hence the issue.

Currently I'm using Out.Contents() instead, but that requires some workaround, so I decide to ask here at the same time.

I'm aware that this behavior is documented, but I fail to figure out the rationale behind.

The documentation of gbytes.Buffer.Close only mentions:

Close signifies that the buffer will no longer be written to

, and it seems that reading from a closed buffer is legit, just like reading from a closed channel.

Please let me know if I'm missing anything, thanks a lot!

The text was updated successfully, but these errors were encountered:

onsi · 2022-08-24T16:34:49Z

I wrote this so long ago and also can't understand why it goes out of its way to deny reads on a closed buffer. It seems like a bug to me and/or a misunderstanding on my part of legit buffer semantics when I wrote it in my youth ;)

Would you be up for a PR to remove the no-read-on closed behavior? There's enough test coverage that things will break if there was an actual reason... so you might learn that in the process ;)

davidhsingyuchen · 2022-08-24T18:23:40Z

Sounds great. Will work on it. Thanks a lot for the prompt reply!

davidhsingyuchen · 2022-08-25T05:46:37Z

Created #575 accordingly.

Running ginkgo -r -p in the gbytes directory succeeded, but running the same command in the repository root failed. As a result, I tried to run the command on the fresh master branch, and there are still some errors. I'm not sure if there are some existing Github issues around this, so I'll just paste the command output below in case you need more information. For what it's worth, here is my setup information:

Go version: go1.19 windows/amd64
OS: Windows 10 with version 10.0.19043 N/A Build 19043 (obtained by systeminfo)

+ [FAILED] [1.016 seconds]
HaveLeaked wrapped around test nodes when leaking [AfterEach] leaks in test
D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:225

  Timed out after 1.015s.
  Expected not to leak 1 goroutines:
      goroutine 20 [syscall, locked to thread]
          syscall.SyscallN(0x1000?, {0xc000375d20?, 0x8000?, 0x13de786ffff?}) at runtime/syscall_windows.go:557
          syscall.Syscall6(0x13de1eed010?, 0x8000?, 0x4?, 0xc000375dd0?, 0xbf8753?, 0x0?, 0x0?, 0x0?) at runtime/syscall_windows.go:501
          syscall.readFile(0xbf808b?, {0xc0000cc000?, 0x8000, 0x800000?}, 0x7ffff800000?, 0x2?) at syscall/zsyscall_windows.go:1024
          syscall.ReadFile(...) at syscall/syscall_windows.go:400
          syscall.Read(0xc0000c2500?, {0xc0000cc000?, 0x0?, 0x1?}) at syscall/syscall_windows.go:379
          internal/poll.(*FD).Read(0xc0000c2500, {0xc0000cc000, 0x8000, 0x8000}) at poll/fd_windows.go:427
          os.(*File).read(...) at os/file_posix.go:31
          os.(*File).Read(0xc00008e060, {0xc0000cc000?, 0xc000098108?, 0x0?}) at os/file.go:118
          io.copyBuffer({0x11c4360, 0xc000098108}, {0x11c46a0, 0xc00008e060}, {0x0, 0x0, 0x0}) at io/io.go:427
          io.Copy(...) at io/io.go:386
          github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1.1() at internal/output_interceptor.go:159
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1 at internal/output_interceptor.go:158
  In [AfterEach] at: D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:222
------------------------------
+ [FAILED] [1.016 seconds]
HaveLeaked failure messages [DeferCleanup] when things go wrong handles filter matcher errors
D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:180

  Timed out after 1.016s.
  Expected not to leak 1 goroutines:
      goroutine 50 [syscall, locked to thread]
          syscall.SyscallN(0x1000?, {0xc00021bd20?, 0x8000?, 0x2e33e9dffff?}) at runtime/syscall_windows.go:557
          syscall.Syscall6(0x2e33ecf0ee8?, 0x8000?, 0x4?, 0xc00021bdd0?, 0xbf8753?, 0x0?, 0x0?, 0x0?) at runtime/syscall_windows.go:501
          syscall.readFile(0xbf808b?, {0xc00021e000?, 0x8000, 0x800000?}, 0x7ffff800000?, 0x2?) at syscall/zsyscall_windows.go:1024
          syscall.ReadFile(...) at syscall/syscall_windows.go:400
          syscall.Read(0xc000388500?, {0xc00021e000?, 0x0?, 0x0?}) at syscall/syscall_windows.go:379
          internal/poll.(*FD).Read(0xc000388500, {0xc00021e000, 0x8000, 0x8000}) at poll/fd_windows.go:427
          os.(*File).read(...) at os/file_posix.go:31
          os.(*File).Read(0xc00000a5e0, {0xc00021e000?, 0xc000210000?, 0x0?}) at os/file.go:118
          io.copyBuffer({0x11c4360, 0xc000210000}, {0x11c46a0, 0xc00000a5e0}, {0x0, 0x0, 0x0}) at io/io.go:427
          io.Copy(...) at io/io.go:386
          github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1.1() at internal/output_interceptor.go:159
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1 at internal/output_interceptor.go:158
  In [DeferCleanup] at: D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:132
------------------------------
+ [FAILED] [1.018 seconds]
HaveLeaked failure messages [DeferCleanup] when things go wrong expects actual to be a slice of Goroutine
D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:170

  Timed out after 1.017s.
  Expected not to leak 1 goroutines:
      goroutine 7 [syscall, locked to thread]
          syscall.SyscallN(0x1000?, {0xc000071d20?, 0x8000?, 0x121d042ffff?}) at runtime/syscall_windows.go:557
          syscall.Syscall6(0x121f6162a00?, 0x8000?, 0x4?, 0xc000071dd0?, 0xbf8753?, 0xbee767?, 0x58?, 0xda680?) at runtime/syscall_windows.go:501
          syscall.readFile(0xbf808b?, {0xc000300000?, 0x8000, 0x800000?}, 0x7ffff800000?, 0x2?) at syscall/zsyscall_windows.go:1024
          syscall.ReadFile(...) at syscall/syscall_windows.go:400
          syscall.Read(0xc000259b80?, {0xc000300000?, 0x2?, 0x0?}) at syscall/syscall_windows.go:379
          internal/poll.(*FD).Read(0xc000259b80, {0xc000300000, 0x8000, 0x8000}) at poll/fd_windows.go:427
          os.(*File).read(...) at os/file_posix.go:31
          os.(*File).Read(0xc0002065d0, {0xc000300000?, 0xc000008858?, 0x11c4880?}) at os/file.go:118
          io.copyBuffer({0x11c4360, 0xc000008858}, {0x11c46a0, 0xc0002065d0}, {0x0, 0x0, 0x0}) at io/io.go:427
          io.Copy(...) at io/io.go:386
          github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1.1() at internal/output_interceptor.go:159
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1 at internal/output_interceptor.go:158
  In [DeferCleanup] at: D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:132
------------------------------
+ [FAILED] [2.003 seconds]
HaveLeaked [It] considers testing and runtime goroutines not to be leaks
D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:77

  Timed out after 2.003s.
  should not find any leaks by default
  Expected not to leak 4 goroutines:
      goroutine 18 [IO wait]
          internal/poll.runtime_pollWait(0x1c27e165de8, 0x72) at runtime/netpoll.go:305
          internal/poll.(*pollDesc).wait(0xc00007fbb8?, 0xfb5c05?, 0x0) at poll/fd_poll_runtime.go:84
          internal/poll.execIO(0xc0001e8798, 0x1139838) at poll/fd_windows.go:175
          internal/poll.(*FD).Read(0xc0001e8780, {0xc000297000, 0x1000, 0x1000}) at poll/fd_windows.go:441
          net.(*netFD).Read(0xc0001e8780, {0xc000297000?, 0xc00001fa34?, 0x16?}) at net/fd_posix.go:55
          net.(*conn).Read(0xc00000a598, {0xc000297000?, 0xc00001fa34?, 0x1057ea0?}) at net/net.go:183
          bufio.(*Reader).Read(0xc000298000, {0xc000288040, 0x1, 0xc00007fd70?}) at bufio/bufio.go:237
          io.ReadAtLeast({0x11c3e80, 0xc000298000}, {0xc000288040, 0x1, 0x9}, 0x1) at io/io.go:332
          io.ReadFull(...) at io/io.go:351
          encoding/gob.decodeUintReader({0x11c3e80?, 0xc000298000?}, {0xc000288040?, 0x9?, 0x9?}) at gob/decode.go:120
          encoding/gob.(*Decoder).recvMessage(0xc00028c080) at gob/decoder.go:81
          encoding/gob.(*Decoder).decodeTypeSequence(0xc00028c080, 0x0) at gob/decoder.go:146
          encoding/gob.(*Decoder).DecodeValue(0xc00028c080, {0x104f760?, 0xc0001eeed0?, 0x1c27c560108?}) at gob/decoder.go:225
          encoding/gob.(*Decoder).Decode(0xc00028c080, {0x104f760?, 0xc0001eeed0?}) at gob/decoder.go:202
          net/rpc.(*gobClientCodec).ReadResponseHeader(0x1079a60?, 0x10496a0?) at rpc/client.go:228
          net/rpc.(*Client).input(0xc0002980c0) at rpc/client.go:109
          created by net/rpc.NewClientWithCodec at rpc/client.go:206
      goroutine 34 [select]
          github.com/onsi/ginkgo/v2/internal.startPipeFactory(0xc000076420, 0xc000240000) at internal/output_interceptor.go:77
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting at internal/output_interceptor.go:143
      goroutine 35 [select]
          github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1() at internal/output_interceptor.go:163
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting at internal/output_interceptor.go:153
      goroutine 50 [syscall, locked to thread]
          syscall.SyscallN(0xc5b465?, {0xc000245d20?, 0x10f32d4?, 0x8?}) at runtime/syscall_windows.go:557
          syscall.Syscall6(0xc000039260?, 0x8000?, 0x4?, 0xc000245dd0?, 0xbf8753?, 0x0?, 0x0?, 0x46840?) at runtime/syscall_windows.go:501
          syscall.readFile(0xbf808b?, {0xc00009c000?, 0x8000, 0x800000?}, 0x7ffff800000?, 0x2?) at syscall/zsyscall_windows.go:1024
          syscall.ReadFile(...) at syscall/syscall_windows.go:400
          syscall.Read(0xc000242000?, {0xc00009c000?, 0x106d2a0?, 0x0?}) at syscall/syscall_windows.go:379
          internal/poll.(*FD).Read(0xc000242000, {0xc00009c000, 0x8000, 0x8000}) at poll/fd_windows.go:427
          os.(*File).read(...) at os/file_posix.go:31
          os.(*File).Read(0xc00020e040, {0xc00009c000?, 0xc000096018?, 0x0?}) at os/file.go:118
          io.copyBuffer({0x11c4360, 0xc000096018}, {0x11c46a0, 0xc00020e040}, {0x0, 0x0, 0x0}) at io/io.go:427
          io.Copy(...) at io/io.go:386
          github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1.1() at internal/output_interceptor.go:159
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1 at internal/output_interceptor.go:158
  In [It] at: D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:79
------------------------------
+ [FAILED] [2.003 seconds]
HaveLeaked when using signals [It] doesn't find leaks
D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:84

  Timed out after 2.003s.
  found signal.Notify leaks
  Expected not to leak 4 goroutines:
      goroutine 34 [IO wait]
          internal/poll.runtime_pollWait(0x244b2d450d0, 0x72) at runtime/netpoll.go:305
          internal/poll.(*pollDesc).wait(0xc00039fbb8?, 0xfb5c05?, 0x0) at poll/fd_poll_runtime.go:84
          internal/poll.execIO(0xc0000b8298, 0x1139838) at poll/fd_windows.go:175
          internal/poll.(*FD).Read(0xc0000b8280, {0xc000219000, 0x1000, 0x1000}) at poll/fd_windows.go:441
          net.(*netFD).Read(0xc0000b8280, {0xc000219000?, 0xc000090188?, 0x16?}) at net/fd_posix.go:55
          net.(*conn).Read(0xc00008e018, {0xc000219000?, 0xc000090188?, 0x1057ea0?}) at net/net.go:183
          bufio.(*Reader).Read(0xc0002060c0, {0xc00020c030, 0x1, 0xc00039fd70?}) at bufio/bufio.go:237
          io.ReadAtLeast({0x11c3e80, 0xc0002060c0}, {0xc00020c030, 0x1, 0x9}, 0x1) at io/io.go:332
          io.ReadFull(...) at io/io.go:351
          encoding/gob.decodeUintReader({0x11c3e80?, 0xc0002060c0?}, {0xc00020c030?, 0x9?, 0x9?}) at gob/decode.go:120
          encoding/gob.(*Decoder).recvMessage(0xc00021a000) at gob/decoder.go:81
          encoding/gob.(*Decoder).decodeTypeSequence(0xc00021a000, 0x0) at gob/decoder.go:146
          encoding/gob.(*Decoder).DecodeValue(0xc00021a000, {0x104f760?, 0xc0000885d0?, 0x2448d110598?}) at gob/decoder.go:225
          encoding/gob.(*Decoder).Decode(0xc00021a000, {0x104f760?, 0xc0000885d0?}) at gob/decoder.go:202
          net/rpc.(*gobClientCodec).ReadResponseHeader(0x1079a60?, 0x10496a0?) at rpc/client.go:228
          net/rpc.(*Client).input(0xc000206180) at rpc/client.go:109
          created by net/rpc.NewClientWithCodec at rpc/client.go:206
      goroutine 38 [select]
          github.com/onsi/ginkgo/v2/internal.startPipeFactory(0xc0000c41e0, 0xc00022e240) at internal/output_interceptor.go:77
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting at internal/output_interceptor.go:143
      goroutine 50 [select]
          github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1() at internal/output_interceptor.go:163
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting at internal/output_interceptor.go:153
      goroutine 43 [syscall, locked to thread]
          syscall.SyscallN(0x1000?, {0xc000241d20?, 0x8000?, 0x244b2a8ffff?}) at runtime/syscall_windows.go:557
          syscall.Syscall6(0x244b2db3280?, 0x8000?, 0x4?, 0xc000241dd0?, 0xbf8753?, 0xc1e287?, 0x0?, 0xc1e2e0?) at runtime/syscall_windows.go:501
          syscall.readFile(0xbf808b?, {0xc0002c4000?, 0x8000, 0x800000?}, 0x7ffff800000?, 0x2?) at syscall/zsyscall_windows.go:1024
          syscall.ReadFile(...) at syscall/syscall_windows.go:400
          syscall.Read(0xc0000b8a00?, {0xc0002c4000?, 0xc0000c41d8?, 0x1?}) at syscall/syscall_windows.go:379
          internal/poll.(*FD).Read(0xc0000b8a00, {0xc0002c4000, 0x8000, 0x8000}) at poll/fd_windows.go:427
          os.(*File).read(...) at os/file_posix.go:31
          os.(*File).Read(0xc00008e048, {0xc0002c4000?, 0xc0002251a0?, 0x1?}) at os/file.go:118
          io.copyBuffer({0x11c4360, 0xc0002251a0}, {0x11c46a0, 0xc00008e048}, {0x0, 0x0, 0x0}) at io/io.go:427
          io.Copy(...) at io/io.go:386
          github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1.1() at internal/output_interceptor.go:159
          created by github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1 at internal/output_interceptor.go:158
  In [It] at: D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:88
------------------------------


Summarizing 5 Failures:
  [FAIL] HaveLeaked wrapped around test nodes when leaking [AfterEach] leaks in test
  D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:222
  [FAIL] HaveLeaked failure messages [DeferCleanup] when things go wrong handles filter matcher errors
  D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:132
  [FAIL] HaveLeaked failure messages [DeferCleanup] when things go wrong expects actual to be a slice of Goroutine
  D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:132
  [FAIL] HaveLeaked [It] considers testing and runtime goroutines not to be leaks
  D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:79
  [FAIL] HaveLeaked when using signals [It] doesn't find leaks
  D:/main/dev/davidhsingyuchen/gomega/gleak/have_leaked_matcher_test.go:88

Ran 34 of 34 Specs in 2.010 seconds
FAIL! -- 29 Passed | 5 Failed | 0 Pending | 0 Skipped

thediveo · 2022-08-25T06:18:13Z

Since I don't have a windows system at hand: does the leak test also bite on the original master? If yes, can you please file a separate issue for it?

davidhsingyuchen · 2022-08-25T06:58:44Z

Created #576 accordingly.

davidhsingyuchen · 2022-08-31T19:09:25Z

Closing this issue because the fix is included in v1.20.2 and tested on my side.

davidhsingyuchen mentioned this issue Aug 25, 2022

Enable reading from a closed gbytes.Buffer #575

Merged

davidhsingyuchen mentioned this issue Aug 25, 2022

gleak tests failing on Windows 10 with go1.19 #576

Closed

davidhsingyuchen closed this as completed Aug 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Why does `gbytes.Buffer.Read` return an error when the buffer is closed? #573

Why does `gbytes.Buffer.Read` return an error when the buffer is closed? #573

davidhsingyuchen commented Aug 24, 2022

onsi commented Aug 24, 2022

davidhsingyuchen commented Aug 24, 2022

davidhsingyuchen commented Aug 25, 2022

thediveo commented Aug 25, 2022

davidhsingyuchen commented Aug 25, 2022

davidhsingyuchen commented Aug 31, 2022

Why does gbytes.Buffer.Read return an error when the buffer is closed? #573

Why does gbytes.Buffer.Read return an error when the buffer is closed? #573

Comments

davidhsingyuchen commented Aug 24, 2022

onsi commented Aug 24, 2022

davidhsingyuchen commented Aug 24, 2022

davidhsingyuchen commented Aug 25, 2022

thediveo commented Aug 25, 2022

davidhsingyuchen commented Aug 25, 2022

davidhsingyuchen commented Aug 31, 2022

Why does `gbytes.Buffer.Read` return an error when the buffer is closed? #573

Why does `gbytes.Buffer.Read` return an error when the buffer is closed? #573