Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed windows compatibility issue, when storage gateway syncs index f… #7148

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Expand Up @@ -15,6 +15,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
- [#7083](https://github.com/thanos-io/thanos/pull/7083) Store Gateway: Fix lazy expanded postings with 0 length failed to be cached.
- [#7080](https://github.com/thanos-io/thanos/pull/7080) Receive: race condition in handler Close() when stopped early
- [#7132](https://github.com/thanos-io/thanos/pull/7132) Documentation: fix broken helm installation instruction
- [#7148](https://github.com/thanos-io/thanos/pull/7148) Store Gateway: Fix windows compatibility in index-header sync.

### Added

Expand Down
25 changes: 17 additions & 8 deletions pkg/block/indexheader/binary_reader.go
Expand Up @@ -13,6 +13,7 @@ import (
"io"
"math"
"os"
"path"
"path/filepath"
"sort"
"sync"
Expand Down Expand Up @@ -112,6 +113,20 @@ func WriteBinary(ctx context.Context, bkt objstore.BucketReader, id ulid.ULID, f
tmpFilename = filename + ".tmp"
}

bw, err := writeAndClose(id, tmpFilename, indexVersion, ir)
if err != nil {
return nil, err
}

if tmpFilename != "" {
// Create index-header in atomic way, to avoid partial writes (e.g during restart or crash of store GW).
return nil, os.Rename(tmpFilename, filename)
}

return bw.Buffer(), nil
}

func writeAndClose(id ulid.ULID, tmpFilename string, indexVersion int, ir *chunkedIndexReader) (*binaryWriter, error) {
// Buffer for copying and encbuffers.
// This also will control the size of file writer buffer.
buf := make([]byte, 32*1024)
Expand Down Expand Up @@ -152,13 +167,7 @@ func WriteBinary(ctx context.Context, bkt objstore.BucketReader, id ulid.ULID, f
if err := bw.writer.Sync(); err != nil {
return nil, errors.Wrap(err, "sync")
}

if tmpFilename != "" {
// Create index-header in atomic way, to avoid partial writes (e.g during restart or crash of store GW).
return nil, os.Rename(tmpFilename, filename)
}

return bw.Buffer(), nil
return bw, nil
}

type chunkedIndexReader struct {
Expand All @@ -170,7 +179,7 @@ type chunkedIndexReader struct {
}

func newChunkedIndexReader(ctx context.Context, bkt objstore.BucketReader, id ulid.ULID) (*chunkedIndexReader, int, error) {
indexFilepath := filepath.Join(id.String(), block.IndexFilename)
indexFilepath := path.Join(id.String(), block.IndexFilename)
attrs, err := bkt.Attributes(ctx, indexFilepath)
if err != nil {
return nil, 0, errors.Wrapf(err, "get object attributes of %s", indexFilepath)
Expand Down