Updating with the upstream master version (#1)

* Return error instead of returning nil * Make Close idempotent * CI update * Auto-discover topics * fix test due to sarama's change in sync_producer's config * fix the doc for Consumer.Notifications * Bump deps * Add HighWaterMarks support ref: HighWaterMarks returns the current high water marks for each topic and partition https://godoc.org/github.com/Shopify/sarama#Consumer * Bump Sarama dependency, use latest Kafka version * Allow to stash offsets when processing batches * Added support method * Subscribe to new topics * Add troubleshooting information regarding consumer not receiving messages * Allow to subscribe to topics via regular expressions * Formatting * Fix troubleshooting formatting. * Added example * Handle authz limited to specific topics * express it a bit simpler * move metadata refresh to consumer method * Fix comparison operator * Allow custom group member UserData to be included when joining the group * Commit every time, even if not dirty. Updated README and CI config. * Updated example * Fix off by one error when marking offset * Updated README * Test with Kafka 0.9 * Partially revert bsm#117. Force-commit when Consumer.Offsets.Retention is set * Remove retention time workaround. Offset retention must be > than log retention. * Allow sarama-cluster offset synchronization dwell time to be configured separately from Sarama MaxProcessingTime * Fix typo and dwell time validation error handling * Remove sleep and replace with select statement. Switch from dwell time as pointer to value. * Switch sleep to select with speedy bailout on close * Better config validation * Ported from glide to dep * Fix CI * Fix CI * Ensure notifications are issued (and consumed) before rebalancing is completed * Fix CI * Bump patch version# * Simpler tests * Fix seeding * Test with Go 1.9 * Remove delay * Reduce memory requirements * Add consumer test * Force rebalance when partition consumers exit with an error * Adding notifications on rebalance start * Stop sharing clients as this is not something Kafka allows us to do * Expose individual partitions * Respond to feedback * Adding whitelist example * Addressing feedback. Always multiplex errors * Updated README * Allow to re-use clients * Adding comment to clarify usage * Update CI settings * Make Close thread-safe * More robust * Fix race, added clarifying comment to CommitOffsets * Fix deps * Fix the kafka source - 0.9.0.1 testing broken * Support Kafka 1.0 * Drain channels on Close * Allow marking of earlier offsets This commit is in alignment with the sarama library commits IBM/sarama@b966238#diff-3ca6d659defd100fe2de43adf2b8f41e IBM/sarama@96fa1c8#diff-3ca6d659defd100fe2de43adf2b8f41e * Address comments and add tests * Ensure we have random topic name and reduce time * Fix test timing for kafka ≥ 0.11.0 * Remove unncessary expect clause * Address comments + gofmt * Unify PartitionConsumer interface * New release * Fix NPE on ResetOffsets and MarkOffsets * Use correct request proto version * Expose offset methods on partition consumers * Fix/remove test * Bump CI config * Fewer builds * add Offset() to PartitionConsumer interface (bsm#221) * add Offset() to PartitionConsumer interface * rename Offset to InitialOffset
ContinuumLLC · May 9, 2018 · 2a07e35 · 2a07e35
1 parent 6d37f56
commit 2a07e35
Show file tree

Hide file tree

Showing 27 changed files with 1,780 additions and 504 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -1,14 +1,19 @@
 sudo: false
 language: go
 go:
-  - 1.7
-  - 1.6.3
-  - 1.5.4
+  - 1.10.x
+  - 1.9.x
+  - 1.8.x
 install:
-  - go get -u github.com/Masterminds/glide
-  - glide install
+  - go get -u github.com/golang/dep/cmd/dep
+  - dep ensure
 env:
-  - SCALA_VERSION=2.11 KAFKA_VERSION=0.9.0.1 GO15VENDOREXPERIMENT=1
-  - SCALA_VERSION=2.11 KAFKA_VERSION=0.10.0.0 GO15VENDOREXPERIMENT=1
+  - SCALA_VERSION=2.12 KAFKA_VERSION=0.10.2.1
+  - SCALA_VERSION=2.12 KAFKA_VERSION=0.11.0.1
+  - SCALA_VERSION=2.12 KAFKA_VERSION=1.0.1
 script:
   - make default test-race
+addons:
+  apt:
+    packages:
+      - oracle-java8-set-default
diff --git a/Gopkg.lock b/Gopkg.lock
diff --git a/Gopkg.toml b/Gopkg.toml
@@ -0,0 +1,26 @@
+
+# Gopkg.toml example
+#
+# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
+# for detailed Gopkg.toml documentation.
+#
+# required = ["github.com/user/thing/cmd/thing"]
+# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
+#
+# [[constraint]]
+#   name = "github.com/user/project"
+#   version = "1.0.0"
+#
+# [[constraint]]
+#   name = "github.com/user/project2"
+#   branch = "dev"
+#   source = "github.com/myfork/project2"
+#
+# [[override]]
+#  name = "github.com/x/y"
+#  version = "2.4.0"
+
+
+[[constraint]]
+  name = "github.com/Shopify/sarama"
+  version = "^1.14.0"
diff --git a/LICENSE b/LICENSE
@@ -0,0 +1,22 @@
+(The MIT License)
+
+Copyright (c) 2017 Black Square Media Ltd
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
diff --git a/Makefile b/Makefile
@@ -1,10 +1,9 @@
-SCALA_VERSION?= 2.11
-KAFKA_VERSION?= 0.10.0.0
+SCALA_VERSION?= 2.12
+KAFKA_VERSION?= 1.0.1
 KAFKA_DIR= kafka_$(SCALA_VERSION)-$(KAFKA_VERSION)
-KAFKA_SRC= http://www.mirrorservice.org/sites/ftp.apache.org/kafka/$(KAFKA_VERSION)/$(KAFKA_DIR).tgz
+KAFKA_SRC= https://archive.apache.org/dist/kafka/$(KAFKA_VERSION)/$(KAFKA_DIR).tgz
 KAFKA_ROOT= testdata/$(KAFKA_DIR)
-
-PKG:=$(shell glide nv)
+PKG=$(shell go list ./... | grep -v vendor)
 
 default: vet test
 
@@ -22,10 +21,15 @@ test-race: testdeps
 
 testdeps: $(KAFKA_ROOT)
 
-.PHONY: test testdeps vet
+doc: README.md
+
+.PHONY: test testdeps vet doc
 
 # ---------------------------------------------------------------------
 
 $(KAFKA_ROOT):
 	@mkdir -p $(dir $@)
 	cd $(dir $@) && curl -sSL $(KAFKA_SRC) | tar xz
+
+README.md: README.md.tpl $(wildcard *.go)
+	becca -package $(subst $(GOPATH)/src/,,$(PWD))