chore: update to go 1.19 (#24119)

* chore: update to go 1.19.6 * chore: gofmt * test: fix tests for sort order change * chore: generate pb * feat: upgrade flux to v0.188.0 (#23911) * feat: upgrade flux to 0.171.0 Tests failing, safety commit First step in #23815 * fix: remove "org" parameter" from writeOptSource I attempted to implement the "orgOpt" argument in a similar fashion to f6669f7. However, it looks like Flux doesn't accept "org" as a parameter to "load". It responds with: Error calling function \"load\" @113:16-113:30: error calling function \"to\" @6:19-6:47: unused arguments [org] This brings us from 194 passing to 570 passing. * fix: temporarily disable broken flux tests These tests expect rows to be stored in a certain order. However, nothing is specifying the sort order. This has been fixed in a later update to flux: (see 3d6f47ded). Temporarily disable these tests until we include a fixed version of the flux tests. * chore: add tests from a492993 This fixes "test-flux.sh" so it runs tests within the "flux/" directory. This uncovered some other issues with the tests located within "flux/". These also needed to be updated to match the newer flux API. * feat: upgrade flux to 0.172.0 This includes changes made in "cbbf4b27da". Since "test.go" in 2.x diverged from 1.x, some modifications were required to make this compatible. * feat: upgrade flux to 0.173.0 * feat: upgrade flux to v0.174.0 * fix: Update the condition when reseting cursor (#23522) Filters that contain `or` may change between cursor resets so we must remember to update the condition in the read cursor. ```flux |> filter(fn: (r) => ((r["_field"] == "field1" and r["_value"]==true) or (r["_field"] == "field2" and r["_value"] == false))) ``` Closes influxdata/flux#4804 * feat: upgrade flux to 0.174.1 * feat: upgrade flux to 0.175.0 * chore: remove end-to-end tests These were removed in a492993 for 2.x. These tests prevent "go test ./..." from completing. As stated in the original commit, these tests should now be handled by the "fluxtest" harness. * feat: upgrade flux to 0.176.0 Some tests needed to be disabled within the flux harness. This is a result of enabling "Optimize Aggregate Window" in flux@05a1065f. These tests are not present in 2.x. Therefore, I am unsure if the breakage is resolved in a later commit. * feat: upgrade flux to 0.177.0 * feat: upgrade flux to 0.178.0 * feat: upgrade flux to v0.179.0 This removes all invocations of "flux.RegisterOpSpec". According to flux@e39096d5, "flux.RegisterOpSpec" does nothing in the current version of flux and was removed. * chore: update fluxtest skip list (#23633) * chore: manually backport 785a465 This removes the reference to "flux.Spec". * build(flux): update flux to v0.181.0 (#23682) * build(flux): update flux to v0.184.2 * chore: skip more Flux acceptance tests There are issues for each skip detailed in test-flux.sh. * feat: upgrade flux to v0.185.0 This adds "FluxTesting" to the "HTTPD" configuration. This option is hidden and disabled by default. When "FluxTesting" is set, it enables the default testing flags for "Flux". These flags allow the "vectorized float tests" and tests requiring the "removeRedundantSortNodes" and "labelPolymorphism" flag enabled to work. These changes are based off of d8553c0. flux@3d6f47ded is included within this version of Flux. Therefore we can now include the "group_*" tests. * feat: upgrade flux to 0.186.0 * feat: upgrade flux to 0.187.0 * feat: upgrade flux to 0.188.0 * fix: re-run ./generate.sh with updated protoc * fix: restrict cores to match CircleCI documentation Co-authored-by: davidby-influx <dbyrne@influxdata.com> Co-authored-by: Markus Westerlind <marwes91@gmail.com> Co-authored-by: Sean Brickley <sean@wabr.io> Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com> Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com> --------- Co-authored-by: Brandon Pfeifer <bpfeifer@influxdata.com> Co-authored-by: davidby-influx <dbyrne@influxdata.com> Co-authored-by: Markus Westerlind <marwes91@gmail.com> Co-authored-by: Sean Brickley <sean@wabr.io> Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com> Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
influxdata · Mar 3, 2023 · fce0d1c · fce0d1c
1 parent 154f23b
commit fce0d1c
Show file tree

Hide file tree

Showing 69 changed files with 869 additions and 699 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -6,7 +6,7 @@ parameters:
     # when updating the go version, should also update the go version in go.mod
     description: docker tag for cross build container from quay.io . Created by https://github.com/influxdata/edge/tree/master/dockerfiles/cross-builder .
     type: string
-    default: go1.18.9-cb1343dd74ecba8ec07fe810195530a0b9055aa9
+    default: go1.19.6-7254b98bc4080dce9c237b698e868e4e0567cee7
 
   workflow:
     type: string
@@ -244,7 +244,7 @@ jobs:
   unit_test_race:
     docker:
       - image: quay.io/influxdb/cross-builder:<< pipeline.parameters.cross-container-tag >>
-    resource_class: large
+    resource_class: xlarge
     steps:
       - checkout
       - restore_cache:
@@ -257,7 +257,17 @@ jobs:
             set -x
             mkdir -p junit-race/
             export GORACE="halt_on_error=1"
-            gotestsum --junitfile junit-race/influxdb.junit.xml -- -race ./...
+            # "resource_class: xlarge" creates a Docker container with eight
+            # virtual cpu cores. However, applications like "nproc" return
+            # the host machine's core count (which in this case is 36).
+            # When less cores are available than advertised, the
+            # race-tests fail.
+            #
+            # We'll manually reduce the number of available cores to what
+            # is specified by the CircleCI documentation:
+            # https://circleci.com/product/features/resource-classes/
+            taskset -c 0-7 \
+              gotestsum --junitfile junit-race/influxdb.junit.xml -- -race ./...
           no_output_timeout: 1500s
       - store_test_results:
           path: junit-race/

diff --git a/cmd/influx/cli/cli.go b/cmd/influx/cli/cli.go
@@ -1461,7 +1461,6 @@ func (f *formatter) valueBuf(i, j int, typ flux.ColType, cr flux.ColReader) []by
 // * common tags sorted by label
 // * other tags sorted by label
 // * value
-//
 type orderedCols struct {
 	indexMap []int
 	cols     []flux.ColMeta

diff --git a/cmd/influx_tools/internal/errlist/errlist.go b/cmd/influx_tools/internal/errlist/errlist.go
@@ -22,7 +22,7 @@ func (el *ErrorList) Add(err error) {
 	el.errs = append(el.errs, err)
 }
 
-//Err returns whether or not an error list is an error.
+// Err returns whether or not an error list is an error.
 func (el *ErrorList) Err() error {
 	if len(el.errs) == 0 {
 		return nil

diff --git a/cmd/influx_tools/internal/format/binary/tools_binary.pb.go b/cmd/influx_tools/internal/format/binary/tools_binary.pb.go
diff --git a/cmd/influxd/backup_util/internal/backup_util.pb.go b/cmd/influxd/backup_util/internal/backup_util.pb.go
diff --git a/cmd/influxd/run/command.go b/cmd/influxd/run/command.go
@@ -280,11 +280,11 @@ type Options struct {
 
 // GetConfigPath returns the config path from the options.
 // It will return a path by searching in this order:
-//   1. The CLI option in ConfigPath
-//   2. The environment variable INFLUXDB_CONFIG_PATH
-//   3. The first influxdb.conf file on the path:
-//        - ~/.influxdb
-//        - /etc/influxdb
+//  1. The CLI option in ConfigPath
+//  2. The environment variable INFLUXDB_CONFIG_PATH
+//  3. The first influxdb.conf file on the path:
+//     - ~/.influxdb
+//     - /etc/influxdb
 func (opt *Options) GetConfigPath() string {
 	if opt.ConfigPath != "" {
 		if opt.ConfigPath == os.DevNull {

diff --git a/cmd/influxd/run/server.go b/cmd/influxd/run/server.go
@@ -15,6 +15,7 @@ import (
 
 	"github.com/influxdata/flux"
 	"github.com/influxdata/flux/dependencies/testing"
+	"github.com/influxdata/flux/execute/executetest"
 	"github.com/influxdata/influxdb"
 	"github.com/influxdata/influxdb/coordinator"
 	influxdb2 "github.com/influxdata/influxdb/flux/stdlib/influxdata/influxdb"
@@ -312,10 +313,16 @@ func (s *Server) appendHTTPDService(c httpd.Config) error {
 		if err != nil {
 			return err
 		}
+
+		deps := []flux.Dependency{storageDep, testing.FrameworkConfig{}}
+		if s.config.HTTPD.FluxTesting {
+			deps = append(deps, executetest.NewDefaultTestFlagger())
+		}
+
 		srv.Handler.Controller, err = control.New(
 			s.config.FluxController,
 			s.Logger.With(zap.String("service", "flux-controller")),
-			[]flux.Dependency{storageDep, testing.FrameworkConfig{}},
+			deps,
 		)
 		if err != nil {
 			return err

diff --git a/coordinator/points_writer.go b/coordinator/points_writer.go
@@ -255,8 +255,8 @@ func (l sgList) Covers(t time.Time) bool {
 // to start time. Therefore, if there are multiple shard groups that match
 // this point's time they will be preferred in this order:
 //
-//  - a shard group with the earliest end time;
-//  - (assuming identical end times) the shard group with the earliest start time.
+//   - a shard group with the earliest end time;
+//   - (assuming identical end times) the shard group with the earliest start time.
 func (l sgList) ShardGroupAt(t time.Time) *meta.ShardGroupInfo {
 	if l.items.Len() == 0 {
 		return nil

diff --git a/flux/stdlib/influxdata/influxdb/buckets.go b/flux/stdlib/influxdata/influxdb/buckets.go
@@ -149,7 +149,7 @@ func (rule LocalBucketsRule) Name() string {
 }
 
 func (rule LocalBucketsRule) Pattern() plan.Pattern {
-	return plan.Pat(influxdb.BucketsKind)
+	return plan.MultiSuccessor(influxdb.BucketsKind)
 }
 
 func (rule LocalBucketsRule) Rewrite(ctx context.Context, node plan.Node) (plan.Node, bool, error) {

diff --git a/flux/stdlib/influxdata/influxdb/filter_test.flux b/flux/stdlib/influxdata/influxdb/filter_test.flux
@@ -33,9 +33,44 @@ testcase filter {
 ,,0,2018-05-22T19:53:36Z,system,host.local,load1,1.63
 ")
 
-    got = testing.loadStorage(csv: input)
+    got = csv.from(csv: input)
+        |> testing.load()
         |> range(start: -100y)
         |> filter(fn: (r) => r._measurement == "system" and r._field == "load1")
         |> drop(columns: ["_start", "_stop"])
     testing.diff(want, got)
 }
+
+
+input_issue_4804 = "#datatype,string,long,dateTime:RFC3339,string,string,string,boolean
+#group,false,false,false,true,true,true,false
+#default,_result,,,,,,
+,result,table,_time,_measurement,host,_field,_value
+,,0,2018-05-22T19:53:26Z,system,host.local,load1,true
+,,0,2018-05-22T19:53:36Z,system,host.local,load1,false
+,,1,2018-05-22T19:53:26Z,system,host.local,load3,false
+,,2,2018-05-22T19:53:26Z,system,host.local,load4,true
+"
+
+testcase flux_issue_4804 {
+    expect.planner(rules: [
+        "influxdata/influxdb.FromStorageRule": 1,
+        "PushDownRangeRule": 1,
+        "PushDownFilterRule": 1,
+    ])
+
+    want = csv.from(csv: "#datatype,string,long,dateTime:RFC3339,string,string,string,boolean
+#group,false,false,false,true,true,true,false
+#default,_result,,,,,,
+,result,table,_time,_measurement,host,_field,_value
+,,0,2018-05-22T19:53:26Z,system,host.local,load1,true
+,,1,2018-05-22T19:53:26Z,system,host.local,load3,false
+")
+
+    got = csv.from(csv: input_issue_4804)
+        |> testing.load()
+        |> range(start: -100y)
+        |> filter(fn: (r) => ((r["_field"] == "load1" and r["_value"] == true) or (r["_field"] == "load3" and r["_value"] == false)))
+        |> drop(columns: ["_start", "_stop"])
+    testing.diff(want, got)
+}