v2.10.2

What's Changed

Critical bug fix:

commit ece844a
Author: George Gensure werkt0@gmail.com
Date: Fri May 31 17:35:30 2024 -0400

Reduce DUPLICATE_OUTPUT_STREAM future to write

Move future completion into the only scope it was actually missing from
- getOutput for Write, rather than inducing all `put` calls into posts
to backplane via onPut with duplicates.

Docker Containers

docker pull bazelbuild/buildfarm-server:2.10.2
docker pull bazelbuild/buildfarm-worker:2.10.2

v2.10.1

What's Changed

Critical bug fix:

commit bb1f766
Author: George Gensure werkt0@gmail.com
Date: Thu May 30 12:36:56 2024 -0400

Prevent write completion starvation for duplicates

When a duplicate output stream is detected, we must signal the
writeWinner (because the write exists) and onInsert (because it was
inserted) for an output stream creation. If we're racing, we should be
eventually convergent, but this absolutely fixes a hang which occurs on
this sentinel stream's return into getOutput, where the future might
never be triggered otherwise.

Docker Containers

docker pull bazelbuild/buildfarm-server:2.10.1
docker pull bazelbuild/buildfarm-worker:2.10.1

v2.10.0

What's Changed

• Dequeue flags expire after timeout effect by @werkt in #1642
• Update .bazelversion to use bazel 7.0.2 by @80degreeswest in #1641
• Disable rules_docker transitions by @80degreeswest in #1648
• style: buildifier by @jasonschroeder-sfdc in #1653
• chore: update CODEOWNERS by @jasonschroeder-sfdc in #1654
• build: pin python-dateutils==2.8.2 by @jasonschroeder-sfdc in #1657
• chore(deps): bump io_grpc_grpc_java to 1.62.2 by @jasonschroeder-sfdc in #1658
• build(protobuf): switch to protobuf from BCR by @jasonschroeder-sfdc in #1656
• refactor(tests): replace ExpectedException with assertThrows by @jasonschroeder-sfdc in #1652
• docs(metrics): Update prometheus metrics by @jasonschroeder-sfdc in #1650
• chore: bump googleapis by @jasonschroeder-sfdc in #1655
• build(bzlmod): remove bzlmod lockfile by @jasonschroeder-sfdc in #1651
• build: add cgroup-tools to the ubuntu base images by @jasonschroeder-sfdc in #1662
• build: remove docker download install pkgs actions by @jasonschroeder-sfdc in #1663
• fix(common): do not log redis password on start-up by @jasonschroeder-sfdc in #1665
• [helm] Support autoscaling/{v2beta2,v2} for HPA. by @monaka in #1666
• chores: bump google-java-format to 1.20.0 by @jasonschroeder-sfdc in #1649
• build(bzlmod): Switch to rules_oci by @jasonschroeder-sfdc in #1667
• build: bump maven version to latest by @jasonschroeder-sfdc in #1670
• ci: Fix rules_oci docker publish by @jasonschroeder-sfdc in #1671
• Fix link to IJ screenshot by @stefanobaghino in #1676
• chore(MODULE.bazel): bump gazelle by @jasonschroeder-sfdc in #1674
• build: declare container_structure_test as dev dependency by @jasonschroeder-sfdc in #1675
• chore(MODULE.bazel): bump rules_go by @jasonschroeder-sfdc in #1673
• build: pin ubuntu-mantic image by @jasonschroeder-sfdc in #1672
• add ingress for the server by @andrewrothstein in #1669
• Switch to jedis-5 by @werkt in #1677
• ci: bump buildifier to 6.4.0 by @jasonschroeder-sfdc in #1679
• Prevent NPE on shutdown with null pipeline by @werkt in #1681
• Update to latest jedis release 5.1.2 by @werkt in #1682
• Update remaining references to redis 5.0.9 by @werkt in #1684
• build(bzlmod): update rules_pkg by @jasonschroeder-sfdc in #1646
• publish Helm chart as OCI artifact to ghcr.io by @andrewrothstein in #1683
• Create codeql.yml by @werkt in #1689
• fix: remove redis hashtags from prometheus metrics by @jasonschroeder-sfdc in #1693
• Rework Queue object pattern hierarchy by @werkt in #1695
• Support for BLAKE3 digest function by @werkt in #1697
• chore(deps): bump bazel to 7.1.1 by @jasonschroeder-sfdc in #1694
• chore: remove bazel binary by @jasonschroeder-sfdc in #1698
• fix(ServerInstance): fix severe log message by @jasonschroeder-sfdc in #1692
• Remove switched_rules_by_language usage by @werkt in #1700
• Fix jedismock tests on mac by @werkt in #1701
• Terminate OperationQueue dequeue when interrupted by @werkt in #1702
• fix(metrics): convert Gauges to Counters by @jasonschroeder-sfdc in #1699
• fix(RedisShardBackplane): fix backplaneStatus by @jasonschroeder-sfdc in #1691
• Go all-in on bzlmod by @jasonschroeder-sfdc in #1678

New Contributors

• @monaka made their first contribution in #1666

Full Changelog: 2.9.0...2.10.0

Docker Containers

docker pull bazelbuild/buildfarm-server:2.10.0
docker pull bazelbuild/buildfarm-worker:2.10.0

v2.9.0

What's Changed

• Consider output_* as relative to WD by @werkt in #1550
• Implement Persistent Workers as an execution path by @wiwa in #1260
• Locate Output Paths relative to WorkingDirectory by @werkt in #1553
• Remove incorrect external resolve of WD on upload by @werkt in #1554
• Instance cleanups by @werkt in #1555
• BuildfarmExecutors moved to its own target by @werkt in #1557
• Update Jedis connection error handling for addWorker by @amishra-u in #1552
• add github action to package and publish the helm chart as a released artifact by @andrewrothstein in #1556
• Assert correct context called for WSO responses by @werkt in #1561
• Write onNext requires onCompleted to follow by @werkt in #1565
• chore: update maven dependencies by @jasonschroeder-sfdc in #1434
• build: support compiling protobuf on macOS by @jasonschroeder-sfdc in #1563
• tests: bump bazelversion for integration tests by @jasonschroeder-sfdc in #1566
• [tests] add unit test for local resources by @luxe in #1558
• refactor: swap HealthStatusManager by @jasonschroeder-sfdc in #1568
• Update log level for blob location adjust by @amishra-u in #1573
• feat: Redis password from file by @jasonschroeder-sfdc in #1569
• docs: update AUTHORS and CONTRIBUTORS by @jasonschroeder-sfdc in #1572
• Identify Graceful Shutdown errors with severe by @werkt in #1574
• Remove orphaned DequeueResults by @werkt in #1575
• Clean up bf-mount usage/refactor by @werkt in #1576
• Clean up bf-mount usage/refactor by @werkt in #1577
• Refactor WriteStreamObserver logging by @werkt in #1578
• Determine PipelineStage exception severity by @werkt in #1579
• Authored sandbox asNobody behavior by @werkt in #1580
• Provide Additional sandbox write paths in config by @werkt in #1581
• Declare stat block information for FuseCAS by @werkt in #1582
• Try a new version of jekyll by @werkt in #1583
• include (un)zip in images by @andrewrothstein in #1590
• Remove rules k8s by @jasonschroeder-sfdc in #1587
• chore: bump rules_license to 0.0.7 by @jasonschroeder-sfdc in #1585
• Bump rules_go and gazelle by @jasonschroeder-sfdc in #1584
• [CI] skip server/worker audit on mac by @luxe in #1592
• Helm Chart: latest image tags not v-prefixed by @andrewrothstein in #1591
• Update helm values to use safe defaults by @werkt in #1597
• Properly balance over nodes, not slot ranges by @werkt in #1598
• fix the shard-worker HPA. it is a StatefulSet and not a Deployment by @andrewrothstein in #1596
• Use integer ids for Sqlite bidirectional index by @werkt in #1599
• Write cleanups by @werkt in #1600
• fix: format without placeholders by @jasonschroeder-sfdc in #1586
• chore: Update proto file styling by @amishra-u in #1594
• fix: Periodically Refresh Active Storage Workers With StartTime by @amishra-u in #1549
• Revert "Use integer ids for Sqlite bidirectional index (#1599)" by @werkt in #1603
• Separate fields for storage and execute workers in BackplaneStatus by @amishra-u in #1605
• Decode nodeId with jedis util SafeEncoder by @werkt in #1607
• build: start adopting bzlmod by @jasonschroeder-sfdc in #1564
• fix(coverage): coverage numbers are not accurate by @jasonschroeder-sfdc in #1609
• Add helm lint CI job. Split tag/release for the Helm chart from the entire repo with tags: helm/X.Y.Z-b1 by @andrewrothstein in #1602
• Publish storage worker and execute worker pool size in prometheus by @amishra-u in #1606
• bf-cat Output salt/platform, deindent command by @werkt in #1615
• Provide hashedName in BackplaneStatus Queue Name by @werkt in #1616
• set expire and drop invocationId by @coder1363691 in #1589
• Add OSSF scorecard by @jasonschroeder-sfdc in #1618
• chore: remove rules_oss_audit by @jasonschroeder-sfdc in #1617
• Changes to upgrade to mockito 5.10.0 by @werkt in #1623
• Fix com_grail_bazel_toolchain fetch by @werkt in #1630
• fix template bugs with with by @andrewrothstein in #1631
• Update llvm-toolchains repo url by @werkt in #1633
• Describe maxSizeBytes default and sentinel by @werkt in #1636
• migrate dependency to Bitnami Redis helm chart by @andrewrothstein in #1637

New Contributors

• @coder1363691 made their first contribution in #1589

Full Changelog: 2.8.0...2.9.0

Docker Containers

docker pull bazelbuild/buildfarm-server:2.9.0
docker pull bazelbuild/buildfarm-worker:2.9.0

v2.8.0

What's Changed

• Re-add missing graceful shutdown functionality by @80degreeswest in #1520
• Incremental changes for read through by @werkt in #1521
• Configured output size operation failure by @werkt in #1524
• Restore abbrev port as -p by @werkt in #1523
• Update zstd-jni for latest version by @jerrymarino in #1518
• Attempt to resolve windows stamping by @werkt in #1528
• Bug: Fix workerSet update logic for RemoteCasWriter by @amishra-u in #1522
• Check for context cancelled before responding to error by @justinwon777 in #1526
• chore(deps): bump com.google.errorprone:error-prone by @jasonschroeder-sfdc in #1503
• Worker name execution properties matching by @werkt in #1533
• [execution] allow tmpfs and cgroups enforcement by @luxe in #1319
• Release resources when not keeping an operation by @werkt in #1535
• Implement custom label header support for Grpc metrics interceptor by @rastenis in #1530
• Specify direct guava dependency usage by @werkt in #1538
• Update lombok dependency for jdk21 by @werkt in #1540
• Reorganize DequeueMatchEvaluator by @werkt in #1537
• Upgrade com_google_protobuf for jvm compatibility by @werkt in #1539
• Create buildfarm-worker-base-build-and-deploy.yml by @80degreeswest in #1534
• Add base image generation scripts by @80degreeswest in #1532
• Fix buildfarm-worker-base-build-and-deploy.yml by @80degreeswest in #1541
• Add public buildfarm image generation actions by @80degreeswest in #1542
• Update base image building action by @80degreeswest in #1544
• Add release image generation action by @80degreeswest in #1545
• Limit workflow to canonical repository by @werkt in #1547
• Check for "cores" exec property as min-cores match by @werkt in #1548

New Contributors

• @justinwon777 made their first contribution in #1526
• @rastenis made their first contribution in #1530

Full Changelog: 2.7.0...2.8.0

Docker Containers

docker pull bazelbuild/buildfarm-server:2.8.0
docker pull bazelbuild/buildfarm-worker:2.8.0

v2.7.0

What's Changed

• Manipulate worker set directly in RSB by @werkt in #1466
• Remove publishTtlMetric option by @werkt in #1467
• Config-compatible behavior for publishTtlMetric by @werkt in #1476
• Correct logging advisements for current Java by @werkt in #1468
• Rename GracefulShutdownTest by @werkt in #1471
• Remove WebController by @werkt in #1472
• Interrupt+Join operationQueuer/dispatchMonitor by @werkt in #1473
• Present operationNames by stage by @werkt in #1474
• Remove subscribeToBackplane, adjust failsafe op by @werkt in #1469
• Removing AWS/GCP Metrics and Admin controls by @werkt in #1470
• Remove unused setOnCancelHandler by @werkt in #1475
• Update BWoB docs for ensureOutputsPresent by @werkt in #1477
• Disable Bzlmod explicitly in .bazelrc by @meteorcloudy in #1480
• Log write errors with worker address by @amishra-u in #1464
• Revert "Use integer ids for Sqlite bidirectional index" by @80degreeswest in #1489
• Common String.format for PipelineStage by @werkt in #1491
• Cleanup matched logic in SWC listener by @werkt in #1492
• Refactor SWC matcher and clarify Nullable by @werkt in #1493
• Interrupt matchStage to induce prepare shutdown by @werkt in #1494
• Specify example config with grpc target by @werkt in #1496
• Remove SpringBoot usage by @werkt in #1495
• Enable graceful shutdown for server by @80degreeswest in #1490
• refactor: code cleanup by @jasonschroeder-sfdc in #1487
• Log paths created on putDirectory by @werkt in #1499
• Switch to positive check for linkInputDirectories by @werkt in #1500
• docs: Configuration docs cleanup by @jasonschroeder-sfdc in #1498
• feat: support --redis_uri command line option by @jasonschroeder-sfdc in #1488
• Example should use container_image instead of java_image by @sousajo-cc in #1497
• chore: deps.bzl updates by @jasonschroeder-sfdc in #1451
• Implement local resources for workers by @luxe in #1282
• fix: Bump protobuf runtime by @jasonschroeder-sfdc in #1504
• Bug: Fix Blocked thread in WriteStreamObserver Caused by CASFile Write by @amishra-u in #1486
• Pin the Java toolchain to remotejdk_17 by @stefanobaghino in #1509
• docs: add markdown language specifiers for code blocks by @jasonschroeder-sfdc in #1506
• Support OutputPaths in OutputDirectory by @werkt in #1511
• Permit Absolute Symlink Targets with configuration by @werkt in #1512
• chore: update bazel to 6.4.0 by @jasonschroeder-sfdc in #1513
• Rename instance types by @luxe in #1514
• Create SymlinkNode outputs during upload by @werkt in #1515
• feat: Implement CAS lease extension by @amishra-u in #1455
• Bump org.json:json from 20230227 to 20231013 in /admin/main by @dependabot in #1516

New Contributors

• @sousajo-cc made their first contribution in #1497
• @stefanobaghino made their first contribution in #1509

Full Changelog: 2.6.0...2.7.0

Docker Containers

docker pull bazelbuild/buildfarm-server:v2.7.0
docker pull bazelbuild/buildfarm-worker:v2.7.0
docker pull bazelbuild/buildfarm-admin:v2.7.0

v2.6.1

This release reverts commit below, which causes issues with remote execution once CAS limit is reached.

1283508 Revert "Use integer ids for Sqlite bidirectional index"

Docker Containers

docker pull bazelbuild/buildfarm-server:v2.6.1
docker pull bazelbuild/buildfarm-worker:v2.6.1
docker pull bazelbuild/buildfarm-admin:v2.6.1

v2.6.0

What's Changed

• Fix io_bytes_read metrics for buildfarm:server by @amishra-u in #1416
• chore(deps): bump Guava from 31.1-jre to 32.1.1-jre by @jasonschroeder-sfdc in #1415
• Upgrade opentelemetry javaagent to 1.28 by @tokongs in #1419
• Update platforms by @keith in #1422
• Add download rate metrics for buildfarm:worker by @amishra-u in #1418
• Add request metadata interceptor to Worker by @amishra-u in #1425
• Separate channel for write api by @amishra-u in #1424
• fix docs about CAS-storages by @Stunkymonkey in #1432
• Fix deadlock when handling Write request by @shirchen in #1442
• Deliver RemoteCasWriter IOExceptions by @werkt in #1438
• build: update maven mirrors by @jasonschroeder-sfdc in #1435
• Add a production-ready helm chart for deploying buildfarm on k8s by @lshmouse in #1431
• Update quick_start.md by @werkt in #1448
• Permit --prometheus_port to override config by @werkt in #1450
• [metrics] Emit operation_exit_code metric to track execution exit codes by @80degreeswest in #1444
• feat(redis): support rediss:// URIs for Redis-SSL by @jasonschroeder-sfdc in #1436
• Update rules_docker dependencies via injection by @werkt in #1454
• Treat '.' working_directory segment as current by @werkt in #1456
• Remove unused setExecuteResponseBuilder by @werkt in #1457
• Log on write errors by @werkt in #1459
• Use integer ids for Sqlite bidirectional index by @werkt in #1460
• Update graceful worker shutdown functionality to better handle worker terminations by @80degreeswest in #1462

New Contributors

• @jasonschroeder-sfdc made their first contribution in #1415
• @tokongs made their first contribution in #1419
• @Stunkymonkey made their first contribution in #1432
• @lshmouse made their first contribution in #1431

Full Changelog: 2.5.0...2.6.0

Docker Containers

docker pull bazelbuild/buildfarm-server:v2.6.0
docker pull bazelbuild/buildfarm-worker:v2.6.0
docker pull bazelbuild/buildfarm-admin:v2.6.0

v2.5.0

What's Changed

• Fix: custom latency buckets #1376 by @amishra-u in #1396
• Supply a basic retrier to remote cas writes by @werkt in #1398
• Update buildfarm-indexer for upstream updates by @werkt in #1400
• Commonize grpc createChannel with TLS selection by @werkt in #1401
• Refactor findMissingBlobs method by @amishra-u in #1395
• Upgrade grpc repo/maven deps for java_common by @werkt in #1404
• Shut down prometheus collector thread on CFC stop by @werkt in #1410
• Output status code name on shard read error by @werkt in #1413
• Reduce log levels to make log files more meaningful by @80degreeswest in #1414

Full Changelog: 2.4.0...2.5.0

Docker Containers

docker pull bazelbuild/buildfarm-server:v2.5.0
docker pull bazelbuild/buildfarm-worker:v2.5.0
docker pull bazelbuild/buildfarm-admin:v2.5.0

v2.4.0

Deployment Notes

Due to the nature of some changes in this release, upgrade to this version will require a full switchover of all servers and workers (blue/green deployment). Do not perform a rolling upgrade.

What's Changed

• Handle null Throwable message in asExecutionStatus by @werkt in #1357
• Update docs to include redis requirement by @werkt in #1359
• Update worker image tag to jammy-java11-gcc by @werkt in #1362
• [docs]Update troubleshooting-bazel-remote-execution.md by @jacobmou in #1361
• Remove unused ActionAmounts by @werkt in #1366
• Correct executeWorkers removal and expire by @werkt in #1367
• Adjust Server/Worker configs after parser by @werkt in #1370
• Handle oversized FMBs in StubInstance by splitting by @werkt in #1377
• Missing directories not visited in validation by @werkt in #1378
• Guard directory revisit with empty/missing checks by @werkt in #1379
• Output additions to RequestMetadata by @werkt in #1380
• SEVERE for app.run failure by @werkt in #1382
• Enable custom latency buckets by @amishra-u in #1376
• Require fileStore for Directories interactions by @werkt in #1385
• Unwrap EEs on PutDirectoryException creation by @werkt in #1386
• Retry downloads in CFC with copyExternalInput by @werkt in #1387
• Skip isReadOnlyExecutable on symlinks by @werkt in #1383
• Support UTF8 Strings in ffiReaddir by @werkt in #1388
• Revert context deadline guarantee from (#1377)" by @werkt in #1389
• Rollback ensure deadline by @werkt in #1390
• Interpret ExecDirException to Status by @werkt in #1391
• Initialize fileStore when skipping load by @werkt in #1392
• Handle inaccurate information in backplane for fmb call by @amishra-u in #1381
• Resolve commit vs expire race by @werkt in #1384
• Exit server on transform token exhaustion by @werkt in #1368

Full Changelog: 2.3.1...2.4.0

Docker Containers

docker pull bazelbuild/buildfarm-server:v2.4.0
docker pull bazelbuild/buildfarm-worker:v2.4.0
docker pull bazelbuild/buildfarm-admin:v2.4.0