CI: check external links

[chalin]

@anuraaga wrote in #989 (comment):

@chalin If it's possible for the docs build to generate a list of referenced external URLs, I think we could add a link checking job to the code samples repo when making changes to it. Does that sound possible?

Yes! I've been working on such a solution (part-time) since the spring. I'll give it more priority in 22Q1.

Related:

• Once we have external link checking in place, we should consider adding a check to guard against Intra-site links should not be external #922.

---

• Fix external links reported by link checker (^a-i) #2349
• CI: check external links (excluding GH) #2362
• CI script changes to fail on diff #2364
• CI: ignore snippet-injected GA link #2365
• Spec link fixes & normalizations:
  • [editorial] Use canonical URL to pkg.go.dev/runtime/debug#Stack opentelemetry-specification#3217
  • [editorial] Use canonical URL to wiki page in metrics/supplementary-guidelines opentelemetry-specification#3230
  • [editorial] Fix Tencent Cloud link opentelemetry-specification#3231
  • [editorial] normalize link to prometheus docs opentelemetry-specification#3232
  • [editorial] Drop Google highlight "hash" from URL in metrics/data-model opentelemetry-specification#3233
  • Bring in spec updates on next release, drop temporary htmltest ignore rules, prune refcache --> CI: external-link check follow up and link cleanup #2429
• Clean up links that shouldn't be external:
  • Relocate non-icon images from iconography #2367
  • Mount iconography 32x32 icons to static/img/logos #2369
  • Don't link to raw.github.com images and resources, deploy them with the site #663
  • Move user-images.githubusercontent.com into this repo #2373 --> CI: external-link check follow up and link cleanup #2429
• Check GitHub links too: CI: check non-resource & non-issue related links into GitHub #2366

[chalin]

In #2346 (review), @reyang wrote:

Curious - is there a CI job validating external links?

No, not as a part of CI currently. There is an NPM script available already that I setup a while back, but as you know, checking external links can be flaky, and it takes quite some time to setup (enabling the check, triaging, fixing actual broken links, adding ignore rules for those that aren't worth fixing or relevant). I just haven't had enough time to bring it online.

E.g. the spec repo has https://github.com/open-telemetry/opentelemetry-specification/actions/runs/4178102356/jobs/7236499014 and I found it useful.

Yes, thanks, I'm aware of the way it is handled in the spec (and of the complaint that it is serial in its processing and so makes it slow :)). I was hoping to avoid bring in yet another tool for link checking, given that we already use htmltest, which, by the way, has nice features for handling external links.

Enabling external link checking is 3rd on my "General top items" list (towards the bottom): #2163.

We'd probably want to run external link checking via a cron job, with some throttling. Otherwise we'd end up hitting 429 errors. Maybe it is worth considering markdown-link-check. I'll give this some more thought. Thanks!

[chalin]

FYI, running npm run check:links:all for a while returned the following:

========================================================================
ecosystem/demo/index.html
  Non-OK status: 404 --- ecosystem/demo/index.html --> https://github.com/open-telemetry/opentelemetry-demo/blob/main/docs/current_architecture.md
ecosystem/vendors/index.html
  Get "https://docs.daocloud.io/en/insight/06UserGuide/01quickstart/otel/otel/": x509: certificate has expired or is not yet valid: current time 2023-02-15T15:28:48-05:00 is after 2023-01-20T23:59:59Z --- ecosystem/vendors/index.html --> https://docs.daocloud.io/en/insight/06UserGuide/01quickstart/otel/otel/
ecosystem/integrations/index.html
  Non-OK status: 404 --- ecosystem/integrations/index.html --> https://www.jaegertracing.io/docs/latest/opentelemetry/
  Non-OK status: 404 --- ecosystem/integrations/index.html --> https://micrometer.io/docs/tracing#_micrometer_tracing_opentelemetry_setup
ecosystem/registry/index.html
  Non-OK status: 404 --- ecosystem/registry/index.html --> https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/fluentbitextension
  Non-OK status: 404 --- ecosystem/registry/index.html --> https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/promtailreceiver
docs/demo/feature-flags/index.html
  Get "http://localhost:8080/feature": dial tcp [::1]:8080: connect: connection refused --- docs/demo/feature-flags/index.html --> http://localhost:8080/feature
docs/demo/docker-deployment/index.html
  request exceeded our ExternalTimeout --- docs/demo/docker-deployment/index.html --> http://ghcr.io/open-telemetry/demo
  Get "http://localhost:8080/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/docker-deployment/index.html --> http://localhost:8080/
  Get "http://localhost:8080/grafana/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/docker-deployment/index.html --> http://localhost:8080/grafana/
  Get "http://localhost:8080/feature/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/docker-deployment/index.html --> http://localhost:8080/feature/
  Get "http://localhost:8080/loadgen/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/docker-deployment/index.html --> http://localhost:8080/loadgen/
  Get "http://localhost:8080/jaeger/ui/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/docker-deployment/index.html --> http://localhost:8080/jaeger/ui/
docs/demo/kubernetes-deployment/index.html
  Get "http://localhost:8080/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/kubernetes-deployment/index.html --> http://localhost:8080/
  Get "http://localhost:8080/grafana/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/kubernetes-deployment/index.html --> http://localhost:8080/grafana/
  Get "http://localhost:8080/feature/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/kubernetes-deployment/index.html --> http://localhost:8080/feature/
  Get "http://localhost:8080/loadgen/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/kubernetes-deployment/index.html --> http://localhost:8080/loadgen/
  Get "http://localhost:8080/jaeger/ui/": dial tcp [::1]:8080: connect: connection refused --- docs/demo/kubernetes-deployment/index.html --> http://localhost:8080/jaeger/ui/
docs/demo/scenarios/recommendation-cache/index.html
  Get "http://localhost:8080/grafana/dashboards": dial tcp [::1]:8080: connect: connection refused --- docs/demo/scenarios/recommendation-cache/index.html --> http://localhost:8080/grafana/dashboards
docs/collector/management/index.html
  Get "http://localhost:4321/": dial tcp [::1]:4321: connect: connection refused --- docs/collector/management/index.html --> http://localhost:4321/
docs/instrumentation/python/index.html
  Non-OK status: 404 --- docs/instrumentation/python/index.html --> https://open-telemetry.github.io/opentelemetry-python/benchmarks/
docs/instrumentation/js/context/index.html
  Non-OK status: 404 --- docs/instrumentation/js/context/index.html --> https://open-telemetry.github.io/opentelemetry-js-api/classes/contextapi.html
docs/instrumentation/js/getting-started/nodejs/index.html
  Get "http://localhost:8080": dial tcp [::1]:8080: connect: connection refused --- docs/instrumentation/js/getting-started/nodejs/index.html --> http://localhost:8080
  Get "http://localhost:8080": dial tcp [::1]:8080: connect: connection refused --- docs/instrumentation/js/getting-started/nodejs/index.html --> http://localhost:8080
docs/instrumentation/rust/index.html
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-api
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-sdk
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-aws
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-contrib
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-datadog
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-dynatrace
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-http
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-jaeger
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-otlp
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-prometheus
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-semantic-conventions
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-stackdriver
  Non-OK status: 404 --- docs/instrumentation/rust/index.html --> https://crates.io/crates/opentelemetry-zipkin
docs/instrumentation/java/index.html
  Non-OK status: 403 --- docs/instrumentation/java/index.html --> https://mvnrepository.com/artifact/io.opentelemetry
docs/instrumentation/java/automatic/agent-config/index.html
  Non-OK status: 404 --- docs/instrumentation/java/automatic/agent-config/index.html --> https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/javaagent-extension-api/src/main/java/io/opentelemetry/javaagent/extension/config/ConfigPropertySource.java
docs/instrumentation/ruby/index.html
  request exceeded our ExternalTimeout --- docs/instrumentation/ruby/index.html --> https://www.fulcrumapp.com/
docs/instrumentation/ruby/getting-started/index.html
  Get "http://localhost:16686": dial tcp [::1]:16686: connect: connection refused --- docs/instrumentation/ruby/getting-started/index.html --> http://localhost:16686
  Get "http://localhost:9411": dial tcp [::1]:9411: connect: connection refused --- docs/instrumentation/ruby/getting-started/index.html --> http://localhost:9411
docs/concepts/signals/metrics/index.html
  Get "https://github.com/open-telemetry/opentelemetry.io/commit/261eb3ccde2d87d268ba8587e78c378d7b331df5": dial tcp: lookup github.com on 10.0.1.1:53: write udp 10.0.1.5:57026->10.0.1.1:53: write: network is down --- docs/concepts/signals/metrics/index.html --> https://github.com/open-telemetry/opentelemetry.io/commit/261eb3ccde2d87d268ba8587e78c378d7b331df5
docs/concepts/signals/index.html
  request exceeded our ExternalTimeout --- docs/concepts/signals/index.html --> https://github.com/open-telemetry/opentelemetry.io/commit/6c7bf4f64b31ac13dd8ad80df1f1660ad9d54c83

[chalin]

Closing as we have external link checking working quite nicely now. For follow up tasks, see:

• CI: external-link check follow up and link cleanup #2429