feat: update variables in metric names as labels

[ssheng]

What does this PR address?

BentoML metrics name used to encode service names, runner names, and worker index etc, into the metric names. This practice however is not friendly to setting up dashboards in Prometheus and Grafana. This PR moves all variables parts in the metric names as labels and hardcode bentoml as the prefix. This PR also maintains backward compatibility by continue to produce metrics in the legacy format.

Legacy metrics:

BENTOML_iris_classifier_request_total
BENTOML_iris_classifier_request_duration_seconds_sum
BENTOML_iris_classifier_request_duration_seconds_bucket
BENTOML_iris_classifier_request_duration_seconds_count

New metrics:

bentoml_request_total
bentoml_request_duration_seconds_sum
bentoml_request_duration_seconds_bucket
bentoml_request_duration_seconds_count

Both sets of the metrics will be supported until we can phase out the legacy one.

Fixes #2963

Before submitting:

• Does the Pull Request follow Conventional Commits specification naming? Here are GitHub's
  guide on how to create a pull request.
• Does the code follow BentoML's code style, both make format and make lint script have passed (instructions)?
• Did you read through contribution guidelines and follow development guidelines?
• Did your changes require updates to the documentation? Have you updated
  those accordingly? Here are documentation guidelines and tips on writting docs.
• Did you write tests to cover your changes?

Who can help review?

Feel free to tag members/contributors who can help review your PR.

[codecov]

Codecov Report

Merging #2969 (42c188b) into main (43c4eae) will decrease coverage by 0.63%.
The diff coverage is 72.34%.

@@            Coverage Diff             @@
##             main    #2969      +/-   ##
==========================================
- Coverage   70.60%   69.96%   -0.64%     
==========================================
  Files         104      104              
  Lines        9538     9683     +145     
==========================================
+ Hits         6734     6775      +41     
- Misses       2804     2908     +104     

Impacted Files	Coverage Δ
bentoml/_internal/server/metrics/prometheus.py	78.94% <ø> (-9.12%)	⬇️
bentoml/_internal/marshal/dispatcher.py	65.58% <70.37%> (-18.85%)	⬇️
bentoml/_internal/server/instruments.py	63.29% <72.22%> (-36.71%)	⬇️
bentoml/_internal/configuration/containers.py	75.98% <100.00%> (-2.53%)	⬇️
bentoml/_internal/utils/metrics.py	96.66% <100.00%> (ø)

[aarnphm]

Where do we hardcode the namespace for metrics?
https://prometheus.io/docs/practices/naming/#metric-names

[aarnphm]

addressed in #3008