Tempo trace query is recursively shifting the time on every new trace search

[AlissonRS]

Describe the bug
When searching for traces, you click on a trace from the list, a right panel appears to query by trace id.

If traceQuery.timeShiftEnabled is enabled, it shifts the time range by subtracting time from the range.from and adding to range.to based on the spanStartTimeShift and spanEndTimeShift. However, on Grafana UI, the original calendar component (from the trace list search on the left side panel) also gets shifted. So every new trace keeps shifting the time recursively.

To Reproduce
Steps to reproduce the behavior:

1. Provision Tempo datasource on Grafana using the configuration below taken from Grafana Docs

  apiVersion: 1
  
  datasources:
    - name: Tempo
      type: tempo
      uid: EbPG8fYoz
      url: http://localhost:3200
      access: proxy
      basicAuth: false
      jsonData:
        tracesToLogsV2:
          # Field with an internal link pointing to a logs data source in Grafana.
          # datasourceUid value must match the uid value of the logs data source.
          datasourceUid: 'loki'
          spanStartTimeShift: '1h'
          spanEndTimeShift: '-1h'
          tags: ['job', 'instance', 'pod', 'namespace']
          filterByTraceID: false
          filterBySpanID: false
          customQuery: true
          query: 'method="${__span.tags.method}"'
        tracesToMetrics:
          datasourceUid: 'prom'
          spanStartTimeShift: '1h'
          spanEndTimeShift: '-1h'
          tags: [{ key: 'service.name', value: 'service' }, { key: 'job' }]
          queries:
            - name: 'Sample query'
              query: 'sum(rate(traces_spanmetrics_latency_bucket{$$__tags}[5m]))'
        tracesToProfiles:
          datasourceUid: 'grafana-pyroscope-datasource'
          tags: ['job', 'instance', 'pod', 'namespace']
          profileTypeId: 'process_cpu:cpu:nanoseconds:cpu:nanoseconds'
          customQuery: true
          query: 'method="${__span.tags.method}"'
        serviceMap:
          datasourceUid: 'prometheus'
        nodeGraph:
          enabled: true
        search:
          hide: false
        lokiSearch:
          datasourceUid: 'loki'
        traceQuery:
          timeShiftEnabled: true
          spanStartTimeShift: '1h'
          spanEndTimeShift: '-1h'
        spanBar:
          type: 'Tag'
          tag: 'http.path'

2. Open to Tempo datasource on Grafana Explorer, search for some traces (e.g. last 15 minutes).
   

3. Click on a trace, the time range on the calendar component will be decreased by one hour on "from" and "to", because the spanStartTimeShift is "1h" and it gets subtracted, and spanEndTimeShift is "-1h" so it adds -1h, essentially subtracting too (in my screenshot, my spanEndTimeShift is "1h" not "-1h", so it's indeed adding 1h to the end instead of subtracting).
   

4. Click another trace from the left panel, the time range should go down by another hour.
   

5. Keep clicking different traces and notice the calendar time being shifted every time.
   

Expected behavior
The time shift should happen based on the original time range used for querying, shifting shouldn't be recursive.

Environment:

• Infrastructure: Kubernetes
• Deployment tool: helm

Additional Context
I can share some screenshots if needed.

[AlissonRS]

On a side note, the issue might be caused by the fact when the UI applies the time shift on the right panel calendar (for the trace query by ID), it also updates the left panel calendar (for the trace list).

So the two calendars seem to be sharing the same state. If I have some free time, I can try running it locally and raising a PR.

[joe-elliott]

Thank you for the detailed issue! This is a Grafana issue though. I will move it over there and tag it appropriately.