fix(broker): do not log error if follower fails to take snapshot when log is not uptodate

[deepthidevaki]

Description

Previously, this case was logged as error. Now AsyncSnapshotDirector treat it as a special case, log it in debug level.

Alternative options where to prevent taking snapshot in such cases. But it is very difficult to identify this case. All options prevented taking snapshots in scenarios where we want to take snapshot.

1. One option was to return -1 as processed position in ReplayStateMachine when no events after snapshot position is replayed. But this also prevents taking snapshot when there are no new records, but the exporter position increases.
2. Another options was to skip snapshot in StateController, when the determined snapshot position <= position in latest snapshot. This would not fix the bug, because if the exporter position has changed, it will still try to take snapshot and enter the same situation where it cannot find the indexed entry. Other options were attempted, but all failed in one or the other edge case.

So, the easiest solution is to treat it as a special case in AsyncSnapshotDirector.

Related issues

closes #7911

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

• The changes are backwards compatibility with previous versions
• If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/1.3) to the PR, in case that fails you need to create backports manually.

Testing:

• There are unit/integration tests that verify all acceptance criterias of the issue
• New tests are written to ensure backwards compatibility with further versions
• The behavior is tested manually
• The change has been verified by a QA run
• The impact of the changes is verified by a benchmark

Documentation:

• The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
• New content is added to the release announcement
• If the PR changes how BPMN processes are validated (e.g. support new BPMN element) then the Camunda modeling team should be informed to adjust the BPMN linting.

Please refer to our review guidelines.

[github-actions]

Unit Test Results

   783 files  ±  0     783 suites  ±0   1h 33m 26s ⏱️ + 4m 44s
5 609 tests  - 54  5 602 ✔️  - 54  7 💤 ±0  0 ❌ ±0 
5 781 runs   - 54  5 774 ✔️  - 54  7 💤 ±0  0 ❌ ±0 

Results for commit 372367d. ± Comparison against base commit 5828b99.

♻️ This comment has been updated with latest results.

[lenaschoenburg]

Thanks @deepthidevaki, nice to have fewer error logs!

Just a couple of optional suggestions, looks good otherwise.

[deepthidevaki]

bors merge

[zeebe-bors-camunda]

Build succeeded:

• Test summary
• Java checks
• Go linting

[backport-action]

Successfully created backport PR #9635 for stable/1.3.

[backport-action]

Successfully created backport PR #9636 for stable/8.0.