Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trying to snapshot all indices fails as Curator accesses system indices #1683

Open
WilliamDEdwards opened this issue Jul 8, 2023 · 0 comments

Comments

@WilliamDEdwards
Copy link

Expected Behavior

When trying to create a snapshot of all indices with Curator, such as with the following config, no error should occur.

---
actions:
  1:
    action: snapshot
    description: >-
      Snapshot selected indices to 'repository' with the snapshot name or name
      pattern in 'name'.  Use all other options as assigned
    options:
      repository: s3-dadup
      # Leaving name blank will result in the default 'curator-%Y%m%d%H%M%S'
      name:
      ignore_unavailable: False
      include_global_state: True
      partial: False
      wait_for_completion: True
      skip_repo_fs_check: False
      disable_action: False
    filters:
    - filtertype: none

Actual Behavior

Elasticsearch 8.x deprecated direct access to system indices. This causes the following error:

2023-07-08 10:33:48,810 ERROR     Failed to complete action: snapshot.  <class 'curator.exceptions.FailedExecution'>: Exception encountered.  Rerun with loglevel DEBUG and/or check Elasticsearch logs formore information. Exception: BadRequestError(400, 'illegal_argument_exception', 'the [indices] parameter includes system indices [.kibana_7.13.3_001, .kibana_7.12.1_001, .kibana_7.14.0_001, .kibana_task_manager_7.17.7_001, .kibana_task_manager_7.14.0_001, .kibana_task_manager_7.12.1_001, .kibana_7.17.6_001, .kibana_task_manager_7.17.1_001, .kibana_7.12.0_001, .kibana_task_manager_7.13.3_001, .kibana_7.15.1_001, .kibana_7.16.3_001, .kibana_7.17.5_001, .kibana_task_manager_7.12.0_001, .security-6-reindexed-for-8, .kibana_task_manager_7.16.3_001, .kibana_7.17.1_001, .kibana_task_manager_7.15.1_001, .kibana_7.17.9_001, .transform-internal-007, .fleet-enrollment-api-keys-7, .kibana_7.17.11_001, .transform-internal-006, .kibana_task_manager_7.17.9_001, .transform-internal-005, .kibana_task_manager_7.17.5_001, .kibana_1-reindexed-for-8, .kibana_task_manager_7.17.11_001, .kibana_task_manager_7.17.3_001, .kibana_7.16.1_001, .kibana_task_manager_7.17.6_001, .kibana_task_manager_7.16.1_001, .kibana_7.17.7_001, .fleet-policies-7, .kibana_7.17.3_001]; to include or exclude system indices from a snapshot, use the [include_global_state] or [feature_states] parameters')

Steps to Reproduce the Problem

  1. Install Curator 8.x
  2. Install Elasticsearch 8.x
  3. Run the snapshot config above

Specifications

  • Version: 8.0.4
  • Platform: Debian 11
  • Subsystem: N/A

Context (Environment)

N/A

Detailed Description

I believe Curator should skip system indices, or make that configurable in case the user created an Elasticsearch user that does have access using the allow_restricted_indices permission. Note that I am using the built-in elastic user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant