Merge 4.6.0 into 4.7.0 (#4349)

* feat(#3693): add cases and configuration files * feat(#3693): add test_registry_wildcards module * feat(#3693): add new callbacks and event_monitor * docs(#3693): update changelog.md * style(#3693): fix whitelines * Merge 4.5.2 into 4.6.0 (#4348) * feat(#4281): New invalid decoder test case for wazuh-logtest * fix(#4281): Fix invalid_decoder_syntax.yaml file line lengths * feat(#4325): upgrade pyyaml to 6.0.1 * feat: bump version 4.5.2 * refactor(#4344): Add space to version json * feat(#4344): add Release section --------- Co-authored-by: Vikman Fernandez-Castro <vmfdez90@gmail.com> Co-authored-by: Victor M. Fernandez-Castro <victor@wazuh.com> Co-authored-by: jnasselle <jnasselle@gmail.com> Co-authored-by: Julia <juliamgnr@gmail.com> Co-authored-by: Julia Magán <80041853+juliamagan@users.noreply.github.com> Co-authored-by: David Jose Iglesias Lopez <davidj.iglesias@wazuh.com> Co-authored-by: Víctor Rebollo Pérez <victorrebollop@gmail.com> * Move 4.5.0 `CHANGELOG.md` changes to 4.6.0 (#4331) * Move changes of 4.5.0 in CHANGELOG.md --------- Co-authored-by: Deblintrake09 <amicalizzi2005@gmail.com> Co-authored-by: Victor M. Fernandez-Castro <victor@wazuh.com> Co-authored-by: Vikman Fernandez-Castro <vmfdez90@gmail.com> Co-authored-by: jnasselle <jnasselle@gmail.com> Co-authored-by: Julia <juliamgnr@gmail.com> Co-authored-by: Julia Magán <80041853+juliamagan@users.noreply.github.com> Co-authored-by: David Jose Iglesias Lopez <davidj.iglesias@wazuh.com> Co-authored-by: Víctor Rebollo Pérez <victorrebollop@gmail.com>
wazuh · Jul 21, 2023 · 2b7adff · 2b7adff
1 parent a20c999
commit 2b7adff
Show file tree

Hide file tree

Showing 15 changed files with 534 additions and 37 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -26,19 +26,6 @@ Release report: TBD
 Wazuh commit: TBD \
 Release report: TBD
 
-### Fixed
-- Update schema database version ([#4128](https://github.com/wazuh/wazuh-qa/pull/4128)) \- (Tests)
-
-## [4.5.1] - TBD
-
-Wazuh commit: TBD \
-Release report: TBD
-
-## [4.5.0] - TBD
-
-Wazuh commit: TBD \
-Release report: TBD
-
 ### Added
 
 - New 'SCA' test suite and framework. ([#3566](https://github.com/wazuh/wazuh-qa/pull/3566)) \- (Framework + Tests)
@@ -52,6 +39,7 @@ Release report: TBD
 - Add new tests for logcollector 'ignore' and 'restrict' options ([#3582](https://github.com/wazuh/wazuh-qa/pull/3582)) \- (Tests)
 - Add 'Force reconnect' feature to agent_simulator tool. ([#3111](https://github.com/wazuh/wazuh-qa/pull/3111)) \- (Tools)
 - Add new module to support migration tool. ([#3837](https://github.com/wazuh/wazuh-qa/pull/3837))
+- Add IT tests FIM registry monitoring using wildcards. ([#4270](https://github.com/wazuh/wazuh-qa/pull/4270)) \- (Framework + Tests)
 
 ### Changed
 
@@ -74,6 +62,8 @@ Release report: TBD
 - Update Authd force_insert tests ([#3379](https://github.com/wazuh/wazuh-qa/pull/3379)) \- (Tests)
 - Update cluster logs in reliability tests ([#2772](https://github.com/wazuh/wazuh-qa/pull/2772)) \- (Tests)
 - Use correct version format in agent_simulator tool ([#3198](https://github.com/wazuh/wazuh-qa/pull/3198)) \- (Tools)
+- Upgrade PyYAML to 6.0.1. ([#4326](https://github.com/wazuh/wazuh-qa/pull/4326)) \- (Framework)
+- Update schema database version ([#4128](https://github.com/wazuh/wazuh-qa/pull/4128)) \- (Tests)
 
 ### Fixed
 
@@ -85,6 +75,16 @@ Release report: TBD
 - Fix an error in the cluster performance tests related to CSV parser ([#2999](https://github.com/wazuh/wazuh-qa/pull/2999)) \- (Framework + Tests)
 - Fix bug in the framework on migration tool ([#4027](https://github.com/wazuh/wazuh-qa/pull/4027)) \- (Framework)
 
+## [4.5.1] - TBD
+
+Wazuh commit: TBD \
+Release report: TBD
+
+## [4.5.0] - TBD
+
+Wazuh commit: TBD \
+Release report: TBD
+
 ## [4.4.5] - 10-07-2023
 
 Wazuh commit: https://github.com/wazuh/wazuh/commit/8d17d2c9c11bc10be9a31c83bc7c17dfbac0d2a0 \

diff --git a/deps/wazuh_testing/wazuh_testing/modules/fim/__init__.py b/deps/wazuh_testing/wazuh_testing/modules/fim/__init__.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2015-2022, Wazuh Inc.
+# Copyright (C) 2015-2023, Wazuh Inc.
 # Created by Wazuh, Inc. <info@wazuh.com>.
 # This program is free software; you can redistribute it and/or modify it under the terms of GPLv2
 

diff --git a/deps/wazuh_testing/wazuh_testing/modules/fim/event_monitor.py b/deps/wazuh_testing/wazuh_testing/modules/fim/event_monitor.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2015-2022, Wazuh Inc.
+# Copyright (C) 2015-2023, Wazuh Inc.
 # Created by Wazuh, Inc. <info@wazuh.com>.
 # This program is free software; you can redistribute it and/or modify it under the terms of GPLv2
 
@@ -7,7 +7,7 @@
 
 from sys import platform
 from datetime import datetime
-from wazuh_testing import LOG_FILE_PATH, logger, T_60
+from wazuh_testing import LOG_FILE_PATH, logger, T_60, T_30
 from wazuh_testing.tools.monitoring import FileMonitor, generate_monitoring_callback
 
 
@@ -43,15 +43,14 @@
 CB_SYNC_INTERVAL_RESET = r".*Previous sync was successful. Sync interval is reset to: '(\d+)s'"
 CB_IGNORING_DUE_TO_SREGEX = r".*?Ignoring path '(.*)' due to sregex '(.*)'.*"
 CB_IGNORING_DUE_TO_PATTERN = r".*?Ignoring path '(.*)' due to pattern '(.*)'.*"
-CB_MAXIMUM_FILE_SIZE = r'.*Maximum file size limit to generate diff information configured to \'(\d+) KB\'.*'
-CB_AGENT_CONNECT = r'.* Connected to the server .*'
 CB_REALTIME_WHODATA_ENGINE_STARTED = r'.*File integrity monitoring (real-time Whodata) engine started.*'
 CB_DISK_QUOTA_LIMIT_CONFIGURED_VALUE = r'.*Maximum disk quota size limit configured to \'(\d+) KB\'.*'
 CB_FILE_EXCEEDS_DISK_QUOTA = r'.*The (.*) of the file size \'(.*)\' exceeds the disk_quota.*'
 CB_FILE_SIZE_LIMIT_REACHED = r'.*File \'(.*)\' is too big for configured maximum size to perform diff operation\.'
 CB_DIFF_FOLDER_DELETED = r'.*Folder \'(.*)\' has been deleted.*'
 CB_FIM_PATH_CONVERTED = r".*fim_adjust_path.*Convert '(.*) to '(.*)' to process the FIM events."
 CB_STARTING_WINDOWS_AUDIT = r'.*state_checker.*(Starting check of Windows Audit Policies and SACLs)'
+CB_FIM_WILDCARD_EXPANDING = r".*Expanding entry '.*' to '(.*)' to monitor FIM events."
 CB_SWITCHING_DIRECTORIES_TO_REALTIME = r'.*state_checker.*(Audit policy change detected.\
                                          Switching directories to realtime)'
 CB_RECIEVED_EVENT_4719 = r'.*win_whodata.*(Event 4719).*Switching directories to realtime'
@@ -227,6 +226,18 @@ def callback_detect_file_integrity_event(line):
     return None
 
 
+def callback_key_event(line):
+    """ Callback that detects if a line contains a registry integrity event for a registry_key
+    Args:
+        line (String): string line to be checked by callback in File_Monitor.
+    """
+    event = callback_detect_event(line)
+    if event is None or event['data']['attributes']['type'] != 'registry_key':
+        return None
+
+    return event
+
+
 def callback_value_event(line):
     event = callback_detect_event(line)
 
@@ -489,6 +500,53 @@ def detect_whodata_start(file_monitor, timeout=T_60):
                        error_message=ERR_MSG_WHODATA_ENGINE_EVENT)
 
 
+def get_messages(callback, timeout=T_30):
+    """Look for as many synchronization events as possible.
+    This function will look for the synchronization messages until a Timeout is raised or 'max_events' is reached.
+    Args:
+        callback (str): Callback to be used to detect the event.
+        timeout (int): Timeout that will be used to get the dbsync_no_data message.
+
+    Returns:
+        A list with all the events in json format.
+    """
+    wazuh_log_monitor = FileMonitor(LOG_FILE_PATH)
+    events = []
+    for _ in range(0, MAX_EVENTS_VALUE):
+        event = None
+        try:
+            event = wazuh_log_monitor.start(timeout=timeout, accum_results=1,
+                                            callback=callback,
+                                            error_message=f"Did not receive expected {callback} event").result()
+        except TimeoutError:
+            break
+        if event is not None:
+            events.append(event)
+    return events
+
+
+def check_registry_crud_event(callback, path,  timeout=T_30, type='added', arch='x32', value_name=None):
+    """Get  all events matching the callback and validate the type, path and architecture of event
+    Args:
+        callback (str): Callback to be used to detect the event.
+        path (str): path to be checked
+        timeout (int): Timeout that will be used to try and get the expected messages
+        type (str): type of event to be checked
+        arch (str): architecture of the event to be checked
+        value_name (str): name of the value to be checked
+    """
+    events = get_messages(callback=callback, timeout=timeout)
+    for event in events:
+        if event['data']['type'] == type and arch in event['data']['arch'] and event['data']['path'] == path:
+            if value_name is not None:
+                if 'value_name' in event and event['data']['value_name'] == value_name:
+                    return event
+            else:
+                return event
+
+    return None
+
+
 def detect_windows_sacl_configured(file_monitor, file='.*'):
     """Detects when windows permision checks have been configured for a given file.
 

diff --git a/deps/wazuh_testing/wazuh_testing/modules/fim/utils.py b/deps/wazuh_testing/wazuh_testing/modules/fim/utils.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2015-2022, Wazuh Inc.
+# Copyright (C) 2015-2023, Wazuh Inc.
 # Created by Wazuh, Inc. <info@wazuh.com>.
 # This program is free software; you can redistribute it and/or modify it under the terms of GPLv2
 
@@ -213,7 +213,8 @@ def modify_registry(key, subkey, arch):
         logger.info(f"Modifying registry key {print_arch}{os.path.join(fim.registry_class_name[key], subkey)}")
 
         modify_key_perms(key, subkey, arch, win32sec.LookupAccountName(None, f"{platform.node()}\\{os.getlogin()}")[0])
-        modify_registry_owner(key, subkey, arch, win32sec.LookupAccountName(None, f"{platform.node()}\\{os.getlogin()}")[0])
+        modify_registry_owner(key, subkey, arch,
+                              win32sec.LookupAccountName(None, f"{platform.node()}\\{os.getlogin()}")[0])
         modify_registry_key_mtime(key, subkey, arch)
 
 
@@ -298,6 +299,14 @@ def calculate_registry_diff_paths(reg_key, reg_subkey, arch, value_name):
 
 
 def transform_registry_list(value_list=['test_value'], value_type=fim.REG_SZ, callback=ev.callback_value_event):
+    """Transform a list of registry values into a dictionary.
+    Args:
+        value list (List): list of string value names
+        value type (str): type of registry value that is expected.
+        Callback (object): Callback to pair with the value to be monitored.
+    Returns:
+        Dict: dictionary with the values and the corresponding callbacks to monitor them.
+    """
     if sys.platform == 'win32':
         if value_type in [win32con.REG_SZ, win32con.REG_MULTI_SZ]:
             value_default_content = ''
@@ -319,6 +328,29 @@ def transform_registry_list(value_list=['test_value'], value_type=fim.REG_SZ, ca
         return aux_dict
 
 
+def transform_registry_key_list(key_list=['test_key'], callback=ev.callback_key_event):
+    """Transform a list of registry keys into a dictionary.
+    Args:
+        key_list list (List): list of strings with the key names names
+        Callback (object): Callback to pair with the key to be monitored.
+    Returns:
+        Dict: dictionary with the keys and the corresponding callbacks to monitor them.
+    """
+    if sys.platform == 'win32':
+        aux_dict = {}
+        if isinstance(key_list, list):
+            for elem in key_list:
+                aux_dict[elem] = ('', callback)
+
+        elif isinstance(key_list, dict):
+            for key, elem in key_list.items():
+                aux_dict[key] = (elem, callback)
+        else:
+            raise ValueError('It can only be a list or dictionary')
+
+        return aux_dict
+
+
 def set_check_options(options):
     """ Return set of check options. If options given is none, it will return check_all"""
     options_set = fim.REQUIRED_REG_VALUE_ATTRIBUTES[fim.CHECK_ALL]

diff --git a/deps/wazuh_testing/wazuh_testing/qa_docs/schema.yaml b/deps/wazuh_testing/wazuh_testing/qa_docs/schema.yaml
@@ -190,6 +190,7 @@ predefined_values:
     - 4.4.5
     - 4.5.0
     - 4.5.1
+    - 4.5.2
     - 4.6.0
   tags:
     - active_response

diff --git a/requirements.txt b/requirements.txt
@@ -23,7 +23,7 @@ pyOpenSSL==19.1.0
 pytest-html==3.1.1
 pytest==6.2.2 ; python_version <= "3.9"
 pytest==7.1.2 ; python_version >= "3.10"
-pyyaml==5.4
+pyyaml==6.0.1
 requests>=2.23.0
 scipy>=1.0; platform_system == "Linux" or platform_system == "Darwin" or platform_system=='Windows'
 seaborn>=0.11.1; platform_system == "Linux" or platform_system == "Darwin" or platform_system=='Windows'

diff --git a/tests/integration/test_fim/test_registry/conftest.py b/tests/integration/test_fim/test_registry/conftest.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2015-2021, Wazuh Inc.
+# Copyright (C) 2015-2023, Wazuh Inc.
 # Created by Wazuh, Inc. <info@wazuh.com>.
 # This program is free software; you can redistribute it and/or modify it under the terms of GPLv2
 

diff --git a/...test_registry_wildcards/data/configuration_template/configuration_registry_wildcards.yaml b/...test_registry_wildcards/data/configuration_template/configuration_registry_wildcards.yaml
@@ -0,0 +1,30 @@
+- sections:
+    - section: syscheck
+      elements:
+        - disabled:
+            value: 'no'
+        - frequency:
+            value: FREQUENCY
+        - windows_registry:
+            value: WINDOWS_REGISTRY
+            attributes:
+              - arch: both
+
+    - section: sca
+      elements:
+        - enabled:
+            value: 'no'
+    - section: rootcheck
+      elements:
+        - disabled:
+            value: 'yes'
+    - section: wodle
+      attributes:
+        - name: syscollector
+      elements:
+        - disabled:
+            value: 'yes'
+    - section: active-response
+      elements:
+        - disabled:
+            value: 'yes'
diff --git a/...m/test_registry/test_registry_wildcards/data/test_cases/cases_registry_key_wildcards.yaml b/...m/test_registry/test_registry_wildcards/data/test_cases/cases_registry_key_wildcards.yaml
@@ -0,0 +1,23 @@
+- name: Test key with question mark wildcard (Scheduled)
+  description: Test path with single question mark wildcard in scheduled mode
+  configuration_parameters:
+    FREQUENCY: 2
+    WINDOWS_REGISTRY: HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMA?
+  metadata:
+    fim_mode: scheduled
+
+- name: Test key with single asterisk wildcard (Scheduled)
+  description: Test path with single asterisk wildcard in scheduled mode
+  configuration_parameters:
+    FREQUENCY: 2
+    WINDOWS_REGISTRY: HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\*
+  metadata:
+    fim_mode: scheduled
+
+- name: Test key with asterisk+question mark (Scheduled)
+  description: Test path with multiple asterisks and question mark wildcards combined in scheduled mode
+  configuration_parameters:
+    FREQUENCY: 2
+    WINDOWS_REGISTRY: HKEY_LOCAL_MACHINE\*\*\PointerClas?
+  metadata:
+    fim_mode: scheduled
diff --git a/...test_registry/test_registry_wildcards/data/test_cases/cases_registry_value_wildcards.yaml b/...test_registry/test_registry_wildcards/data/test_cases/cases_registry_value_wildcards.yaml
@@ -0,0 +1,23 @@
+- name: Test value with question mark wildcard (Scheduled)
+  description: Test path with single question mark wildcard
+  configuration_parameters:
+    FREQUENCY: 2
+    WINDOWS_REGISTRY: HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\PointerClas?
+  metadata:
+    fim_mode: scheduled
+
+- name: Test value with single asterisk wildcard (Scheduled)
+  description: Test path with single asterisk wildcard in scheduled mode
+  configuration_parameters:
+    FREQUENCY: 2
+    WINDOWS_REGISTRY: HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\*
+  metadata:
+    fim_mode: scheduled
+
+- name: Test3 value with asterisk+question mark (Scheduled)
+  description: Test path with multiple asterisks and question mark wildcards combined in scheduled mode
+  configuration_parameters:
+    FREQUENCY: 2
+    WINDOWS_REGISTRY: HKEY_LOCAL_MACHINE\*\*\PointerClas?
+  metadata:
+    fim_mode: scheduled