Release 4.8.0 - RC 2 - Integration tests

[Rebits]

Description

Main release stage issue	#23405
Version	4.8.0
Release stage	RC 2
Wazuh QA: Tag	https://github.com/wazuh/wazuh-qa/tree/v4.8.0-rc2
Wazuh QA: Branch	v4.8.0-rc2
Wazuh: Tag	https://github.com/wazuh/wazuh/tree/v4.8.0-rc2
Wazuh: Branch	v4.8.0-rc2

The tests will be performed in Jenkins using centOS as the manager OS. As for the agents, Linux, Windows, Solaris, and macOS will be used as required.

Build

Build: https://ci.wazuh.info/job/Test_integration_launcher/253/

Results of the integration tests

Color	Details
🟢	The pipeline has been executed successfully
🔴	The pipeline has failed or it has errors

Test suite	Target	CentOS	Ubuntu	Windows	macOS	Solaris
active_response	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
active_response	Agent	🟢 Download evidence	N/A	🟢 Download evidence	N/A	N/A
agentd	Agent	🟢 Download evidence	N/A	🔴 Download evidence	N/A	N/A
analysisd	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
api	Manager	🔴 Download evidence	N/A	N/A	N/A	N/A
authd	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
enrollment	Agent	🟢 Download evidence	N/A	🟢 Download evidence	N/A	N/A
fim	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
fim	Agent	🟢 Download evidence	N/A	🟢 Download evidence	N/A	N/A
fim	Agent	N/A	N/A	N/A	🟢	🟢
gcloud	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
github	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
github	Agent	🟢 Download evidence	N/A	N/A	N/A	N/A
logcollector	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
logcollector	Agent	🟢 Download evidence	N/A	🔴 Download evidence	🟢 Download evidence	🟢 Download evidence
logtest	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
office_365	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
office_365	Agent	🟢 Download evidence	N/A	N/A	N/A	N/A
remoted	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
rids	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
rootcheck	Manager	🔴 Download evidence	N/A	N/A	N/A	N/A
wazuh_db	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
syscollector	Manager	🟢 Download evidence	N/A	N/A	N/A	N/A
syscollector	Agent	🟢 Couldn't get evidence. Check the pipeline.	N/A	🔴 Couldn't get evidence. Check the pipeline.	🟢 Couldn't get evidence. Check the pipeline.	N/A

Conclusion 🔴

New Issues

• Windows Agentd Integration tests fails for 4.8.0 RC2  #23465

Known issues

• Windows Logcollector agent IT fails due to errors during directory creation/removal wazuh-qa#5096
• LogCollector Test Fails for Windows Agents wazuh-qa#4840
• Failed Windows Logcollector tests on 4.8.0 Beta 2 wazuh-qa#5034
• Missing vulnerability value in the statistics in API IT  #22495
• Rootcheck test fails in 4.7.3-rc1 integration tests #22058
• Syscollector scanning IT fails for Windows agents wazuh-qa#4230

[Rebits]

GA IT

List of GA

• API:
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-api-tier-0-1.yml
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-api-tier-2.yml
• Analysisd
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-analysisd-tier-0-1.yml
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-analysisd-tier-2.yml
• Execd:
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-execd-tier-0-1-lin.yml
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-execd-tier-0-1-win.yml
• Github
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-github-tier-0-1-lin.yml
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-github-tier-0-1-win.yml
• Office365
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-office365-tier-0-1-lin.yml
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-office365-tier-0-1-win.yml
• MSGraph
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-msgraph-tier-0-1-lin.yml
• Vulnerability Detection (Disabled)
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-vulnerability_detector-tier-0-1.yml
  • https://github.com/wazuh/wazuh/actions/workflows/integration-tests-vulnerability_detector-tier-2.yml

[Rebits]

GA Executions

API 🟢

• https://github.com/wazuh/wazuh/actions/runs/9099696421
• https://github.com/wazuh/wazuh/actions/runs/9099718436

Analysisd 🟢

• https://github.com/wazuh/wazuh/actions/runs/9099727142
• https://github.com/wazuh/wazuh/actions/runs/9099732719

Execd 🟢

• https://github.com/wazuh/wazuh/actions/runs/9099738391
• https://github.com/wazuh/wazuh/actions/runs/9099743610

Github 🟢

• https://github.com/wazuh/wazuh/actions/runs/9099758667
• https://github.com/wazuh/wazuh/actions/runs/9099763896

Office365 🟢

• https://github.com/wazuh/wazuh/actions/runs/9099772613
• https://github.com/wazuh/wazuh/actions/runs/9099779649

MSGraph 🟢

• https://github.com/wazuh/wazuh/actions/runs/9099785809

[Rebits]

Results of IT from Jenkins Build

Syscollector 🟡

Failure during deployment of instances. Known issue. Reported https://github.com/wazuh/wazuh-jenkins/issues/5539
Rerun in https://ci.wazuh.info/job/Test_integration/47302

Tests fail due to a known issue: wazuh/wazuh-qa#4230

Error trace

11:42:04  Command: ["import", "/var/root/.vagrant.d/boxes/development-VAGRANTSLASH-macos-catalina/0/virtualbox/box.ovf", "--vsys", "0", "--vmname", "new_box_macos_fim_test_1582644530271_78262_1715769345026_74264", "--vsys", "0", "--unit", "12", "--disk", "/Users/jenkins/VirtualBox VMs/new_box_macos_fim_test_1582644530271_78262_1715769345026_74264/box-disk001.vmdk"]
11:42:04  
11:42:04  Stderr: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

Rootcheck 🟡

Known issue: #22058
Already migrated to GHA: #22058.

Rootcheck ITs were integrated in analysisd, which was migrated to GHA. Unmarking it from the launcher #23434 (comment)

Logcollector 🟡

Failure during teardonwn in test_logcollector/test_location/test_win_location_wildcards.py::test_win_location_wildcards[test_mixed_wildcards-]::teardown:

E           OSError: [WinError 145] The directory is not empty: 'c:\\testfolder\\subfolder'

Logcollector IT has already been migrated ti GHA. However, these tests are not present in 4.8.0 branch

Known issue: wazuh/wazuh-qa#5096

Fim specials 🟢

Failure during deployment of instances. Known issue. Reported https://github.com/wazuh/wazuh-jenkins/issues/5539
Rerun in https://ci.wazuh.info/job/Test_integration/47303/

Error trace

11:42:04  Command: ["import", "/var/root/.vagrant.d/boxes/development-VAGRANTSLASH-macos-catalina/0/virtualbox/box.ovf", "--vsys", "0", "--vmname", "new_box_macos_fim_test_1582644530271_78262_1715769345026_74264", "--vsys", "0", "--unit", "12", "--disk", "/Users/jenkins/VirtualBox VMs/new_box_macos_fim_test_1582644530271_78262_1715769345026_74264/box-disk001.vmdk"]
11:42:04  
11:42:04  Stderr: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

FIM tests passed after solving deployment error. Marking as 🟢

API 🟡

Already migrated to GHA. I suggest removing them from the launcher. Known issue #22495
Unmarking it from the launcher #23434 (comment)

Agentd Windows 🔴

Unexpected failure in Windows agentd test test_agentd/test_agentd_reconnection.py::test_agentd_reconection_enrollment_no_keys_file[tcp]

                    self.stop()
                    if error_message:
                        logger.error(error_message)
                        logger.error(f"Results accumulated: "
                                     f"{len(result) if isinstance(result, list) else 0}")
                        logger.error(f"Results expected: {accum_results}")
>                   raise TimeoutError(error_message)
E                   TimeoutError: Agent never enrolled for the first time.

Agentd IT seems to have already been migrated. However, these are not present for the 4.8.0 branch. For this reason a manual review is required. Reported in #23465

[Rebits]

I have disabled by default from the IT launcher the following IT modules:

• API
• Analysisd
• Github
• Office
• Rootcheck

All these modules have been migrated in 4.7 and 4.8 to GHA, so there is no need to run these tests in Jenkins

Approved by

• @juliamagan

[MARCOSD4]

LGTM

[juliamagan]

LGTM