Skip to content

Commit

Permalink
Doc:Update qa readme with BUILD_JAVA_HOME setting info [8.1] (#13778)
Browse files Browse the repository at this point in the history
Backports #11985 to 8.1
Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
  • Loading branch information
karenzone committed Feb 16, 2022
1 parent ea1c8f6 commit afd5aee
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 33 deletions.
10 changes: 5 additions & 5 deletions qa/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
## Acceptance test Framework

Welcome to the acceptance test framework for logstash, in this small
README we're going to describe it's features and the necessary steps you will need to
follow to setup your environment.
Welcome to the acceptance test framework for Logstash. In this small README we
describe its features and the steps necessary for setting up your environment.

### Setup your environment

Expand All @@ -22,6 +21,7 @@ Is important to notice that the first time you set everything up, or when a
new VM is added, there is the need to download the box (this will
take a while depending on your internet speed).


### Running Tests

It is possible to run the full suite of the acceptance test with the codebase by
Expand Down Expand Up @@ -192,10 +192,10 @@ tests, a collection of them using filtering, etc.

Check https://relishapp.com/rspec/rspec-core/v/3-4/docs/command-line for more details, but here is a quick cheat sheet to run them:

# Run the examples that get "is installed" in their description
#### Run the examples that get "is installed" in their description

* bundle exec rspec acceptance/spec -e "is installed"

# Run the example defined at line 11
#### Run the example defined at line 11

* bundle exec rspec acceptance/spec/lib/artifact_operation_spec.rb:11
83 changes: 55 additions & 28 deletions qa/integration/README.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,77 @@
## Logstash Integration Tests aka RATS

These set of tests are full integration tests as in: they can start LS from a binary, run configs using `-e` and can use any external services like Kafka, ES and S3. This framework is hybrid -- a combination of bash scripts (to mainly setup services), Ruby service files, and RSpec. All test assertions are done in RSpec.
These test sets are full integration tests. They can:

* start Logstash from a binary,
* run configs using `-e`, and
* use external services such as Kafka, Elasticsearch, and Beats.

This framework is hybrid -- a combination of bash scripts (to mainly setup services), Ruby service files, and RSpec. All test assertions are done in RSpec.

## Running integration tests locally (Mac/Linux)
## Environment setup

### Directory Layout

* `fixtures`: Specify services to run, Logstash config, and test specific scripts ala `.travis.yml`. You test settings in form of `test_name.yml`.
* `services`: This directory has bash scripts that download and bootstrap binaries for services. This is where services like Elasticsearch will be downloaded and run. Service can have 3 files: `<service>_setup.sh`, `<service>_teardown.sh` and `<service>`.rb. The bash scripts deal with downloading and bootstrapping, but the ruby source will trigger them from the test as a shell out (using backticks). The tests are blocked until the setup/teardown completes. For example, Elasticsearch service has `elasticsearch_setup.sh`, `elasticsearch_teardown.sh` and `elasticsearch.rb`. The service name in yml is "elasticsearch".
* `framework`: Test framework source code.
* `specs`: Rspec tests that use services and validates stuff

### Setup Java

The integration test scripts use `gradle` to run the tests.
Gradle requires a valid version of Java either on the system path, or specified using the `JAVA_HOME` environment variable pointing to the location of a valid JDK.

To run integration tests using a different version of Java, set the `BUILD_JAVA_HOME` environment variable to the location of the JDK that you wish to test with.
## Testing on Mac/Linux

### Dependencies
* `JRuby`
* `rspec`
* `rake`
* `bundler`

From the Logstash root directory:
### Running integration tests locally (Mac/Linux)
Run tests from the Logstash root directory.

* Run all tests:

`ci/integration_tests.sh`

* Run a single test:

* Run all tests: `ci/integration_tests.sh`
* Run a single test: `ci/integration_tests.sh specs/es_output_how_spec.rb`
`ci/integration_tests.sh specs/es_output_how_spec.rb`
* Debug tests:
```
ci/integration_tests.sh setup
cd qa/integration
bundle exec rspec specs/es_output_how_spec.rb (single test)
bundle exec rspec specs/* (all tests)
```
## Running integration tests locally via Docker
```
ci/integration_tests.sh setup
cd qa/integration
bundle exec rspec specs/es_output_how_spec.rb (single test)
bundle exec rspec specs/* (all tests)
```

## Testing with Docker

### Dependencies
* `Docker`

From the Logstash root directory:
### Running integration tests locally using Docker

Run tests from the Logstash root directory.

* Run all tests:
```
docker build -t logstash-integration-tests .
docker run -it --rm logstash-integration-tests ci/integration_tests.sh
```

```
docker build -t logstash-integration-tests .
docker run -it --rm logstash-integration-tests ci/integration_tests.sh
```

* Run a single test:
```
docker build -t logstash-integration-tests .
docker run -it --rm logstash-integration-tests ci/integration_tests.sh specs/es_output_how_spec.rb
```

* Debug tests:
```
(Mac/Linux) docker ps --all -q -f status=exited | xargs docker rm
Expand All @@ -54,24 +86,19 @@ exit
docker kill debug
docker rm debug
```
## Running integration tests locally from Windows

The integration tests need to be run from MacOS or Linux. However, the tests may be run locally within Docker.

## Docker clean up (Mac/Linux)
### Docker clean up (Mac/Linux)

! Warning this will remove all images and containers except for the `logstash-base` container !
WARNING: Docker cleanup removes all images and containers except for the `logstash-base` container!

* `ci/docker_prune.sh`

### Directory Layout
## Testing on Windows

The integration tests should be run from MacOS or Linux. However, the tests can be run locally within Docker on Windows.

* `fixtures`: In this dir you will test settings in form of `test_name.yml`. Here you specify services to run, LS config, test specific scripts ala `.travis.yml`
* `services`: This dir has bash scripts that download and bootstrap binaries for services. This is where services like ES will be downloaded and run from. Service can have 3 files: `<service>_setup.sh`, `<service>_teardown.sh` and `<service>`.rb. The bash scripts deal with downloading and bootstrapping, but the ruby source will trigger them from the test as a shell out (using backticks). The tests are blocked until the setup/teardown completes. For example, Elasticsearch service has `elasticsearch_setup.sh`, `elasticsearch_teardown.sh` and `elasticsearch.rb`. The service name in yml is "elasticsearch".
* `framework`: Test framework source code.
* `specs`: Rspec tests that use services and validates stuff

### Adding a new test
## Adding a new test

1. Creating a new test -- lets use as example. Call it "test_file_input" which brings up LS to read from a file and assert file contents (file output) were as expected.
2. You'll have to create a yml file in `fixtures` called `test_file_input_spec.yml`. Here you define any external services you need and any LS config.
Expand Down

0 comments on commit afd5aee

Please sign in to comment.