Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test workflows #372

Merged
merged 26 commits into from Oct 6, 2023
Merged

Add test workflows #372

merged 26 commits into from Oct 6, 2023

Conversation

valentin-seitz
Copy link
Member

@valentin-seitz valentin-seitz commented Sep 5, 2023

This PR does some things:

  • adopt the dockerfile to work on ALL machines by specifing the host uid upon building time
  • fetches the tutorial ref before trying to check that out (checkout-action doesnt do it)
  • splits the testsuites runs into manual and reusable triggering
  • introduce caching via GHA cache backend

@valentin-seitz valentin-seitz marked this pull request as ready for review September 26, 2023 06:10
Copy link
Member

@MakisH MakisH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions, mostly on documentation. The main aspect I am worried about is the UID/GID hard-coding.

Caching seems to work locally. Running the same test case twice, I get:

First time:

./tools/tests [add-test-workflows]$ python3 ./systemtests.py --suites=fenics_test
Using log-level: INFO
INFO: About to run the following systemtest in the directory /home/gc/repos/precice/tutorials/runs:
 [Flow over heated plate (fluid-openfoam, solid-fenics)]
INFO: Started running Flow over heated plate (fluid-openfoam, solid-fenics),  0/1
INFO: Running Flow over heated plate (fluid-openfoam, solid-fenics) took 1524.1612270610058 seconds
INFO: Success running Flow over heated plate (fluid-openfoam, solid-fenics)
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+
| systemtest                                              | success | building time [s] | solver time [s] | fieldcompare time [s] |
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+
| Flow over heated plate (fluid-openfoam, solid-fenics)   |    1    |      1272.33      |     157.36      |         92.69         |
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+

Second time:

./tools/tests [add-test-workflows]$ python3 ./systemtests.py --suites=fenics_test
Using log-level: INFO
INFO: About to run the following systemtest in the directory /home/gc/repos/precice/tutorials/runs:
 [Flow over heated plate (fluid-openfoam, solid-fenics)]
INFO: Started running Flow over heated plate (fluid-openfoam, solid-fenics),  0/1
INFO: Running Flow over heated plate (fluid-openfoam, solid-fenics) took 178.49133943999186 seconds
INFO: Success running Flow over heated plate (fluid-openfoam, solid-fenics)
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+
| systemtest                                              | success | building time [s] | solver time [s] | fieldcompare time [s] |
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+
| Flow over heated plate (fluid-openfoam, solid-fenics)   |    1    |       1.64        |     161.08      |         14.06         |
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+

This is also reflected in the ./runs/<id>/stdout.log file, where I see CACHED in all layers before the actual solver execution.

On GitHub Actions, while the cache is updated, (see the versions next to each name), we do not hit it, so the jobs do take a long time to run (rebuilding the images). @valentin-seitz could you please post a link to the bug you mentioned?

tools/tests/docker-compose.field_compare.template.yaml Outdated Show resolved Hide resolved
.github/workflows/run_testsuite_manual.yml Show resolved Hide resolved
.github/workflows/run_testsuite_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/run_testsuite_workflow.yml Outdated Show resolved Hide resolved
@valentin-seitz
Copy link
Member Author

A few suggestions, mostly on documentation. The main aspect I am worried about is the UID/GID hard-coding.

Caching seems to work locally. Running the same test case twice, I get:

First time:

./tools/tests [add-test-workflows]$ python3 ./systemtests.py --suites=fenics_test
Using log-level: INFO
INFO: About to run the following systemtest in the directory /home/gc/repos/precice/tutorials/runs:
 [Flow over heated plate (fluid-openfoam, solid-fenics)]
INFO: Started running Flow over heated plate (fluid-openfoam, solid-fenics),  0/1
INFO: Running Flow over heated plate (fluid-openfoam, solid-fenics) took 1524.1612270610058 seconds
INFO: Success running Flow over heated plate (fluid-openfoam, solid-fenics)
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+
| systemtest                                              | success | building time [s] | solver time [s] | fieldcompare time [s] |
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+
| Flow over heated plate (fluid-openfoam, solid-fenics)   |    1    |      1272.33      |     157.36      |         92.69         |
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+

Second time:

./tools/tests [add-test-workflows]$ python3 ./systemtests.py --suites=fenics_test
Using log-level: INFO
INFO: About to run the following systemtest in the directory /home/gc/repos/precice/tutorials/runs:
 [Flow over heated plate (fluid-openfoam, solid-fenics)]
INFO: Started running Flow over heated plate (fluid-openfoam, solid-fenics),  0/1
INFO: Running Flow over heated plate (fluid-openfoam, solid-fenics) took 178.49133943999186 seconds
INFO: Success running Flow over heated plate (fluid-openfoam, solid-fenics)
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+
| systemtest                                              | success | building time [s] | solver time [s] | fieldcompare time [s] |
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+
| Flow over heated plate (fluid-openfoam, solid-fenics)   |    1    |       1.64        |     161.08      |         14.06         |
+---------------------------------------------------------+---------+-------------------+-----------------+-----------------------+

This is also reflected in the ./runs/<id>/stdout.log file, where I see CACHED in all layers before the actual solver execution.

On GitHub Actions, while the cache is updated, (see the versions next to each name), we do not hit it, so the jobs do take a long time to run (rebuilding the images). @valentin-seitz could you please post a link to the bug you mentioned?

I think this issues also describes our problem quite well:
docker/build-push-action#829

@MakisH MakisH merged commit fddebee into develop Oct 6, 2023
5 checks passed
@MakisH MakisH deleted the add-test-workflows branch October 6, 2023 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants