Unified testsuite #4334

MSVoss · 2022-10-28T06:35:10Z

This is my suggestion for the generalization of the test suite. As already stated, the current version of the KHI test is very geared towards this. In order to make the test suite usable for different test cases, specific components had to be replaced. Among them are the following aspects in particular:

Labeling and description within the results (plot and .log file), including e.g. title, author, ... (see TestKHI.py file, in which these are statically defined)
Function for calculating the theoretical values (see TheoryGrowthrate.py)
Calculation of the values from the simulation data
Limited use of other functions for the general (e.g. function for calculating the deviation)

In order to remedy these shortcomings, the following has been changed (here only the essential aspects):

New structure of the test suite for better encapsulation and clarity
Introduction of Data.py: This file serves as a parameter file and contains essential parameters of the test, such as author, title, theory calculation function,.. . This eliminates the static character of the old structure.

There are a few things to keep in mind:

This is only the first version of the generalization, extensions will follow later
Deterioration compared to the previous version: no case distinction, it follows that the test case of the KHI (i.e. MI and ESKHI) currently has to be treated separately
Some functions have not yet been completed, such as the 2D plot

Still under development or included in the next "version":

Simple case distinction: Using a parameter, the test suite should be able to distinguish individual different theoretical cases from each other (e.g. see KHI: use the Lorentz factor to distinguish whether ESKHI or MI is present)
Plotting and evaluating 2D plots
Evaluation of openPMD data

Currently integrated tests:

MI
ESKHI
Both applicable for 3D as well as for 2D (however, the corresponding .param files have to be adapted to the case). Please note the dependency on the Lorenz factor in 3D. The included ci.bash script uses the MI. For more information, refer to the test case documentation.

Resulting advantage for the user compared to the old version.

The result is easily customizable
Test developer can freely define functions of the theory
...

Only the most important aspects are mentioned in the list above. Further information can be found in the individual documentation (function documentation)
Documentation will be added later for more information

MSVoss · 2022-10-28T06:38:31Z

Flake8 will be corrected shortly

steindev · 2022-10-28T08:06:00Z

@MSVoss There are about 300 lines of complaints by flake8, this is the reason the pull request validation failed. See report.

EDIT: I did not see your second comment, sorry for bothering you again 🙈

EDIT2: Would you please update the PR description with an explanation of 'generalization'? That is, what is the aim of the generalization and how do you achieve this? (Just a few lines)

PrometheusPi · 2022-10-28T11:25:39Z

lib/python/test/setups/ESKHI/Data.py

+    s of KHI currently have to be treated differently.
+"""
+import sys
+sys.path.insert(1, "../../")


why do you replace the 2nd entry of the path list?

It's almost random. I could also have used append and simply added it afterwards. But here I'm looking for another solution anyway because flake8 is not fulfilled here.

@MSVoss I think sth like this could possibly also work and would maybe not trigger flake8. But, no idea if this is any better.

from importlib.machinery import SourceFileLoader testsuite = SourceFileLoader("testsuite", "../../testsuite/__init__.py").load_module()

MSVoss · 2023-03-28T13:00:16Z

I hope I haven't overlooked anything. I made a note of the inheritance for the next pull request.

PrometheusPi

Some open questions, but this looks good to me

PrometheusPi · 2023-03-30T12:29:41Z

@psychocoderHPC @pordyna You had some open issues and questions on this pull request. What's the status?

PrometheusPi · 2023-04-21T08:41:58Z

only open questions by @psychocoderHPC
#4334 (comment)

PrometheusPi

I am blocking merging for now because when executing the CI via

./picongpu/share/picongpu/tests/KHI_growthRate/bin/ci.sh picongpu/share/picongpu/tests/KHI_growthRate/ ./KHI

(on hemera v100) I get the error 42.
Thus, when adding this test, the CI will always fail.

@MSVoss could you clarify what I am doing wrong or what the error 42 is causing.

PrometheusPi · 2023-04-21T15:57:55Z

The python code validate.sh throws the following error:

<class 'ValueError'> This should never happen <traceback object at 0x2aab462ff708>

PrometheusPi · 2023-05-22T10:32:25Z

@MSVoss the PIP check of our CI is still unhappy with:

./lib/python/test/testsuite/_manager.py:13:1: E302 expected 2 blank lines, found 1

psychocoderHPC · 2023-06-02T10:51:50Z

share/picongpu/tests/KHI_growthRate/include/picongpu/param/particle.param

@@ -27,7 +27,7 @@
 #include <pmacc/math/operation.hpp>

 #ifndef PARAM_GAMMA
-#    define PARAM_GAMMA 1.021
+#    define PARAM_GAMMA 1.2


please add to cmakeFlags

flags[3]="-DPARAM_OVERWRITES:LIST='-DPARAM_GAMMA=1.2' # Depending on the GPU vendor and version compile can fail because of limited amound of shared memory. # Please enable the test by hand! # flags[4]="-DPARAM_OVERWRITES:LIST='-DPARAM_PRECISION=precision64Bit;-DPARAM_PARTICLESHAPE=PQS'"

psychocoderHPC · 2023-06-02T11:00:19Z

share/picongpu/tests/KHI_growthRate/include/picongpu/param/particle.param

@@ -27,7 +27,7 @@
 #include <pmacc/math/operation.hpp>

 #ifndef PARAM_GAMMA
-#    define PARAM_GAMMA 1.021
+#    define PARAM_GAMMA 1.2


offline discussed with @PrometheusPi: we should not change the default behavior of this test. We have users which using this test case and we should not silently invalidate the possibility that old and now runs can be compared.

psychocoderHPC · 2023-06-02T12:52:12Z

lib/python/test/testsuite/Reader/paramReader.py

+
+            for line in all_Lines:
+                if "define" + search_u in line and parameter is None:
+                    parameter = float(line[line.find(search_u) +


@MSVoss and @PrometheusPi please discuss this comment

Issue introduced with ComputationalRadiationPhysics#4334 - remove slash at the end of a path (do not enforce a slash) - add a slash before concadinate a filename with a directory path

Issue introduced with ComputationalRadiationPhysics#4334 - remove the slash at the end of a path (do not enforce a slash) - add a slash before concatenate a filename with a directory path

MSVoss force-pushed the dev branch 3 times, most recently from 3a8a88b to c28d6be Compare October 28, 2022 07:49

steindev requested review from steindev, PrometheusPi and psychocoderHPC October 28, 2022 08:23

steindev added refactoring code change to improve performance or to unify a concept but does not change public API component: tests unit tests labels Oct 28, 2022

steindev added this to the 0.7.0 / 1.0.0: Next Stable milestone Oct 28, 2022

MSVoss force-pushed the dev branch 2 times, most recently from 0f4bed1 to c78a37b Compare October 28, 2022 09:36

PrometheusPi reviewed Oct 28, 2022

View reviewed changes

MSVoss force-pushed the dev branch 4 times, most recently from c90329a to da5a2f0 Compare October 28, 2022 13:36

PrometheusPi mentioned this pull request Nov 3, 2022

PICMI: silence pyflakes #4091

Closed

MSVoss force-pushed the dev branch 8 times, most recently from 58c3749 to a9925f5 Compare November 7, 2022 15:16

MSVoss force-pushed the dev branch 3 times, most recently from 972b8c7 to bf9addf Compare November 11, 2022 15:23

MSVoss force-pushed the dev branch from c413270 to bd73857 Compare March 28, 2023 13:13

PrometheusPi approved these changes Mar 30, 2023

View reviewed changes

pordyna previously approved these changes Apr 21, 2023

View reviewed changes

PrometheusPi requested changes Apr 21, 2023

View reviewed changes

MSVoss dismissed pordyna’s stale review via 9b3f87f April 28, 2023 12:05

MSVoss force-pushed the dev branch 2 times, most recently from 9b3f87f to 9e8a1af Compare May 5, 2023 08:35

MSVoss force-pushed the dev branch from 9e8a1af to 54f1cdd Compare May 12, 2023 09:05

MSVoss force-pushed the dev branch from 54f1cdd to 496bd95 Compare May 19, 2023 14:45

Unified test and first example

840a48d

MSVoss force-pushed the dev branch from 496bd95 to 840a48d Compare May 26, 2023 08:34

PrometheusPi self-requested a review June 2, 2023 10:38

PrometheusPi previously approved these changes Jun 2, 2023

View reviewed changes

PrometheusPi added the CI:no-compile CI is skipping compile/runtime tests but runs PICMI tests label Jun 2, 2023

MSVoss dismissed PrometheusPi’s stale review via ad8623d June 2, 2023 12:27

psychocoderHPC previously approved these changes Jun 2, 2023

View reviewed changes

MSVoss dismissed psychocoderHPC’s stale review via 03a25b1 June 2, 2023 13:48

MSVoss force-pushed the dev branch from ad8623d to 03a25b1 Compare June 2, 2023 13:48

adjust to previous test case as required by @psychocoderHPC

9913a79

MSVoss force-pushed the dev branch from 03a25b1 to 9913a79 Compare June 2, 2023 14:01

PrometheusPi approved these changes Jun 6, 2023

View reviewed changes

PrometheusPi merged commit 7247171 into ComputationalRadiationPhysics:dev Jun 6, 2023
6 checks passed

psychocoderHPC mentioned this pull request Jul 20, 2023

fix broken KHI growth rate test tools #4637

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unified testsuite #4334

Unified testsuite #4334

MSVoss commented Oct 28, 2022 •

edited

MSVoss commented Oct 28, 2022

steindev commented Oct 28, 2022 •

edited

PrometheusPi Oct 28, 2022

MSVoss Oct 28, 2022

pordyna Nov 24, 2022

MSVoss commented Mar 28, 2023

PrometheusPi left a comment

PrometheusPi commented Mar 30, 2023

PrometheusPi commented Apr 21, 2023

PrometheusPi left a comment •

edited

PrometheusPi commented Apr 21, 2023

PrometheusPi commented May 22, 2023

psychocoderHPC Jun 2, 2023

psychocoderHPC Jun 2, 2023

psychocoderHPC Jun 2, 2023

Unified testsuite #4334

Unified testsuite #4334

Conversation

MSVoss commented Oct 28, 2022 • edited

MSVoss commented Oct 28, 2022

steindev commented Oct 28, 2022 • edited

PrometheusPi Oct 28, 2022

Choose a reason for hiding this comment

MSVoss Oct 28, 2022

Choose a reason for hiding this comment

pordyna Nov 24, 2022

Choose a reason for hiding this comment

MSVoss commented Mar 28, 2023

PrometheusPi left a comment

Choose a reason for hiding this comment

PrometheusPi commented Mar 30, 2023

PrometheusPi commented Apr 21, 2023

PrometheusPi left a comment • edited

Choose a reason for hiding this comment

PrometheusPi commented Apr 21, 2023

PrometheusPi commented May 22, 2023

psychocoderHPC Jun 2, 2023

Choose a reason for hiding this comment

psychocoderHPC Jun 2, 2023

Choose a reason for hiding this comment

psychocoderHPC Jun 2, 2023

Choose a reason for hiding this comment

MSVoss commented Oct 28, 2022 •

edited

steindev commented Oct 28, 2022 •

edited

PrometheusPi left a comment •

edited