#################################################

Overview of hipermark Architecture

#################################################

Hipermark works with this hierarchy:

1. Benchmark: A specific mathematical problem or function (in the mathematical sense) which can be solved by a computer. The specific solutions to this problem can be written in many different languages and can use parallel or sequential logic. It is called a benchmark because it provides a basis for which to compare different implementations of solutions to this problem.

2. Implementation: A specific solution to a problem. An implementation contains the source code of the solution to a specific problem. Each problem can have many different implementations which can use e.g. openMP or openCL and can be written in C, C++, futhark, Haskell etc.

3. Static Configuration: An implementation plus compile-time variables. A compile-time variable is for example REAL_TYPE which can be float or double.

4. Case: A dynamic configuration plus input and expected (correct) output data. The different input data can for example be a small, medium, and a large dataset.

5. Dynamic configuration: A case plus any run-time variables. NUM_THREADS is an example of a run-time variable which an cpp_openmp implementation uses.

6. A run: Each setup (dynamic configuration) can be executed a number of times. This means that the mean runtime and its uncertainty can be found.

#################################################

API and Functionality of hipermark

#################################################

In order to run HIPERmark the user must organize the source files in a specific way. The source files are the implementations, the input/output data, and files describing the static and dynamic configurations. The user must also provide a Makefile for each implementation.

Hipermark should be called from the root directory of the structure presented in "Data Structure" below. Besides from the directory structure, the user must provide these files: instantiate, instantiate_data, and Makefile

instantiate

An executable that follows these principles:

• These arguments are available within the program since the program is executed with these environment variables. The paths are specified relative to the root folder described above.
  • HIPERMARK_LIB_DIR -- /lib
  • HIPERMARK_BENCHMARK_LIB_DIR -- /benchmarks/<benchmark>/implementations/<implementation>/lib
  • HIPERMARK_IMPLEMENTATION -- /benchmarks/<benchmark>/implementations/<implementation>
  • HIPERMARK_PLATFORM --
• Copy all required files from source destination to instantiate destination. The instantiate destination is the current folder ("./") since hipermark will start a process in which the program is executed and this process' path will be the place to store the binary of the source code of the implementation.
• The /lib/generate_platform_mk.py Python script must be executed with the argument of the desired platform and the output must be placed in ./platform.mk (relative to the folder of the execution of "instantiate". Relatively to the root directory of the benchmark, the same path is: /instantiations/"-"/).
• A run executable which can take an argument and pipe this argument into the executable must be generated by the execution of the instantiate program.
• The user (in the OS sense) creating the "run" must be able to execute "run". "run" is also placed in the folder ./ compared to the execution of instantiate.

instantiate_data

hipermark runs instantiate_data one time for each dataset for each implementation.

instantiate_data is an executable that the user can choose to use in order to copy the input data into a position accessible for the above mentioned run script, and instantiate_data can also be used to format the input data into another format if that is required by the compiled implementation. The run script should call the executable and allow an argument for the input data to be given.

Makefile

Use of hipermark

The program runs only one benchmark at a time. This is specified when the program is executed.

The implementations are compiled one time for each case meaning that m implementations and n datasets will lead to m*n compilations in total and n compilations for each implementation. The source code should only be compiled once for every implementation. See "To Do".

The implementation and benchmark concepts are implemented as classes in Hipermark, the case concept is not.

Th hipermark program will not run implementations which contain a file called "disable" in /benchmarks//implementations/, i.e., the root folder of the specific implementation.

#################################################

Functionality of hipermark

#################################################

1. When the program is executed, it creates an "overview" (contained in some datastructures) of all benchmarks (objects), implementations (objects), and datasets. It also checks that the necessary files and folders are present.

2. For each implementation and for each dataset it then runs the function implementation.instantiate. This function makes the target directory for the running of a case and sets some variables found in the "instantiate" file which is located in implementation folder. It then creates a new process which runs the instantiate file and switches the folder of the child process to the target directory. There exists one instantiation file and one make file per implementation. So the same file is run once for each input.

3. The instantiate script copies the needed library and configuration files to the target directory (in /instantiations/). It creates a run file where the number of cores (and othr run-time parameters?) are set. When the run file is executed, it runs the binary file with the input found in input.data. The instantiate then calls these procedures:

4. creates a input.data which formats the input.json data in a implementation/benchmark-dependant way.

5. generates a platform.mk file from /lib/generate_platform_mk.py with the argument /config/platform_example.json.

6. Sets NCORES with /lib/json_get.py

7. Runs the make file that has been copied to the target folder.

#################################################

Design Standards

#################################################

Documentation in this program follows the Docstring convention found in PEP 0257: https://www.python.org/dev/peps/pep-0257/#what-is-a-docstring

#################################################

To Do

#################################################

So the new structure of /instantiations/ will become:

• instantiations
  • "-/" ()
    • binary for implementation (i.e., compiled source code)
    • run file that can take arguments (extended run file) -- $run + other?
    • datasets/
      • * input.data * output.data

#################################################

Data Structure

#################################################

The data should be structured as follows:

• Root folder
  • benchmarks
    • benchmark1
      • datasets
        • dataset1
          • input.json
          • output.json
        • dataset2 ...
      • implementations
        • implementation1
          • instantiate
          • Makefile
          • source file(s)
        • implementation2 ...
      • lib
        • include
          • various library files for this specific benchmark (benchmark1)
    • benchmark2 ...
  • config
    • platform_example.json -- what is this?
  • DOCS
  • instantiations (created by running program)
    • instance1 (benchmark + implementation + dataset)
      • Makefile
      • needed library file(s)
      • source code file(s)
      • setup.mk
      • input.data
      • output.data
      • run
      • stderr.log
      • stdout.log
      • binary file for this impl.
    • instance2 ...
  • lib
    • include
      • ParserC.h
      • SDK_stub.h
      • Util.h
      • Utilities.cl
    • generate_platform_mf.py
    • json_get.py
    • linearise_data.py
    • platform.macbookpro.intel.mk
    • platform.macbookpro.mk
    • setup.mk