lstMCpipe

Scripts to ease the reduction of MC data on the LST cluster at La Palma. With this package, the analysis/creation of R1/DL0/DL1/DL2/IRFs can be orchestrated.

Contact: Thomas Vuillaume, thomas.vuillaume [at] lapp.in2p3.fr Enrique Garcia, garcia [at] lapp.in2p3.fr Lukas Nickel, lukas.nickel [at] tu-dortmund.de

Cite us 📝

If lstMCpipe was used for your analysis, please cite:

https://doi.org/10.48550/arXiv.2212.00120

@misc{garcia2022lstmcpipe,
      title={The lstMCpipe library},
      author={Enrique Garcia and Thomas Vuillaume and Lukas Nickel},
      year={2022},
      eprint={2212.00120},
      archivePrefix={arXiv},
      primaryClass={astro-ph.IM}
}

in addition to the exact lstMCpipe version used from https://doi.org/10.5281/zenodo.6460727

You may also want to include the config file with your published code for reproducibility.

Install 💻

As an user:

For lstmcpipe >= 0.10.3, the preferred installation should be conda:

conda install lstmcpipe

Former versions:

VERSION=0.10.1  # change as desired
wget https://raw.githubusercontent.com/cta-observatory/lstmcpipe/$VERSION/environment.yml
conda env create -f environment.yml
conda activate lstmcpipe
pip install lstmcpipe==$VERSION

As a developer:

git clone https://github.com/cta-observatory/lstmcpipe.git
cd lstmcpipe
conda env create -n lstmcpipe_dev -f environment.yml
conda activate lstmcpipe_dev
pip install -e .
pre-commit install

This will setup a pre-commit hook: Given that you are in the right enviroment, it will run and format files you are about to commit with black. (You need to stage the changes again after that). This ensures the formatting of the code follows our guidelines and there is less work dealing with the code checker in the CI.

Requesting a MC analysis 📊

You may find a longer, more detailed, version of these steps in our documentation.

You may find the list of already run productions in the documentation. Please check in this list that the request you are about to make does not exist already!

To request a MC analysis:

1. Make sure to be part of the github cta-observatory/lst-dev team. If not, ask one of the admins.
2. Clone the repository in the cluster at La Palma.
3. Create a new branch named with you prodID
4. Make a new directory named date_ProdID in the production_configs dir (have a look at the production_configs/template_prod as an example)
5. Generate your config (see below)
6. Open a pull request into lstMCpipe with a clear description (probably the same as in the readme of your config dir)
7. The requested config must contain:

• a lstchain config file (please provide an exhaustive config that will help others and provide a more explicit provenance information)
• a lstmcpipe config file (to generate it, please refer to the documentation)
• a readme with a short description of why you require this analysis to be run. Do not add information that should not appear publicly (such as source names) here. If you are requesting a production for a specific new source, please edit this table on LST wiki. Also add the command line to generate the lstmcpipe config, that will help debugging.

The proposed configuration will be tested for validity by the continuous integration tests and we will interact with you to run the analysis on the cluster at La Palma.

Depending on the number of requests, we may give priorities.

Need help? Join the CTA North slack and ask for help in the

Launch jobs 🚀

To generate your lstmcpipe configuration file, use lstmcpipe_generate_config command. If the type of production you want is not listed in the existing ones, you may create your own PathConfig class from an existing one, or generate a config from an existing prod type and edit the file manually.

Once you have your configuration file, you way launch the pipeline with the described stages in the config using:

lstmcpipe -c config_MC_prod.yml -conf_lst lstchain_*.json [-conf_cta CONFIG_FILE_CTA] [-conf_rta CONFIG_FILE_RTA] [--debug] [--log-file LOG_FILE]

lstmcpipe is the orchestrator of the pipeline, it schedules the stages specified in the config_MC_prod.yml file. All the configuration related with the MC pipe must be declared in this file (stages, particles to be analysed, zenith, pointing, type of MC production...).

Pipeline-specific configuration options (such as cleaning or model parameters) are declared in a different configuration file, which is passed via the options -conf_lst/-conf_cta/-conf_rta.

Note: You can always launch this command without fear; there is an intermediate step that verifies and shows the configuration that you are passing to the pipeline.

Note that a complete pipeline still requires quite a lot of resources. Think about other LP-IT cluster users.