Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example Snakemake rule wrapper attribute for DADA2_QUALITY_PROFILES is incorrect #622

Open
mtmcgowan opened this issue Oct 25, 2022 · 5 comments
Labels
bug Something isn't working

Comments

@mtmcgowan
Copy link

Snakemake version
Snakemake = 7.16.0
Snakemake-wrapper = v1.17.3

Describe the bug
Using the example wrapper attribute for DADA2_QUALITY_PROFILES ("v1.17.3/bio/dada2/quality-profile") causes a 404 error when Conda tries to pull the wrapper file.

The http link that causes the 404 error appears like it points to a python wrapper instead of an R wrapper.

Updating the wrapper attribute in my Snakefile to "v1.17.3/bio/dada2/quality-profile/wrapper.R" appears to solve the problem.

Logs
Did not collect logs

Minimal example
rule dada2_quality_profile_pe:
input:
expand("{project_dir}/{sample_name}-{rep}_{orientation}_001.fastq.gz",
project_dir=config['project_dir'],
orientation=[1, 2],
allow_missing=True)
output:
"reports/dada2/quality-profile/{sample_name}-{rep}-quality-profile.png"
log:
"reports/dada2/quality-profile/{sample_name}-{rep}-quality-profile.log"
wrapper:
"v1.17.3/bio/dada2/quality-profile"

Additional context
None

@mtmcgowan mtmcgowan added the bug Something isn't working label Oct 25, 2022
@fgvieira
Copy link
Collaborator

Tried running the test case for that wrapper, and it works fine.
Can you give it a try?

@rdenise
Copy link

rdenise commented Mar 24, 2023

Snakemake version
Snakemake = 7.24.2
Snakemake-wrapper = v1.25.0:

Description of the bug

I encountered the same problem

rule dada2_quality_profile_pe:
    input:
        # FASTQ file without primer sequences
        expand(os.path.join(
                OUTPUT_FOLDER,
                "databases",
                "reads_trimmed",
                "paired",
                "{{sample}}_R{orientation}.fastq.gz",
            ),
            orientation=[1,2]
        )
    output:
        os.path.join(
                OUTPUT_FOLDER,
                "reports",
                "dada2",
                "quality-profile",
                "{sample}-quality-profile.png"
        )
    log:
        os.path.join(
                OUTPUT_FOLDER,
                "logs",
                "dada2",
                "quality-profile",
                "{sample}-quality-profile.log"
        )
    wrapper:
        "v1.25.0/bio/dada2/quality-profile"

It said that it tried to find the Python wrapper and not the R wrapper:

Failed to open source file https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/dada2/quality-profile/wrapper.py
HTTPError: 404 Client Error: Not Found for url: https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/dada2/quality-profile/wrapper.py, attempt 1/3 failed - retrying in 3 seconds...
Failed to open source file https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/dada2/quality-profile/wrapper.py
HTTPError: 404 Client Error: Not Found for url: https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/dada2/quality-profile/wrapper.py, attempt 2/3 failed - retrying in 6 seconds...
Failed to open source file https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/dada2/quality-profile/wrapper.py
HTTPError: 404 Client Error: Not Found for url: https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/dada2/quality-profile/wrapper.py, attempt 3/3 failed - giving up!

@fgvieira
Copy link
Collaborator

fgvieira commented Mar 27, 2023

I tried to run the example but, even though if gave the same error, it end up founding the wrapper:

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job                         count    min threads    max threads
------------------------  -------  -------------  -------------
dada2_quality_profile_pe        1              1              1
total                           1              1              1

Select jobs to execute...

[Mon Mar 27 08:50:56 2023]
rule dada2_quality_profile_pe:
    input: trimmed/a.1.fastq, trimmed/a.2.fastq
    output: reports/dada2/quality-profile/a-quality-profile.png
    log: logs/dada2/quality-profile/a-quality-profile-pe.log
    jobid: 0
    reason: Forced execution
    wildcards: sample=a
    resources: tmpdir=/tmp

Failed to open source file https://github.com/snakemake/snakemake-wrappers/raw/master/bio/dada2/quality-profile/wrapper.py
HTTPError: 404 Client Error: Not Found for url: https://github.com/snakemake/snakemake-wrappers/raw/master/bio/dada2/quality-profile/wrapper.py, attempt 1/3 failed - retrying in 3 seconds...
Failed to open source file https://github.com/snakemake/snakemake-wrappers/raw/master/bio/dada2/quality-profile/wrapper.py
HTTPError: 404 Client Error: Not Found for url: https://github.com/snakemake/snakemake-wrappers/raw/master/bio/dada2/quality-profile/wrapper.py, attempt 2/3 failed - retrying in 6 seconds...
Failed to open source file https://github.com/snakemake/snakemake-wrappers/raw/master/bio/dada2/quality-profile/wrapper.py
HTTPError: 404 Client Error: Not Found for url: https://github.com/snakemake/snakemake-wrappers/raw/master/bio/dada2/quality-profile/wrapper.py, attempt 3/3 failed - giving up!
Rscript --vanilla ./snakemake-wrappers/bio/dada2/quality-profile/test/.snakemake/scripts/tmpgd_lus2z.wrapper.R
Activating conda environment: .snakemake/conda/39215bc162aa14d49e07c3e5e7f3f64a_

It still fails, but with the error:

Loading required package: Rcpp
Error: package or namespace load failed for ‘dada2’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object './snakemake-wrappers/bio/dada2/quality-profile/test/.snakemake/conda/39215bc162aa14d49e07c3e5e7f3f64a_/lib/R/library/jpeg/libs/jpeg.so':
  libjpeg.so.9: cannot open shared object file: No such file or directory
Execution halted

Copy link
Contributor

github-actions bot commented Nov 1, 2023

This issue was marked as stale because it has been open for 6 months with no activity.

@github-actions github-actions bot added the Stale label Nov 1, 2023
@fgvieira
Copy link
Collaborator

fgvieira commented Nov 1, 2023

I think that what is happening is that snakemake looks for the wrapper.py file and, only after failing three times, tries the wrapper.R file.
@johanneskoester can you confirm this behaviour?

If so, it is just a bit misleading error message...

@github-actions github-actions bot removed the Stale label Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants