Overwriting module rules leads to unexpected behavior

[domonik]

Describe the bug
Hi,
When i tried to reuse code from previous workflows via module imports i realized that overwriting input and output sometimes leads to unexpected behavior where output of rules points to wrong file names. (See minimal example)

Snakemake version
8.5.3 from bioconda.

Minimal example
I created a minimal example with two files:

The first one is called setup.smk

rule generateAnnotationFromCounts:
    input:
        counts = "foo.tsv",
    output:
        annotation = "Old/Annotation.csv"
    run:
        with open(output.annotation) as handle:
            handle.write("annotation")

rule dropUnusedSamples:
    input:
        counts = "foo.tsv",
        annotation = rules.generateAnnotationFromCounts.output.annotation,
    output:
        counts = "Old/DroppedCounts.tsv",
    run:
        with open(output.counts) as handle:
            handle.write("counts")

The second is the one where i import this workflow and overwrite inputs of the two rules:

module other_workflow:
    snakefile: "setup.smk"
    config: config

use rule * from other_workflow as other_*

use rule generateAnnotationFromCounts from other_workflow as other_generateAnnotationFromCounts with:
    input:
        counts = "foo2.tsv",
    output:
        annotation = "New/Annotation.csv"


use rule dropUnusedSamples from other_workflow as other_dropUnusedSamples with:
    input:
        counts="foo2.tsv",
        annotation=rules.other_generateAnnotationFromCounts.output.annotation,
    output:
        counts = "New/DroppedCounts.tsv",


rule all:
    default_target: True
    input:
        file = rules.other_dropUnusedSamples.output

The file foo.tsv and foo2.tsv are just empty files in the same directory.
Running the second workflow via: snakemake -s newSetup.smk --cores 1 -n

results in the following error:

Traceback (most recent call last):
  File "/home/rabsch/miniconda/envs/snakemaketest/lib/python3.12/site-packages/snakemake/cli.py", line 1933, in args_to_api
    dag_api = workflow_api.dag(
              ^^^^^^^^^^^^^^^^^
  File "/home/rabsch/miniconda/envs/snakemaketest/lib/python3.12/site-packages/snakemake/api.py", line 328, in dag
    return DAGApi(
           ^^^^^^^
  File "<string>", line 6, in __init__
  File "/home/rabsch/miniconda/envs/snakemaketest/lib/python3.12/site-packages/snakemake/api.py", line 438, in __post_init__
    self.workflow_api._workflow.dag_settings = self.dag_settings
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rabsch/miniconda/envs/snakemaketest/lib/python3.12/site-packages/snakemake/api.py", line 385, in _workflow
    workflow.include(
  File "/home/rabsch/miniconda/envs/snakemaketest/lib/python3.12/site-packages/snakemake/workflow.py", line 1386, in include
    exec(compile(code, snakefile.get_path_or_uri(), "exec"), self.globals)
  File "/home/rabsch/PythonProjects/snakemakeBug/newSetup.smk", line 39, in <module>
AttributeError: 'Namedlist' object has no attribute 'annotation'

Adding a print(rules.other_generateAnnotationFromCounts.output) After overwriting the corresponding outputs results in: Old/DroppedCounts.tsv, which is weirdly the output of the dropUnusedSamples rule.

I just wonder what is going on here. maybe i am using the module directive in an unintended way?

[hunter-cameron]

I think I might be having a similar issue (with both v7.32.4 and v8.10.6) where when I try to overwrite modules in a similar fashion.

I've observed that overwritten rules end up referencing the last rule in the file of the module that has been imported rather than whatever rule name was supplied. I tested this by changing which rule was last in the module's snakefile and it always pointed to the last one. It seems to be an error with how the rule proxy object is being set up but I don't understand the architecture well enough to fix it.

I tested by looking at the rule name e.g. for the example above: print(rules.other_generateAnnotationFromCounts.rule.name)

[Hocnonsense]

I think #2838 is similar to this, as module other_workflow will be executed every time when use rule .... from it