[traced-graph][sparse] propagate sparsity metadata into traced graph

[aartbik]

Propagate sparsity metadata from sparse tensors of torch.sparse into the traced graph representation (with would be useful for a JIT backend that supports a "sparse compiler"). This is a first careful attempt, since the actual "meta" feature seem still incomplete for coo and completely lacking for csr/csc/bsr/bsc.

For background see forum postings (with examples):
https://discuss.pytorch.org/t/connecting-pytorch-sparse-tensors-with-mlir/195145
https://dev-discuss.pytorch.org/t/connecting-pytorch-sparse-tensors-with-mlir/1803

And feature request:
#117188

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @fegin @XilunWu @wanchaol @fduwjj @wz337 @tianyu-l @wconstab @yf225 @chauhang @d4l3k @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @voznesenskym @EikanWang @Guobing-Chen @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @rohan-varma @aakhundov

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/117907

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit b313230 with merge base 1b29c16 ():

NEW FAILURE - The following job has failed:

• inductor / cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.

UNSTABLE - The following job failed but was likely due to flakiness present on trunk and has been marked as unstable:

• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (inductor_torchbench_cpu_smoketest_perf, 1, 1, linux.24xl.spr-metal) (gh) (#126993)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: aartbik / name: Aart Bik (a34ef36, b313230, c973cc5, 1830983)

[pytorch-bot]

Please seek CI approval before scheduling CIFlow labels

[pytorch-bot]

Please seek CI approval before scheduling CIFlow labels

[aartbik]

@ezyang

[ezyang]

I am in principle in favor of make fake tensor work on sparse tensors, and adjusting our infra so that this works. I didn't carefully review the implementation details in this PR. What should next steps be?

[ezyang]

cc @eellison @pearu

[aartbik]

As for next steps, I can definitely work (and with my team) on bringing this to a higher quality, but we would really like to have a core PyTorch developer assist and look over our shoulders, since we are new to this repo and don't want to miss essential design principles when making first changes. Of course, if a core developer is willing and interested in helping out with actual code, that would be welcome too!

[pearu]

I added a couple of comments. While the aim here is to support sparse tensor layouts, I think the corresponding changes ought address the layout-specific issues (such as the lack of strides, for instance) in a more sparse-layout-agnostic way whenever possible.

[pytorch-bot]

Please seek CI approval before scheduling CIFlow labels

[pytorch-bot]

Please seek CI approval before scheduling CIFlow labels

[pearu]

A related issue is #99404 that suggests looking at the problem from the perspective of creating FakeTensors using the fake tensor converter class.

[ezyang]

@pearu are you willing to shepherd this PR to completion?

[aartbik]

Thanks all!

Note that I am not stalling this PR ;-) but I am wrapping-up the required torch-mlir work so that we are pretty certain of all the metadata that should propagated into the traced graph. Also, we plan to do some "sanity" performance runs, just to see if this new path really has potential.

So please stay tuned, I will get back to this PR really soon....

[pearu]

@pytorchbot merge -f "with two unrelated CI failures"

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[aartbik]

Thanks again, @pearu , for you truly exemplary mentorship to me getting this in!
I am enthusiastically starting on the next tasks for this feature!

[pearu]

@aartbik , congratulations to getting this PR merged! And thanks for your patience, landing a PR in 5 months requires plenty of it :) Looking forward to your next contributions!