【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API #45849

zrr1999 · 2022-09-07T10:41:28Z

PR types

New features

PR changes

APIs

Describe

为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API

rfc: 【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API community#250
docs: 【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API docs#5272

paddle-bot · 2022-09-07T10:41:32Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

# Conflicts: # paddle/phi/api/yaml/sparse_bw_api.yaml

zhwesky2010 · 2022-09-27T11:14:54Z

paddle/phi/kernels/sparse/cpu/transpose_kernel.cc

+  const DenseTensor& x_values = x.values();
+  // return a copy of x
+  if (perm[0] == 0 && perm[1] == 1 && (n_dim == 2 || perm[2] == 2)) {
+    out->SetMember(x_crows, x_cols, x_values, x.dims());


用DenseTensor a=b; 触发左值构造，底层会共享Allocation内存，不要直接就完全用同一个DenseTensor，这个可能会引入风险

https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/kernels/sparse/impl/unary_kernel_impl.h#L138

好的，已经修改

zhwesky2010

LGTM

Ligoml · 2022-09-29T07:44:45Z

python/paddle/incubate/sparse/unary.py

+    """
+    Changes the perm order of ``x`` without changing its data, requiring x to be a SparseCooTensor or SparseCsrTensor.
+
+    .. math::


中英文API文档需要保持一致哈，例如API描述和公式，最好不要一边有一边没有

好的，这里已经在中文文档中修改

paddle/phi/kernels/sparse/unary_kernel.h

paddle/phi/kernels/sparse/unary_grad_kernel.h

Ligoml

LGTM for docs

jeff41404

LGTM

chenwhql

LGTM for fluid header using

XiaoguangHu01

LGTM

OccupyMars2025 · 2022-10-10T08:25:18Z

paddle/phi/api/yaml/sparse_ops.yaml

+  args : (Tensor x, int[] perm)
+  output : Tensor(out)
+  infer_meta :
+    func : TransposeInferMeta


I think the used " func : TransposeInferMeta" is actually the one for dense tensor. But the TransposeInferMeta for dense tensor is also applicable to sparse tensor. "func : TransposeGradInferMeta" has the same situation. So I submit a PR to delete maybe unused code in paddle\phi\infermeta\sparse\unary.h

#46844

OccupyMars2025 · 2022-10-14T08:17:45Z

python/paddle/fluid/tests/unittests/test_sparse_transpose_op.py

+    def check_result(self, x_shape, dims, format):
+        with _test_eager_guard():
+            mask = paddle.randint(0, 2, x_shape).astype("float32")
+            origin_x = paddle.rand(x_shape, dtype='float32') * mask


You need to make sure that all zero elements in "origin_x" is caused by multiplying by "mask",
or the following backward checks may fail:

np.testing.assert_allclose(sp_x.grad.to_dense().numpy(), (dense_x.grad * mask).numpy(), rtol=1e-05)

Here is an example:
origin_x = [2, 0, 3, 4, 5] * [1, 1, 0, 1, 1]
so origin_x is [2, 0, 0, 4, 5]
sp_x.grad.to_dense().numpy() will be [1, 0, 0, 1, 1] ,
while (dense_x.grad * mask).numpy() will be [1, 1, 0, 1, 1],
then the backward check fails.

I have stated a PR to modify it.
#47017

感谢修改

zhwesky2010 · 2022-10-17T07:02:19Z

@zrr1999 你好，还需要在 https://github.com/PaddlePaddle/docs 中增加transpose的中文文档

zrr1999 · 2022-10-17T08:09:39Z

@zrr1999 你好，还需要在 https://github.com/PaddlePaddle/docs 中增加transpose的中文文档

您好，这部分之前已经提交pr，，但是一直还没有merge，已经解决了最新的冲突

…addlePaddle#45849)

zkh2016

#53008 单测出现随机现象可能是使用错误的cudaMalloc和cudaMemcpy导致的，麻烦改下后再测下。 @zrr1999

zkh2016 · 2023-04-18T06:14:40Z

paddle/phi/kernels/sparse/gpu/transpose_kernel.cu

+  hipMemcpy(
+      d_perm, perm.data(), sizeof(int) * perm.size(), hipMemcpyHostToDevice);
+#else
+  cudaMalloc(reinterpret_cast<void **>(&d_perm), sizeof(int) * perm.size());


这里使用的有点问题，如果用cudaMalloc应该手动调用cudaFree释放。但是建议使用paddle::memory::Alloc。

zkh2016 · 2023-04-18T06:18:58Z

paddle/phi/kernels/sparse/gpu/transpose_kernel.cu

+      d_perm, perm.data(), sizeof(int) * perm.size(), hipMemcpyHostToDevice);
+#else
+  cudaMalloc(reinterpret_cast<void **>(&d_perm), sizeof(int) * perm.size());
+  cudaMemcpy(


这里需要使用异步的方式，否则这里和kernel执行不在一个stream上。可以用 GpuMemcpyAsync。

#53008 单测出现随机现象可能是使用错误的cudaMalloc和cudaMemcpy导致的，麻烦改下后再测下。 @zrr1999

我添加了一个新的pr用来修复这个bug，看起来reshape算子好像也存在类似的问题 https://github.com/PaddlePaddle/Paddle/pull/53038/files

好的，辛苦一块改下呗

zrr1999 added 2 commits September 7, 2022 16:24

add sparse transpose op

9f403dc

add sparse transpose op

2275d63

paddle-bot bot added contributor External developers status: proposed labels Sep 7, 2022

luotao1 assigned luotao1, Ligoml and zhwesky2010 Sep 7, 2022

Merge branch 'develop' into sparse_transpose

c6c33ff

# Conflicts: # paddle/phi/api/yaml/sparse_bw_api.yaml

zrr1999 force-pushed the sparse_transpose branch from a3d62a1 to b4cb032 Compare September 8, 2022 11:34

complete 2D CSR sparse transpose op

f36cc0e

zrr1999 force-pushed the sparse_transpose branch from 62cad30 to f36cc0e Compare September 8, 2022 14:12

zrr1999 added 3 commits September 8, 2022 22:45

add more unitests

5158792

fix bugs

10f1682

complete 3D CSR sparse transpose op

1a7684a

zrr1999 force-pushed the sparse_transpose branch from ab579e2 to 618f43c Compare September 9, 2022 05:39

zrr1999 added 4 commits September 9, 2022 16:15

fix bugs

1bcdfac

add grad

553a792

fix bugs

744667a

Merge branch 'develop' into sparse_transpose

0135acc

zrr1999 force-pushed the sparse_transpose branch from afa5275 to 4bef593 Compare September 9, 2022 10:18

zrr1999 marked this pull request as ready for review September 9, 2022 10:20

optimize unittest

c6060b9

zrr1999 force-pushed the sparse_transpose branch 3 times, most recently from 3390642 to 368aaf8 Compare September 10, 2022 08:55

zrr1999 mentioned this pull request Sep 12, 2022

【PaddlePaddle Hackathon 第三期】任务总览 #43938

Closed

zrr1999 force-pushed the sparse_transpose branch from 504e574 to 754a4bf Compare September 13, 2022 16:16

zrr1999 added 2 commits September 14, 2022 16:10

fix bugs

4d0e052

add cuda impl

129a2f0

zrr1999 force-pushed the sparse_transpose branch 4 times, most recently from 10161e9 to a0d092c Compare September 24, 2022 16:11

fix bugs

535cb46

zrr1999 force-pushed the sparse_transpose branch from 9b786ed to 535cb46 Compare September 25, 2022 16:06

zrr1999 requested a review from zhwesky2010 September 26, 2022 01:34

zhwesky2010 reviewed Sep 27, 2022

View reviewed changes

fix bugs

6522826

zhwesky2010 approved these changes Sep 28, 2022

View reviewed changes

Ligoml reviewed Sep 29, 2022

View reviewed changes

jeff41404 reviewed Sep 29, 2022

View reviewed changes

paddle/phi/kernels/sparse/unary_kernel.h Show resolved Hide resolved

jeff41404 reviewed Sep 29, 2022

View reviewed changes

paddle/phi/kernels/sparse/unary_grad_kernel.h Show resolved Hide resolved

Ligoml approved these changes Sep 29, 2022

View reviewed changes

jeff41404 approved these changes Sep 29, 2022

View reviewed changes

chenwhql approved these changes Sep 29, 2022

View reviewed changes

XiaoguangHu01 approved these changes Sep 29, 2022

View reviewed changes

luotao1 merged commit 2b879a6 into PaddlePaddle:develop Sep 30, 2022

zrr1999 mentioned this pull request Sep 30, 2022

【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API PaddlePaddle/docs#5272

Merged

OccupyMars2025 reviewed Oct 10, 2022

View reviewed changes

OccupyMars2025 reviewed Oct 14, 2022

View reviewed changes

This was referenced Oct 14, 2022

[Hackathon 3rd No.22 ] add paddle.incubate.sparse.reshape #46694

Merged

[hidden trouble] Update test_sparse_transpose_op.py to get rid of a hidden trouble. #47017

Merged

luotao1 mentioned this pull request Oct 17, 2022

delete maybe unused code in paddle\phi\infermeta\sparse\unary.h #46844

Merged

zhwesky2010 pushed a commit to zhwesky2010/Paddle that referenced this pull request Oct 17, 2022

【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API (P…

ee539dd

…addlePaddle#45849)

zhwesky2010 mentioned this pull request Oct 17, 2022

[cherry-pick 2.4] add sparse api transpose/reshape/is_same_shape #47076

Merged

zkh2016 reviewed Apr 18, 2023

View reviewed changes

zrr1999 mentioned this pull request Jul 12, 2023

【社区治理】zrr1999 发起 Committer 身份申请 #55384

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API #45849

【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API #45849

zrr1999 commented Sep 7, 2022 •

edited

paddle-bot bot commented Sep 7, 2022

zhwesky2010 Sep 27, 2022 •

edited

zhwesky2010 Sep 27, 2022

zrr1999 Sep 28, 2022

zhwesky2010 left a comment

Ligoml Sep 29, 2022

zrr1999 Sep 29, 2022

Ligoml left a comment

jeff41404 left a comment

chenwhql left a comment

XiaoguangHu01 left a comment

OccupyMars2025 Oct 10, 2022 •

edited

OccupyMars2025 Oct 14, 2022 •

edited

zrr1999 Oct 17, 2022

zhwesky2010 commented Oct 17, 2022

zrr1999 commented Oct 17, 2022 •

edited

zkh2016 left a comment •

edited

zkh2016 Apr 18, 2023

zkh2016 Apr 18, 2023

zrr1999 Apr 18, 2023

zkh2016 Apr 18, 2023

【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API #45849

【Hackathon No.21】为 Paddle 新增 paddle.incubate.sparse.transpose 稀疏 API #45849

Conversation

zrr1999 commented Sep 7, 2022 • edited

PR types

PR changes

Describe

paddle-bot bot commented Sep 7, 2022

zhwesky2010 Sep 27, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhwesky2010 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ligoml left a comment

Choose a reason for hiding this comment

jeff41404 left a comment

Choose a reason for hiding this comment

chenwhql left a comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

OccupyMars2025 Oct 10, 2022 • edited

Choose a reason for hiding this comment

OccupyMars2025 Oct 14, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhwesky2010 commented Oct 17, 2022

zrr1999 commented Oct 17, 2022 • edited

zkh2016 left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zrr1999 commented Sep 7, 2022 •

edited

zhwesky2010 Sep 27, 2022 •

edited

OccupyMars2025 Oct 10, 2022 •

edited

OccupyMars2025 Oct 14, 2022 •

edited

zrr1999 commented Oct 17, 2022 •

edited

zkh2016 left a comment •

edited