Add IntermediateLayerGetter #47908

DrRyanHuang · 2022-11-12T13:16:16Z

PR types

New features

PR changes

APIs

Describe

Add IntermediateLayerGetter just as torchvision.models._utils.IntermediateLayerGetter

paddle-bot · 2022-11-12T13:16:20Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

luotao1 · 2022-11-14T03:33:38Z

需要补充单测来通过 PR-CI-Coverage

DrRyanHuang · 2022-11-14T03:38:56Z

@luotao1 意思是提供一个这样的测试用例？

import paddle
m = paddle.vision.models.resnet18(pretrained=True)
# extract layer1 and layer3, giving as names `feat1` and feat2`
new_m = paddle.vision.models.utils.IntermediateLayerGetter(m,
    {'layer1': 'feat1', 'layer3': 'feat2'})
out = new_m(paddle.rand([1, 3, 224, 224]))
print([(k, v.shape) for k, v in out.items()])

[('feat1', [1, 64, 56, 56]), 
 ('feat2', [1, 256, 14, 14])]

jerrywgz · 2022-11-14T05:49:44Z

python/paddle/vision/models/utils.py

@@ -12,6 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

+import paddle.nn as nn
+from collections import OrderedDict
+from typing import Dict



#43611 可以参考这个pr增加单测

jerrywgz · 2022-11-14T05:54:31Z

python/paddle/vision/models/utils.py

+            if name in return_layers:
+                del return_layers[name]
+            if not return_layers:
+                break


这里的layers表示return layer前所有layer？能否给出示例代码

是的

IntermediateLayerGetter 的目的就是运行 model 的前向传播，并将中间的某些层的输出拿出来

layers 之所以包括传入 model 的所有 sublayer，是因为要通过传入model 的前向传播

之后执行：

super(IntermediateLayerGetter, self).__init__(layers)

由于继承自 nn.LayerDict 则它包含的子层将被注册和添加

所以在 self.forward 中，能通过 for name, module in self.items(): 来迭代每一层操作

如果某一层的输出，是我们想要的，即if name in self.return_layers，则添加到返回out = OrderedDict() 中

self.return_layers 字典变量用来指示哪些层的输出需要返回，返回之后的名字叫什么

jerrywgz · 2022-11-14T05:55:52Z

python/paddle/vision/models/utils.py

+    Examples:
+        .. code-block:: python
+
+        >>> m = paddle.vision.models.resnet18(pretrained=True)


测试代码需要加import paddle，保证代码可以单独运行

2022-11-29 14:26:01 subprocess return code: 1 2022-11-29 14:26:01 Error Raised from Sample Code: 2022-11-29 14:26:01 stderr: File "samplecode_temp/paddle.vision.models.utils.IntermediateLayerGetter_example.py", line 4 2022-11-29 14:26:01 >>> import paddle 2022-11-29 14:26:01 ^ 2022-11-29 14:26:01 SyntaxError: invalid syntax 2022-11-29 14:26:01 2022-11-29 14:26:01 stdout:

https://xly.bce.baidu.com/paddlepaddle/paddle/newipipe/detail/7275600/job/20709077
示例代码跑不过，把>>>都去掉吧

add `import paddle`

jerrywgz · 2022-11-14T09:19:55Z

请在提交的代码前使用pre-commit 做下格式检查

pip install pre-commit==2.17.0
pre-commit install
pre-commit run --files python/paddle/vision/models/utils.py

…develop Nothing important.

jerrywgz · 2022-11-14T10:59:55Z

还请下添加单测~

luotao1

"/home/Paddle/build/python/paddle/fluid/tests/unittests/test_IntermediateLayerGetter.py", line 18, in <module>
2022-11-25 02:39:14     from paddle.vision.models import IntermediateLayerGetter
2022-11-25 02:39:14 ImportError: cannot import name 'IntermediateLayerGetter' from 'paddle.vision.models'

现在CI上流水线都挂在这个单测上，请先修一下。需要在 utils.py里加一个__all__ 来加上这个新的API

luotao1 · 2022-11-28T07:56:22Z

2022-11-28 14:20:37 	344 - test_IntermediateLayerGetter (Timeout)

PR-CI-Py3 和 PR-CI-Windows-OPENBLAS 上都是 CPU 环境，单测默认时间是15s，目前是超时的。
请减少下单测时间，或设置下单测时长，类似

Paddle/python/paddle/fluid/tests/unittests/CMakeLists.txt

Lines 1026 to 1030 in 89b5fc7

    
           if(WIN32) 
        
             set_tests_properties(test_static_save_load_large PROPERTIES TIMEOUT 900) 
        
             set_tests_properties(test_paddle_save_load PROPERTIES TIMEOUT 250) 
        
           else() 
        
             set_tests_properties(test_static_save_load_large PROPERTIES TIMEOUT 600)

if(NOT WITH_GPU)
   set_tests_properties(test_IntermediateLayerGetter PROPERTIES TIMEOUT xxx)
endif()

DrRyanHuang · 2022-11-28T13:19:40Z

您好，打扰了，这是需要QA的同学的许可吧，我需要给他们发邮件吗？

You must have one QA (XieYunshen(Recommend) or chalsliu) approval for setting parameter RUN_TYPE as EXCLUSIVE, 
DIST, NIGHTLY, EXCLUSIVE:NIGHTLY or DISTNIGHTLY, or setting parameter SERIAL, or setting TIMEOUT properties.

luotao1 · 2022-11-29T03:16:42Z

单测时间太长了，特别是windows这条要跑15分钟：

Py3流水线的时间140s左右

2022-11-28 22:59:47     Start 344: test_IntermediateLayerGetter
2022-11-28 23:02:11     Test #344: test_IntermediateLayerGetter .....   Passed  143.95 sec
2022-11-28 23:02:11     Start 344: test_IntermediateLayerGetter
2022-11-28 23:04:28     Test #344: test_IntermediateLayerGetter .....   Passed  136.54 sec
2022-11-28 23:04:28     Start 344: test_IntermediateLayerGetter
2022-11-28 23:06:52 1/1 Test #344: test_IntermediateLayerGetter .....   Passed  144.56 sec

Windows-openblas流水线的时间840s左右

2022-11-29 02:25:02 Test project C:/home/workspace/Paddle/build
2022-11-29 02:25:02     Start 306: test_IntermediateLayerGetter
2022-11-29 02:39:07 1/1 Test #306: test_IntermediateLayerGetter .....   Passed  844.64 sec

luotao1 · 2022-11-29T03:17:19Z

python/paddle/fluid/tests/unittests/test_IntermediateLayerGetter.py

+    @paddle.no_grad()
+    def test_inter_result(self):
+
+        inp = paddle.randn([32, 3, 224, 224])


输入数据减少是否会降低单测时间？

嗯嗯是的，在AIstudio CPU环境测试，由75s下降至6s 🥳🥳🥳

目前的单测时间

py3：12s左右

2022-11-29 23:08:58 Start 344: test_IntermediateLayerGetter 2022-11-29 23:09:11 Test #344: test_IntermediateLayerGetter ..... Passed 12.55 sec 2022-11-29 23:09:11 Start 344: test_IntermediateLayerGetter 2022-11-29 23:09:22 Test #344: test_IntermediateLayerGetter ..... Passed 11.40 sec 2022-11-29 23:09:22 Start 344: test_IntermediateLayerGetter 2022-11-29 23:09:34 1/1 Test #344: test_IntermediateLayerGetter ..... Passed 11.31 sec

windows-openblas: 50s左右

2022-11-29 19:00:37 325/1433 Test #298: test_IntermediateLayerGetter ................................ Passed 51.91 sec

py3:

2022-11-30 14:25:47 Start 344: test_IntermediateLayerGetter 2022-11-30 14:25:58 Test #344: test_IntermediateLayerGetter ..... Passed 10.94 sec 2022-11-30 14:25:58 Start 344: test_IntermediateLayerGetter 2022-11-30 14:26:05 Test #344: test_IntermediateLayerGetter ..... Passed 7.32 sec 2022-11-30 14:26:05 Start 344: test_IntermediateLayerGetter 2022-11-30 14:26:13 1/1 Test #344: test_IntermediateLayerGetter ..... Passed 7.76 sec

windows-openblas

2022-11-30 17:17:55 Start 307: test_adam_op 2022-11-30 17:17:56 300/1433 Test #298: test_IntermediateLayerGetter ....... Passed 14.71 sec

终于算是卡着边界通过了😂😂😂

luotao1

LGTM

luotao1 · 2022-12-01T08:02:04Z

@jeff41404 的意见：
torchvision.models._utils.IntermediateLayerGetter 不是公开API，我们是否也写成内部API就行。如果加公开API需要和黑客松一样，在docs repo 中写设计文档并review，同时 paddle.vision.models.utils 这个路径之前是没有的，是否放这里也需要在设计文档中着重review

jerrywgz · 2022-12-01T09:07:50Z

我们和torchvision保持一致吧，把utils.py改成_utils.py 注意示例代码也做下相应修改

DrRyanHuang · 2022-12-01T09:25:35Z

@jerrywgz
请问下，utils.py 下还有_make_divisible 这个函数

def _make_divisible(v, divisor=8, min_value=None):
    """
    This function ensures that all layers have a channel number that is divisible by divisor
    You can also see at https://github.com/keras-team/keras/blob/8ecef127f70db723c158dbe9ed3268b3d610ab55/keras/applications/mobilenet_v2.py#L505
    Args:
        divisor (int): The divisor for number of channels. Default: 8.
        min_value (int, optional): The minimum value of number of channels, if it is None,
                the default is divisor. Default: None.
    """
    if min_value is None:
        min_value = divisor
    new_v = max(min_value, int(v + divisor / 2) // divisor * divisor)
    # Make sure that round down does not go down by more than 10%.
    if new_v < 0.9 * v:
        new_v += divisor
    return new_v

请问 _make_divisible 也要一同迁移吗？还是只是新建一个 _utils.py 放入 IntermediateLayerGetter 即可

luotao1 · 2022-12-01T10:44:20Z

请问 _make_divisible 也要一同迁移吗？

因为这个API只在内部使用，可以一同迁移

DrRyanHuang · 2022-12-05T04:29:38Z

打扰了，还是有一个单测没有通过，请问是我操作不对吗？还是我需要给RD和TPM发邮件得到许可呢 ?

Please find RD for approval first, and then find TPM for approval.

......

2022-12-03 17:21:57 There are 3 approved errors.
2022-12-03 17:21:57 **************************************************************
2022-12-03 17:21:58 API Difference is: 
2022-12-03 17:21:58 + paddle.vision.models._utils.IntermediateLayerGetter (ArgSpec(), ('document', 'f743c6e12bae41b9da3175f725b62106'))
2022-12-03 17:21:58 + paddle.vision.models._utils.IntermediateLayerGetter.forward (ArgSpec(args=['self', 'x'], varargs=None, varkw=None, defaults=None, kwonlyargs=[], kwonlydefaults=None, annotations={}), ('document', 'a8313a8e60085775349df5535ec8174b'))
2022-12-03 17:21:58 + approval_error=6

luotao1 · 2022-12-05T07:21:12Z

python/paddle/vision/models/_utils.py

+import paddle
+import paddle.nn as nn
+
+__all__ = ["IntermediateLayerGetter"]


打扰了，还是有一个单测没有通过，请问是我操作不对吗？还是我需要给RD和TPM发邮件得到许可呢 ?

因为是非公开API，你试试去掉21行后：

流水线是否能通过

你能否使用paddle.vision.models._utils.IntermediateLayerGetter

luotao1

LGTM

IntermediateLayerGetter

d12605e

paddle-bot bot added contributor External developers status: proposed labels Nov 12, 2022

Update utils.py

39b0f2c

luotao1 self-assigned this Nov 14, 2022

DrRyanHuang changed the title ~~Add IntermediateLayerGetter~~ del comment Nov 14, 2022

DrRyanHuang changed the title ~~del comment~~ Add IntermediateLayerGetter Nov 14, 2022

luotao1 assigned jerrywgz Nov 14, 2022

Update utils.py

bc9bd78

jerrywgz reviewed Nov 14, 2022

View reviewed changes

Update utils.py

cc210b0

add `import paddle`

DrRyanHuang added 2 commits November 14, 2022 18:02

pre-commit

31bd115

Merge branch 'develop' of https://github.com/DrRyanHuang/Paddle into …

1ca8277

…develop Nothing important.

add unitest and fix linear error

970ecbe

luotao1 reviewed Nov 28, 2022

View reviewed changes

DrRyanHuang added 2 commits November 28, 2022 03:56

fix can't import bug

27f7655

add

89b5fc7

add timeout PROPERTIES

17293f9

luotao1 reviewed Nov 29, 2022

View reviewed changes

DrRyanHuang added 2 commits November 29, 2022 05:51

decrease batchsize

4e8fcaf

del >>>

e33237b

switch big model and del AlexNet

fe1850c

luotao1 previously approved these changes Dec 1, 2022

View reviewed changes

jerrywgz previously approved these changes Dec 1, 2022

View reviewed changes

luotao1 requested a review from jeff41404 December 1, 2022 06:12

luotao1 assigned Ligoml Dec 1, 2022

utils => _utils

2670de7

DrRyanHuang dismissed stale reviews from jerrywgz and luotao1 via 2670de7 December 2, 2022 07:18

DrRyanHuang and others added 3 commits December 2, 2022 15:22

Merge branch 'develop' into develop

cf9e020

fix docstring codestyle Error: utils => _utils

09d254e

update pre-commit version

37a6ad1

jerrywgz previously approved these changes Dec 5, 2022

View reviewed changes

luotao1 reviewed Dec 5, 2022

View reviewed changes

del __all__

dc0ccc8

DrRyanHuang dismissed jerrywgz’s stale review via dc0ccc8 December 5, 2022 08:40

luotao1 approved these changes Dec 6, 2022

View reviewed changes

luotao1 merged commit 39adb22 into PaddlePaddle:develop Dec 6, 2022

lxsbupt pushed a commit to lxsbupt/Paddle that referenced this pull request Dec 17, 2022

Add IntermediateLayerGetter (PaddlePaddle#47908)

1059564

kevinfu1717 mentioned this pull request Jan 22, 2023

paddlepaddle保存的静态模型通过paddle.jit.load载入成 translated_layer，关于translated_layer的使用问题 #50003

Closed

DrRyanHuang mentioned this pull request Dec 10, 2023

WAVE SUMMIT+2023下半年飞桨开源之星评选-信息征集 PaddlePaddle/community#765

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add IntermediateLayerGetter #47908

Add IntermediateLayerGetter #47908

DrRyanHuang commented Nov 12, 2022

paddle-bot bot commented Nov 12, 2022

luotao1 commented Nov 14, 2022

DrRyanHuang commented Nov 14, 2022 •

edited

jerrywgz Nov 14, 2022

DrRyanHuang Nov 24, 2022

jerrywgz Nov 14, 2022

DrRyanHuang Nov 14, 2022 •

edited

jerrywgz Nov 14, 2022

DrRyanHuang Nov 14, 2022

luotao1 Nov 29, 2022

DrRyanHuang Nov 29, 2022

jerrywgz commented Nov 14, 2022 •

edited

jerrywgz commented Nov 14, 2022

luotao1 left a comment

luotao1 commented Nov 28, 2022

DrRyanHuang commented Nov 28, 2022 •

edited

luotao1 commented Nov 29, 2022

luotao1 Nov 29, 2022

DrRyanHuang Nov 29, 2022

luotao1 Nov 30, 2022

DrRyanHuang Nov 30, 2022

luotao1 left a comment

luotao1 commented Dec 1, 2022

jerrywgz commented Dec 1, 2022

DrRyanHuang commented Dec 1, 2022 •

edited

luotao1 commented Dec 1, 2022

DrRyanHuang commented Dec 5, 2022 •

edited

luotao1 Dec 5, 2022

DrRyanHuang Dec 6, 2022

luotao1 left a comment

Add IntermediateLayerGetter #47908

Add IntermediateLayerGetter #47908

Conversation

DrRyanHuang commented Nov 12, 2022

PR types

PR changes

Describe

paddle-bot bot commented Nov 12, 2022

luotao1 commented Nov 14, 2022

DrRyanHuang commented Nov 14, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DrRyanHuang Nov 14, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jerrywgz commented Nov 14, 2022 • edited

jerrywgz commented Nov 14, 2022

luotao1 left a comment

Choose a reason for hiding this comment

luotao1 commented Nov 28, 2022

DrRyanHuang commented Nov 28, 2022 • edited

luotao1 commented Nov 29, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luotao1 left a comment

Choose a reason for hiding this comment

luotao1 commented Dec 1, 2022

jerrywgz commented Dec 1, 2022

DrRyanHuang commented Dec 1, 2022 • edited

luotao1 commented Dec 1, 2022

DrRyanHuang commented Dec 5, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luotao1 left a comment

Choose a reason for hiding this comment

DrRyanHuang commented Nov 14, 2022 •

edited

DrRyanHuang Nov 14, 2022 •

edited

jerrywgz commented Nov 14, 2022 •

edited

DrRyanHuang commented Nov 28, 2022 •

edited

DrRyanHuang commented Dec 1, 2022 •

edited

DrRyanHuang commented Dec 5, 2022 •

edited