Add configurable options to improve `output.screen` node #547

leeping-ng · 2021-12-01T08:28:24Z

closes #543

Summary of improvements:

Gives users the option to resize the output window via configs, for situations such as high-res videos on a small laptop screen, where not the entire output window can be seen
The window size can now also be adjusted by clicking and dragging the output window
The top left corner of the output window will now be at the top left corner of the screen by default. It can also be changed in the configs. Previously, the top left corner of the output window would be around the middle of my screen, and I always had to drag the output window up to see the whole thing.
Gives users the option to change the name of the output window, when previously it was fixed as "PeekingDuck"

ongtw

Provided some suggestions.

ongtw · 2021-12-02T02:44:13Z

peekingduck/configs/output/screen.yml

+window_height: 720
+window_x_coord: 0
+window_y_coord: 0


Suggest OOP-ing the above with a window object:

window: { name: "PeekingDuck", x: 0, y: 0, width: 1280, height: 720, }

Facilitates future attribute extension.

I was thinking of this at first, but thought it would be clunky when users read our RTD page. I'd have to lump all the descriptions under one Dict type which can look messy with different descriptions. Also there's a mix of int and str types that get lost when all are lumped under Dict.

Example from resize config from input.live - I think this is simple enough to be nested though.

What do you think?

Alternatively, I could do it this way, which is a good compromise:

window_name: "PeekingDuck" window_loc: { x: 0, y: 0 } window_size: { width: 1280, height: 720 }

The resize (Dict) example also relies on the user to infer the bool and int types, so it actually servers to highlight the point made on lost type info.

The alternative you suggested is a compromise that I am neutral about.

Another suggestion is to remove window_ totally, just have

inputs: [...] outputs: [...] name: "PeekingDuck" x: 0 y: 0 width: 1280 height: 720

and the code will be self.height, self.width, etc.
Since, after all, the screen is the display.

Tried the last suggestion, but the keyword name causes an AttributeError: can't set attribute.

In the end I went for the in-between solution, the RTD looks like this:

Cool, let's go with the in-between solution.
Good reminder that name is reserved, for future similar cases, we'll have to use title or label.

resizeWindow() doesn't work for opencv >= 4.5.x, only applying resize() to the image works. Unfortunately, it is the opposite case for opencv == 4.1.2.30. To avoid discrepancies in behaviour, in this PR, will go with the solution that works for 4.5.x, and will create another PR to upgrade our openCV version (issue #556).

Added a boolean config for resizing window that's consistent with input.live and input.recorded.

ongtw · 2021-12-02T02:48:17Z

peekingduck/pipeline/nodes/output/screen.py

    """

    def __init__(self, config: Dict[str, Any] = None, **kwargs: Any) -> None:
        super().__init__(config, node_path=__name__, **kwargs)
+        cv2.namedWindow(self.window_name, cv2.WINDOW_NORMAL)
+        cv2.moveWindow(self.window_name, self.window_x_coord, self.window_y_coord)
+        cv2.resizeWindow(self.window_name, self.window_width, self.window_height)


In line with the suggested window object above, Node configs will accept a window object and code will reference its attributes via self.window.name, etc.

Updated accordingly.

peekingduck/pipeline/nodes/output/screen.py

ongtw · 2021-12-02T10:23:43Z

Tested your feature branch by setting window loc to (10, 10) and size to 720x360. Works correctly with input.recorded but not with input.live.

This is using input.live: window is shifted to (10, 10) but size is larger than 720x360.
cf. my screen is 1440x900. run_config.yml contents in top-right window.

This is using input.recorded: window correctly shifted to (10, 10) and size correctly set to 720x360.
cf. my screen is 1440x900. run_config.yml contents in top-right window.

leeping-ng · 2021-12-09T04:41:01Z

Tested your feature branch by setting window loc to (10, 10) and size to 720x360. Works correctly with input.recorded but not with input.live.

This is using input.live: window is shifted to (10, 10) but size is larger than 720x360. cf. my screen is 1440x900. run_config.yml contents in top-right window.

This is using input.recorded: window correctly shifted to (10, 10) and size correctly set to 720x360. cf. my screen is 1440x900. run_config.yml contents in top-right window.

Looks like the issue is caused by the opencv version. I had no issues when using opencv-python-4.1.2.30, but it didn't resize for both input.recorded and input.live when I upgraded to opencv-python-4.5.1.48. Will dig deeper into this.

ongtw · 2021-12-09T05:53:44Z

Looks like the issue is caused by the opencv version. I had no issues when using opencv-python-4.1.2.30, but it didn't resize for both input.recorded and input.live when I upgraded to opencv-python-4.5.1.48. Will dig deeper into this.

opencv for M1 Mac is 4.5.1. Looks like behavioural changes from earlier versions.

leeping-ng · 2021-12-16T01:18:37Z

Looks like the issue is caused by the opencv version. I had no issues when using opencv-python-4.1.2.30, but it didn't resize for both input.recorded and input.live when I upgraded to opencv-python-4.5.1.48. Will dig deeper into this.

opencv for M1 Mac is 4.5.1. Looks like behavioural changes from earlier versions.

Added issue #556 - I'll raise another PR to update our opencv version to 4.5.x, and the solution in this PR will tackle behaviour for 4.5.x and higher.

ongtw

Suggested a minor refactoring.

ongtw · 2021-12-16T10:34:13Z

peekingduck/pipeline/nodes/output/screen.py

+            resized_img = cv2.resize(
+                inputs["img"], (self.window_size["width"], self.window_size["height"])
+            )
+            cv2.imshow(self.window_name, resized_img)


Would like to propose streamlining the code segment in lines 55-61 as:

img = inputs["img"] if self.window_size["do_resizing"]: img = cv2.resize(img, (self.window_size["width"], self.window_size["height"])) cv2.imshow(self.window_name, img)

This looks better, updated.

ongtw · 2021-12-16T10:58:40Z

Tested the following scenario (M1 Mac opencv 4.5.1):

set window resizing = True, width = 720, height = 360
run PKD: 720x360 window appears with recorded input
manually resize PKD's window

The PKD window is now the new size, even though the cv2.resize(...) portion still runs (added print statement to confirm), i.e. the window size is now different from the specified 720x360.

I suppose this is an "acceptable" behavior?

…g and dragging

leeping-ng · 2021-12-17T01:15:52Z

Tested the following scenario (M1 Mac opencv 4.5.1):

set window resizing = True, width = 720, height = 360

run PKD: 720x360 window appears with recorded input

manually resize PKD's window

The PKD window is now the new size, even though the cv2.resize(...) portion still runs (added print statement to confirm), i.e. the window size is now different from the specified 720x360.

I suppose this is an "acceptable" behavior?

Yes, I think this is acceptable. To let users know that this may happen (and that it's an option to manually resize), I added to the window_size config description: "The size of the displayed window can also be adjusted by clicking and dragging."

ongtw

Updates 👍🏼

feat: add configs to output screen

901b4a7

leeping-ng requested a review from ongtw December 1, 2021 08:42

ongtw added the enhancement New feature or request label Dec 2, 2021

ongtw reviewed Dec 2, 2021

View reviewed changes

feat: nested related configs for cleaner code

9fe2def

feat: update resizing on opencv version >= 4.5

aafeb28

ongtw reviewed Dec 16, 2021

View reviewed changes

leeping-ng added 2 commits December 17, 2021 09:10

feat: minor refactor to streamline code

98d3f78

feat: added note in config that window size can be changed by clickin…

9762dc8

…g and dragging

ongtw approved these changes Dec 17, 2021

View reviewed changes

ongtw merged commit 6527660 into aisingapore:dev Dec 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add configurable options to improve `output.screen` node #547

Add configurable options to improve `output.screen` node #547

leeping-ng commented Dec 1, 2021

ongtw left a comment

ongtw Dec 2, 2021

leeping-ng Dec 2, 2021

leeping-ng Dec 2, 2021

ongtw Dec 2, 2021 •

edited

leeping-ng Dec 2, 2021

ongtw Dec 2, 2021

leeping-ng Dec 16, 2021

ongtw Dec 2, 2021

leeping-ng Dec 2, 2021

ongtw commented Dec 2, 2021 •

edited

leeping-ng commented Dec 9, 2021 •

edited

ongtw commented Dec 9, 2021

leeping-ng commented Dec 16, 2021

ongtw left a comment

ongtw Dec 16, 2021 •

edited

leeping-ng Dec 17, 2021

ongtw commented Dec 16, 2021 •

edited

leeping-ng commented Dec 17, 2021

ongtw left a comment

Add configurable options to improve output.screen node #547

Add configurable options to improve output.screen node #547

Conversation

leeping-ng commented Dec 1, 2021

ongtw left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ongtw Dec 2, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ongtw commented Dec 2, 2021 • edited

leeping-ng commented Dec 9, 2021 • edited

ongtw commented Dec 9, 2021

leeping-ng commented Dec 16, 2021

ongtw left a comment

Choose a reason for hiding this comment

ongtw Dec 16, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ongtw commented Dec 16, 2021 • edited

leeping-ng commented Dec 17, 2021

ongtw left a comment

Choose a reason for hiding this comment

Add configurable options to improve `output.screen` node #547

Add configurable options to improve `output.screen` node #547

ongtw Dec 2, 2021 •

edited

ongtw commented Dec 2, 2021 •

edited

leeping-ng commented Dec 9, 2021 •

edited

ongtw Dec 16, 2021 •

edited

ongtw commented Dec 16, 2021 •

edited