Fix: Download image to cache_path #23

SilverRainZ · 2021-06-16T13:04:33Z

No description provided.

terencehonles

I believe I noticed this too and possibly started doing something with it, but I either broke too much or got distracted with something else. I belive this change makes sense, so thanks!

P.S. you have a typo for "image" in your PR title

terencehonles · 2021-06-17T17:39:22Z

sphinxcontrib/images.py

@@ -157,17 +157,16 @@ def run(self):
        if self.is_remote(self.arguments[0]):
            img['remote'] = True
            if download:
-                img['uri'] = os.path.join('_images', hashlib.sha1(self.arguments[0].encode()).hexdigest())
+                reluri = os.path.join(conf['cache_path'], hashlib.sha1(self.arguments[0].encode()).hexdigest())
+                img['uri'] = os.path.join('/', reluri) # relative to source dir


I'm a little confused about this, do you need to os.path.join the relative path to /? Is this actually the URL that Sphinx will use to reference the image in the HTML? If it's actually that it might make sense to use urllib.parse.urljoin in order to make it more obvious what you're doing.

# *nix >>> os.path.join('/', 'test/key') '/test/key' >>> urllib.parse.urljoin('/', 'test/key') '/test/key'

I think I wrote a confusing comment :'(.

The prefixed / means the uri is "relative to source dir"(rootof sphinx project) but not "relative to current document's dir". Without it, sphinx will try load image from srcdir/foo/bar/_images, while our cache_path is srcdir/_images/.

And the URI is actualy a filesystem path here, so I think we need os.path.join.

My testing shows me thumbnail directives with :download: true were copied correctly to _build/html/_images/ and the URL set correctly before this PR (e.g. in 0.9.3).

Thumbnail directives (local or remote) in index.html turn into _images/<imagefile>-URLs and thumbnail directives in subdir/subfile.rst turn into ../_images/<imagefile>-URLs (because the html-file resides in a subdirectory compared to index.html ../_images are used).

Edit: clarity

terencehonles · 2021-06-17T17:40:46Z

sphinxcontrib/images.py

            else:
                img['uri'] = self.arguments[0]
                img['remote_uri'] = self.arguments[0]
        else:
            img['uri'] = self.arguments[0]
            img['remote'] = False
-            env.images.add_file('', img['uri'])


Is this change needed? I'm not completely up to speed about env.images since I've not looked at this code in quite awhile.

We don't need to add file by self -- because we return a image node, it will be processed by sphinx's env collector.

In contrast to line 163 which seems necessary (see https://github.com/sphinx-contrib/images/pull/23/files/4d656516ca9453f46d158ca29a20fe8e22ff5674#r660153218) line 170 does not appear to be necessary, local images are copied to the _build/html/_images/ without this line. See also my comment here #23 (comment)

Edit: clarity

jonascj · 2021-06-17T20:17:14Z

Thank you for contributing @SilverRainZ and thank you for reviewing @terencehonles. I'll have a look this weekend or next Wednesday at the latest.

SilverRainZ · 2021-06-19T03:47:36Z

Thank you for contributing @SilverRainZ and thank you for reviewing @terencehonles. I'll have a look this weekend or next Wednesday at the latest.

Please also see my co-maintainer request at #20, thank you!

jonascj · 2021-06-28T22:18:18Z

@SilverRainZ I tested your PR with sphinx 4.0.2 and sphinxcontrib-images from this PR (commit id 4d65651 [1]), a fresh sphinx project [2] and the following content:

conf.py:

extensions = [                                                                  
        'sphinxcontrib.images',                                                     
        ]
images_config = {'cache_path': '_custom_cache' }

index.html

Test remote images download
===========================

.. thumbnail:: https://upload.wikimedia.org/wikipedia/meta/0/08/Wikipedia-logo-v2_1x.png
    :width: 50%
    :download: false

.. thumbnail:: https://upload.wikimedia.org/wikipedia/meta/0/08/Wikipedia-logo-v2_1x.png
    :width: 50%
    :download: true

I get a directory _custom_cache created next to conf.py and the Wiki logo is downloaded to that directory as f453ce6fa37bfa485a753fc8b2286dee1df0f620. So far so good.

I however do not get the downloaded image copied to _build/html/_images/ and hence when I view index.html in a web browser the image with :download: trueis not shown.

The file is copied if you add env.images.add_file('', reluri) back to images.py:

if self.is_remote(self.arguments[0]):                                   
    img['remote'] = True                                                
    if download:                                                        
        reluri = os.path.join(conf['cache_path'], hashlib.sha1(self.arguments[0].encode()).hexdigest())
        img['uri'] = os.path.join('/', reluri) # relative to source dir 
        img['remote_uri'] = self.arguments[0]                           
        env.remote_images[img['remote_uri']] = reluri                   
        env.images.add_file('', reluri) # Needed here                                
    else:                                                               
        img['uri'] = self.arguments[0]                                  
        img['remote_uri'] = self.arguments[0]                           
else:                                                                   
    img['uri'] = self.arguments[0]                                      
    img['remote'] = False
    # Not needed here

Why remote images need to be added with add_file explicitly when local images do not (I tested with a png-file next to index.html and it gets copied correctly by your PR, it is only remote downloaded images which do not get copied).

Could you verify these findings and if you find the same as I do, update your PR.

[1] git clone (remember git submodule update --init --recursive for the submodule), checkout the PR, pip install -e . (or pip install -e git+https://github.com/sphinx-contrib/images@4d656516ca9453f46d158ca29a20fe8e22ff5674#egg=sphinxcontrib_images)

[2] sphinx-quickstart -q -p PR23 -a Test --extensions sphinxcontrib.images

jonascj

See also #23 (comment) as already mentioned.

jonascj · 2021-06-28T22:21:00Z

sphinxcontrib/images.py

                img['remote_uri'] = self.arguments[0]
-                env.remote_images[img['remote_uri']] = img['uri']
-                env.images.add_file('', img['uri'])


I find that remote images with :download: true are downloaded to the custom cache directory but not copied to _build/html/_images/ upon building. Hence they are not shown in the built HTML-docs. Adding env.images.add_file() back fixes this.

See also my comment here #23 (comment)

jonascj · 2021-06-28T22:22:53Z

sphinxcontrib/images.py

            else:
                img['uri'] = self.arguments[0]
                img['remote_uri'] = self.arguments[0]
        else:
            img['uri'] = self.arguments[0]
            img['remote'] = False
-            env.images.add_file('', img['uri'])


In contrast to line 163 which seems necessary (see https://github.com/sphinx-contrib/images/pull/23/files/4d656516ca9453f46d158ca29a20fe8e22ff5674#r660153218) line 170 does not appear to be necessary, local images are copied to the _build/html/_images/ without this line. See also my comment here #23 (comment)

Edit: clarity

jonascj · 2021-06-28T22:35:56Z

sphinxcontrib/images.py

@@ -157,17 +157,16 @@ def run(self):
        if self.is_remote(self.arguments[0]):
            img['remote'] = True
            if download:
-                img['uri'] = os.path.join('_images', hashlib.sha1(self.arguments[0].encode()).hexdigest())
+                reluri = os.path.join(conf['cache_path'], hashlib.sha1(self.arguments[0].encode()).hexdigest())
+                img['uri'] = os.path.join('/', reluri) # relative to source dir


My testing shows me thumbnail directives with :download: true were copied correctly to _build/html/_images/ and the URL set correctly before this PR (e.g. in 0.9.3).

Thumbnail directives (local or remote) in index.html turn into _images/<imagefile>-URLs and thumbnail directives in subdir/subfile.rst turn into ../_images/<imagefile>-URLs (because the html-file resides in a subdirectory compared to index.html ../_images are used).

Edit: clarity

SilverRainZ · 2021-08-13T06:22:58Z

Sorry for delay, I have add the missing env.images.add_file call for downloaded image.

Fix: Download iamge to cache_path

1f705bd

SilverRainZ mentioned this pull request Jun 17, 2021

Incremental build is not working when the plugin enabled #8

Closed

SilverRainZ added 2 commits June 17, 2021 21:50

URI of image node should be prefixed with '/'

f9084e0

No need to add file to env by self

4d65651

SilverRainZ force-pushed the fix-cache-path branch from c933d80 to 4d65651 Compare June 17, 2021 14:17

terencehonles reviewed Jun 17, 2021

View reviewed changes

SilverRainZ changed the title ~~Fix: Download iamge to cache_path~~ Fix: Download image to cache_path Jun 19, 2021

jonascj reviewed Jun 28, 2021

View reviewed changes

jonascj mentioned this pull request Jun 28, 2021

Add tests #25

Open

Manually copy downloaded image to env

747bf48

rlaphoenix mentioned this pull request Jul 19, 2022

Ensure download destination directory exists #31

Closed

jonascj mentioned this pull request Aug 1, 2022

Change the thumbnail cursor #29

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix: Download image to cache_path #23

Fix: Download image to cache_path #23

SilverRainZ commented Jun 16, 2021

terencehonles left a comment

terencehonles Jun 17, 2021

SilverRainZ Jun 19, 2021

jonascj Jun 28, 2021 •

edited

terencehonles Jun 17, 2021

SilverRainZ Jun 19, 2021 •

edited

jonascj Jun 28, 2021 •

edited

jonascj commented Jun 17, 2021

SilverRainZ commented Jun 19, 2021

jonascj commented Jun 28, 2021

jonascj left a comment

jonascj Jun 28, 2021

jonascj Jun 28, 2021 •

edited

jonascj Jun 28, 2021 •

edited

SilverRainZ commented Aug 13, 2021

Fix: Download image to cache_path #23

Are you sure you want to change the base?

Fix: Download image to cache_path #23

Conversation

SilverRainZ commented Jun 16, 2021

terencehonles left a comment

Choose a reason for hiding this comment

terencehonles Jun 17, 2021

Choose a reason for hiding this comment

SilverRainZ Jun 19, 2021

Choose a reason for hiding this comment

jonascj Jun 28, 2021 • edited

Choose a reason for hiding this comment

terencehonles Jun 17, 2021

Choose a reason for hiding this comment

SilverRainZ Jun 19, 2021 • edited

Choose a reason for hiding this comment

jonascj Jun 28, 2021 • edited

Choose a reason for hiding this comment

jonascj commented Jun 17, 2021

SilverRainZ commented Jun 19, 2021

jonascj commented Jun 28, 2021

jonascj left a comment

Choose a reason for hiding this comment

jonascj Jun 28, 2021

Choose a reason for hiding this comment

jonascj Jun 28, 2021 • edited

Choose a reason for hiding this comment

jonascj Jun 28, 2021 • edited

Choose a reason for hiding this comment

SilverRainZ commented Aug 13, 2021

jonascj Jun 28, 2021 •

edited

SilverRainZ Jun 19, 2021 •

edited

jonascj Jun 28, 2021 •

edited

jonascj Jun 28, 2021 •

edited

jonascj Jun 28, 2021 •

edited