Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Self update (win_x86): "ERROR: Unable to remove the old version" #5632

Closed
6 of 9 tasks
Vangelis66 opened this issue Nov 24, 2022 · 10 comments · Fixed by #5970
Closed
6 of 9 tasks

Self update (win_x86): "ERROR: Unable to remove the old version" #5632

Vangelis66 opened this issue Nov 24, 2022 · 10 comments · Fixed by #5970
Assignees
Labels
bug Bug that is not site-specific help-wanted Extra attention is needed regression Works in youtube-dl/older yt-dlp

Comments

@Vangelis66
Copy link

Vangelis66 commented Nov 24, 2022

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

  • I'm reporting a bug unrelated to a specific site
  • I've verified that I'm running yt-dlp version 2022.11.11 (update instructions) or later (specify commit)
  • I've checked that all provided URLs are playable in a browser with the same IP and same login details
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
  • I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood

My older (but still worthy) laptop is still on a 32-bit Windows OS, thus the variant of yt-dlp that I use is yt-dlp_x86.exe (built on CPython 3.7).
Adjacent to the executable, I've placed batch file update_yt-dlp.cmd, whose content reads:

yt-dlp_x86 --ffmpeg-location ".." -vU

pause

Running that file while being on the release/stable channel of yt-dlp would update it as expected, e.g. from tag 2022.10.04 to (latest) 2022.11.11:

[debug] Command-line config: ['--ffmpeg-location', '..', '-vU']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2022.10.04 [4e0511f] (win32_exe)
[debug] Python 3.7.9 (CPython 32bit) - Windows-Vista-6.0.6003-SP2
[debug] Checking exe version: "..\ffmpeg" -bsfs
[debug] Checking exe version: "..\ffprobe" -bsfs
[debug] exe versions: ffmpeg 5.0 (fdk,setts), ffprobe 5.0, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Loaded 1690 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.10.04
[debug] Downloading _update_spec from https://github.com/yt-dlp/yt-dlp/releases/download/2022.11.11/_update_spec
Current Build Hash 609ab5d8e23313209db5a7a063e1890347f7e71d11a323e569023cac27bb8da1
Updating to version 2022.11.11 ...
[debug] Downloading yt-dlp_x86.exe from https://github.com/yt-dlp/yt-dlp/releases/download/2022.11.11/yt-dlp_x86.exe
[debug] Downloading SHA2-256SUMS from https://github.com/yt-dlp/yt-dlp/releases/download/2022.11.11/SHA2-256SUMS
Updated yt-dlp to version 2022.11.11

During the update process, the "old/current" executable is being renamed to yt-dlp_x86.exe.old and once the "new/updated" executable has been fully downloaded, the former is then deleted by the updating process (that much detail is not displayed inside the verbose log) ...

Let's now move on to the "dev" channel (aka "daily builds") of yt-dlp, which offers builds compiled from the master branch (many thanks Lesmiscore 😄 ):
https://github.com/ytdl-patched/yt-dlp/tags

Due to a number of consecutive Rebase on Upstream workflow run failures,
https://github.com/ytdl-patched/yt-dlp/actions,
tag 2022.11.14.334 is identical code-wise with tag 2022.11.09.810, see
ytdl-patched@1bf17ad
i.e. 2022.11.14.334 is built from below yt-dlp code snapshot:
[extractor/stripchat] Improve error message (yt-dlp#5475)

Running the aforementioned batch file on yt-dlp_x86.exe version 2022.11.14.334, will update it to the latest release (at the time of writing, that would be 2022.11.23.43044) in a normal/expected fashion:

[debug] Command-line config: ['--ffmpeg-location', '..', '-vU']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] ytdl-patched/yt-dlp version 2022.11.14.334 [1bf17ad] (win32_exe)
[debug] ** This build is unofficial daily builds, provided for ease of use.
[debug] ** Please do not ask for any support.
[debug] Python 3.7.9 (CPython 32bit) - Windows-Vista-6.0.6003-SP2 (OpenSSL 1.1.1g  21 Apr 2020)
[debug] Checking exe version: "..\ffmpeg" -bsfs
[debug] Checking exe version: "..\ffprobe" -bsfs
[debug] exe versions: ffmpeg 5.0 (fdk,setts), ffprobe 5.0, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1718 extractors
[debug] Fetching release info: https://api.github.com/repos/ytdl-patched/yt-dlp/releases/latest
Latest version: 2022.11.23.43044, Current version: 2022.11.14.334
[debug] Downloading _update_spec from https://github.com/ytdl-patched/yt-dlp/releases/download/2022.11.23.43044/_update_spec
Current Build Hash 7273160c27f375aebfa160ca92013c03c741fc10d9ac9be884dd9d3cf27897dc
Updating to version 2022.11.23.43044 ...
[debug] Downloading yt-dlp_x86.exe from https://github.com/ytdl-patched/yt-dlp/releases/download/2022.11.23.43044/yt-dlp_x86.exe
[debug] Downloading SHA2-256SUMS from https://github.com/ytdl-patched/yt-dlp/releases/download/2022.11.23.43044/SHA2-256SUMS
Updated yt-dlp to version 2022.11.23.43044

However, when the version to be updated is newer than v2022.11.14.334, this is where the new bug manifests itself; e.g., tag 2022.11.15.40298 (first BAD), see ytdl-patched@b9e88f3, is built from below yt-dlp code snapshot:
[extractor/youtube] Consider language in format de-duplication ;
running my batch file on that version will produce the verbose log that follows:

[debug] Command-line config: ['--ffmpeg-location', '..', '-vU']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] ytdl-patched/yt-dlp version 2022.11.15.40298 [b9e88f3] (win_x86_exe)
[debug] ** This build is unofficial daily builds, provided for ease of use.
[debug] ** Please do not ask for any support.
[debug] Python 3.7.9 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 (OpenSSL 1.1.1g  21 Apr 2020)
[debug] exe versions: ffmpeg 5.0 (fdk,setts), ffprobe 5.0, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1724 extractors
[debug] Fetching release info: https://api.github.com/repos/ytdl-patched/yt-dlp/releases/latest
Latest version: 2022.11.23.43044, Current version: 2022.11.15.40298
[debug] Downloading _update_spec from https://github.com/ytdl-patched/yt-dlp/releases/download/2022.11.23.43044/_update_spec
Current Build Hash ec1a0273b4dc9adf9fe118d3597f152ab1aa772223d1837a8c42f9b596397b88
Updating to version 2022.11.23.43044 ...
[debug] Downloading yt-dlp_x86.exe from https://github.com/ytdl-patched/yt-dlp/releases/download/2022.11.23.43044/yt-dlp_x86.exe
[debug] Downloading SHA2-256SUMS from https://github.com/ytdl-patched/yt-dlp/releases/download/2022.11.23.43044/SHA2-256SUMS
ERROR: Unable to remove the old version
Traceback (most recent call last):
  File "yt_dlp\update.py", line 272, in update
PermissionError: [WinError 5] Access is denied: 'D:\\Vangelis\\Installers\\youtube-dl\\yt-dlp\\yt-dlp_x86.exe.old'

... the end result being that yt-dlp does get updated to the latest available version, but previous renamed version, file yt-dlp_x86.exe.old, remains on disk undeleted 😞 ...
This isn't a deal-breaker after all, I can still delete that file manually via Windows Explorer, but isn't anymore the planned/expected behaviour...

From a brief dissection,
ytdl-patched/yt-dlp@3b87f4d...a4894d3
my bet is on
[cleanup] Misc
as being the culprit, because it was the most recent commit to "touch" file yt-dlp/update.py ...

Thanks for your attention 😄 , keep up improving yt-dlp 👍 !

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

For verbose logs, please see inside the description field; it seemed to me more apt to place them there in the case of this issue; apologies I have broken the template flow...
@Vangelis66 Vangelis66 added bug Bug that is not site-specific triage Untriaged issue labels Nov 24, 2022
@pukkandan pukkandan removed the triage Untriaged issue label Nov 24, 2022
@pukkandan pukkandan self-assigned this Nov 24, 2022
@pukkandan
Copy link
Member

The bug is introduced in 17fc3dc 🤔

@pukkandan pukkandan added the regression Works in youtube-dl/older yt-dlp label Nov 24, 2022
@pukkandan
Copy link
Member

For some odd reason, this fixes the issue 😕

diff --git a/yt_dlp/update.py b/yt_dlp/update.py
index ac3e28057..94a000105 100644
--- a/yt_dlp/update.py
+++ b/yt_dlp/update.py
@@ -43,7 +43,7 @@ def _get_variant_and_executable_path():
             # Ref: https://en.wikipedia.org/wiki/Uname#Examples
             if machine[1:] in ('x86', 'x86_64', 'amd64', 'i386', 'i686'):
                 machine = '_x86' if platform.architecture()[0][:2] == '32' else ''
-        return f'{remove_end(sys.platform, "32")}{machine}_exe', path
+        return f'{sys.platform}{machine}_exe', path

     path = os.path.dirname(__file__)
     if isinstance(__loader__, zipimporter):
@@ -74,8 +74,8 @@ def current_git_head():
 _FILE_SUFFIXES = {
     'zip': '',
     'py2exe': '_min.exe',
-    'win_exe': '.exe',
-    'win_x86_exe': '_x86.exe',
+    'win32_exe': '.exe',
+    'win32_x86_exe': '_x86.exe',
     'darwin_exe': '_macos',
     'darwin_legacy_exe': '_macos_legacy',
     'linux_exe': '_linux',

@Vangelis66
Copy link
Author

I wrote:

From a brief dissection,
ytdl-patched/yt-dlp@3b87f4d...a4894d3,
my bet is on
[cleanup] Misc
as being the culprit, because it was the most recent commit to "touch" file yt-dlp/update.py ...

... but as it turned out:

The bug is introduced in 17fc3dc 🤔

... IOW, I had 50-50 chances to spot the culprit:
https://github.com/ytdl-patched/yt-dlp/commits/7aaf4cd/yt_dlp/update.py
(either ytdl-patched@7aaf4cd or ytdl-patched@17fc3dc),
but I miserably failed there 😿 ...

One other thing I only noticed today is the way the executable yt-dlp_x86.exe advertises/identifies itself in the verbose logs:
v2022.11.14.334 (last GOOD):
[debug] ytdl-patched/yt-dlp version 2022.11.14.334 [1bf17ad] (win32_exe)

whereas v2022.11.15.40298 (first BAD):
[debug] ytdl-patched/yt-dlp version 2022.11.15.40298 [b9e88f3] (win_x86_exe)

i.e. win32_exe vs win_x86_exe, which was probably a valuable clue 😉 ...

Thanks for the fix 👍 , eventually it'll hit the master branch... 😄

@pukkandan pukkandan added the help-wanted Extra attention is needed label Dec 31, 2022
pukkandan added a commit that referenced this issue Dec 31, 2022
@Vangelis66
Copy link
Author

Vangelis66 commented Jan 3, 2023

Most unfortunately, I can't verify the original issue as fixed 😞 😭 ; not, at least, inside the "dev" yt-dlp "daily" update channel...
https://github.com/ytdl-patched/yt-dlp/tags

STR:

https://github.com/ytdl-patched/yt-dlp/releases/tag/2023.01.01.1919

That release was built on code ytdl-patched@b394e67, parent commit being:
ytdl-patched@2fb0f85, i.e. the one that "fixes/workarounds" original issue:
https://github.com/ytdl-patched/yt-dlp/commits/b394e67

If one updates that "release" to the latest available (2023.01.02.40298 at the time of writing), original issue persists:

[debug] Command-line config: ['--ffmpeg-location', '..', '-vU']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] ytdl-patched/yt-dlp version 2023.01.01.1919 [b394e67] (win32_x86_exe)
[debug] ** This build is unofficial daily builds, provided for ease of use.
[debug] ** Please do not ask for any support.
[debug] Plugins: ['Youtube_AgeGateBypassIE']
[debug] Python 3.7.9 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 (OpenSSL 1.1.1g  21 Apr 2020)
[debug] exe versions: ffmpeg 5.0 (fdk,setts), ffprobe 5.0, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.16.0, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1752 extractors
[debug] Fetching release info: https://api.github.com/repos/ytdl-patched/yt-dlp/releases/latest
Latest version: 2023.01.02.40298, Current version: 2023.01.01.1919
[debug] Downloading _update_spec from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.02.40298/_update_spec
Current Build Hash 5c64588d128bd2a90f588867bfda4bf38fe0827d846c1a897485ba325d1e2555
Updating to version 2023.01.02.40298 ...
[debug] Downloading yt-dlp_x86.exe from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.02.40298/yt-dlp_x86.exe
[debug] Downloading SHA2-256SUMS from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.02.40298/SHA2-256SUMS
ERROR: Unable to remove the old version
Traceback (most recent call last):
  File "yt_dlp\update.py", line 272, in update
PermissionError: [WinError 5] Access is denied: 'D:\\Vangelis\\Installers\\youtube-dl\\yt-dlp\\yt-dlp_x86.exe.old'

Updated yt-dlp to version 2023.01.02.40298

Any additional help will be highly welcome! 😉

Kind regards, Happy New Year 😄 !

@pukkandan pukkandan reopened this Jan 3, 2023
@pukkandan
Copy link
Member

The x64 version is fixed. Since I don't have an x86 system, debugging this is gonna be a pain

@honx4amiga
Copy link

honx4amiga commented Jan 3, 2023

i just wanted to update yt-dlp.exe 2022.11.11 to 2023.01.02 on windows 7 and i got the same error.
"ERROR: Unable to remove the old version"
yt-dlp.exe itsself is now updated to 2023.01.02 but i had to delete the .old file myself.

i'm not using any developer or beta builds, i'm only using update releases yt-dlp itsself finds and downloads.

i'm using the standalone .exe for windows 7, which has python included. i'm using 64bit version of yt-dlp standalone.

to check if i am using 64bit or 32bit exe i downloaded both binaries (yt-dlp.exe and yt-dlp_x86.exe) and compared file size between my today updated yt-dlp.exe and newly downloaded yt-dlp.exe and yt-dlp_x86.
yt-dlp.exe (auto updated) i am using is 13mb big (13.702.209 bytes) which is the same size as newly downloaded 64bit yt-dlp. downloaded 32bit yt-dlp only is 10,8mb big (11.399.003 bytes), which is smaller than the version i have. consequently i'm using 64bit version of yt-dlp standalone.

The x64 version is fixed. Since I don't have an x86 system, debugging this is gonna be a pain

since i'm using x64 version, and i got that error today after updating from 2022.11.11 to 2023.01.02, i don't think x64 version is fixed. :D

@pukkandan
Copy link
Member

When updating from old version to current, the updater running is that of the old one. So the fix will only be effective in next release

@Grub4K
Copy link
Member

Grub4K commented Jan 6, 2023

diff --git a/yt_dlp/update.py b/yt_dlp/update.py
index a3a731aef..c38bf1ef1 100644
--- a/yt_dlp/update.py
+++ b/yt_dlp/update.py
@@ -264,7 +264,7 @@ def update(self):
                 self._report_error('Unable to overwrite current version')
                 return os.rename(old_filename, self.filename)

-        if detect_variant() in ('win32_exe', 'py2exe'):
+        if detect_variant() in ('win32_exe', 'win32_x86_exe', 'py2exe'):
             atexit.register(Popen, f'ping 127.0.0.1 -n 5 -w 1000 & del /F "{old_filename}"',
                             shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
         elif old_filename:

Since it explicitly checked in ('win32_exe', 'py2exe') this would never be true for win32 32bit (which returned 'win32_x86_exe') and therefore fall through to the elif old_filename: branch, resulting in the error. I have tested this to work on Windows7 32bit.

@Vangelis66
Copy link
Author

Since I don't have an x86 system

FWIW, file yt-dlp_x86.exe.old fails to be deleted by the updating process even when the host is 64-bit:

yt-dlp_x86 -vU => 

[debug] Command-line config: ['-vU']
[debug] User config: []
[debug] System config: []
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] ytdl-patched/yt-dlp version 2023.01.02.40298 [c3224a2] (win32_x86_exe)
[debug] ** This build is unofficial daily builds, provided for ease of use.
[debug] ** Please do not ask for any support.
[debug] Python 3.7.9 (CPython AMD64 32bit) - Windows-7-6.1.7601-SP1 (OpenSSL 1.1.1g  21 Apr 2020)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.16.0, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1751 extractors
[debug] Fetching release info: https://api.github.com/repos/ytdl-patched/yt-dlp/releases/latest
Latest version: 2023.01.03.19419, Current version: 2023.01.02.40298
[debug] Downloading _update_spec from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.03.19419/_update_spec
Current Build Hash f93e52c0a5f310016e3933e3ce67788bb901540c02409cd3992510838fc5b260
Updating to version 2023.01.03.19419 ...
[debug] Downloading yt-dlp_x86.exe from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.03.19419/yt-dlp_x86.exe
[debug] Downloading SHA2-256SUMS from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.03.19419/SHA2-256SUMS
ERROR: Unable to remove the old version
Traceback (most recent call last):
  File "yt_dlp\update.py", line 272, in update
PermissionError: [WinError 5] Access is denied: 'D:\\Vangelis\\yt-dlp\\2023.01.02.40298\\x86\\yt-dlp_x86.exe.old'

Updated yt-dlp to version 2023.01.03.19419

(the log is two days old, when I got access to sister's Win7 SP1 x64 laptop 😉 ...)

pukkandan pushed a commit that referenced this issue Jan 6, 2023
@Vangelis66
Copy link
Author

When updating from old version to current,
the updater running is that of the old one.
So the fix will only be effective in next release

I can confirm the issue is fixed when updating from v2023.01.07.19419 ("daily" update channel) to latest "daily" build 32-bit (v2023.01.08.1919 as of this writing):

[debug] Command-line config: ['--ffmpeg-location', '..', '-vU']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] ytdl-patched/yt-dlp version 2023.01.07.19419 [7d5a286] (win_x86_exe)
[debug] ** This build is unofficial daily builds, provided for ease of use.
[debug] ** Please do not ask for any support.
[debug] Python 3.7.9 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 (OpenSSL 1.1.1g  21 Apr 2020)
[debug] exe versions: ffmpeg 5.0 (fdk,setts), ffprobe 5.0, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.16.0, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Extractor Plugins: AGB+NSIG (YoutubeIE)
[debug] Plugin directories: {<redacted>}
[debug] Loaded 1760 extractors
[debug] Fetching release info: https://api.github.com/repos/ytdl-patched/yt-dlp/releases/latest
Latest version: 2023.01.08.1919, Current version: 2023.01.07.19419
[debug] Downloading _update_spec from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.08.1919/_update_spec
Current Build Hash c38fdd344249afd7b86cd2e8064592f557f8925e74509a3047969a2f29e7e3fc
Updating to version 2023.01.08.1919 ...
[debug] Downloading yt-dlp_x86.exe from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.08.1919/yt-dlp_x86.exe
[debug] Downloading SHA2-256SUMS from https://github.com/ytdl-patched/yt-dlp/releases/download/2023.01.08.1919/SHA2-256SUMS
Updated yt-dlp to version 2023.01.08.1919

... with v2023.01.07.19419 being the first that incorporates the latest fix 5be214a 😉 ; job well executed everyone 🎉 , many thanks! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug that is not site-specific help-wanted Extra attention is needed regression Works in youtube-dl/older yt-dlp
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants