Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Won't upload program fully - stops uploading at random percentages (ESPTOOL-622) #842

Closed
1 task done
bheestandLMT opened this issue Feb 16, 2023 · 8 comments
Closed
1 task done

Comments

@bheestandLMT
Copy link

bheestandLMT commented Feb 16, 2023

Operating System

Windows 11 Pro

Esptool Version

4.4.0

Python Version

3.9.10

Chip Description

ESP32-S3

Device Description

DevKitC-1 v1.0
Using the USB port (not UART)

Hardware Configuration

Nothing else connected to the dev board

How is Esptool Run

PlatformIO Core 6.1.6 Home 3.4.3

Full Esptool Command Line that Was Run

esptool.py --chip esp32s3 --port "COM16" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 4MB 0x0 D:\Github_Desktop\smart_config.pio\build\esp32s3\bootloader.bin 0x8000 D:\Github_Desktop\smart_config.pio\build\esp32s3\partitions.bin 0x10000 .pio\build\esp32s3\firmware.bin

Esptool Output

PS D:\Github_Desktop\smart_config> python.exe "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool.py" --chip esp32s3 --port "COM16" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 4MB 0x0 D:\Github_Desktop\smart_config\.pio\build\esp32s3\bootloader.bin 0x8000 D:\Github_Desktop\smart_config\.pio\build\esp32s3\partitions.bin 0x10000 .pio\build\esp32s3\firmware.bin
esptool.py v4.4
Serial port COM16
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: 7c:df:a1:e2:3e:90
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00005fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00010000 to 0x000c4fff...
Compressed 20816 bytes to 13276...
Wrote 20816 bytes (13276 compressed) at 0x00000000 in 0.3 seconds (effective 491.3 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 103...
Wrote 3072 bytes (103 compressed) at 0x00008000 in 0.1 seconds (effective 428.0 kbit/s)...
Hash of data verified.
Compressed 740672 bytes to 473378...
Writing at 0x0002008a... (10 %)Traceback (most recent call last):
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool.py", line 34, in <module>
    esptool._main()
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\__init__.py", line 1026, in _main
    main()
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\__init__.py", line 826, in main
    operation_func(esp, args)
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\cmds.py", line 556, in write_flash
    esp.flash_defl_block(block, seq, timeout=timeout)
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 109, in inner
    return func(*args, **kwargs)
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 991, in flash_defl_block
    self.check_command(
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 407, in check_command
    val, data = self.command(op, data, chk, timeout=timeout)
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 376, in command
    p = self.read()
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 308, in read
    return next(self._slip_reader)
StopIteration
PS D:\Github_Desktop\smart_config> python.exe "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool.py" --chip esp32s3 --port "COM16" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 4MB 0x0 D:\Github_Desktop\smart_config\.pio\build\esp32s3\bootloader.bin 0x8000 D:\Github_Desktop\smart_config\.pio\build\esp32s3\partitions.bin 0x10000 .pio\build\esp32s3\firmware.bin
esptool.py v4.4
Serial port COM16
Connecting....
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: 7c:df:a1:e2:3e:90
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00005fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00010000 to 0x000c4fff...
Compressed 20816 bytes to 13276...
Wrote 20816 bytes (13276 compressed) at 0x00000000 in 0.3 seconds (effective 506.5 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 103...
Wrote 3072 bytes (103 compressed) at 0x00008000 in 0.1 seconds (effective 484.6 kbit/s)...
Hash of data verified.
Compressed 740672 bytes to 473378...
Writing at 0x00038254... (20 %)Traceback (most recent call last):
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool.py", line 34, in <module>
    esptool._main()
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\__init__.py", line 1026, in _main
    main()
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\__init__.py", line 826, in main
    operation_func(esp, args)
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\cmds.py", line 556, in write_flash
    esp.flash_defl_block(block, seq, timeout=timeout)
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 109, in inner
    return func(*args, **kwargs)
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 991, in flash_defl_block
    self.check_command(
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 407, in check_command
    val, data = self.command(op, data, chk, timeout=timeout)
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 376, in command
    p = self.read()
  File "C:\Users\b123\.platformio\packages\tool-esptoolpy\esptool\loader.py", line 308, in read
    return next(self._slip_reader)
StopIteration

More Information

I have tried different baud rates via the command line and in the Windows port settings - no difference.
I tried a dozen different USB cables with various lengths and manufacturers to no avail.
The command will work about 1 out of 25 tries - it is random. When it works, I never have an issue with the serial monitor on the same port - it always works fine if the program is able to successfully upload.
The percentage of upload completed is also random - stops at a different percentage almost every time (from 3% to 97%)

Other Steps to Reproduce

This was not an issue on my windows 10 machine that died - it may not have been same esptool version though.

I Have Read the Troubleshooting Guide

  • I confirm I have read the troubleshooting guide.
@github-actions github-actions bot changed the title Won't upload program fully - stops uploading at random percentages Won't upload program fully - stops uploading at random percentages (ESPTOOL-622) Feb 16, 2023
@dobairoland
Copy link
Collaborator

Could you please check if the issue persists with the latest master branch, i.e. 7b28699? You can find information in the documentation how to install it. I think this has been addressed since the v4.4 release.

@bheestandLMT
Copy link
Author

I updated to version 4.5 and am still having the same issues.

My computer is:
HP ZBook Fury 16 G9 Mobile Workstation PC
12th Gen Intel(R) Core(TM) i9-12950HX, 2300 Mhz, 16 Core(s), 24 Logical Processor(s)
128GB RAM

Here is the log:

`(["upload"], [".pio\build\esp32s3\firmware.bin"])
AVAILABLE: cmsis-dap, esp-bridge, esp-builtin, esp-prog, espota, esptool, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa
CURRENT: upload_protocol = esptool
BeforeUpload(["upload"], [".pio\build\esp32s3\firmware.bin"])
Auto-detected: COM17
"C:\Users\bheestand.platformio\penv\Scripts\python.exe" "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool.py" --chip esp32s3 --port "COM17" --baud 115200 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 8MB 0x0 D:\Github_Desktop\smart_config.pio\build\esp32s3\bootloader.bin 0x8000 D:\Github_Desktop\smart_config.pio\build\esp32s3\partitions.bin 0x10000 .pio\build\esp32s3\firmware.bin
esptool.py v4.5
Serial port COM17
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:51:57:e0
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00005fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00010000 to 0x000c2fff...
Compressed 20576 bytes to 13099...
Writing at 0x00000000... (100 %)
Wrote 20576 bytes (13099 compressed) at 0x00000000 in 0.5 seconds (effective 349.5 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 103...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (103 compressed) at 0x00008000 in 0.1 seconds (effective 337.8 kbit/s)...
Hash of data verified.
Compressed 730848 bytes to 467400...
Writing at 0x00010000... (3 %)
Writing at 0x0001ae46... (6 %)
Traceback (most recent call last):
File "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool_init_.py", line 1031, in main
main()
File "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool_init
.py", line 831, in main
operation_func(esp, args)
File "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool\cmds.py", line 580, in write_flash
esp.flash_defl_block(block, seq, timeout=timeout)
File "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool\loader.py", line 131, in inner
return func(*args, **kwargs)
File "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool\loader.py", line 1030, in flash_defl_block
self.check_command(
File "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool\loader.py", line 435, in check_command
val, data = self.command(op, data, chk, timeout=timeout)
File "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool\loader.py", line 404, in command
p = self.read()
File "C:\Users\bheestand.platformio\packages\tool-esptoolpy\esptool\loader.py", line 337, in read
return next(self._slip_reader)
StopIteration

A fatal error occurred: The chip stopped responding.
*** [upload] Error 2
============================================== [FAILED] Took 12.78 seconds ==============================================`

@bheestandLMT
Copy link
Author

bheestandLMT commented Feb 18, 2023

I was reading through some of the other responses and decided to try esptool version 4.3 and it uploaded fine - no issues.
Also I'm not sure I have the correct version of esptool with the changes needed. I did a 'pip install esptool' to get the latest but I'm don't think it's correct.
What pip install command should I run to ensure I have the correct version? (thanks)

@dobairoland
Copy link
Collaborator

There was a regression what we have already fixed. In order to check this, you cannot use pip but need to install from Github. Instructions are here:

https://docs.espressif.com/projects/esptool/en/latest/esp32/contributing.html#development-setup

@mrdude2478
Copy link

Here you go, this should fix it for you.
(modded)esptool.zip

It has a modded stub file which has a mod for esp32-s3 clock register address.

@JunOllyLi
Copy link

Here you go, this should fix it for you. (modded)esptool.zip

It has a modded stub file which has a mod for esp32-s3 clock register address.

@mrdude2478 , thanks! This esptool.exe seems working, so far. I tried to use pip to install the latest esptool from the repo, but it did not fix the problem.
So now I have a question, how can I use the esptool.exe in the "idf.py flash"?

Thanks!

@radimkarnis
Copy link
Collaborator

@JunOllyLi please see this explanation why that esptool.exe version works and why you probably shouldn't use it. We will release esptool v4.5.1 with a proper fix soon, it will be automatically available in your IDF installation.

espressif-bot pushed a commit that referenced this issue Feb 28, 2023
@radimkarnis
Copy link
Collaborator

I believe this should be now fully fixed in the latest esptool release. The USB-Serial/JTAG mode on the S3 is now stable.
Feel free to reopen this if issues persist. Thank you!

Jason2866 added a commit to Jason2866/esptool that referenced this issue Nov 4, 2023
* docs: espsecure remote signing using a HSM broken link fix

* fix(rfc2217_server): Use new reset sequences

* fix(ESP32-S3): Lower CPU freq to improve flasher stub stability

Closes espressif#832

* fix: Unknown chip (ID or magic number) error

* pyinstaller: fix glibc dependency on gnu/linux

pyinstaller package for linux is built within the ubuntu-latest image in
github workflow. This may cause prbolem with glibc symbol versions on
older distributions, where the new symbol versions are not available.
Fix this by building on the older ubuntu version.

Closes espressif#843

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>

* tests: Create custom `host_test` marker for tests without real chip connected

Closes espressif#838

* fix(ESP32-S3): Temporarily disable increasing CPU freq

Related to espressif#848

Related to espressif#842

* build: add esp_rfc2217_server to published scripts

Closes espressif#846

* Update version to v4.5.1

* Update version to v4.6-dev

* espefuse: Hide sensitive info by default during burning burn_key and burn_key_digest

Adds --show-sensitive-info flag for two commands:
burn_key and burn_key_digest.

* flasher_stub: pass -mabi=ilp32 to the RISC-V compiler

This is a no-op change for the upstream toolchain (compiled stubs are
binary identical), but is required when building with Debian's
riscv64-unknown-elf-gcc compiler.

* flasher_stub: allow passing extra CFLAGS

The flasher_stub Makefile allows for some system-local configuration,
either through local.mk, or through environment variables.

For example, the compiler prefix can be overridden, by defining e.g.
CROSS_ESPRISCV32. However, passing additional flags to the compiler
isn't possible right now. Add EXTRA_CFLAGS and EXTRA_CFLAGS_ESPRISCV32
to allow for that option.

* flasher_stub: collect all targets at the top, DRY

* flasher_stub: make target selection more modular

Rather than a special "make esp32", create WITHOUT_* variables to
selectively disable chip families. Currently, WITHOUT_ESP8266,
WITHOUT_ESP32_XTENSA and WITHOUT_ESP32_RISCV32 are defined, but the code
can be easily adjusted to allow for all kinds of other
sets/combinations.

* flasher_stub: create %.json targets, make all a proper PHONY

* flasher_stub: drop --embed from wrap_stub.py

Since commit 94f29a5 the flasher stub is not embedded in the Python
source, but rather included as simple json files.

As such, wrap_stub.py --embed was converted to basically just vary the
build dir. Rather than keep this indirection and for better clarity,
remove that piece of code and replace it by a simple "cp" in the
Makefile.

While at it, replace the target name from "embed" to "install", as this
more akin to a "make install" step.

* espefuse: Support burning ECDSA_KEY from pem file

- fix some assert check in test_espefuse.py
- add tests to cover the new functionality

* espefuse(c2): Fix BLOCK_KEY0 view for summary cmd when SB + FE keys are burnt

For C2 secure boot + flash enc block, we saw that in summary cmd
"0's" from secure boot digest part (upper 128 bit) were translated
into "?'s" when the block was read protected.
For C2, we should apply this translation for lower 128 bits only.

* fix(ESP32-C6): Fix get_pkg_version and get_{major,minor}_chip_version

* image_info: removed check that reserved bytes in image header are zero

IDF may start using parts of the reserved bytes in the extended header at any time,
which will break chip auto-detect in image_info.

* build: limit max cryptography version to 40

* fix: Set flash parameters even with --flash_size keep

Related to espressif/esp-idf#10788

Related to espressif/esp-idf#10959

* build: add arm and arm64 as build target

Closes espressif#845

* Fix typo in serial protocol docs

Fixes misspelling of `triggered` in serial protocol docs.

Signed-off-by: hasheddan <georgedanielmangum@gmail.com>

* Support more recent reedsolo packages

- https://github.com/tomerfiliba-org/reedsolomon/releases/tag/v1.6.1 - this seems to be related to licenses only.
- https://github.com/tomerfiliba-org/reedsolomon/releases/tag/v1.7.0 - this is related to installation.

Closes espressif#872

* build(arm): add pip extra url for github action build

* ci: Fix libffi symlinks for cryptography>=40

* espefuse: Prevent burning XTS_AES and ECDSA keys into BLOCK9 (BLOCK_KEY5)

eFuse module has a hardware bug.
It is related to ESP32-C3, C6, S3, H2 chips:
    - BLOCK9 (BLOCK_KEY5) can not be used by XTS_AES keys.
For H2 chips, the BLOCK9 (BLOCK_KEY5) can not be used by ECDSA keys.
S2 does not have such a hardware bug.

* image_info: Display disabled WP pin as disabled

The image formats know about the special value 0xee used to disable WP.
Display this with image_info.

E.g.:

ESP32-C3 extended image header
==============================
WP pin: 0xee (disabled)

* image_info: Print chip ID's name if known

Example:

Flash pins drive settings: clk_drv: 0x0, q_drv: 0x0, d_drv: 0x0, cs0_drv: 0x0, hd_drv: 0x0, wp_drv: 0x0
Chip ID: 5 (ESP32-C3)
Minimal chip revision: v0.0, (legacy min_rev = 0)
Maximal chip revision: v655.35

An unknown ID will be printed as:

Chip ID: 42 (Unknown ID)

* tests: Make the testsuite Windows compatible

* espefuse: Adds external esp instance

Closes espressif#873

* espefuse: Improve efuse error viewing

* espefuse: Explicit setting of efuse time settings

EFUSE_PWR_ON_NUM in C3 has default value = 0x2880, now = 0x3000

* docs(Boot log): Add all esp targets to cover boot troubleshooting

Closes espressif#732

* fix: USB-JTAG-Serial PID detection error

* esptool: Move bootdesc on the top of the ram segment

* espefuse: Move some vars under init method to speedup tool after adding yaml support

* espefuse: Adds yaml efuse description files for all chip

- esptool: Updates eFuses wafer major&minor versions
- esptool(esp32c6): Adds package versions
- espefuse(esp32c6): Replace PKG_VERSION BLK_VERSION_MINOR BLK_VERSION_MAJOR
- espefuse(esp32c6): Adds adc calib efuses
- espefuse: Adds yaml files for Build with PyInstaller

* efuse(H2): Adds RF Calibration Information

* espsecure: Improve error message for incorrect PEM format

Closes espressif#881

* bugfix(usb_jtag_serial): Autofeed super watchdog (SWD) to avoid resets during flashing

* esptool: Read 64-bit MAC address on C6 and H2

* bugfix: Adjust wrapper scripts to not import themselves

* bugfix(espsecure): Print a clear error message if incompatible OpenSSL backend is used

Closes espressif#878

* fix: inconsistent usage of dirs separator

* feat(esptool): add option to dump whole flash based on detected size

Closes espressif#461

* Update version to v4.6

* Update version to v4.7-dev

* fix(ESP32-S3): Correct RTC WDT registers to fix resets during flashing

* Update version to v4.6.1

* Update version to v4.7-dev

* docs: add explanation for flash_id example to avoid confusion

* docs(boot-log): fix list formatting

* docs: add c2, c6 and h2 as build targets

* fix(compressed upload): Accept short data blocks with only Adler-32 bytes

* fix(CH9102F): Suggest to install new serial drivers if writing to RAM fails

* esptool & espefuse: Fix byte order in MAC (for C6 and H2)

MAC: 60:55:f9:ff:fe:f7:2c:a2 (EUI64, used for IEEE802154)
BASE MAC: 60:55:f9:f7:2c:a2 (used for BT)
MAC_EXT: ff:fe

* Update version to v4.6.2

* Update version to v4.7

* change: Add conventional precommit linter

* ci(pre-commit): Update version of `conventional-precommit-linter`

* feat(get_security_info): Improved the output format and added more
details

* fix(esp32-c2): Enable flashing in secure download mode

Closes espressif#895

* ci: Add DangerJS checks to GL and GH

* feat(esptool): Add PICO package for ESP32S3 and flash/psram efuses

* feat(esptool): Add tests for get_chip_features

* feat(esptool): Add new packages for ESP32C3 and flash efuses

* fix(expand file args): Correctly print the expanded command

* feat(espsecure): Allow prompting for HSM PIN in read_hsm_config

If hsm_config does not contain "credentials" the user will be
prompted for the HSM PIN.

This avoids the need to have HSM PINs typed in config files
which is not a good security practice.

ADJUNCT: Updated documentation to reflect new usage

Closes espressif#900

* fix(dangerGH): Update token permissions - allow Danger to add comments to PR

* fix(elf2image): fix text/rodata mapping overlap issue on uni-idrom bus chips

* fix: assert in esp32 exclusive workaround

* docs: Add other resources page

* fix(autodetection): Remove the ESP32-S2 ROM class from get_security_info autodetection

* change(pre-commit): Bump version conventional-precommit-linter to 1.2.1

* feat(esptool): added target to esp32p4

* feat(espefuse): Add support for esp32p4 chip

* fix: Fix redirection of STDOUT

Closes espressif#904

* fix(danger-github): Fir Danger GitHub token permission

* ci(danger-github): Fix github-action-bot permissions for posting Danger output

* ci: Shared danger to local stage (remove possible double CI pipelines)

* ci: add 'flake8-import-order' as a dependecy to flake8

* fix(bin_image): Check only ELF sections when searching for .flash.appdesc

Closes espressif#917

* feat(efuse): ESP32P4 adds ecdsa_key support

* feat(efuse): Update key purpose table and tests

* feat(esp32-s3): Support >16MB quad flash chips

Adds support for the W25Q256 and GD25Q256 flash chips.

Closes espressif#883

* ci(dev_release): Upload dev releases to PyPI with GH Actions

* ci: fix pipeline for building docs

* feat(merge_bin): add support for uf2 format

* feat(esp32c3): Support ECO6 and ECO7 magic numbers

* ci(gitlab_ci): Change only/except syntax to rules

* fix(flasher_stub): fix usb-serial-jtag enabled non-related intr source

* fix(loader): Could not open serial port message adjusted

* ci(gitlab): Fix deploying docs to production

* ci(github): Fix pyinstaller builds on ubuntu

* docs(basic-commands): added note for PowerShell users for merge_bin command

Closes espressif#923

* feat: Add support for Python 3.12

* feat(loader): Added hints for some serial port issues when rising port error

Closes espressif/esp-idf#12366

* feat: add support for get_security_info on esp32c3 ECO7

* docs(troubleshooting): Explain issues when flashing with USB-Serial/JTAG or USB-OTG

Closes espressif#924

* feat(espefuse): Update the way to complete the operation

* docs(boot_mode_selection): Correct secondary strapping pin boot mode levels

Closes espressif#928

* feat(espefuse): Adds efuse ADC calibration data for ESP32H2

* feat(rfc2217_server): Add hard reset sequence

* feat(elf2image): add ram-only-header argument

The ram-only-header configuration makes only
the RAM segments visible to the ROM bootloader placing
them at the beginning of the file and altering the
segment count from the image header with the quantity
of these segments, and also writing only their
checksum. This segment placement also may not result
as optimal as the standard way regarding the padding
gap use among the flash segments that could result
in a less fragmented binary.

The image built must then handle the basic hardware
initialization and the flash mapping for code execution
after ROM bootloader boot it.

Signed-off-by: Marek Matej <marek.matej@espressif.com>
Signed-off-by: Almir Okato <almir.okato@espressif.com>

* feat(esp32p4): Stub flasher support

* refactor(stub_flasher): Cleanup, make adding new targets easier

* feat: add support for intel hex format

---------

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Signed-off-by: Almir Okato <almir.okato@espressif.com>
Co-authored-by: harshal.patil <harshal.patil@espressif.com>
Co-authored-by: radim.karnis <radim.karnis@espressif.com>
Co-authored-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Co-authored-by: Peter Dragun <peter.dragun@espressif.com>
Co-authored-by: KonstantinKondrashov <konstantin@espressif.com>
Co-authored-by: Faidon Liambotis <paravoid@debian.org>
Co-authored-by: XiNGRZ <hi@xingrz.me>
Co-authored-by: Marius Vikhammer <marius.vikhammer@espressif.com>
Co-authored-by: hasheddan <georgedanielmangum@gmail.com>
Co-authored-by: Roland Dobai <roland@espressif.com>
Co-authored-by: Trent Piepho <tpiepho@gmail.com>
Co-authored-by: Dean Gardiner <me@dgardiner.net>
Co-authored-by: Massimiliano Montagni <massimiliano@solutiontech.tech>
Co-authored-by: Tomas Sebestik <tomas.sebestik@espressif.com>
Co-authored-by: Aditya Patwardhan <aditya.patwardhan@espressif.com>
Co-authored-by: Richard Retanubun <richard.retanubun@mmbnetworks.com>
Co-authored-by: wuzhenghui <wuzhenghui@espressif.com>
Co-authored-by: Armando <douyiwen@espressif.com>
Co-authored-by: Jakub Kocka <jakub.kocka@espressif.com>
Co-authored-by: 20162026 <36726858+20162026@users.noreply.github.com>
Co-authored-by: Almir Okato <almir.okato@espressif.com>
Jason2866 added a commit to Jason2866/esptool that referenced this issue Nov 7, 2023
* docs: espsecure remote signing using a HSM broken link fix

* fix(rfc2217_server): Use new reset sequences

* fix(ESP32-S3): Lower CPU freq to improve flasher stub stability

Closes espressif#832

* fix: Unknown chip (ID or magic number) error

* pyinstaller: fix glibc dependency on gnu/linux

pyinstaller package for linux is built within the ubuntu-latest image in
github workflow. This may cause prbolem with glibc symbol versions on
older distributions, where the new symbol versions are not available.
Fix this by building on the older ubuntu version.

Closes espressif#843

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>

* tests: Create custom `host_test` marker for tests without real chip connected

Closes espressif#838

* fix(ESP32-S3): Temporarily disable increasing CPU freq

Related to espressif#848

Related to espressif#842

* build: add esp_rfc2217_server to published scripts

Closes espressif#846

* Update version to v4.5.1

* Update version to v4.6-dev

* espefuse: Hide sensitive info by default during burning burn_key and burn_key_digest

Adds --show-sensitive-info flag for two commands:
burn_key and burn_key_digest.

* flasher_stub: pass -mabi=ilp32 to the RISC-V compiler

This is a no-op change for the upstream toolchain (compiled stubs are
binary identical), but is required when building with Debian's
riscv64-unknown-elf-gcc compiler.

* flasher_stub: allow passing extra CFLAGS

The flasher_stub Makefile allows for some system-local configuration,
either through local.mk, or through environment variables.

For example, the compiler prefix can be overridden, by defining e.g.
CROSS_ESPRISCV32. However, passing additional flags to the compiler
isn't possible right now. Add EXTRA_CFLAGS and EXTRA_CFLAGS_ESPRISCV32
to allow for that option.

* flasher_stub: collect all targets at the top, DRY

* flasher_stub: make target selection more modular

Rather than a special "make esp32", create WITHOUT_* variables to
selectively disable chip families. Currently, WITHOUT_ESP8266,
WITHOUT_ESP32_XTENSA and WITHOUT_ESP32_RISCV32 are defined, but the code
can be easily adjusted to allow for all kinds of other
sets/combinations.

* flasher_stub: create %.json targets, make all a proper PHONY

* flasher_stub: drop --embed from wrap_stub.py

Since commit 94f29a5 the flasher stub is not embedded in the Python
source, but rather included as simple json files.

As such, wrap_stub.py --embed was converted to basically just vary the
build dir. Rather than keep this indirection and for better clarity,
remove that piece of code and replace it by a simple "cp" in the
Makefile.

While at it, replace the target name from "embed" to "install", as this
more akin to a "make install" step.

* espefuse: Support burning ECDSA_KEY from pem file

- fix some assert check in test_espefuse.py
- add tests to cover the new functionality

* espefuse(c2): Fix BLOCK_KEY0 view for summary cmd when SB + FE keys are burnt

For C2 secure boot + flash enc block, we saw that in summary cmd
"0's" from secure boot digest part (upper 128 bit) were translated
into "?'s" when the block was read protected.
For C2, we should apply this translation for lower 128 bits only.

* fix(ESP32-C6): Fix get_pkg_version and get_{major,minor}_chip_version

* image_info: removed check that reserved bytes in image header are zero

IDF may start using parts of the reserved bytes in the extended header at any time,
which will break chip auto-detect in image_info.

* build: limit max cryptography version to 40

* fix: Set flash parameters even with --flash_size keep

Related to espressif/esp-idf#10788

Related to espressif/esp-idf#10959

* build: add arm and arm64 as build target

Closes espressif#845

* Fix typo in serial protocol docs

Fixes misspelling of `triggered` in serial protocol docs.

Signed-off-by: hasheddan <georgedanielmangum@gmail.com>

* Support more recent reedsolo packages

- https://github.com/tomerfiliba-org/reedsolomon/releases/tag/v1.6.1 - this seems to be related to licenses only.
- https://github.com/tomerfiliba-org/reedsolomon/releases/tag/v1.7.0 - this is related to installation.

Closes espressif#872

* build(arm): add pip extra url for github action build

* ci: Fix libffi symlinks for cryptography>=40

* espefuse: Prevent burning XTS_AES and ECDSA keys into BLOCK9 (BLOCK_KEY5)

eFuse module has a hardware bug.
It is related to ESP32-C3, C6, S3, H2 chips:
    - BLOCK9 (BLOCK_KEY5) can not be used by XTS_AES keys.
For H2 chips, the BLOCK9 (BLOCK_KEY5) can not be used by ECDSA keys.
S2 does not have such a hardware bug.

* image_info: Display disabled WP pin as disabled

The image formats know about the special value 0xee used to disable WP.
Display this with image_info.

E.g.:

ESP32-C3 extended image header
==============================
WP pin: 0xee (disabled)

* image_info: Print chip ID's name if known

Example:

Flash pins drive settings: clk_drv: 0x0, q_drv: 0x0, d_drv: 0x0, cs0_drv: 0x0, hd_drv: 0x0, wp_drv: 0x0
Chip ID: 5 (ESP32-C3)
Minimal chip revision: v0.0, (legacy min_rev = 0)
Maximal chip revision: v655.35

An unknown ID will be printed as:

Chip ID: 42 (Unknown ID)

* tests: Make the testsuite Windows compatible

* espefuse: Adds external esp instance

Closes espressif#873

* espefuse: Improve efuse error viewing

* espefuse: Explicit setting of efuse time settings

EFUSE_PWR_ON_NUM in C3 has default value = 0x2880, now = 0x3000

* docs(Boot log): Add all esp targets to cover boot troubleshooting

Closes espressif#732

* fix: USB-JTAG-Serial PID detection error

* esptool: Move bootdesc on the top of the ram segment

* espefuse: Move some vars under init method to speedup tool after adding yaml support

* espefuse: Adds yaml efuse description files for all chip

- esptool: Updates eFuses wafer major&minor versions
- esptool(esp32c6): Adds package versions
- espefuse(esp32c6): Replace PKG_VERSION BLK_VERSION_MINOR BLK_VERSION_MAJOR
- espefuse(esp32c6): Adds adc calib efuses
- espefuse: Adds yaml files for Build with PyInstaller

* efuse(H2): Adds RF Calibration Information

* espsecure: Improve error message for incorrect PEM format

Closes espressif#881

* bugfix(usb_jtag_serial): Autofeed super watchdog (SWD) to avoid resets during flashing

* esptool: Read 64-bit MAC address on C6 and H2

* bugfix: Adjust wrapper scripts to not import themselves

* bugfix(espsecure): Print a clear error message if incompatible OpenSSL backend is used

Closes espressif#878

* fix: inconsistent usage of dirs separator

* feat(esptool): add option to dump whole flash based on detected size

Closes espressif#461

* Update version to v4.6

* Update version to v4.7-dev

* fix(ESP32-S3): Correct RTC WDT registers to fix resets during flashing

* Update version to v4.6.1

* Update version to v4.7-dev

* docs: add explanation for flash_id example to avoid confusion

* docs(boot-log): fix list formatting

* docs: add c2, c6 and h2 as build targets

* fix(compressed upload): Accept short data blocks with only Adler-32 bytes

* fix(CH9102F): Suggest to install new serial drivers if writing to RAM fails

* esptool & espefuse: Fix byte order in MAC (for C6 and H2)

MAC: 60:55:f9:ff:fe:f7:2c:a2 (EUI64, used for IEEE802154)
BASE MAC: 60:55:f9:f7:2c:a2 (used for BT)
MAC_EXT: ff:fe

* Update version to v4.6.2

* Update version to v4.7

* change: Add conventional precommit linter

* ci(pre-commit): Update version of `conventional-precommit-linter`

* feat(get_security_info): Improved the output format and added more
details

* fix(esp32-c2): Enable flashing in secure download mode

Closes espressif#895

* ci: Add DangerJS checks to GL and GH

* feat(esptool): Add PICO package for ESP32S3 and flash/psram efuses

* feat(esptool): Add tests for get_chip_features

* feat(esptool): Add new packages for ESP32C3 and flash efuses

* fix(expand file args): Correctly print the expanded command

* feat(espsecure): Allow prompting for HSM PIN in read_hsm_config

If hsm_config does not contain "credentials" the user will be
prompted for the HSM PIN.

This avoids the need to have HSM PINs typed in config files
which is not a good security practice.

ADJUNCT: Updated documentation to reflect new usage

Closes espressif#900

* fix(dangerGH): Update token permissions - allow Danger to add comments to PR

* fix(elf2image): fix text/rodata mapping overlap issue on uni-idrom bus chips

* fix: assert in esp32 exclusive workaround

* docs: Add other resources page

* fix(autodetection): Remove the ESP32-S2 ROM class from get_security_info autodetection

* change(pre-commit): Bump version conventional-precommit-linter to 1.2.1

* feat(esptool): added target to esp32p4

* feat(espefuse): Add support for esp32p4 chip

* fix: Fix redirection of STDOUT

Closes espressif#904

* fix(danger-github): Fir Danger GitHub token permission

* ci(danger-github): Fix github-action-bot permissions for posting Danger output

* ci: Shared danger to local stage (remove possible double CI pipelines)

* ci: add 'flake8-import-order' as a dependecy to flake8

* fix(bin_image): Check only ELF sections when searching for .flash.appdesc

Closes espressif#917

* feat(efuse): ESP32P4 adds ecdsa_key support

* feat(efuse): Update key purpose table and tests

* feat(esp32-s3): Support >16MB quad flash chips

Adds support for the W25Q256 and GD25Q256 flash chips.

Closes espressif#883

* ci(dev_release): Upload dev releases to PyPI with GH Actions

* ci: fix pipeline for building docs

* feat(merge_bin): add support for uf2 format

* feat(esp32c3): Support ECO6 and ECO7 magic numbers

* ci(gitlab_ci): Change only/except syntax to rules

* fix(flasher_stub): fix usb-serial-jtag enabled non-related intr source

* fix(loader): Could not open serial port message adjusted

* ci(gitlab): Fix deploying docs to production

* ci(github): Fix pyinstaller builds on ubuntu

* docs(basic-commands): added note for PowerShell users for merge_bin command

Closes espressif#923

* feat: Add support for Python 3.12

* feat(loader): Added hints for some serial port issues when rising port error

Closes espressif/esp-idf#12366

* feat: add support for get_security_info on esp32c3 ECO7

* docs(troubleshooting): Explain issues when flashing with USB-Serial/JTAG or USB-OTG

Closes espressif#924

* feat(espefuse): Update the way to complete the operation

* docs(boot_mode_selection): Correct secondary strapping pin boot mode levels

Closes espressif#928

* feat(espefuse): Adds efuse ADC calibration data for ESP32H2

* feat(rfc2217_server): Add hard reset sequence

* feat(elf2image): add ram-only-header argument

The ram-only-header configuration makes only
the RAM segments visible to the ROM bootloader placing
them at the beginning of the file and altering the
segment count from the image header with the quantity
of these segments, and also writing only their
checksum. This segment placement also may not result
as optimal as the standard way regarding the padding
gap use among the flash segments that could result
in a less fragmented binary.

The image built must then handle the basic hardware
initialization and the flash mapping for code execution
after ROM bootloader boot it.

Signed-off-by: Marek Matej <marek.matej@espressif.com>
Signed-off-by: Almir Okato <almir.okato@espressif.com>

* feat(esp32p4): Stub flasher support

* refactor(stub_flasher): Cleanup, make adding new targets easier

* feat: add support for intel hex format

* feat(xip_psram): support xip psram feature on esp32p4

Expanded IROM / DROM range to include psram space as well

* Delete docs directory

* Delete .gitlab-ci.yml

* Delete .pre-commit-config.yaml

* Delete MANIFEST.in

* Update build_esptool.yml

* Delete .github/workflows/test_esptool.yml

---------

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Signed-off-by: Almir Okato <almir.okato@espressif.com>
Co-authored-by: harshal.patil <harshal.patil@espressif.com>
Co-authored-by: radim.karnis <radim.karnis@espressif.com>
Co-authored-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Co-authored-by: Peter Dragun <peter.dragun@espressif.com>
Co-authored-by: KonstantinKondrashov <konstantin@espressif.com>
Co-authored-by: Faidon Liambotis <paravoid@debian.org>
Co-authored-by: XiNGRZ <hi@xingrz.me>
Co-authored-by: Marius Vikhammer <marius.vikhammer@espressif.com>
Co-authored-by: hasheddan <georgedanielmangum@gmail.com>
Co-authored-by: Roland Dobai <roland@espressif.com>
Co-authored-by: Trent Piepho <tpiepho@gmail.com>
Co-authored-by: Dean Gardiner <me@dgardiner.net>
Co-authored-by: Massimiliano Montagni <massimiliano@solutiontech.tech>
Co-authored-by: Tomas Sebestik <tomas.sebestik@espressif.com>
Co-authored-by: Aditya Patwardhan <aditya.patwardhan@espressif.com>
Co-authored-by: Richard Retanubun <richard.retanubun@mmbnetworks.com>
Co-authored-by: wuzhenghui <wuzhenghui@espressif.com>
Co-authored-by: Armando <douyiwen@espressif.com>
Co-authored-by: Jakub Kocka <jakub.kocka@espressif.com>
Co-authored-by: 20162026 <36726858+20162026@users.noreply.github.com>
Co-authored-by: Almir Okato <almir.okato@espressif.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants