FATAL RangeError: Invalid string length #4922

dimaslanjaka · 2022-03-27T17:20:52Z

Check List

Please check followings before submitting a new issue.

I have already read Docs page & Troubleshooting page
I have already searched existing issues and they are not help to me
I examined error or warning messages and it's difficult to solve
Using the latest version of Hexo (run hexo version to check)
Node.js is higher than minimum required version

I don't know why this error occurs. Even though yesterday it was okay.

I didn't change anything in the code
I didn't modify the theme, even if I changed the theme many times, the error still persists.
I didn't modify any plugins
LOG Workflow
This error persist on hexo generate

FATAL {
  err: RangeError: Invalid string length
      at JSON.stringify (<anonymous>)
      at _Model._export (/media/dimaslanjaka/DATA/Repositories/gh-pages/node_modules/warehouse/lib/model.js:932:17)
      at exportAsync (/media/dimaslanjaka/DATA/Repositories/gh-pages/node_modules/warehouse/lib/database.js:50:44)
} Something's wrong. Maybe you can find the solution here: %s https://hexo.io/docs/troubleshooting.html

Environtment

hexo: 6.0.0
hexo-cli: 4.3.0
os: linux 5.13.0-37-generic Zorin OS 16.1
node: 14.18.1
v8: 8.4.371.23-node.84
uv: 1.42.0
zlib: 1.2.11
brotli: 1.0.9
ares: 1.17.2
modules: 83
nghttp2: 1.42.0
napi: 8
llhttp: 2.1.4
openssl: 1.1.1l
cldr: 39.0
icu: 69.1
tz: 2021a
unicode: 13.0

The text was updated successfully, but these errors were encountered:

dimaslanjaka · 2022-03-31T10:31:59Z

Hellooooo why still getting errors ?

FATAL {
  err: RangeError: Invalid string length
      at JSON.stringify (<anonymous>)
      at _Model._export (D:\Repositories\gh-pages-old\node_modules\warehouse\lib\model.js:932:17)
      at exportAsync (D:\Repositories\gh-pages-old\node_modules\warehouse\lib\database.js:50:44)
} Something's wrong. Maybe you can find the solution here: %s https://hexo.io/docs/troubleshooting.html
FATAL RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
    at _Model._export (D:\Repositories\gh-pages-old\node_modules\warehouse\lib\model.js:932:17)
    at exportAsync (D:\Repositories\gh-pages-old\node_modules\warehouse\lib\database.js:50:44)

i've tried remove all plugins, change themes.

yoshinorin · 2022-03-31T13:55:05Z

@dimaslanjaka

It seems Out of memory exception and it is from v8.

nodejs/node-v0.x-archive#14170
https://stackoverflow.com/questions/29175877/json-stringify-throws-rangeerror-invalid-string-length-for-huge-objects

dimaslanjaka · 2022-04-08T17:19:51Z

@dimaslanjaka

It seems Out of memory exception and it is from v8.

nodejs/node-v0.x-archive#14170
https://stackoverflow.com/questions/29175877/json-stringify-throws-rangeerror-invalid-string-length-for-huge-objects

ive tried with this, same result

name: Build

on:
  push:
    branches:
      - compiler # run when this branch pushed

# cancel previous workflows, run only one workflow
concurrency:
  group: build-${{ github.event.push.number || github.event.pull_request.number || github.ref }}
  #cancel-in-progress: true

jobs:
  build:
    if: {{ $false }}
    runs-on: ubuntu-latest
    timeout-minutes: 120
    env:
      NODE_OPTIONS: "--max_old_space_size=8192" #8192 4096 --expose-gc
      GITFLOW: true
    steps:
      - name: Checkout repository and submodules
        uses: actions/checkout@v2
        with:
          submodules: recursive
          token: "${{ secrets.GITHUB_TOKEN }}"
      - name: Setup Node.js 16.x
        uses: actions/setup-node@v1
        with:
          node-version: "16.x"
      - ....

dimaslanjaka · 2022-04-08T17:21:07Z

i cant increase "--max_old_space_size=8192" more than 8192. its limited.

dimaslanjaka · 2022-04-08T17:22:36Z

and my computer only have 8GB RAM

dimaslanjaka · 2022-04-08T17:24:09Z

now i fork some hexojs functions to create my own generator https://github.com/dimaslanjaka/dimaslanjaka.github.io/tree/compiler

my own generator work for huge posts. might helpful for hexojs dev to improve the HexoJS package

dimaslanjaka · 2022-04-08T17:27:35Z

HexoJS need 16GB RAM to process 1000+ posts i guess.
ive tried forked hexojs/site repository and copied all of my posts, same result (error).

HexoJS is not yet qualified for an article writer like me.
i hope, i can use HexoJS in future. so i dont worry to creating article without a lot coding.

dimaslanjaka · 2022-04-08T17:39:27Z

SUGGEST:

make hexo api separated to reduce memory usages
just like gulp task example:

hexo generate tag # generating only tags archives
hexo generate categories # generating only categories archives
hexo generate posts # generating posts
hexo generate index # generating index
hexo generate page # generating all page except `source/_posts`
hexo generate sitemap # generating sitemaps, rss

dimaslanjaka · 2022-04-08T17:41:56Z

nah from generator, caching all posts object and reuse them on next process, its good idea

yoshinorin · 2022-04-09T03:05:14Z

Perhaps this issue does not caused by number of posts, but the size of the file. Do you have a large size of article or page? Isn't it?

nodejs/node#35973

HexoJS need 16GB RAM to process 1000+ posts i guess.

I have a 1200+ post and I try to generate them with NODE_OPTIONS=--max_old_space_size=512. It works.

set NODE_OPTIONS=--max_old_space_size=512 && hexo generate

INFO  Validating config
INFO  Start processing
INFO  Files loaded in 20 s

.....

INFO  Generated: 2016/06/24/jenkins-install/1.jpg
INFO  Generated: 2018/12/01/update-site-layout/header.gif
INFO  Generated: 2019/01/02/create-photo-gallery/gallery.gif
INFO  3918 files generated in 28 s

256MB is failed.

set NODE_OPTIONS=--max_old_space_size=256 && hexo generate   
INFO  Validating config
INFO  Start processing
INFO  Files loaded in 19 s

<--- Last few GCs --->

[7404:000001F892D401D0]    30883 ms: Scavenge (reduce) 241.0 (260.3) -> 240.8 (261.0) MB, 1.4 / 0.0 ms  (average mu = 0.780, current mu = 0.194) allocation failure
[7404:000001F892D401D0]    31016 ms: Mark-sweep (reduce) 242.5 (261.8) -> 241.0 (261.8) MB, 130.3 / 0.1 ms  (average mu = 0.661, current mu = 0.121) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF68A2CE3EF v8::internal::CodeObjectRegistry::~CodeObjectRegistry+111951
 2: 00007FF68A25DA36 v8::internal::WebSnapshotDeserializer::context_count+65446
 3: 00007FF68A25E8ED node::OnFatalError+301

dimaslanjaka · 2022-04-09T03:09:59Z

I have dynamic posts (such as the legend of Neverland quiz) that post actually generated by user input (based on game update). I thought maybe because of that. Try check site: webmanajemen.com quiz the legend of Neverland from google

dimaslanjaka · 2022-04-09T03:28:49Z

Perhaps this issue does not caused by number of posts, but the size of the file. Do you have a large size of article or page? Isn't it?

nodejs/node#35973

HexoJS need 16GB RAM to process 1000+ posts i guess.

I have a 1200+ post and I try to generate them with NODE_OPTIONS=--max_old_space_size=512. It works.

set NODE_OPTIONS=--max_old_space_size=512 && hexo generate

INFO  Validating config
INFO  Start processing
INFO  Files loaded in 20 s

.....

INFO  Generated: 2016/06/24/jenkins-install/1.jpg
INFO  Generated: 2018/12/01/update-site-layout/header.gif
INFO  Generated: 2019/01/02/create-photo-gallery/gallery.gif
INFO  3918 files generated in 28 s

256MB is failed.

set NODE_OPTIONS=--max_old_space_size=256 && hexo generate   
INFO  Validating config
INFO  Start processing
INFO  Files loaded in 19 s

<--- Last few GCs --->

[7404:000001F892D401D0]    30883 ms: Scavenge (reduce) 241.0 (260.3) -> 240.8 (261.0) MB, 1.4 / 0.0 ms  (average mu = 0.780, current mu = 0.194) allocation failure
[7404:000001F892D401D0]    31016 ms: Mark-sweep (reduce) 242.5 (261.8) -> 241.0 (261.8) MB, 130.3 / 0.1 ms  (average mu = 0.661, current mu = 0.121) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF68A2CE3EF v8::internal::CodeObjectRegistry::~CodeObjectRegistry+111951
 2: 00007FF68A25DA36 v8::internal::WebSnapshotDeserializer::context_count+65446
 3: 00007FF68A25E8ED node::OnFatalError+301

i dont know the detail, but full log detail here https://github.com/dimaslanjaka/dimaslanjaka.github.io/runs/5711205131?check_suite_focus=true

ive tried fork hexojs/site and copy all my posts and hexo generate same error result

the errors is random: sometime invalid string length, also sometime like this screenshot

now i backup the hexo project on https://github.com/dimaslanjaka/dimaslanjaka.github.io/tree/hexo-compiler

dimaslanjaka · 2022-04-10T20:35:16Z

Memory leaks have been confirmed by me because hexo doesn't separate all processes but wraps them all into one process which makes resource usage (memory/cpu) increase instantly.
I made a project like hexo https://github.com/dimaslanjaka/dimaslanjaka.github.io/tree/compiler (if I wrap everything into 1 command gulp, then the result will be error memory leaks, but if I run task gulp ` one by one, the result is perfect even though there are hundreds of posts/pages in large numbers).

HEXO NEED TO UPDATE

refs: hexojs/hexo#4922

- refs: hexojs/hexo#4922

* fix(workaround): RangeError: Invalid string length - refs: hexojs/hexo#4922 * fix lint

yoshinorin · 2022-05-10T20:48:41Z

We released hexo 6.2.0 just now. It includes this issue workaround.

https://github.com/hexojs/hexo/releases/tag/6.2.0

stevenjoezhang · 2022-11-28T07:55:30Z

I found that the external_link filter is consuming a huge amount of memory. You can try to disable it in _config.yml

external_link:
  enable: false

See also #3886

You can use Heap Profiler to inspect memory issues.

hexo/lib/plugins/filter/after_render/external_link.js

Line 21 in ca51e15

return data.replace(rATag, (str, href) => {

This function takes up about 2G of memory and I'm not sure what the reason is

dimaslanjaka · 2022-12-02T11:32:50Z

Now iam using my own task and removing a lot of plugins from hexo project. Now i can fix seo, external links (anonymizing), optimize images automatically with separated task using gulp. disvantages is the process taking more times, but works.

posts total: 1419 (exclude pages)
RAM my pc: 8 GB
Hardisk type: HDD
processor engine: https://github.com/dimaslanjaka/static-blog-generator-hexo/tree/master/packages/gulp-sbg
deployed to: https://github.com/dimaslanjaka/dimaslanjaka.github.io -> https://www.webmanajemen.com/

stevenjoezhang · 2022-12-04T13:47:12Z

@dimaslanjaka How many tags and categories are there for your 1400+ posts? I have also found that Hexo takes a lot of time querying posts and tags, because the database query algorithm is not optimized.

dimaslanjaka · 2022-12-05T06:01:53Z

@dimaslanjaka How many tags and categories are there for your 1400+ posts? I have also found that Hexo takes a lot of time querying posts and tags, because the database query algorithm is not optimized.

emm around 100+, you can check it on sidebar https://webmanajemen.com

there has counter of post, tag, category.

dimaslanjaka · 2022-12-05T06:04:44Z

because this blog is imported from blogger using my plugin hexo-blogger-xml. So old tags and uncategorized still exists.

dimaslanjaka · 2022-12-05T06:07:13Z

Now iam using my own task and removing a lot of plugins from hexo project. Now i can fix seo, external links (anonymizing), optimize images automatically with separated task using gulp. disvantages is the process taking more times, but works.

posts total: 1419 (exclude pages) RAM my pc: 8 GB Hardisk type: HDD processor engine: https://github.com/dimaslanjaka/static-blog-generator-hexo/tree/master/packages/gulp-sbg deployed to: https://github.com/dimaslanjaka/dimaslanjaka.github.io -> https://www.webmanajemen.com/

hexo project on: https://github.com/dimaslanjaka/static-blog-generator-hexo

yoshinorin added a commit to yoshinorin/warehouse that referenced this issue Apr 12, 2022

fix(workaround): RangeError: Invalid string length

84e62d4

refs: hexojs/hexo#4922

yoshinorin added a commit to yoshinorin/warehouse that referenced this issue Apr 12, 2022

fix(workaround): RangeError: Invalid string length

5cba9f6

- refs: hexojs/hexo#4922

yoshinorin mentioned this issue Apr 12, 2022

fix(workaround): RangeError: Invalid string length hexojs/warehouse#123

Merged

yoshinorin added a commit to hexojs/warehouse that referenced this issue Apr 13, 2022

fix(workaround): RangeError: Invalid string length (#123)

b5a3ba7

* fix(workaround): RangeError: Invalid string length - refs: hexojs/hexo#4922 * fix lint

yoshinorin mentioned this issue Apr 14, 2022

chore(deps): bump warehouse from 4.0.0 to 4.0.1 #4943

Merged

2 tasks

stevenjoezhang added the #perfmatters label Nov 27, 2022

stevenjoezhang mentioned this issue Jul 14, 2023

Hexo eats all the memory when generating #3886

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FATAL RangeError: Invalid string length #4922

FATAL RangeError: Invalid string length #4922

dimaslanjaka commented Mar 27, 2022 •

edited

dimaslanjaka commented Mar 31, 2022

yoshinorin commented Mar 31, 2022

dimaslanjaka commented Apr 8, 2022 •

edited

dimaslanjaka commented Apr 8, 2022

dimaslanjaka commented Apr 8, 2022

dimaslanjaka commented Apr 8, 2022

dimaslanjaka commented Apr 8, 2022 •

edited

dimaslanjaka commented Apr 8, 2022 •

edited

dimaslanjaka commented Apr 8, 2022

yoshinorin commented Apr 9, 2022 •

edited

dimaslanjaka commented Apr 9, 2022 •

edited

dimaslanjaka commented Apr 9, 2022 •

edited

dimaslanjaka commented Apr 10, 2022 •

edited

yoshinorin commented May 10, 2022

stevenjoezhang commented Nov 28, 2022

dimaslanjaka commented Dec 2, 2022 •

edited

stevenjoezhang commented Dec 4, 2022

dimaslanjaka commented Dec 5, 2022 •

edited

dimaslanjaka commented Dec 5, 2022 •

edited

dimaslanjaka commented Dec 5, 2022

FATAL RangeError: Invalid string length #4922

FATAL RangeError: Invalid string length #4922

Comments

dimaslanjaka commented Mar 27, 2022 • edited

Check List

dimaslanjaka commented Mar 31, 2022

yoshinorin commented Mar 31, 2022

dimaslanjaka commented Apr 8, 2022 • edited

dimaslanjaka commented Apr 8, 2022

dimaslanjaka commented Apr 8, 2022

dimaslanjaka commented Apr 8, 2022

dimaslanjaka commented Apr 8, 2022 • edited

dimaslanjaka commented Apr 8, 2022 • edited

dimaslanjaka commented Apr 8, 2022

yoshinorin commented Apr 9, 2022 • edited

dimaslanjaka commented Apr 9, 2022 • edited

dimaslanjaka commented Apr 9, 2022 • edited

dimaslanjaka commented Apr 10, 2022 • edited

yoshinorin commented May 10, 2022

stevenjoezhang commented Nov 28, 2022

dimaslanjaka commented Dec 2, 2022 • edited

stevenjoezhang commented Dec 4, 2022

dimaslanjaka commented Dec 5, 2022 • edited

dimaslanjaka commented Dec 5, 2022 • edited

dimaslanjaka commented Dec 5, 2022

dimaslanjaka commented Mar 27, 2022 •

edited

dimaslanjaka commented Apr 8, 2022 •

edited

dimaslanjaka commented Apr 8, 2022 •

edited

dimaslanjaka commented Apr 8, 2022 •

edited

yoshinorin commented Apr 9, 2022 •

edited

dimaslanjaka commented Apr 9, 2022 •

edited

dimaslanjaka commented Apr 9, 2022 •

edited

dimaslanjaka commented Apr 10, 2022 •

edited

dimaslanjaka commented Dec 2, 2022 •

edited

dimaslanjaka commented Dec 5, 2022 •

edited

dimaslanjaka commented Dec 5, 2022 •

edited