Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to libv8-node 20.x #284

Closed
wants to merge 8 commits into from
Closed

Update to libv8-node 20.x #284

wants to merge 8 commits into from

Conversation

lloeki
Copy link
Collaborator

@lloeki lloeki commented May 27, 2023

Depends on #283: uses libv8-node-19 as a base, which should be merged first.

@lloeki lloeki mentioned this pull request May 27, 2023
37 tasks
context = MiniRacer::Context.new(timeout: 5)
context.eval("let a='testing';")
# Failure: expecting the isolate to have values for all the vals
# {:total_physical_size=>835584, :total_heap_size_executable=>0, :total_heap_size=>1392640, :used_heap_size=>345736, :heap_size_limit=>1518338048}
Copy link
Collaborator Author

@lloeki lloeki Apr 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

total_heap_size_executable seems to always be 0 (at least as tested on arm64-darwin)


assert(stats.values.all?{|v| v > 0}, "expecting the isolate to have values for all the vals")
end
# File.open('testlog', 'wb') { |f| f << stats.inspect }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Writing things to a file to debug

# cuts after c + cuts before d + never e => likely to be Isolate#pump_message_loop
# def test_webassembly
# skip "TruffleRuby does not enable WebAssembly by default" if RUBY_ENGINE == "truffleruby"
# log = File.open('testlog2', 'wb')
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Writing things to a file to debug

Comment on lines 1002 to 1010
# Fatal error in HandleScope::HandleScope
# Entering the V8 API without proper locking in place
# log has been seen as (in order):
# abcdcdcdcdcdcdcdcdcdcdcdcdc
# abcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdc
# abcdcdcdcdcdcdcdcdcdcdcdcdc
# abcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdc
# abcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdc
# cuts after c + cuts before d + never e => likely to be Isolate#pump_message_loop
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one seems to happen randomly after a few loops. Exact cause is yet unknown. Could be some timing, could be message loop doing no work but had some before, could be message loop having some work and none before, could be any other unknown thing really since I did not dig.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From: #283 (comment) this was fixed by @Fayti1703 in Fayti1703@1e43a49, which I've cherry-picked.

Comment on lines +548 to +550
# segfault
# CFUNC :init_with_snapshot
# METHOD /Users/loic.nageleisen/Source/github.com/rubyjs/libv8-node/test/mini_racer/lib/mini_racer.rb:81 (Isolate#initialize)
Copy link
Collaborator Author

@lloeki lloeki Apr 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initialising the isolate with a snapshot burns into flames. Note that it apparently doesn't reach warmup! so probably either unrelated to the above or indirectly related but for yet unknown, deeper reasons.

    def initialize(snapshot = nil)
      unless snapshot.nil? || snapshot.is_a?(Snapshot)
        raise ArgumentError, "snapshot must be a Snapshot object, passed a #{snapshot.inspect}"
      end

      # defined in the C class
      init_with_snapshot(snapshot)
    end
static VALUE rb_isolate_init_with_snapshot(VALUE self, VALUE snapshot) {
    IsolateInfo* isolate_info;
    TypedData_Get_Struct(self, IsolateInfo, &isolate_type, isolate_info);

    init_v8();

    SnapshotInfo* snapshot_info = nullptr;
    if (!NIL_P(snapshot)) {
        TypedData_Get_Struct(snapshot, SnapshotInfo, &snapshot_type, snapshot_info);
    }

    isolate_info->init(snapshot_info);
    isolate_info->hold();

    return Qnil;
}

Comment on lines +453 to +455
# segfault
# CFUNC :warmup_unsafe!
# METHOD /Users/loic.nageleisen/Source/github.com/rubyjs/libv8-node/test/mini_racer/lib/mini_racer.rb:452 (def warmup!)
Copy link
Collaborator Author

@lloeki lloeki Apr 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warmup_unsafe! indeed! Either we don't do something that we should to make it safe, or we pass something that was okay before and is not now.

    def warmup!(src)
      # we have to do something here
      # we are bloating memory a bit but it is more correct
      # than hitting an exception when attempty to compile invalid source
      begin
        ctx = MiniRacer::Context.new
        ctx.eval(@source)
        ctx.eval(src)
      rescue MiniRacer::RuntimeError => e
        raise MiniRacer::SnapshotError.new, e.message, e.backtrace
      end

      warmup_unsafe!(src)
    end
static VALUE rb_snapshot_warmup_unsafe(VALUE self, VALUE str) {
    SnapshotInfo* snapshot_info;
    TypedData_Get_Struct(self, SnapshotInfo, &snapshot_type, snapshot_info);

    Check_Type(str, T_STRING);

    init_v8();

    StartupData cold_startup_data = {snapshot_info->data, snapshot_info->raw_size};
    StartupData warm_startup_data = warm_up_snapshot_data_blob(cold_startup_data, RSTRING_PTR(str));

    if (warm_startup_data.data == NULL && warm_startup_data.raw_size == 0) {
        rb_raise(rb_eSnapshotError, "Could not warm up snapshot, most likely the source is incorrect");
    } else {
        delete[] snapshot_info->data;

        snapshot_info->data = warm_startup_data.data;
        snapshot_info->raw_size = warm_startup_data.raw_size;
    }

    return self;
}

@lloeki
Copy link
Collaborator Author

lloeki commented Apr 6, 2024

Heh, just noticed *-linux-gnu has bundler electing to fetch the ruby platform gem instead of the binary ones, so spends time building libv8, which works for x86_64 and probably also for aarch64 if it did not get killed by GHA timeout.

@lloeki
Copy link
Collaborator Author

lloeki commented Apr 6, 2024

Echoing @Fayti1703 notes here coz I'm tired of clicking around :D

#283 (comment)

  • Something deep in v8 is segfaulting when trying to read a snapshot -- specifically the constructor of > v8::internal::StringTableInsertionKey. Unsure what the cause is at the moment.
  • crash in test_snapshots_can_be_warmed_up_with_no_side_effects
  • crash in test_isolates_from_snapshot_dont_get_corrupted_if_the_snapshot_gets_warmed_up_or_GCed

#283 (comment)

With a debug build of V8 (via a local libv8-node gem build), I get even more snapshot-related crashes due to invalid checksums. V8 says (for example):

#
# Fatal error in ../deps/v8/src/heap/read-only-spaces.cc, line 71
# Check failed: read_only_blob_checksum_ == snapshot_checksum (<unprintable> vs. 3847049538).
#

This smells like a V8 bug, since as far as I can tell MR doesn't touch the snapshot data whatsoever.

Might also be that we do (or fail to do) something and we end up corrupting memory / pointer / whatever.

Hence why rr might be of some help to see what actually touches the memory.

lloeki added a commit to rubyjs/libv8-node that referenced this pull request Apr 8, 2024
lloeki added a commit to rubyjs/libv8-node that referenced this pull request Apr 8, 2024
@lloeki
Copy link
Collaborator Author

lloeki commented Apr 22, 2024

Merging this as is as per the plan.

@lloeki lloeki marked this pull request as ready for review April 22, 2024 20:05
@lloeki lloeki closed this Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants