8331911: Reconsider locking for recently disarmed nmethods #19285

neethu-prasad · 2024-05-17T13:51:57Z

Notes
We are spending significant time on acquiring the per-nmethod as all the
threads are in same nmethod.
Adding double-check lock by calling is_armed before lock acquisition.

Verification

Shenendoah

% /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
[0.706s][info][gc] GC(0) Concurrent marking roots 11.519ms
[0.752s][info][gc] GC(1) Concurrent marking roots 9.833ms
[0.814s][info][gc] GC(2) Concurrent marking roots 10.000ms
[0.855s][info][gc] GC(3) Concurrent marking roots 9.314ms
[0.895s][info][gc] GC(4) Concurrent marking roots 8.937ms
[1.213s][info][gc] GC(5) Concurrent marking roots 12.582ms
[1.340s][info][gc] GC(6) Concurrent marking roots 9.574ms
[1.465s][info][gc] GC(7) Concurrent marking roots 12.791ms

ZGC

% /home/neethp/Development/opensource/jdk/build/linux-x86_64-server-release/images/jdk/bin/java -Xmx1g -Xms1g -XX:+UseShenandoahGC -Xlog:gc ManyThreadsStacks.java 2>&1 | grep "marking roots"
[0.732s][info][gc] GC(0) Concurrent marking roots 10.694ms
[0.782s][info][gc] GC(1) Concurrent marking roots 14.614ms
[0.825s][info][gc] GC(2) Concurrent marking roots 12.700ms
[0.863s][info][gc] GC(3) Concurrent marking roots 9.622ms
[0.904s][info][gc] GC(4) Concurrent marking roots 12.892ms
[1.244s][info][gc] GC(5) Concurrent marking roots 12.422ms
[1.375s][info][gc] GC(6) Concurrent marking roots 12.756ms
[1.503s][info][gc] GC(7) Concurrent marking roots 12.265ms
[1.628s][info][gc] GC(8) Concurrent marking roots 12.309ms
[1.754s][info][gc] GC(9) Concurrent marking roots 12.996ms
[1.879s][info][gc] GC(10) Concurrent marking roots 9.416ms

Issue
https://bugs.openjdk.org/browse/JDK-8331911

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8331911: Reconsider locking for recently disarmed nmethods (Enhancement - P4)

Reviewers

Aleksey Shipilev (@shipilev - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/19285/head:pull/19285
$ git checkout pull/19285

Update a local copy of the PR:
$ git checkout pull/19285
$ git pull https://git.openjdk.org/jdk.git pull/19285/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 19285

View PR using the GUI difftool:
$ git pr show -t 19285

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/19285.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2024-05-17T13:52:33Z

Hi @neethu-prasad, welcome to this OpenJDK project and thanks for contributing!

We do not recognize you as Contributor and need to ensure you have signed the Oracle Contributor Agreement (OCA). If you have not signed the OCA, please follow the instructions. Please fill in your GitHub username in the "Username" field of the application. Once you have signed the OCA, please let us know by writing /signed in a comment in this pull request.

If you already are an OpenJDK Author, Committer or Reviewer, please click here to open a new issue so that we can record that fact. Please use "Add GitHub user neethu-prasad" as summary for the issue.

If you are contributing this work on behalf of your employer and your employer has signed the OCA, please let us know by writing /covered in a comment in this pull request.

openjdk · 2024-05-17T13:53:13Z

@neethu-prasad This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8331911: Reconsider locking for recently disarmed nmethods

Reviewed-by: shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 145 new commits pushed to the master branch:

bc7d9e3: 8333013: Update vmTestbase/nsk/share/LocalProcess.java to don't use finalization
03b7a85: 8332259: JvmtiTrace::safe_get_thread_name fails if current thread is in native state
43a2f17: 8333149: ubsan : memset on nullptr target detected in jvmtiEnvBase.cpp get_object_monitor_usage
fed2b56: 8320999: RISC-V: C2 RotateLeftV
6cda4c5: 8321543: Update NSS to version 3.96
c003c12: 8331865: Consolidate size and alignment checks in LayoutPath
6d718ae: 8324341: Remove redundant preprocessor #if's checks
9b64ece: 8332904: ubsan ppc64le: c1_LIRGenerator_ppc.cpp:581:21: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long int'
3d4eb15: 8302744: Refactor Hotspot container detection code
2cca83b: 8332880: JFR GCHelper class recognizes "Archive" regions as valid
... and 135 more: https://git.openjdk.org/jdk/compare/d4c2edf2c91a790874c80f1a7bea5bfd4f438bde...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@shipilev) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

openjdk · 2024-05-17T13:53:33Z

@neethu-prasad The following labels will be automatically applied to this pull request:

hotspot-gc
shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

neethu-prasad · 2024-05-17T13:57:46Z

/covered

bridgekeeper · 2024-05-17T13:58:24Z

Thank you! Please allow for a few business days to verify that your employer has signed the OCA. Also, please note that pull requests that are pending an OCA check will not usually be evaluated, so your patience is appreciated!

shipilev

I like this. Some stylistic comments.

shipilev · 2024-05-22T08:50:57Z

src/hotspot/share/gc/shenandoah/shenandoahBarrierSetNMethod.cpp

@@ -36,6 +36,13 @@
 #include "runtime/threadWXSetters.inline.hpp"

 bool ShenandoahBarrierSetNMethod::nmethod_entry_barrier(nmethod* nm) {
+


Here and later: no need for new line at the beginning of the method.

shipilev · 2024-05-22T08:55:57Z

src/hotspot/share/gc/shenandoah/shenandoahBarrierSetNMethod.cpp

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Red Hat, Inc. All rights reserved.
+ * Copyright (c) 2019, 2024, Red Hat, Inc. All rights reserved.


This update is unnecessary.

shipilev · 2024-05-22T08:56:36Z

src/hotspot/share/gc/z/zBarrierSetNMethod.cpp

@@ -37,6 +37,14 @@
 #include "runtime/threadWXSetters.inline.hpp"

 bool ZBarrierSetNMethod::nmethod_entry_barrier(nmethod* nm) {
+
+  if (!is_armed(nm)) {
+    log_develop_trace(gc, nmethod)("nmethod: " PTR_FORMAT " visited by entry (disarmed)", p2i(nm));


Should it be "(disarmed before lock)" to disambiguate against "(disarmed)" later?

shipilev · 2024-05-22T09:01:38Z

src/hotspot/share/gc/shenandoah/shenandoahBarrierSetNMethod.cpp

+    // Some other thread got here first and healed the oops
+    // and disarmed the nmethod.


Suggestion for the comment (here and later):

// Some other thread got here first and healed the oops // and disarmed the nmethod. No need to continue.

...and then later, under the lock:

// Some other thread managed to complete while we were // waiting for lock. No need to continue.

mlbridge · 2024-05-23T11:19:13Z

Webrevs

shipilev

I think this is good.

Yes, we could have restructured the code so that nmethod_entry_barrier was not called when nmethod is already disarmed. There are already some places where we check it externally, but the reproducer in the bug shows that it is easy to miss. So checking right here in the method looks appropriate.

@fisk might want to take a look as well.

openjdk · 2024-05-23T18:07:25Z

⚠️ @neethu-prasad the full name on your profile does not match the author name in this pull requests' HEAD commit. If this pull request gets integrated then the author name from this pull requests' HEAD commit will be used for the resulting commit. If you wish to push a new commit with a different author name, then please run the following commands in a local repository of your personal fork:

$ git checkout JDK-8331911
$ git commit --author='Preferred Full Name <you@example.com>' --allow-empty -m 'Update full name'
$ git push

shipilev · 2024-05-29T15:56:57Z

@fisk @stefank -- are you good with this for ZGC?

fisk

It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.

neethu-prasad · 2024-05-29T17:14:31Z

It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.

Thanks for the feedback! Looking around the code, I think there are a few places where we can do more changes.

First, remove check here:

jdk/src/hotspot/share/code/nmethod.cpp

Lines 852 to 855 in bc7d9e3

    
           if (bs_nm->is_armed(nm)) { 
        
             bool alive = bs_nm->nmethod_entry_barrier(nm); 
        
             assert(alive, "should be alive"); 
        
           }

This would force us to add the check in super-class implementation here:

jdk/src/hotspot/share/gc/shared/barrierSetNMethod.cpp

Line 83 in bc7d9e3

bool BarrierSetNMethod::nmethod_entry_barrier(nmethod* nm) {

Second, we can remove the check here:

jdk/src/hotspot/share/gc/shared/barrierSetNMethod.cpp

Lines 175 to 181 in bc7d9e3

    
           if (!bs_nm->is_armed(nm)) { 
        
             return 0; 
        
           } 
        
           assert(!nm->is_osr_method(), "Should not reach here"); 
        
           // Called upon first entry after being armed 
        
           bool may_enter = bs_nm->nmethod_entry_barrier(nm);

But it does not seem straightforward, because we currently skip cross-modification fence based on is_armed(...) check. Unfortunately, we cannot easily know if nmethod_entry_barrier acted or not, we only know if method is safe or not. Can we / should we do these refactoring separately?

fisk · 2024-05-30T16:04:04Z

It seems fine to me that the GC backends are responsible for checking if the nmethod is disarmed outside the lock. However, we have some callers that now check it redundantly. I think those callers should stop doing that now. Otherwise, this looks good to me.

Thanks for the feedback! Looking around the code, I think there are a few places where we can do more changes.

First, remove check here:

jdk/src/hotspot/share/code/nmethod.cpp

Lines 852 to 855 in bc7d9e3

if (bs_nm->is_armed(nm)) {

bool alive = bs_nm->nmethod_entry_barrier(nm);

assert(alive, "should be alive");

}

This would force us to add the check in super-class implementation here:

jdk/src/hotspot/share/gc/shared/barrierSetNMethod.cpp

Line 83 in bc7d9e3

bool BarrierSetNMethod::nmethod_entry_barrier(nmethod* nm) {

Second, we can remove the check here:

jdk/src/hotspot/share/gc/shared/barrierSetNMethod.cpp

Lines 175 to 181 in bc7d9e3

if (!bs_nm->is_armed(nm)) {

return 0;

}

assert(!nm->is_osr_method(), "Should not reach here");

// Called upon first entry after being armed

bool may_enter = bs_nm->nmethod_entry_barrier(nm);

But it does not seem straightforward, because we currently skip cross-modification fence based on is_armed(...) check. Unfortunately, we cannot easily know if nmethod_entry_barrier acted or not, we only know if method is safe or not. Can we / should we do these refactoring separately?

I see your point. However, this PR is refactoring the code to iron out who is responsible for checking is_armed, so I would prefer if we got that right in this PR. We say it should be the backend code doing that, so the callers shouldn't. I agree with all the changes you just listed and if you make them I would be happy.

Regarding the cross modifying fence, I strongly prefer to not try and be clever. Just run the cross modifying fence unconditionally after calling the backend code. We get there because the barrier was armed anyway.

8331911: Reconsider locking for recently disarmed nmethods

f4cdbf6

bridgekeeper bot added the oca Needs verification of OCA signatory status label May 17, 2024

openjdk bot added hotspot-gc hotspot-gc-dev@openjdk.org shenandoah shenandoah-dev@openjdk.org labels May 17, 2024

bridgekeeper bot added the oca-verify Needs verification of OCA signatory status label May 17, 2024

8331911: Fix whitespace

5a0d556

shipilev reviewed May 22, 2024

View reviewed changes

bridgekeeper bot removed oca Needs verification of OCA signatory status oca-verify Needs verification of OCA signatory status labels May 23, 2024

openjdk bot added the rfr Pull request is ready for review label May 23, 2024

Address feedback on whitespace and comments

8c5bad2

shipilev approved these changes May 23, 2024

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label May 23, 2024

Update full name

c59034d

fisk suggested changes May 29, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8331911: Reconsider locking for recently disarmed nmethods #19285

8331911: Reconsider locking for recently disarmed nmethods #19285

neethu-prasad commented May 17, 2024 •

edited by openjdk bot

bridgekeeper bot commented May 17, 2024

openjdk bot commented May 17, 2024 •

edited

openjdk bot commented May 17, 2024

neethu-prasad commented May 17, 2024

bridgekeeper bot commented May 17, 2024

shipilev left a comment

shipilev May 22, 2024

shipilev May 22, 2024

shipilev May 22, 2024

shipilev May 22, 2024

mlbridge bot commented May 23, 2024 •

edited

shipilev left a comment

openjdk bot commented May 23, 2024

shipilev commented May 29, 2024

fisk left a comment

neethu-prasad commented May 29, 2024

fisk commented May 30, 2024

		@@ -36,6 +36,13 @@
		#include "runtime/threadWXSetters.inline.hpp"

		bool ShenandoahBarrierSetNMethod::nmethod_entry_barrier(nmethod* nm) {

		// Some other thread got here first and healed the oops
		// and disarmed the nmethod.

8331911: Reconsider locking for recently disarmed nmethods #19285

Are you sure you want to change the base?

8331911: Reconsider locking for recently disarmed nmethods #19285

Conversation

neethu-prasad commented May 17, 2024 • edited by openjdk bot

Progress

Issue

Reviewers

Reviewing

Webrev

bridgekeeper bot commented May 17, 2024

openjdk bot commented May 17, 2024 • edited

openjdk bot commented May 17, 2024

neethu-prasad commented May 17, 2024

bridgekeeper bot commented May 17, 2024

shipilev left a comment

Choose a reason for hiding this comment

shipilev May 22, 2024

Choose a reason for hiding this comment

shipilev May 22, 2024

Choose a reason for hiding this comment

shipilev May 22, 2024

Choose a reason for hiding this comment

shipilev May 22, 2024

Choose a reason for hiding this comment

mlbridge bot commented May 23, 2024 • edited

Webrevs

shipilev left a comment

Choose a reason for hiding this comment

openjdk bot commented May 23, 2024

shipilev commented May 29, 2024

fisk left a comment

Choose a reason for hiding this comment

neethu-prasad commented May 29, 2024

fisk commented May 30, 2024

neethu-prasad commented May 17, 2024 •

edited by openjdk bot

openjdk bot commented May 17, 2024 •

edited

mlbridge bot commented May 23, 2024 •

edited