xgboost4j_2.12:1.7.6 's (ml/dmlc/xgboost4j/java/XGBoostJNI.XGBoosterPredict) much slower than 0.90 in some model

[hanfengatonline]

When the concurrency exceeds 100, the function ml/dmlc/xgboost4j/java/XGBoostJNI.XGBoosterPredict occupies more than 49% of the CPU computation time.
this model featureNum = 11, and the model file is attached. which is .json file by 1.7.6 saved
6cdf5ed2e74d9cb614a293dbb1b91dc4_xgb_model_20230331_1_7_6.json

[hcho3]

@trivialfis Was the implementation for the predict function changed substantially?

[trivialfis]

From 0.9 to 1.7.6, yes. Specifically, in later versions, there's a block-based parallelism.

[SoulH-qqq]

From 0.9 to 1.7.6, yes. Specifically, in later versions, there's a block-based parallelism.

It stands to reason that the prediction speed of the model should be faster. But we found that prediction took longer.

[trivialfis]

Are you calling the prediction on a single sample or small batch of samples? On that case, it can be slower.

[SoulH-qqq]

Are you calling the prediction on a single sample or small batch of samples? On that case, it can be slower.

Make predictions on a single sample. Our scenario is to predict every time a piece of data comes in, so it is a single sample prediction. Is there any way to improve the prediction speed using 1.7.6, in the case of a single sample.

[trivialfis]

Hmm, for JVM packages, we might have to build out a new streaming inference library.

@wbo4958 Are you familiar with the JVM land for inference?