Add getData function for DMatrix #9379
Conversation
jinmfeng001
force-pushed
the
add_get_data
branch
3 times, most recently
from
b170d57
to
caa4d9f
Compare
jinmfeng001
force-pushed
the
add_get_data
branch
5 times, most recently
from
16e5ee7
to
dcb94ac
Compare
|
cc @wbo4958 . |
|
I will check it today. sorry for late |
|
|
||
| for (int row = 0; row < rowNum; row++) { | ||
| for(int col = 0; col < colNum; col++){ | ||
| denseMatrix.set(row, col, 0.0f); |
There was a problem hiding this comment.
is it possible to set the default to 0.0f when creating BigDenseMatrix
There was a problem hiding this comment.
sorry for the late response. No such functions provided by BigDenseMatrix.
There was a problem hiding this comment.
Can we just return the CSR instead? Converting a CSR to a dense can be extremely memory hungry when input is sparse. Some datasets can span a few thousands of features with only a small portion of valid values due to encoding.
| int[] featureIndex = new int[nonMissingNum]; | ||
| float[] featureValue = new float[nonMissingNum]; | ||
|
|
||
| XGBoostJNI.checkCall(XGBoostJNI.XGDMatrixGetDataAsCSR(handle, "{}", |
There was a problem hiding this comment.
@trivialfis, is there an API to get the dense matrix directly instead of converting CSR data to the dense dmatrix?
There was a problem hiding this comment.
After checking the source code, I didn't find such functions. But @trivialfis can confirm it.
There was a problem hiding this comment.
No, DMatrix itself is a CSR matrix.
| BigDenseMatrix denseMatrix = new BigDenseMatrix(rowNum, colNum); | ||
|
|
||
| for (int row = 0; row < rowNum; row++) { | ||
| for(int col = 0; col < colNum; col++){ |
There was a problem hiding this comment.
for(int col = 0; col < colNum; col++) {
|
some minor comments, Overall, LGTM. |
jinmfeng001
force-pushed
the
add_get_data
branch
from
dcb94ac
to
3ebf9f6
Compare
The python API has a get_data function, add a similar api for jvm-package too.