Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add model_uuid to Java Model #5165

Merged
merged 8 commits into from Dec 15, 2021
Merged

Add model_uuid to Java Model #5165

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a93307d
add model_uuid
harupy Dec 14, 2021
209eca4
rename
harupy Dec 14, 2021
c659a89
test
harupy Dec 14, 2021
3451c24
uncomment
harupy Dec 14, 2021
05ac51b
add test
harupy Dec 15, 2021
6b246d4
update tests
harupy Dec 15, 2021
98ebc1e
set model uuid
harupy Dec 15, 2021
290a9d6
address comments
harupy Dec 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 10 additions & 0 deletions mlflow/java/scoring/src/main/java/org/mlflow/models/Model.java
View file
Open in desktop
Expand Up @@ -6,6 +6,7 @@
import java.io.IOException;
import java.util.Map;
import java.util.Optional;
import java.util.UUID;
import org.mlflow.Flavor;
import org.mlflow.utils.FileUtils;
import org.mlflow.utils.SerializationUtils;
Expand Down Expand Up @@ -64,6 +65,11 @@ public static Model fromRootPath(String modelRootPath) throws IOException {
public static Model fromConfigPath(String configPath) throws IOException {
File configFile = new File(configPath);
Model model = SerializationUtils.parseYamlFromFile(configFile, Model.class);
// Set the model uuid if it's absent.
if (!model.getModelUuid().isPresent()) {
String uuid = UUID.randomUUID().toString().replace("-", "");
model.setModelUuid(uuid);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we set a new ID under this case ? I prefer to keep it empty because the ID should be generated while logging model (and after model logged, the ID should be immutable)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we set new generated ID here, suppose we load a old version model twice, we will get 2 in-memory model with different model ID, this break the rule of model ID being immutable

Copy link
Member Author

@harupy harupy Dec 15, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we set new generated ID here, suppose we load a old version model twice, we will get 2 in-memory model with different model ID, this break the rule of model ID being immutable.

@WeichenXu123 If so, I think we need to fix the python code.

import tempfile
import os

from mlflow.models import Model

with tempfile.TemporaryDirectory() as tmp_dir:
    path = os.path.join(tmp_dir, "MLmodel")

    with open(path, "w") as f:
        f.write(
            """
artifact_path: model
flavors:
  python_function:
    env: conda.yaml
    loader_module: mlflow.sklearn
    model_path: model.pkl
    python_version: 3.7.9
  sklearn:
    pickled_model: model.pkl
    serialization_format: cloudpickle
    sklearn_version: 0.24.1
"""
        )

    print(Model.load(path).model_uuid)
    print(Model.load(path).model_uuid)

output:

cac277097e54410c8d191a717e640cf6
560075c0351f43dca8554f8ab6eb96f4

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I create a fixing PR for python side:
#5167

}
// Set the root path to the directory containing the configuration file.
// This will be used to create an absolute path to the serialized model
model.setRootPath(configFile.getParentFile().getAbsolutePath());
Expand Down Expand Up @@ -112,4 +118,8 @@ public <T extends Flavor> Optional<T> getFlavor(String flavorName, Class<T> flav
private void setRootPath(String rootPath) {
this.rootPath = rootPath;
}

private void setModelUuid(String modelUuid) {
this.modelUuid = modelUuid;
}
}
2 changes: 1 addition & 1 deletion mlflow/java/scoring/src/test/java/org/mlflow/ModelTest.java
View file
Open in desktop
Expand Up @@ -29,7 +29,7 @@ public void testModelIsLoadedCorrectlyWhenModelUuidDoesNotExist() {
String configPath = getClass().getResource("sample_model_root/MLmodel.no.model_uuid").getFile();
try {
Model model = Model.fromConfigPath(configPath);
Assert.assertFalse(model.getModelUuid().isPresent());
Assert.assertTrue(model.getModelUuid().isPresent());
} catch (IOException e) {
e.printStackTrace();
Assert.fail("Encountered an exception while reading the model from a configuration path!");
Expand Down