Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scale_float does not honor limits for long values #105361

Closed
carlosdelest opened this issue Feb 9, 2024 · 3 comments · Fixed by #107966
Closed

scale_float does not honor limits for long values #105361

carlosdelest opened this issue Feb 9, 2024 · 3 comments · Fixed by #107966
Assignees
Labels
>bug :Search/Mapping Index mappings, including merging and defining field types Team:Search Meta label for search team

Comments

@carlosdelest
Copy link
Member

Elasticsearch Version

7.17 / 8.x

Installed Plugins

No response

Java Version

bundled

OS Version

N/A

Problem Description

When using scaled_float, there is no range check on the value and scaling factor to be outside the long value range.

According to documentation for scaled_float:

scaled_float - A floating point number that is backed by a long, scaled by a fixed double scaling factor

However, that assumption is not checked on document ingestion. The following mapping:

"properties": {
"a_scaled_float": {
"scaling_factor": 1000000,
"type": "scaled_float"
}
There's a scaled_float mapping configuration with value 1000000.
So, why did the number 1707339753294418 become 9223372036854.775? Shouldn't it be 1707339753294418000000?

{
"_index": "test-index",
"_id": "Z5t_iY0BLRoRMvLbP6e3",
"_score": 1,
"_source": {
"a_scaled_float": 1707339753294418
},
"fields": {
"a_scaled_float": [
9223372036854.775
]
}
}

This is due to the rounding up done here, which produces a long value which is outside range, and thus returns MAX_LONG.

As this provides an incorrect result, and the value is outside of the long range, the correct behaviour should be to error on the value as other numeric types do.

Steps to Reproduce

Set a mapping with properties:

PUT test-index
{
      "mappings": {
        "properties": {
            "scaled_float": {
              "type": "scaled_float",
              "scaling_factor": 1000000
            }
        }
      }
}

Ingest a document:

PUT test-index/_doc/1
{
  "scaled_float": 1707339753294418
}

This should return an error, as the number is outside the long range.

Logs (if relevant)

No response

@carlosdelest carlosdelest added >bug :Search/Mapping Index mappings, including merging and defining field types Team:Search Meta label for search team needs:triage Requires assignment of a team area label labels Feb 9, 2024
@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Feb 9, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@carlosdelest
Copy link
Member Author

We need to update docs here to provide proper guidance.

Also, the new behaviour should be corrected depending on the IndexVersion the index was created on. In case it's greater or equal than the fix version, it should return an error for a value that is not indexable using a long - unless ignore_malformed is specified as true.

@benwtrent benwtrent self-assigned this Feb 23, 2024
@benwtrent
Copy link
Member

To move the point that saturation should probably be disabled by default.

PUT scaled_float
{
	"mappings": {
		"properties": {
			"value": {
			    "type": "scaled_float",
			    "scaling_factor": 1000000
			}
		}
	}
}

POST scaled_float/_doc
{
	"value": 100000000000000.0
}

GET scaled_float/_search
{
	"fields": [ "value" ],
	"query": {
		"range": {
			"value": {
				"lt": 10000000000000.0,
				"gt": 100000000000000.0
			}
		}
	}
}

Returns hits, even though its impossible to have a number that is less than 10000000000000.0 and greater than 100000000000000.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Mapping Index mappings, including merging and defining field types Team:Search Meta label for search team
Projects
None yet
3 participants