L.Util.formatNum round

[inkstak]

• I've looked at the documentation to make sure the behavior is documented and expected
• I'm sure this is a Leaflet code issue, not an issue with my own code nor with the framework I'm using (Cordova, Ionic, Angular, React…)
• I've searched through the issues to make sure it's not yet reported

Environment

• Leaflet : 1.3.1 & 1.4.0
• Browser : Firefox 66 & Chrome 73

To reproduce

L.Util.formatNum(1.005, 2)
> 1
// Returns 1 instead of 1.01

L.Util.formatNum(-1.4837191022531273, 18) 
> -1.4837191022531275
// Returns -1...75 instead of -1...73

The way formatNum performs rounding can result in unexpected changes to coordinates.
For example :

L.GeoJSON.latLngToCoords({  lng: -1.4837191022531273, lat: 43.49222084042808 }, 18)
> [-1.4837191022531275, 43.49222084042808]

The precision argument for latLngToCoords is not documented, but it may be.
I play with french cadastre, and such approximation leads to various mistakes.

Solution

According to this article on stackoverflow, this rounding method should be better :

L.Util.formatNum = formatNum

function formatNum(num, digits) {
  return +(Math.round(num + ('e+' + digits)) + ('e-' + digits))
}

[IvanSanchez]

I play with french cadastre, and such approximation leads to various mistakes.

That error between -1.4837191022531273 and -1.4837191022531275 is roughly equal to... checks notes... one tenth of an atom. I know that cadastre applications rely on precision, but I don't think one needs that much precision.

In all seriousness now: I acknowledge that it can lead to misdetection of features with the same coordinates.

If you could make that suggestion into a pull request, and see it all the unit tests still pass nicely, I don't think there should be any problem making the change.

[inkstak]

is roughly equal to... checks notes... one tenth of an atom

Sure ! But maths dont lie : Geos & Postgis are really sensible.
Take two polygons that touch, side by side. Sending the GeoJson to a back-end app by changing the 16th decimal, I'll fall into one polygon or another.

Pull request in progress.

[johnd0e]

Just for completeness:

#6587 was reverted because of performance issues, but there also was reported more serious flow with numbers in scientific notation:

var line = L.polyline([[0,-7.993322e-10],[10,20],[20,30]]);
L.geoJson(line.toGeoJSON());//errors on NaN
L.Util.formatNum(-7.993322e-10); //Returns NaN

But you may consider "2nd best" approach with epsilon: #7129

With this function we get almost same precision as in your original PR:

L.Util.formatNum(1.005, 2) === 1.01
L.Util.formatNum(-1.005, 2) === -1.01
L.Util.formatNum(1.555, 2) === 1.56
L.Util.formatNum(-1.555, 2) === -1.56
L.Util.formatNum(-7.993322e-10, 16) === -7.993322e-10
L.Util.formatNum(-1.4837191022531273, 18) === -1.4837191022531275 // Round-off error

Performance is great: https://jsperf.com/leaflet-issue-6668/2