Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve MurmurHash string hash memory footprint #5028

Merged
merged 3 commits into from May 24, 2021
Merged

Improve MurmurHash string hash memory footprint #5028

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f697381
Reduce memory footprint
Arkatufus May 21, 2021
19bd4f3
Fix enumerator, off by 1 error
Arkatufus May 21, 2021
213ea3c
Merge branch 'dev' into FIX_DData_cluster_sharding_pruning
Arkatufus May 24, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/Akka.sln
View file
Open in desktop
Expand Up @@ -244,7 +244,7 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Samples.Akka.AspNetCore", "
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "SerializationBenchmarks", "benchmark\SerializationBenchmarks\SerializationBenchmarks.csproj", "{2E4B9584-42CC-4D17-B719-9F462B16C94D}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DDataStressTest", "examples\Cluster\DData\DDataStressTest\DDataStressTest.csproj", "{44B3DDD6-6103-4E8F-8AC2-0F4BA3CF6B50}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DDataStressTest", "examples\Cluster\DData\DDataStressTest\DDataStressTest.csproj", "{44B3DDD6-6103-4E8F-8AC2-0F4BA3CF6B50}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Expand Down
6 changes: 3 additions & 3 deletions src/contrib/cluster/Akka.DistributedData/ORSet.cs
View file
Open in desktop
Expand Up @@ -712,9 +712,9 @@ private ORSet<T> MergeRemoveDelta(RemoveDeltaOperation delta)
{
while (deleteDots.MoveNext())
{
var curr = deleteDots.Current;
deleteDotNodes.Add(curr.Key);
deleteDotsAreGreater &= (thisDot != null && (thisDot.VersionAt(curr.Key) <= curr.Value));
var current = deleteDots.Current;
deleteDotNodes.Add(current.Key);
deleteDotsAreGreater &= (thisDot != null && (thisDot.VersionAt(current.Key) <= current.Value));
}
}

Expand Down
2 changes: 1 addition & 1 deletion src/core/Akka/Actor/Address.cs
View file
Open in desktop
Expand Up @@ -40,7 +40,7 @@ public int Compare(Address x, Address y)
if (result != 0) return result;
result = string.CompareOrdinal(x.System, y.System);
if (result != 0) return result;
result = string.CompareOrdinal(x.Host ?? "", y.Host ?? "");
result = string.CompareOrdinal(x.Host ?? string.Empty, y.Host ?? string.Empty);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

if (result != 0) return result;
result = (x.Port ?? 0).CompareTo(y.Port ?? 0);
return result;
Expand Down
6 changes: 3 additions & 3 deletions src/core/Akka/Util/MurmurHash.cs
View file
Open in desktop
Expand Up @@ -202,20 +202,20 @@ public static int StringHash(string s)
{
unchecked
{
var sChar = s.ToCharArray();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the biggest offending code. MurmurHash is called once for every DataEnvelope that are going to be sent as a gossip. For a system that are trying to gossip thousands of gossips, this can add up quite significantly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Murmur_string_hash 60.87 ns 1.240 ns 2.389 ns 0.0229 - - 96 B
Jenkins_string_hash 221.14 ns 0.767 ns 0.680 ns 0.0153 - - 64 B
Murmur_binary_hash 166.72 ns 2.177 ns 2.036 ns - - - -
Jenkins_binary_hash 370.40 ns 0.709 ns 0.554 ns - - - -

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19041.985 (2004/May2020Update/20H1)
AMD Ryzen 7 1700, 1 CPU, 16 logical and 8 physical cores
.NET SDK=5.0.203
  [Host]     : .NET Core 3.1.15 (CoreCLR 4.700.21.21202, CoreFX 4.700.21.21402), X64 RyuJIT
  DefaultJob : .NET Core 3.1.15 (CoreCLR 4.700.21.21202, CoreFX 4.700.21.21402), X64 RyuJIT
Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Murmur_string_hash 40.39 ns 0.082 ns 0.077 ns - - - -
Jenkins_string_hash 218.54 ns 0.467 ns 0.390 ns 0.0153 - - 64 B
Murmur_binary_hash 166.06 ns 2.224 ns 2.080 ns - - - -
Jenkins_binary_hash 372.00 ns 2.132 ns 1.664 ns - - - -

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is definitely a big improvement.

var span = s.AsSpan();
var h = StartHash((uint)s.Length * StringSeed);
var c = HiddenMagicA;
var k = HiddenMagicB;
var j = 0;
while (j + 1 < s.Length)
{
var i = (uint)((sChar[j] << 16) + sChar[j + 1]);
var i = (uint)((span[j] << 16) + span[j + 1]);
h = ExtendHash(h, i, c, k);
c = NextMagicA(c);
k = NextMagicB(k);
j += 2;
}
if (j < s.Length) h = ExtendHash(h, sChar[j], c, k);
if (j < s.Length) h = ExtendHash(h, span[j], c, k);
return (int)FinalizeHash(h);
}
}
Expand Down