Investigate the cause for the max round increase during testing #196

masih · 2024-05-07T09:42:57Z

The expectation is that the tests should converge in smaller number of rounds than the current max. It was increased from 10 to 18, and again from 18 to 30 (though only needing 21) .

Ideally, it should take fewer rounds for the consensus to occur. Currently it is unclear if this is a side effect of the gPBFT algorithm changes or some unintended behaviour in simulations.

Use `heap.Interface` to simplify the implementation of message passing during simulation by implementing a priority queue. The messages in-flight are priorities in ascending order of their `deliverAt` time, i.e. messages with earlier delivery time have higher priority. Additionally, explicitly introduce an option to set Global Stabilization Time (GST). GST duration is one of the fundamental assumptions made by gPBFT, beyond which message delivery is assumed to be guaranteed. Enhance Withold adversary tests with varying GST time. Relates to #196

When the sender and receiver of a message is the same, always return zero as the latency sample. Relates to #196

Set the default simulation latency to `None` when unset to avoid surprising the user. The default was arbitrarily chosen from the simulation CLI and has no significance. Change the way max round is calculated, where all participants are checked. Fail if simulation go beyond max. Relates to #196

Use `heap.Interface` to simplify the implementation of message passing during simulation by implementing a priority queue. The messages in-flight are priorities in ascending order of their `deliverAt` time, i.e. messages with earlier delivery time have higher priority. Additionally, explicitly introduce an option to set Global Stabilization Time (GST). GST duration is one of the fundamental assumptions made by gPBFT, beyond which message delivery is assumed to be guaranteed. Enhance Withold adversary tests with varying GST time. Relates to #196

The latency model in simulations is generated using a Log Normal distribution with some configured mean. The latency generation occurs every time a message is sent from any one participant to another participant. This effectively simulates an adhoc network with volatile latency which is typically observed in mobile networks. Compared to Filecoin network, this approach is unrealistic and too volatile. Because, in Filecoin SPs are typically well-connected servers with high bandwidth low latency connection to their peers. As a result, tests regularly exceed the maximum number of rounds where, so far, they require at least 21 rounds to succeed. A more realistic latency model is one that uses some randomly generated but fixed point-to-point latency that remains constant or varies very slightly over the lifetime of a participant. The changes here introduce consistent P2P modeling of latency for Log Normal latency model, where the latency from one participant to another is generated once and remains constant for the lifetime of a simulation. The latency model uses an asymmetric model where, latency from one node to another is not necessarily the same other way around. This change reduces the maximum number of rounds required for tests from 30 to 10. Fixes #196

Set the default simulation latency to `None` when unset to avoid surprising the user. The default was arbitrarily chosen from the simulation CLI and has no significance. Change the way max round is calculated, where all participants are checked. Fail if simulation go beyond max. Relates to #196

Use `heap.Interface` to simplify the implementation of message passing during simulation by implementing a priority queue. The messages in-flight are priorities in ascending order of their `deliverAt` time, i.e. messages with earlier delivery time have higher priority. Additionally, explicitly introduce an option to set Global Stabilization Time (GST). GST duration is one of the fundamental assumptions made by gPBFT, beyond which message delivery is assumed to be guaranteed. Enhance Withold adversary tests with varying GST time. Relates to #196

The latency model in simulations is generated using a Log Normal distribution with some configured mean. The latency generation occurs every time a message is sent from any one participant to another participant. This effectively simulates an adhoc network with volatile latency which is typically observed in mobile networks. Compared to Filecoin network, this approach is unrealistic and too volatile. Because, in Filecoin SPs are typically well-connected servers with high bandwidth low latency connection to their peers. As a result, tests regularly exceed the maximum number of rounds where, so far, they require at least 21 rounds to succeed. A more realistic latency model is one that uses some randomly generated but fixed point-to-point latency that remains constant or varies very slightly over the lifetime of a participant. The changes here introduce consistent P2P modeling of latency for Log Normal latency model, where the latency from one participant to another is generated once and remains constant for the lifetime of a simulation. The latency model uses an asymmetric model where, latency from one node to another is not necessarily the same other way around. This change reduces the maximum number of rounds required for tests from 30 to 10. Fixes #196

Use `heap.Interface` to simplify the implementation of message passing during simulation by implementing a priority queue. The messages in-flight are priorities in ascending order of their `deliverAt` time, i.e. messages with earlier delivery time have higher priority. Additionally, explicitly introduce an option to set Global Stabilization Time (GST). GST duration is one of the fundamental assumptions made by gPBFT, beyond which message delivery is assumed to be guaranteed. Enhance Withold adversary tests with varying GST time. Relates to #196

The latency model in simulations is generated using a Log Normal distribution with some configured mean. The latency generation occurs every time a message is sent from any one participant to another participant. This effectively simulates an adhoc network with volatile latency which is typically observed in mobile networks. Compared to Filecoin network, this approach is unrealistic and too volatile. Because, in Filecoin SPs are typically well-connected servers with high bandwidth low latency connection to their peers. As a result, tests regularly exceed the maximum number of rounds where, so far, they require at least 21 rounds to succeed. A more realistic latency model is one that uses some randomly generated but fixed point-to-point latency that remains constant or varies very slightly over the lifetime of a participant. The changes here introduce consistent P2P modeling of latency for Log Normal latency model, where the latency from one participant to another is generated once and remains constant for the lifetime of a simulation. The latency model uses an asymmetric model where, latency from one node to another is not necessarily the same other way around. This change reduces the maximum number of rounds required for tests from 30 to 10. Fixes #196

#209) Use `heap.Interface` to simplify the implementation of message passing during simulation by implementing a priority queue. The messages in-flight are priorities in ascending order of their `deliverAt` time, i.e. messages with earlier delivery time have higher priority. Additionally, explicitly introduce an option to set Global Stabilization Time (GST). GST duration is one of the fundamental assumptions made by gPBFT, beyond which message delivery is assumed to be guaranteed. Enhance Withold adversary tests with varying GST time. Relates to #196

masih self-assigned this May 9, 2024

masih mentioned this issue May 10, 2024

Simplify simulation messaging with priority queue and parameterise GST #209

Merged

masih added a commit that referenced this issue May 10, 2024

Adjust latency model to always return zero for local messages

aa6cd20

When the sender and receiver of a message is the same, always return zero as the latency sample. Relates to #196

masih mentioned this issue May 10, 2024

Adjust latency model to always return zero for local messages #210

Merged

github-merge-queue bot pushed a commit that referenced this issue May 10, 2024

Adjust latency model to always return zero for local messages (#210)

f628e68

When the sender and receiver of a message is the same, always return zero as the latency sample. Relates to #196

masih mentioned this issue May 10, 2024

Set default simulation latency to None #211

Merged

masih mentioned this issue May 13, 2024

Consistently model P2P latency to reduce max rounds in tests #212

Merged

masih closed this as completed in #212 May 14, 2024

This was referenced May 14, 2024

Fuzz RNG seed in simulation tests #218

Closed

Comprehensive Simulation Testing #226

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Investigate the cause for the max round increase during testing #196

Investigate the cause for the max round increase during testing #196

masih commented May 7, 2024

Investigate the cause for the max round increase during testing #196

Investigate the cause for the max round increase during testing #196

Comments

masih commented May 7, 2024