Commits on May 31, 2022

1. Add consensus_decode_from_finite_reader optimization …

   As things are right now, memory exhaustion protection in `Decodable`
   is based on checking input-decoded lengths against arbitrary limits,
   and ad-hoc wrapping collection deserialization in `Take`.
   
   The problem with that are two-fold:
   
   * Potential consensus bugs due to incorrect limits.
   * Performance degradation when decoding nested structured,
     due to recursive `Take<Take<..>>` readers.
   
   This change introduces a systematic approach to the problem.
   
   A concept of a "size-limited-reader" is introduced to rely on
   the input data to finish at enforced limit and fail deserialization.
   
   Memory exhaustion protection is now achived by capping allocations
   to reasonable values, yet allowing the underlying collections
   to grow to accomodate rare yet legitmately oversized data (with tiny
   performance cost), and reliance on input data size limit.
   
   A set of simple rules allow avoiding recursive `Take` wrappers.
   
   Fix #997

   

   dpc committed May 31, 2022
   Copy the full SHA

   082e185 View commit details

   Browse the repository at this point in the history