More efficient ArraySeq iteration

[szeiger]

• Use ArrayIterator for ArraySeq: As discovered in
  Scala 2.13 Iterator .take() method converts dropped values into null bug#11658, ArraySeq (both mutable
  and immutable) used the generic IndexedSeqView-based iterators even
  though the more efficient ArrayIterator is available.

• Avoid runtime type checks for ArraySeq steppers: ArraySeq.stepper
  had to perform a runtime type check for the array. Now there are
  separate implementations for all specialized ArraySeq types that
  avoid the extra checks.

• Add a useful knownSize implementation to ArrayIterator.

[lrytz]

+java.io.NotSerializableException: scala.collection.ArrayOps$ArrayIterator$mcI$sp

in test/files/run/serialize-stream.scala

[lrytz]

Stepper change LGTM

[szeiger]

The serialization change is problematic. ArrayIterator (along with the other array-specific iterator implementations) is not serializable. It should have been. Making it serializable now is not forwards binary compatible, suddenly returning a non-serializable implementation is not strictly incompatible (since we never promised that it would be serializable) but can break code (as observed in the test failure).

[lrytz]

We could have a private, Serailizable subclass of ArrayIterator and use that (without exposing its static type).

Is there a general story / spec about iterators being serializable?

[szeiger]

Hm, right. On second thought, making those classes Serializable should be OK. They are private anyway.

[lrytz]

Ah right, ArrayIterator is private, so even better.

[szeiger]

The MiMa errors are strange:

* abstract method stepper(scala.collection.StepperShape)scala.collection.Stepper in class scala.collection.immutable.ArraySeq does not have a correspondent in current version
* in current version there is abstract method stepper(scala.collection.StepperShape)scala.collection.Stepper in class scala.collection.immutable.ArraySeq, which does not have a correspondent
* abstract method stepper(scala.collection.StepperShape)scala.collection.Stepper in class scala.collection.mutable.ArraySeq does not have a correspondent in current version
* in current version there is abstract method stepper(scala.collection.StepperShape)scala.collection.Stepper in class scala.collection.mutable.ArraySeq, which does not have a correspondent

The signatures are the same but it complains about the methods both ways.

(And contrary to what I originally suspected, relaxing accessibility of ArrayIterator is not a binary compatibility issue at all because it ends up being public in bytecode either way)

[szeiger]

I examined the signatures manually. The MiMa errors are confusing and the wording of the first one ("abstract method ... does not have a correspondent in current version") is plain wrong. The actual change is as follows:

Old (concrete stepper implementation in ArraySeq):

  public <S extends scala.collection.Stepper<?>> S stepper(scala.collection.StepperShape<A, S>);
    descriptor: (Lscala/collection/StepperShape;)Lscala/collection/Stepper;

New (abstract stepper declaration):

  public abstract <S extends scala.collection.Stepper<?>> S stepper(scala.collection.StepperShape<A, S>);
    descriptor: (Lscala/collection/StepperShape;)Lscala/collection/Stepper;

So whereas the first error message gives the impression that the old version had an abstract method, it is in fact the new version.

We could leave out the abstract declaration to satisfy MiMa but it wouldn't be source compatible. It exists to refine the return type from S to S with EfficientSplit (which is erased in bytecode). Since ArraySeq is sealed, it doesn't matter if the method is abstract or concrete.

[szeiger]

@lrytz Any further comments on this?

[dwijnand]

@szeiger Just saw this. In case you want to revisit this, this might be fixed in current MiMa but if not we could improve it.