`Collectors.toMap` handling for streams #938

msridhar · 2024-03-22T22:47:05Z

Fixes #934

The key new thing with the support here is we have further nesting. Rather than a map method, where the relevant lambda is passed directly:

stream.filter(foo -> foo.bar != null).map(foo -> foo.bar.baz)

In this case we have a collect call, which gets as its argument the result of Collectors.toMap, and the relevant lambdas are passed to toMap:

stream
  .filter(foo -> foo.bar != null)
  .collect(Collectors.toMap(foo -> foo.bar.baz, foo -> foo.bar.other))

Supporting this requires some new types of logic in our streams handler (particularly because there are multiple relevant lambdas for a single collect call). We do also handle anonymous inner classes. I only added support for collecting into toMap for now; I'm not sure if there are other important cases.

I did a couple drive-by refactorings in this PR as I couldn't help it. I can undo them if they really make review more difficult, but I hope it's ok.

codecov · 2024-03-22T22:57:09Z

Codecov Report

Attention: Patch coverage is 96.51163% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 87.13%. Comparing base (ce892d7) to head (3d985b7).

Files	Patch %	Lines
...nullaway/handlers/StreamNullabilityPropagator.java	95.16%	0 Missing and 3 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #938      +/-   ##
============================================
+ Coverage     87.04%   87.13%   +0.08%     
- Complexity     1995     2011      +16     
============================================
  Files            77       78       +1     
  Lines          6447     6513      +66     
  Branches       1252     1264      +12     
============================================
+ Hits           5612     5675      +63     
  Misses          422      422              
- Partials        413      416       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lazaroclapp

Approach makes sense to me, mostly a few requests to update the docs :)

lazaroclapp · 2024-03-23T21:48:45Z

nullaway/src/main/java/com/uber/nullaway/handlers/StreamNullabilityPropagatorFactory.java

+                "<T,K,U>toMap(java.util.function.Function<? super T,? extends K>,java.util.function.Function<? super T,? extends U>)",
+                ImmutableSet.of(0, 1),
+                "apply",
+                ImmutableSet.of(0))


Does Rx have an equivalent? See getRxStreamNullabilityPropagator()

RxJava 3 does have an Observable.collect method that takes a Collector:

http://reactivex.io/RxJava/3.x/javadoc/io/reactivex/rxjava3/core/Observable.html#collect-java.util.stream.Collector-

And there was a previous collect API that acts like reduce or fold where this support may be relevant. But I'd rather address our immediate need in this PR, and maybe file a follow-up task on systematically going through the Stream and RxJava APIs to add support for further methods. Sound ok?

lazaroclapp · 2024-03-23T21:51:33Z

nullaway/src/main/java/com/uber/nullaway/handlers/stream/StreamModelBuilder.java

+    return this;
+  }
+
+  public StreamModelBuilder withCollectMethodFromSignature(


Javadoc, please, specially because this has a lot of arguments with complex to understand semantics (e.g. argsToCollectorFactoryMethod vs argsToCollectorFactoryMethod). An example in the docs might even be a good idea :)

Edit: After reading a bit more, an alternative is to link to the docs on CollectLikeMethodRecord, but I also wouldn't mind a bit of redundant documentation of this arguments here.

nullaway/src/main/java/com/uber/nullaway/handlers/stream/StreamTypeRecord.java

lazaroclapp · 2024-03-23T22:09:54Z

nullaway/src/main/java/com/uber/nullaway/handlers/StreamNullabilityPropagator.java

+              streamType.getCollectlikeMethodRecord(methodSymbol);
+          if (collectlikeMethodRecord != null && methodSymbol.getParameters().length() == 1) {
+            handleCollectCall(tree, collectlikeMethodRecord);
+          }


Not for this PR, but wonder if as a follow up you want to extract the other two cases of this if into their own methods too, with onMatchMethodInvocation(...) as just a dispatcher.

Opened #944 to keep this PR simpler

lazaroclapp · 2024-03-23T22:12:29Z

nullaway/src/main/java/com/uber/nullaway/handlers/StreamNullabilityPropagator.java

+
+  /**
+   * Handles a call to a collect-like method. If the argument to the method is supported, updates
+   * the {@link #collectCallToInnerMethodsOrLambdas} map appropriately.


Adding a description of the arguments here would make this method easier to follow. What method invocation is tree referring to on entry? (collect(...) itself, I presume)

lazaroclapp · 2024-03-23T22:14:06Z

nullaway/src/main/java/com/uber/nullaway/handlers/StreamNullabilityPropagator.java

+      MethodInvocationTree tree, CollectLikeMethodRecord collectlikeMethodRecord) {
+    ExpressionTree argTree = tree.getArguments().get(0);
+    if (argTree instanceof MethodInvocationTree) {
+      MethodInvocationTree collectInvokeArg = (MethodInvocationTree) argTree;


This one is the Collectors.toMap(...) invocation, right?

Yup, 8b3f984

lazaroclapp · 2024-03-23T22:25:51Z

nullaway/src/main/java/com/uber/nullaway/handlers/StreamNullabilityPropagator.java

Not required, but should we update the docs in lines 74-100 and the list of examples in the docs for observableCallToInnerMethodOrLambda to include info on .collect() calls?

Added a few lines in 8b3f984

lazaroclapp · 2024-03-23T22:29:25Z

nullaway/src/main/java/com/uber/nullaway/handlers/StreamNullabilityPropagator.java

+      LinkedHashMultimap.create();
+
+  // Map from map or collect method (or lambda) to corresponding previous filter method (e.g.
+  // B.apply => A.filter)


Probably need an example here for the collect case, specially if we update the global docs above.

msridhar · 2024-03-26T17:18:05Z

@lazaroclapp this is ready for another look. I added support for a couple more collector factory methods based on feedback in #934 (comment). This required some generalization, as we couldn't support multiple factory methods for a single collect method before; see d317d0e and c6ac215. But the overall logical structure is the same.

lazaroclapp

LGTM. One minor comment about a potential refactoring, but entirely up to you at this point :)

nullaway/src/main/java/com/uber/nullaway/handlers/StreamNullabilityPropagator.java

msridhar added 7 commits March 22, 2024 13:21

test

68d5fcd

WIP

94639cc

Merge branch 'master' into issue-934

dabd0f9

more

9959485

WIP

fc03dc4

fix cast

b747e93

fix and another test

aa99bd2

msridhar added 6 commits March 22, 2024 16:35

handle anonymous classes, more tests

96d057f

renaming and docs

cad63a0

rename

b473e5c

more refactoring and docs

ed01dcd

typo

363fc06

one more comment

30ecc4c

msridhar changed the title ~~[WIP] Collectors.toMap handling for streams~~ Collectors.toMap handling for streams Mar 23, 2024

msridhar marked this pull request as ready for review March 23, 2024 00:20

msridhar requested a review from lazaroclapp March 23, 2024 00:20

msridhar mentioned this pull request Mar 23, 2024

Control flow analysis fails within a stream collector #934

Closed

lazaroclapp reviewed Mar 23, 2024

View reviewed changes

msridhar added 2 commits March 26, 2024 08:45

Merge branch 'master' into issue-934

809c5d6

add docs

3923589

msridhar mentioned this pull request Mar 26, 2024

Extract methods in StreamNullabilityPropagator #944

Open

msridhar added 6 commits March 26, 2024 09:15

docs and rename

2724339

more docs

8b3f984

docs

1b57a96

fix Javadoc error

ff1b191

Add support for Collectors.groupingBy; required some refactoring

d317d0e

Support toImmutableMap

c6ac215

msridhar requested a review from lazaroclapp March 26, 2024 17:16

Merge branch 'master' into issue-934

24b0275

lazaroclapp approved these changes Mar 27, 2024

View reviewed changes

nullaway/src/main/java/com/uber/nullaway/handlers/StreamNullabilityPropagator.java Show resolved Hide resolved

comments

3d985b7

msridhar enabled auto-merge (squash) March 27, 2024 18:36

msridhar disabled auto-merge March 27, 2024 18:36

msridhar enabled auto-merge (squash) March 27, 2024 18:36

msridhar merged commit 76f0f77 into uber:master Mar 27, 2024
12 checks passed

msridhar deleted the issue-934 branch March 27, 2024 18:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`Collectors.toMap` handling for streams #938

`Collectors.toMap` handling for streams #938

msridhar commented Mar 22, 2024 •

edited

codecov bot commented Mar 22, 2024 •

edited

lazaroclapp left a comment

lazaroclapp Mar 23, 2024

msridhar Mar 26, 2024

lazaroclapp Mar 23, 2024

msridhar Mar 26, 2024

lazaroclapp Mar 23, 2024

msridhar Mar 26, 2024

lazaroclapp Mar 23, 2024

msridhar Mar 26, 2024

lazaroclapp Mar 23, 2024

msridhar Mar 26, 2024

lazaroclapp Mar 23, 2024

msridhar Mar 26, 2024

lazaroclapp Mar 23, 2024

msridhar Mar 26, 2024

msridhar commented Mar 26, 2024

lazaroclapp left a comment

Collectors.toMap handling for streams #938

Collectors.toMap handling for streams #938

Conversation

msridhar commented Mar 22, 2024 • edited

codecov bot commented Mar 22, 2024 • edited

Codecov Report

lazaroclapp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msridhar commented Mar 26, 2024

lazaroclapp left a comment

Choose a reason for hiding this comment

`Collectors.toMap` handling for streams #938

`Collectors.toMap` handling for streams #938

msridhar commented Mar 22, 2024 •

edited

codecov bot commented Mar 22, 2024 •

edited