Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Annotate impl/codegen with IWYU pragmas #27252

Merged
merged 12 commits into from
Sep 8, 2021
Merged

Annotate impl/codegen with IWYU pragmas #27252

merged 12 commits into from
Sep 8, 2021

Conversation

ctiller
Copy link
Member

@ctiller ctiller commented Sep 6, 2021

Help users of this tool land on the right headers.

Add a tool to keep this up to date.

@nicolasnoble

@ctiller ctiller added the release notes: yes Indicates if PR needs to be in release notes label Sep 6, 2021
@ctiller ctiller requested a review from drfloob September 6, 2021 04:07
@ctiller ctiller changed the title Add a tool to annotate impl/codegen with IWYU pragmas Annotate impl/codegen with IWYU pragmas Sep 6, 2021
@@ -19,6 +19,8 @@
#ifndef GRPC_IMPL_CODEGEN_ATM_H
#define GRPC_IMPL_CODEGEN_ATM_H

// IWYU pragma: private, include <grpc/impl/codegen/atm.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what this accomplishes, the specified alternative file is the <...> way of naming itself. Is this so that IWYU suggests our preferred system-header-style include format?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah it's got the wrong path... good catch!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed now

@drfloob
Copy link
Member

drfloob commented Sep 7, 2021

Do I understand correctly that the goal is to replace most (maybe all?) #include ".*/codegen/.*" instances with their support path equivalents? For example, we have ~120 places that use #include <grpc/impl/codegen/port_platform.h>, but ~900 places that use the <grpc/support/port_platform.h> version. Is there anywhere that internal code should be including from the codegen folder directly, maybe within the codegen folder itself?

@ctiller
Copy link
Member Author

ctiller commented Sep 7, 2021 via email

Copy link
Member

@drfloob drfloob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm missing something in the pragma script, but for example, I don't see what would prevent IWYU from suggesting that the includes in include/grpc/impl/codegen/sync.h need to change to use <grpc/support/sync_generic.h> instead of the current include path <grpc/impl/codegen/sync_generic.h>.

def fix_tree(tree):
"""Fix one include tree"""
# Map of filename --> paths including that filename
reverse_map = {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggest using defaultdict

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@ctiller
Copy link
Member Author

ctiller commented Sep 8, 2021

The sync_generic example is interesting... and yeah I think it needs handling too. I don't think this is the end state for these annotations, rather I expect to be iterating on them for a while - as we can grab real code and see what it recommends. I'm also trying to bound time here a little bit - this isn't high high priority, but would help some folks out.

@ctiller
Copy link
Member Author

ctiller commented Sep 8, 2021

ok nerd sniped... added some hacks to the scripts to get better behavior with atm, sync

@@ -18,7 +18,7 @@
#ifndef GRPCPP_IMPL_CODEGEN_SYNC_STREAM_H
#define GRPCPP_IMPL_CODEGEN_SYNC_STREAM_H

// IWYU pragma: private, include <grpcpp/support/sync_stream.h>
// IWYU pragma: private, include <grpc/support/sync.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs an exception, sync_stream is not a sync alternative.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think done before this review?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you're right, I wasn't looking at the complete diff.

@drfloob
Copy link
Member

drfloob commented Sep 8, 2021

The sync_generic example is interesting... and yeah I think it needs handling too

My only hold-up is whether IWYU would be useful if it does the wrong thing on our own codebase, and needs manual intervention every time we run it (adding back impl/codegen to includes). I agree this would be a nice win for gRPC users, and AFAICT these pragmas would work for IWYU runs on projects that depend on gRPC as a library. Maybe we could caution gRPC devs (in comments/documentation somewhere) that IWYU may do the wrong thing if run on our include folders?

@ctiller
Copy link
Member Author

ctiller commented Sep 8, 2021

We've (AFAICT) never run IWYU on our codebase - I tried early on - the OSS version is hard to set up and marked as alpha quality software -- and so the likelihood that we embrace it soon is low. If we do so, often it ends up being a substantial project the first time through adding pragma's to necessary headers to make sure that those inclusions are not touched.

In the meantime, we've got a redacted huge number of misdirected include directives from customers who are using this tooling and getting bad results.

I can add a comment somewhere, but I don't know where would be useful.

@ctiller
Copy link
Member Author

ctiller commented Sep 8, 2021

(also note that we plan to eliminate impl/codgen entirely early next year, so this is a bandaid at best)

Copy link
Member

@drfloob drfloob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. Any gRPC devs searching git for IWYU will likely come across this PR, so maybe the issue is documented enough.

@ctiller ctiller merged commit 2831634 into grpc:master Sep 8, 2021
@ctiller ctiller deleted the iwyu branch September 8, 2021 16:13
@copybara-service copybara-service bot added the imported Specifies if the PR has been imported to the internal repository label Sep 8, 2021
lidizheng pushed a commit to lidizheng/grpc that referenced this pull request Sep 23, 2021
* Add a tool to annotate impl/codegen with IWYU pragmas

* xx

* oops

* fmt

* x

* fix wrong direction bug

* use defaultdict

* better annotations

* better annotations

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
imported Specifies if the PR has been imported to the internal repository lang/c++ lang/core release notes: yes Indicates if PR needs to be in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants