gRPC-Core fails to archive for Catalyst

[paulb777]

What version of gRPC and what language are you using?

gRPC-Core 1.21.0

What operating system (Linux, Windows,...) and version?

macOS 10.15 Beta

What runtime / compiler are you using (e.g. python version or version of gcc)

Xcode 11.1

What did you do?

Archive a project that includes the gRPC-Core pod.

What did you expect to see?

Successful archive

What did you see instead?

Failed to find ssl.h in openssl_grpc framework.

Full details and repro instructions at firebase/firebase-ios-sdk#4007

Anything else we should know about your project / environment?

[sabiland]

I have the same issue when archiving for macOS Catalyst:
pod 'SwiftGRPC'

../Pods/gRPC-Core/src/core/tsi/ssl/session_cache/ssl_session.h:29:10: 'openssl_grpc/ssl.h' file not found

[paulb777]

I did a bit more investigation and found that Xcode uses at least two different sets of symbolic links when doing Catalyst archives and that seems to cause a problem with the header_mappings_dir or header_dir attributes in the BoringSSL podspec.

[paulb777]

I made an smaller repro case at https://github.com/paulb777/CatalystArchiveGRPC

• git clone git@github.com:paulb777/CatalystArchiveGRPC.git
• cd CatalystArchiveGRPC
• open CatalystArchiveGRPC.xcworkspace
• Choose MyMac scheme
• Product -> Archive

See the build failure

[muxi]

Thanks Paul. We don't have a mac with Catalina for debug right now and I'm working on getting one. Will check back.

[muxi]

Reproduced the problem. Looking for the problem

[muxi]

I observed the difference between build and archive: openssl_grpc.framework generated during build has the symbolic link openssl_grpc.framework/Headers -> openssl_grpc.framework/Versions/Current/Headers, while archive dose not create this symbolic link. If I manually use command line to create this symbolic link before openssl_grpc.framework finishes archiving, the archive process becomes successful. So it looks like this symbolic link is the key.

I still have no idea why the symbolic link is not generated during archive. @paulb777 any ideas? Does this look like an Xcode issue?

[paulb777]

@muxi It does look like an Xcode issue. It still would be interesting to understand what is different between openssl_grpc.framework and the frameworks that successfully archive for Catalyst.

[muxi]

That is good point. Checked the other frameworks in the same artifact directory for archive. nanopb.framework has the Headers symbolic link while grpc.framework does not (but it turned out ok, probably because in your sample project there's no external dependency onto it). There might be some sort of switch. Still confusing.

[hajjD]

Pardon my ignorance but is there a way a workout could be detailed step by step. I would greatly appreciate it, I don't know enough about symbolic links to mess with it in terminal.

[michael-mansour]

Same I am a bit lost a step by step instructions would be greatly appreciated!
Thanks

[muxi]

This is what I did:

• Archive the project in Xcode. It fails with error "openssl_grpc/xxx.h file not found".
• In the issue navigator, find the particular error, right click, click "Reveal in Log"
• In the build log that's showing the error, there's a parentheses with words loaded from '/Users/xxx/Library/Developer/Xcode/DerivedData/....'. Copy the full path and append it with /openssl_grpc.framework for use in the next step; should look something like /Users/xxx/Library/Developer/Xcode/DerivedData/......../BoringSSL-GRPC/openssl_grpc.framework
• Archive the project again. When Xcode shows it's building "BoringSSL-GRPC", go to terminal, cd into the directory in the above step, then run command ln -s Versions/Current/Headers Headers. This needs to be done before Xcode finishes building "BoringSSL-GRPC".
• Expect the file-not-found error not showing up this time.

[emreoktem]

@muxi, I tried this couple of times but didn't help :(

[muxi]

hmm... I was doing it with @paulb777's reproduction project; maybe it's different in yours? @emreoktem, could you share which step did you find not working?

I guess maybe I'll need to try with another project for further investigation.

[muxi]

So I tried with gRPC's hello world app with gRPC v1.21.0 and was able to archive with Catalyst successfully... I'm still not figured out why it just fails for archiving, or why it fails on @paulb777's sample project but not for the hello world.

[kajensen]

I was able to get a Catalyst build archived with @muxi 's steps (Thank you! -- needed to do it for multiple frameworks). What needs to be done for a permanent, non-hacky solution?

[muxi]

Thanks for the update. For now I don't have much idea on the root cause; doesn't look to me like anything's wrong in gRPC or Cocoapods. @paulb777 filed an issue with Apple on the difference of behaviors between build andI'l archive in Xcode, and I'll wait to see if that points us to a direction.

@paulb777 - sorry that it got stuck at this place. Did you get any reply from Apple?

[paulb777]

No reply from Apple. Maybe it will help increase the priority if others file the same issue?

[muxi]

I'll do that.

[honghaoz]

The solution provided by @muxi is helpful! In my case, I need to fix the Headers for:

• BoringSSL-GRPC/openssl_grpc.framework
• gRPC-Core/grpc.framework

[joekim]

@honghaoz - Can you provide the exact terminal commands? I understand that the directory will be different for each archive build, but I'm confused as to what the exact symlinks need to be created where.

[honghaoz]

@joekim Hi Joe,

To be exact, I did this:

1. when Xcode is archiving, cd to this folder (follow the pattern, the username/appname is different)

/Users/Username/Library/Developer/Xcode/DerivedData/YourApp-dowcxbourtsinleeggnghcwwuqhk/Build/Intermediates.noindex/ArchiveIntermediates/YourApp/BuildProductsPath/Release-maccatalyst/

You will see there're some frameworks are being built.

2. When Xcode is showing building BoringSSL-GRPC. Find the BoringSSL-GRPC in the folder and you will find there's a symlink openssl_grpc.framework. You should cd into this framework directory and run ln -s Versions/Current/Headers Headers

Basically, this creates a new symlink called Header which points to Versions/Current/Headers

Before:

After:

You should do this before Xcode reports a failure.

Similarly, I did the same thing in gRPC-Core/grpc.framework

Hope this helps!

[hajjD]

@joekim Hi Joe,

To be exact, I did this:

1. when Xcode is archiving, cd to this folder (follow the pattern, the username/appname is different)

/Users/Username/Library/Developer/Xcode/DerivedData/YourApp-dowcxbourtsinleeggnghcwwuqhk/Build/Intermediates.noindex/ArchiveIntermediates/YourApp/BuildProductsPath/Release-maccatalyst/

You will see there're some frameworks are being built.

1. When Xcode is showing building BoringSSL-GRPC. Find the BoringSSL-GRPC in the folder and you will find there's a symlink openssl_grpc.framework. You should cd into this framework directory and run ln -s Versions/Current/Headers Headers

Basically, this creates a new symlink called Header which points to Versions/Current/Headers

Before:

After:

You should do this before Xcode reports a failure.

Similarly, I did the same thing in gRPC-Core/grpc.framework

Hope this helps!

I felt like I was diffusing a bomb, but after several attempts I was able to get the timing down pat. I may just create an Apple Script or something temporarily until Apple/Firebase fixes the issue

[appfrilans]

Any fix for this?

Edit: Got it working. However, still does not feel like a long-term solution. Anyone making a script for this?

[appfrilans]

Cannot get the workaround to work. Anyone having a script for this?

[zummenix]

No automatic script to run it on CI. I've thrown together a script to run manually. It finds broken frameworks and patches them in the loop. I run it after starting archiving in xcode, after successful build hit Ctrl+C to stop it. Not pretty, but...

#!/usr/bin/env bash

# Semiautomatic workaround for the issue described in
# https://github.com/grpc/grpc/issues/20500

trap "exit" INT

while true; do
    sleep 0.1
    _boring_ssl=$(find /Users/zummenix/Library/Developer/Xcode/DerivedData/MyApp* \
        -name "BoringSSL-GRPC" | head -n1)

    cd "$_boring_ssl/openssl_grpc.framework" && ln -s Versions/Current/Headers Headers &&
        echo "Patched openssl_grpc"

    _grpc_core=$(find /Users/zummenix/Library/Developer/Xcode/DerivedData/MyApp* \
        -name "gRPC-Core" | head -n1)

    cd "$_grpc_core/grpc.framework" && ln -s Versions/Current/Headers Headers &&
        echo "Patched grpc"
done

Change paths in find commands for your project. And remove lines related to patching grpc.framework if you don't have it.

[stale]

This issue/PR has been automatically marked as stale because it has not had any update (including commits, comments, labels, milestones, etc) for 30 days. It will be closed automatically if no further update occurs in 7 day. Thank you for your contributions!

[muxi]

not stale

[jlaws]

Any updates?

[xurei]

I noticed that running the archive command in the CLI twice sometimes works.
This makes me think that it is some kind of race condition...

xcodebuild -quiet -workspace ios/Runner.xcworkspace -scheme Runner -archivePath builder/build.xcarchive -config Release archive || \
xcodebuild -quiet -workspace ios/Runner.xcworkspace -scheme Runner -archivePath builder/build.xcarchive -config Release archive 

EDIT : I may have found something. When I try to build with a fixed derived data path in /tmp, it seems to be working every time.

xcodebuild -quiet -derivedDataPath "/tmp/derived-data" -workspace ios/Runner.xcworkspace -scheme Runner -archivePath builder/build.xcarchive -config Release archive

Can someone try and confirm this ?

[levous]

@xurei xcodebuild -quiet -derivedDataPath "/tmp/derived-data" -workspace ios/Runner.xcworkspace -scheme Runner -archivePath builder/build.xcarchive -config Release archive

Did not work for me

In file included from /Users/username/dev/project-dir/Pods/gRPC-Core/src/core/ext/filters/client_channel/xds/xds_channel_secure.cc:34:
In file included from /Users/username/dev/project-dir/Pods/gRPC-Core/src/core/lib/security/credentials/credentials.h:35:
In file included from /Users/username/dev/project-dir/Pods/gRPC-Core/src/core/lib/security/security_connector/security_connector.h:33:
/Users/username/dev/project-dir/Pods/gRPC-Core/src/core/tsi/ssl_transport_security.h:29:12: fatal error: 
      'openssl_grpc/x509.h' file not found
  #include <openssl_grpc/x509.h>
           ^~~~~~~~~~~~~~~~~~~~~
/Users/username/dev/project-dir/Pods/gRPC-Core/src/core/tsi/ssl_transport_security.h:29:12: note: 
      did not find header 'x509.h' in framework 'openssl_grpc' (loaded from
      '/tmp/derived-data/Build/Intermediates.noindex/ArchiveIntermediates/App-Debug-Development/BuildProductsPath/Debug-Development-maccatalyst/BoringSSL-GRPC')

[lukemmtt]

Still an issue as of Jan 2021, but zummenix's script is an effective workaround for me.

[appfrilans]

Is this related https://fantashit.com/i-am-getting-multiple-command-produce-grpccertificates-bundle-error-in-firestore-when-archiving/ ?

In that case, is the issue solved in a grpc-version but not updated by Firebase? Running pod outdated gives me this:

- BoringSSL-GRPC 0.0.7 -> 0.0.7 (latest version 0.0.14)
- gRPC-C++ 1.28.2 -> 1.28.2 (latest version 1.34.0)
- gRPC-Core 1.28.2 -> 1.28.2 (latest version 1.34.0)
- nanopb 2.30906.0 -> 2.30906.0 (latest version 2.30907.0)

[appfrilans]

Nevermind, seems to be an old answer: https://stackoverflow.com/questions/53406947/xcode-10-ios-firebase-firestore-sdk-multiple-command-produce-grpccertificates

[appfrilans]

The script above works fine, but for automating with fastlane, this workaround wont work. Would super appreciate a real solution here.

[joeldrotleff]

Still an issue for me, would really appreciate if this was fixed for real (the workaround didn't work for me)

[yulin-liang]

It's really weird, when building the project above, openssl_grpc.framework creates a new symlink called Header which points to Versions/Current/Headers

But archiving doesn't create the symbolic link.

I still don't have any idea, will keep looking into it. Also filed an issue(CocoaPods/CocoaPods#10354) on the Cocoapods repo and see if they have any idea.

[yulin-liang]

It's a bug with Cocoapods, the PR(CocoaPods/CocoaPods#10224) should fix this.

[joeldrotleff]

Thanks for figuring this out @yulin-liang! I'm going to try and test with that Cocoapods PR when I have a chance to verify

[thecoolwinter]

The script above works fine, but for automating with fastlane, this workaround wont work. Would super appreciate a real solution here.

I'm using a small .sh script like this that daemons the archive fix, then runs the fastlane lane and then stops the archive fix process.

#!/usr/bin/env bash

./fix-archive.sh >/dev/null 2>&1 & pid=$!
echo Archive fix running on $pid

echo Archiving and notarizing MacOS build
bundle exec fastlane notarize_and_upload

kill -9 $pid
echo Killed archive fix script, exiting...

But this is still a super annoying issue. would appreciate a real fix.

[paulb777]

The fix is merged for the upcoming CocoaPods 1.10.2 release - see CocoaPods/CocoaPods#10224

[lukemmtt]

The CocoaPods version with the fix has yet to be released as of today.

Meanwhile, @zummenix 's workaround script above recently stopped working for me, but Paul's answer on StackOverflow helped me figure out what the issue was.

As of 6/12/21, it seems that the script only looks for the first instance of the specified "BoringSSL-GRPC" and "gRPC-Core" directories, which is problematic because my DerivedData folder suddenly contain several directories with those names, even after pod deintegrate / pod install and cleaning the build folder.

To be specific, the script was failing because (for example) it was finding the directory

/Users/USERNAME/Library/Developer/Xcode/DerivedData/APPNAME-fasmxxymfmziyecbxjrcyrieukzs/Index/Build/Products/Debug-maccatalyst/BoringSSL-GRPC/openssl_grpc.framework

instead of the directory that the archive compiler was complaining about:

/Users/USERNAME/Library/Developer/Xcode/DerivedData/APPNAME-fasmxxymfmziyecbxjrcyrieukzs/Build/Intermediates.noindex/ArchiveIntermediates/TimeFinder/BuildProductsPath/Release-maccatalyst

I'm not familiar with the script syntax, so I simply hardcoded my pathname in to get the script working again, and the project Archives without a problem now. It feels silly to be refining this script when the CocoaPods team is sitting on a release with the real fix, but if someone is familiar with the scripting syntax, we would just need to modify the script to patch the necessary files in ALL directories with the specified names, not just the first directories with the specified names.

Here's my hardcoded script for reference (with placeholder values):

#!/usr/bin/env bash

# Semiautomatic workaround for the issue described in
# https://github.com/grpc/grpc/issues/20500

trap "exit" INT

while true; do
    sleep 1
    
    _actualPath="/Users/USERNAME/Library/Developer/Xcode/DerivedData/APPNAME-fasmxxymfmziyecbxjrcyrieukzs/Build/Intermediates.noindex/ArchiveIntermediates/TimeFinder/BuildProductsPath/Release-maccatalyst"

    cd "$_actualPath/BoringSSL-GRPC/openssl_grpc.framework" && ln -s Versions/Current/Headers Headers &&
        echo "Patched openssl_grpc"

    cd "$_actualPath/gRPC-Core/grpc.framework" && ln -s Versions/Current/Headers Headers &&
        echo "Patched grpc"
done

[appfrilans]

The fix for me was just to change from cocoapods to SPM for the containing framework (firebase). Works like a charm!

[delasign]

Can confirm @appfrilans's comment & this is not an issue in SPM.