Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using firebase-admin leads to "Container Healthcheck failed." when deploying function #7053

Closed
sebastianrueckerai opened this issue Apr 26, 2024 · 8 comments

Comments

@sebastianrueckerai
Copy link

[REQUIRED] Environment info

firebase-tools: 13.7.3

Platform: Ubuntu

[REQUIRED] Test case

// Content of "index.ts"
// Change "XXX" for actual credentials or path 
import { onCall } from "firebase-functions/v2/https"
import admin from "firebase-admin"
import { getFirestore } from "firebase-admin/firestore"
import fs from "fs"

const credentialPath = ".../XXX-firebase-adminsdk-gsnow-238e5320b0.json"
const credentialsFile = fs.readFileSync(credentialPath, "utf-8")
const serviceAccountCredentials = JSON.parse(credentialsFile) as admin.ServiceAccount

const prodConfig = {
  apiKey: "XXX",
  authDomain: "XXX",
  projectId: "XXX",
  storageBucket: "XXX",
  messagingSenderId: "XXX",
  appId: "XXX",
  measurementId: "XXX",
  credential: admin.credential.cert(serviceAccountCredentials),
}

admin.initializeApp(prodConfig)
const firestore = getFirestore()

const testUseAdmin = onCall({ region: "europe-west3" }, async () => {
  console.log("--> testUseAdmin running!")
  await firestore.collection("test").add({ name: "HeyItWorks!" })
})
export { testUseAdmin }

[REQUIRED] Steps to reproduce

Setup a TS firebase functions project with index.ts as above and deploy with:
firebase deploy --only functions:testUseAdmin

[REQUIRED] Expected behavior

Function should deploy without issue.

[REQUIRED] Actual behavior

Below are the logs without the --debug flag.

Adding --debug gives a lot more output and is leaking a lot of tokens and pathnames which are difficult to cut out. The error message at the end remains the same. Also: the error message is pointing me to some logs in GCP but following the URL, there are not logs to be found.

What I learned about the error:

  • The deploy works if I don't use firebase-admin
  • The code runs fine on the emulator.
  • I am currently trying to move my codebase to firebase function V2 and discovered this issue. This is my attempt to provide a minimal version of the error.

Any help would be greatly appreciated since this is a critical blocking issue for my team and me!

LOGS:

✖ 7 problems (0 errors, 7 warnings)

✔  functions: Finished running predeploy script.
i  functions: preparing codebase default for deployment
i  functions: ensuring required API cloudfunctions.googleapis.com is enabled...
i  functions: ensuring required API cloudbuild.googleapis.com is enabled...
i  artifactregistry: ensuring required API artifactregistry.googleapis.com is enabled...
✔  artifactregistry: required API artifactregistry.googleapis.com is enabled
✔  functions: required API cloudbuild.googleapis.com is enabled
✔  functions: required API cloudfunctions.googleapis.com is enabled
i  functions: Loading and analyzing source code for codebase default to determine what to deploy
Serving at port 8297

i  functions: Loaded environment variables from .env.no-bovine.
i  functions: preparing ts-functions directory for uploading...
i  functions: packaged /.../ts-functions (4.72 MB) for uploading
i  functions: ensuring required API run.googleapis.com is enabled...
i  functions: ensuring required API eventarc.googleapis.com is enabled...
i  functions: ensuring required API pubsub.googleapis.com is enabled...
i  functions: ensuring required API storage.googleapis.com is enabled...
✔  functions: required API storage.googleapis.com is enabled
✔  functions: required API run.googleapis.com is enabled
✔  functions: required API eventarc.googleapis.com is enabled
✔  functions: required API pubsub.googleapis.com is enabled
i  functions: generating the service identity for pubsub.googleapis.com...
i  functions: generating the service identity for eventarc.googleapis.com...
✔  functions: ts-functions folder uploaded successfully
i  functions: creating Node.js 20 (2nd Gen) function testUseAdmin(europe-west3)...
Could not create or update Cloud Run service testuseadmin, Container Healthcheck failed. Revision 'testuseadmin-00001-hoc' is not ready and cannot serve traffic. The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable. Logs for this revision might contain more information.

Logs URL: https://console.cloud.google.com/logs/viewer?...

Functions deploy had errors with the following functions:
        testUseAdmin(europe-west3)
i  functions: cleaning up build files...
⚠  functions: Unhandled error cleaning up build images. This could result in a small monthly bill if not corrected. You can attempt to delete these images by redeploying or you can delete them manually at https://console.cloud.google.com/gcr/images/...

Error: There was an error deploying functions
@aalej
Copy link
Contributor

aalej commented Apr 26, 2024

Hey @sebastianrueckerai, sorry to hear you’re encountering this issue. I’m currently trying to reproduce this, but so far I haven’t encountered any errors during deployment of functions.

I tried running the code you provided, but I encountered some errors related to imports, so I updated the imports to the ff:

import {onCall} from "firebase-functions/v2/https";
import * as admin from "firebase-admin";
import {getFirestore} from "firebase-admin/firestore";
import * as fs from "fs";

From what I can tell, the Firebase Admin SDK does not use some of the provided parameters, so I changed prodConfig to:

const prodConfig: admin.AppOptions = {
 projectId: "PROJECT_ID",
 storageBucket: "PROJECT_ID.appspot.com",
 credential: admin.credential.cert(serviceAccountCredentials),
};

I created this repo using the code snippets provided, which also includes the changes above. In case I’m missing anything to properly replicate the issue or if I made a mistake in the setup, please let me know. Also, could you share with me the version of firebase-functions and firebase-admin you’re using?

@aalej aalej added the Needs: Author Feedback Issues awaiting author feedback label Apr 26, 2024
@sebastianrueckerai
Copy link
Author

@aalej thanks for the prompt response!

I found that if I do admin.initializeApp() WITHOUT any config, it works.
I guess this is because GOOGLE_APPLICATION_CREDENTIALS is automatically set in the functions cloud runtime and points to valid credentials? (https://firebase.google.com/docs/admin/setup)

I am confused by this, should this not work with explicit credentials as well?

It also leaves me with further questions:

  • The error message is super unhelpful, any chance to make this more helpful for the next person?
  • The link that was supposed to point to relevant logs did not work (it was going to GCP Logs Explorer and showed nothing). In fact, Logs Explorer shows no logs at all for any of my functions. What I have to do instead is go to cloud functions, click on the function in question and then click on the logs tab, which is a different view (not logs explorer). Very strange and confusing!
  • I also still get the error: functions: Unhandled error cleaning up build images. with some files I have to manually clean up. Why does this happen?

@google-oss-bot google-oss-bot added Needs: Attention and removed Needs: Author Feedback Issues awaiting author feedback labels Apr 26, 2024
@serialito74
Copy link

serialito74 commented Apr 26, 2024

I have the same issue in Next.js app

✔  functions[firebase-frameworks-click-menu-dev-firebase:ssrclickmenudevfirebase(us-east1)] Successful create operation.
Function URL (firebase-frameworks-click-menu-dev-firebase:ssrclickmenudevfirebase(us-east1)): https://ssrclickmenudevfirebase-kxxxxxx-ue.a.run.app/
i  functions: cleaning up build files...
⚠  functions: Unhandled error cleaning up build images. This could result in a small monthly bill if not corrected. You can attempt to delete these images by redeploying or you can delete them manually at https://console.cloud.google.com/artifacts/docker/xxxx-xxxx-dev-firebase/us-east1/gcf-artifacts
i  hosting[click-menu-dev-firebase]: finalizing version...

Error: Failed to replace Run service projects/xxxxxxx/locations/us-east1/services/ssrclickmenudevfirebase

with --debug

{"error":{"code":400,"message":"spec.template.spec.runtimeClassName: When annotation [run.googleapis.com/base-images] set, runtimeClassName must be set to run.googleapis.com/linux-base-image-update","status":"INVALID_ARGUMENT","details":[{"@type":"type.googleapis.com/google.rpc.BadRequest","fieldViolations":
[2024-04-26T16:25:42.194Z] FirebaseError: HTTP Error: 400, spec.template.spec.runtimeClassName: When annotation [run.googleapis.com/base-images] set, runtimeClassName must be set to run.googleapis.com/linux-base-image-update
    at responseToError (/usr/local/lib/node_modules/firebase-tools/lib/responseToError.js:49:12)
    at RetryOperation._fn (/usr/local/lib/node_modules/firebase-tools/lib/apiv2.js:297:77)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 

I manually deleted the cloud function, cleaned the artifact repository, and removed files from storage, but without success.

@sebastianrueckerai
Copy link
Author

@aalej any news on this?

@joehan
Copy link
Contributor

joehan commented May 1, 2024

Hey @sebastianrueckerai - this error is almost always indicative of your code crashing on load. My best guess from you've shared is that the credentials path you've provided is not accessible when your code is uploaded to GCF. When you deploy to GCF, we zip up your functions directory and upload it to GCF - anything outside of that directory will not be accessible.

In general, I'd recommend that you use the application default credentials provided by GCF to authenticate calls from your functions. If you need to use different credentials, ensure that the service account key file is included in your functions directory

@sebastianrueckerai
Copy link
Author

@joehan Thanks, that was indeed the issue! :)

The reason it took me longer to figure that out it because as described above the link to the logs is not working. Any idea why that is?

Also it would be interesting to know how to get rid of this error (details above): Unhandled error cleaning up build images. This could result in a small monthly bill if not corrected. You can attempt to delete these images by redeploying or you can delete them manually at

This still show up even of the deploy is working!

@joehan
Copy link
Contributor

joehan commented May 3, 2024

@sebastianrueckerai Glad that cleared up the main issue! For the other issues:

'unhandled error cleaning up images' - We'll need to see the debug logs to help with one (or at the very least, the last bit of them printed from 'cleaning up build files' down.

Broken Logs URL link: That link comes back as part of an error message from the Cloud Functions API, and only says it 'may' contain relevant logs, so this might just be WAI. Unfortunately, I don't really have any visibility to help here - consider filing a bug on the Google Cloud Functions public issue tracker

@joehan
Copy link
Contributor

joehan commented May 7, 2024

Closing this for now since we address the primary issue.

@joehan joehan closed this as completed May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants