amplify/backend/amplify-meta.json' does not exist on Build

[KuzonFyre]

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

18.18.2

Amplify CLI Version

12.10.1

What operating system are you using?

Windows

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made. I did have to make some changes locally when I was having issues with my local cli env. The issue is #13667

Describe the bug

Any pull or push command works perfectly fine on my local enviorment. When I deploy my application I get this error, 🛑 Failed to pull the backend. amplify-meta.json does not exist. . I can confirm I have that file locally. I have not modified any build settings. This issue started occuring after I added an S3 storage bucket to amplify. I have pulled and pushed any potential updates to my repo.

Expected behavior

I expect the build to finish sucessfully. I have confirmed I have no node build issues in my application.

Reproduction steps

Not sure how to reproduce this error in a blank directory.

Project Identifier

b833d96149f7eee5c40e4281e8701b1b

Log output

2024-03-31T17:30:48.347Z [INFO]: Successfully cleaned up Git credentials
2024-03-31T17:30:48.348Z [INFO]: # Checking for Git submodules at: /codebuild/output/src856760570/src/napky/.gitmodules
2024-03-31T17:30:48.357Z [INFO]: # Retrieving environment cache...
2024-03-31T17:30:48.419Z [INFO]: # Retrieved environment cache
2024-03-31T17:30:48.419Z [INFO]: ---- Setting Up SSM Secrets ----
2024-03-31T17:30:48.419Z [INFO]: SSM params {"Path":"/amplify/d1f2rx89u9ycx/dev/","WithDecryption":true}
2024-03-31T17:30:52.199Z [INFO]: # Patching NPM package '@aws-amplify/cli' from 12.8.2 to 12.10.3...
2024-03-31T17:31:18.838Z [INFO]: # Installed '@aws-amplify/cli@12.10.3'
2024-03-31T17:31:18.959Z [INFO]: # Done patching NPM package '@aws-amplify/cli' to version 12.10.3
2024-03-31T17:31:18.973Z [INFO]: # Retrieving cache...
2024-03-31T17:31:21.706Z [INFO]: # Extracting cache...
2024-03-31T17:31:22.733Z [INFO]: # Extraction completed
2024-03-31T17:31:22.858Z [INFO]: # Retrieved cache
2024-03-31T17:31:27.114Z [INFO]: ## Starting Backend Build
                                 # Starting phase: build
2024-03-31T17:31:27.121Z [INFO]: ## Running amplify pull to generate aws-exports.js file for frontend
2024-03-31T17:31:29.506Z [INFO]: �[0mAmplify AppID found: [MYAPPID]. Amplify App name is: [MYAPPNAME]
2024-03-31T17:31:29.542Z [INFO]: �[0mBackend environment dev found in Amplify Console app: [MYAPPNAME]
2024-03-31T17:31:31.054Z [WARNING]: - Fetching updates to backend environment: dev from the cloud.
2024-03-31T17:31:31.604Z [WARNING]: - Building resource api/[APPNAME]
2024-03-31T17:31:31.604Z [WARNING]: - Building resource auth/[APPNAME]
2024-03-31T17:31:31.604Z [WARNING]: - Building resource storage/[NAME]
2024-03-31T17:31:31.712Z [WARNING]: ✔ Successfully pulled backend environment dev from the cloud.
2024-03-31T17:31:31.732Z [WARNING]: ✖ There was an error initializing your environment.
2024-03-31T17:31:31.742Z [INFO]: 🛑 Failed to pull the backend.
2024-03-31T17:31:31.743Z [INFO]: Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/
2024-03-31T17:31:31.753Z [INFO]: Session Identifier: 095c4be4-2f4a-4b43-8091-65843eb4674f
2024-03-31T17:31:31.754Z [WARNING]: - Creating Zip
2024-03-31T17:31:31.767Z [INFO]: ✅ Report saved: /tmp/{NAME]/report-1711906291757.zip
2024-03-31T17:31:31.767Z [INFO]: 
2024-03-31T17:31:31.767Z [WARNING]: - Sending zip
2024-03-31T17:31:31.770Z [INFO]: File at path: '/codebuild/output/src856760570/src/[NAME]/amplify/backend/amplify-meta.json' does not exist
2024-03-31T17:31:31.771Z [WARNING]: ✖ Sending zip
2024-03-31T17:31:31.799Z [ERROR]: !!! Build failed
2024-03-31T17:31:31.800Z [INFO]: Please read more about Amplify Hosting's support for SSR frameworks to find if your build failure is related to an unsupported feature: https://docs.aws.amazon.com/amplify/latest/userguide/amplify-ssr-framework-support.html. You may also find this troubleshooting guide useful: https://docs.aws.amazon.com/amplify/latest/userguide/troubleshooting-ssr-deployment.html
2024-03-31T17:31:31.800Z [ERROR]: !!! Error: Command failed with exit code 1
2024-03-31T17:31:31.800Z [INFO]: # Starting environment caching...
2024-03-31T17:31:31.800Z [INFO]: # Uploading environment cache artifact...
2024-03-31T17:31:31.879Z [INFO]: # Uploaded environment cache artifact
2024-03-31T17:31:31.879Z [INFO]: # Environment caching completed
Terminating logging...

Additional information

No response

Before submitting, please confirm:

• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

[ykethan]

Hey @KuzonFyre, thank you for reaching out. Could you download the #current-cloud-backend.zip from the S3 deployment bucket on the AWS console. The bucket name should end with deployment and contain the app name.
Could you verify if the current-cloud-backend contains the amplify-meta.json?

if you do not observe a amplify-meta.json in the zip, we could try carefully re-zipping the contents with the local copy of amplify-meta.json without creating a nested folder similar to as follows, then upload the zip and try pulling the env in a new empty folder outside your existing project directory to test.

cd /#current-cloud-backend/
zip -r "../#current-cloud-backend.zip" *

[KuzonFyre]

@ykethan I do see an amplify-meta.json file in the root dir of #current-cloud-backend.zip.

[ykethan]

@KuzonFyre could you try adding AMPLIFY_ENABLE_DEBUG_OUTPUT as true in the Amplify hosting environment variables and provide us the output?

[KuzonFyre]

                                 # Starting phase: build
2024-04-02T15:07:55.770Z [INFO]: ## Running amplify pull to generate aws-exports.js file for frontend
2024-04-02T15:07:58.003Z [INFO]: �[0mAmplify AppID found: d1f2rx89u9ycx. Amplify App name is: Napky�[0m
2024-04-02T15:07:58.052Z [INFO]: �[0mBackend environment dev found in Amplify Console app: Napky�[0m
2024-04-02T15:07:59.526Z [WARNING]: - Fetching updates to backend environment: dev from the cloud.
2024-04-02T15:08:00.228Z [INFO]: Overrides functionality is not implemented for this category
2024-04-02T15:08:00.229Z [WARNING]: - Building resource api/napky
2024-04-02T15:08:00.229Z [WARNING]: - Building resource auth/napkyff72303c
2024-04-02T15:08:00.229Z [WARNING]: - Building resource storage/s3korkbostoragephotos
2024-04-02T15:08:00.282Z [WARNING]: ✔ Successfully pulled backend environment dev from the cloud.
2024-04-02T15:08:00.297Z [WARNING]: ✖ There was an error initializing your environment.
2024-04-02T15:08:00.306Z [INFO]: 🛑 Failed to pull the backend.
2024-04-02T15:08:00.306Z [INFO]: Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/
                                 PullBackendFault: Failed to pull the backend.
                                 at attachBackend (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend.js:55:15)
                                 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
                                 at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/commands/pull.js:64:9)
                                 at async Object.executeAmplifyCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:194:9)
                                 at async executePluginModuleCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:139:5)
                                 at async executeCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:37:9)
                                 at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:121:5)
                                 File at path: '/codebuild/output/src3871774277/src/napky/amplify/backend/auth/napkyff72303c/parameters.json' does not exist
                                 Error: File at path: '/codebuild/output/src3871774277/src/napky/amplify/backend/auth/napkyff72303c/parameters.json' does not exist
                                 at JSONUtilities.readJson (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-cli-core/lib/jsonUtilities.js:44:19)
                                 at StateManager.getData (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-cli-core/lib/state-manager/stateManager.js:327:56)
                                 at StateManager.getResourceParametersJson (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-cli-core/lib/state-manager/stateManager.js:149:25)
                                 at ensureAmplifyMetaFrontendConfig (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/extensions/amplify-helpers/on-category-outputs-change.js:105:60)
                                 at AmplifyToolkit.onCategoryOutputsChange (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/extensions/amplify-helpers/on-category-outputs-change.js:46:53)
                                 at initializeEnv (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/initialize-env.js:143:31)
                                 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
                                 at async onSuccess (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend.js:99:9)
                                 at async attachBackend (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend.js:50:9)
                                 at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/commands/pull.js:64:9)
                                 at async Object.executeAmplifyCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:194:9)
                                 at async executePluginModuleCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:139:5)
                                 at async executeCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:37:9)
                                 at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:121:5)
2024-04-02T15:08:00.315Z [INFO]: 
2024-04-02T15:08:00.315Z [INFO]: Session Identifier: 0263abb0-021f-480d-9e12-006ca4853c94
2024-04-02T15:08:00.316Z [WARNING]: - Creating Zip
2024-04-02T15:08:00.324Z [INFO]: 
2024-04-02T15:08:00.324Z [INFO]: ✅ Report saved: /tmp/Napky/report-1712070480318.zip
2024-04-02T15:08:00.324Z [WARNING]: - Sending zip
2024-04-02T15:08:00.324Z [INFO]: File at path: '/codebuild/output/src3871774277/src/napky/amplify/backend/amplify-meta.json' does not exist
2024-04-02T15:08:00.326Z [WARNING]: ✖ Sending zip
2024-04-02T15:08:00.352Z [ERROR]: !!! Build failed
2024-04-02T15:08:00.352Z [INFO]: Please read more about Amplify Hosting's support for SSR frameworks to find if your build failure is related to an unsupported feature: https://docs.aws.amazon.com/amplify/latest/userguide/amplify-ssr-framework-support.html. You may also find this troubleshooting guide useful: https://docs.aws.amazon.com/amplify/latest/userguide/troubleshooting-ssr-deployment.html
2024-04-02T15:08:00.352Z [ERROR]: !!! Error: Command failed with exit code 1
2024-04-02T15:08:00.352Z [INFO]: # Starting environment caching...
2024-04-02T15:08:00.353Z [INFO]: # Uploading environment cache artifact...
2024-04-02T15:08:00.417Z [INFO]: # Uploaded environment cache artifact
2024-04-02T15:08:00.417Z [INFO]: # Environment caching completed
Terminating logging...

[ykethan]

@KuzonFyre from the logs it appears the build failed due to

Error: File at path: '/codebuild/output/src3871774277/src/napky/amplify/backend/auth/napkyff72303c/parameters.json' does not exist

from the comment #13667 (comment) it appears you were able to run a push.
Is the Live package updates on the Amplify hosting Build setting currently set to latest or an older version on Amplify CLI? or in the amplify.yml are you manually installing an older version of Amplify CLI?

[KuzonFyre]

I double checked the current cloud backup file that I downloaded earlier and their is a parameters.json file but it is in the build folder like so auth/napkyff72303c/build/parameters.json.

Live Package updates is set to the latest version of the cli.

Not seeing anything in the yml file that is set to an older version. I'm working on a team so it's possible someone else might have modified the backend. I think that's why I got #13667 in the first place.

[ykethan]

@KuzonFyre interesting, the issue does show symptoms of this.
To ensure the project is in the correct state, could you zip the Amplify folder and send it to amplify-cli@amazon.com?

[KuzonFyre]

Sent an email.

[KuzonFyre]

@ykethan Any updates on this?

[edwardfoyle]

Hi @KuzonFyre taking a look at the stack trace you provided, it looks like the condition that is not being hit is here

amplify-cli/packages/amplify-cli-core/src/state-manager/pathManager.ts

Lines 161 to 162 in 88f6e23

	!fs.existsSync(path.join(resourceDirPath, PathConstants.ParametersJsonFileName)) &&
	fs.existsSync(path.join(resourceDirPath, PathConstants.CLIInputsJsonFileName)) &&

Looking at the amplify .zip archive you provided, I don't see what would be causing that condition to not hit. In addition to double checking that amplify/backend/auth/napkyff72303c/parameters.json does NOT exist and that amplify/backend/auth/napkyff72303c/cli-inputs.json DOES exist, can you also check that the same is true within the #current-cloud-backend directory?

[KuzonFyre]

I can confirm that amplify/backend/auth/napkyff72303c/parameters.json does not exist and that amplify/backend/auth/napkyff72303c/cli-inputs.json does exist in my local amplify dir. I can also confirm that is also true in the #current-cloud-backend dir that I downloaded from S3. I have sent another zip file to the email that was provided. This is the amplify folder for a previous version of the application that deployed correctly before we added a storage bucket for images. As far as I can tell there is nothing about adding storage to the Amplify folders that would have messed up Auth but it might be helpful.

[KuzonFyre]

While I would prefer to fix this without having to just delete the amplify folder and backend env on AWS and starting fresh, that is an acceptable work around at this point. I may test this on a git branch and update this comment if I observe the same issue on a fresh setup.

Update: Okay so I am going to send yet another zipped amplify folder that I set up on fresh build with the same setup, api, auth and storage. I ended up with the same error in AWS after I added the AMPLIFY_ENABLE_DEBUG_OUTPUT env variable.

Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/
                                 PullBackendFault: Failed to pull the backend.
                                 at attachBackend (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend.js:55:15)
                                 at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/commands/pull.js:64:9)
                                 at async Object.executeAmplifyCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:194:9)
                                 at async executePluginModuleCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:139:5)
                                 at async executeCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:37:9)
                                 at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:121:5)
                                 File at path: '/codebuild/output/src626817708/src/Napky/amplify/backend/auth/korkbo39ee6db5/parameters.json' does not exist
                                 Error: File at path: '/codebuild/output/src626817708/src/Napky/amplify/backend/auth/korkbo39ee6db5/parameters.json' does not exist
                                 at JSONUtilities.readJson (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-cli-core/lib/jsonUtilities.js:44:19)
                                 at StateManager.getData (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-cli-core/lib/state-manager/stateManager.js:327:56)
                                 at StateManager.getResourceParametersJson (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-cli-core/lib/state-manager/stateManager.js:149:25)
                                 at ensureAmplifyMetaFrontendConfig (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/extensions/amplify-helpers/on-category-outputs-change.js:105:60)
                                 at AmplifyToolkit.onCategoryOutputsChange (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/extensions/amplify-helpers/on-category-outputs-change.js:46:53)
                                 at initializeEnv (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/initialize-env.js:143:31)
                                 at async onSuccess (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend.js:99:9)
                                 at async attachBackend (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend.js:50:9)
                                 at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/commands/pull.js:64:9)
                                 at async Object.executeAmplifyCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:194:9)
                                 at async executePluginModuleCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:139:5)
                                 at async executeCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:37:9)
                                 at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:121:5)
2024-04-08T22:27:55.718Z [INFO]: 

In this new setup I also double checked the #current-cloud-backup dir and my local dir and made sure I had cli-inputs.json and not parameters.json. You would have seen this when you looked at the first zip I sent but their was a parameters.json in the auth/napky/build dir. I will try a new setup with just the api and auth enabled and see if that changes anything.
@edwardfoyle

[KuzonFyre]

Did one more check and it doesn't look like adding storage caused any issues. @edwardfoyle @ykethan

[ykethan]

@KuzonFyre from the information provided it appears issue doesnt appear be on the current-cloud-backend.zip. If are able to run amplify push locally and the issue is only occurring on the hosting builds with the live package override for Amplify CLI is set to latest, could you check if the local project root package.json and provide us the amplify packages installed here?
could you provide us the amplify.yml file here?

[KuzonFyre]

Here is the relevant dependencies in the package.json file. ```
"dependencies": {
"@aws-amplify/adapter-nextjs": "^1.0.21",
"@aws-amplify/ui-react": "^6.1.5",
"aws-amplify": "^6.0.18",
}

I do not have an amplify.yml file in the local dir but I do have a `.graphqlconfig.yml file`. 
```projects:
  korkbo:
    schemaPath: src/graphql/schema.json
    includes:
      - src/graphql/**/*.ts
    excludes:
      - ./amplify/**
    extensions:
      amplify:
        codeGenTarget: typescript
        generatedFileName: src/API.ts
        docsFilePath: src/graphql
extensions:
  amplify:
    version: 3

We had a version of the app deploying succesffully so I am going to go through all of the changes that occured after that point.

[ykethan]

@KuzonFyre you can find the amplify.yml on the Amplify console under build setting

[KuzonFyre]

@ykethan

version: 1
backend:
  phases:
    build:
      commands:
        - '# Execute Amplify CLI with the helper script'
        - amplifyPush --simple
frontend:
  phases:
    preBuild:
      commands:
        - npm ci --cache .npm --prefer-offline
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: .next
    files:
      - '**/*'
  cache:
    paths:
      - .next/cache/**/*
      - .npm/**/*

[ykethan]

Hey @KuzonFyre , thank you for the information. Would you be open for a quick chat. I am available on discord, my handle is ykethan. Wanted to dive deeper into the issue over a screenshare.