"Request is missing Authentication Token" after upgrade to 3.25.0

[andrmoel]

Message from maintainers:

Upgrade to v3.25.1

This issue is caused by recent regression in AWS SDK. Please increase visibility by adding thumbs up at aws/aws-sdk-js#4286 and aws/aws-sdk-js#4285

---

Are you certain it's a bug?

• Yes, it looks like a bug

Is the issue caused by a plugin?

• It is not a plugin issue

Are you using the latest v3 release?

• Yes, I'm using the latest v3 release

Is there an existing issue for this?

• I have searched existing issues, it hasn't been reported yet

Issue description

After I have upgraded from 2.24.1 to 3.25.0 I do get the following error while I deploy:

"Request is missing Authentication Token"

Service configuration (serverless.yml) content

N/A

Command name and used flags

serverless deploy --stage=staging

Command output

yarn run v1.22.19
$ cd microservices/core && serverless deploy --verbose

Deploying portal to stage dev (eu-central-1)

Packaging
Using configuration:
{
  "webpackConfig": "./../../webpack.config.js",
  "includeModules": false,
  "packager": "yarn",
  "packagerOptions": {},
  "keepOutputDirectory": false,
  "concurrency": 8
}
Removing /home/test/Projekte/test-project/.webpack
[Webpack] Building with Webpack
[Webpack] Packaging modules
Zip service: /home/test/Projekte/test-project/.webpack/service [324 ms]
[Webpack] Copying existing artifacts
Setting artifact for function 'info' to '.serverless/portal.zip'
Setting artifact for function 'version' to '.serverless/portal.zip'
Setting artifact for function 'onPreSignUp' to '.serverless/portal.zip'
Setting artifact for function 'onPostConfirmation' to '.serverless/portal.zip'
Setting artifact for function 'onCustomMessage' to '.serverless/portal.zip'
Remove /home/test/Projekte/test-project/.webpack
Generating custom CloudFormation resources
Removing /home/test/Projekte/test-project/.webpack done
Retrieving CloudFormation stack

✖ Stack portal-dev failed to deploy (32s)
Environment: linux, node 14.19.0, framework 3.25.0 (local), plugin 6.2.2, SDK 4.3.2
Credentials: Local, "default" profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Request is missing Authentication Token

1 deprecation found: run 'serverless doctor' for more details
error Command failed with exit code 1.

Environment information

Running "serverless" from node_modules
Framework Core: 3.25.0 (local) 3.1.1 (global)
Plugin: 6.2.2
SDK: 4.3.2

[sbmvirdi]

Hi even i am also facing the same issue while using serverless.
It was working fine few days back.

[modanwalrahul31]

Hi, even I am also facing the same issue while using serverless.
From Today.

[JunBeum-Cho]

I am also getting the same issue ughhhhh spent whole day for this

[modanwalrahul31]

I don't think this issue comes after the upgrade, even existing projects are still working with the upgraded.

[andrmoel]

I don't think this issue comes after the upgrade, even existing projects are still working with the upgraded.

I tested it with 2.24.1 -> works as expected
Only 2.25.0 is facing that issue

[luksfarris]

I tested with several versions of serverless, from 1.61 to 3.21, all have this error.
I don't think it's related to the version

[nikhiltechahead]

I'm getting the same error when deploying the code to AWS lambda from today only. I've a serverless version 3.25.0.

[sbmvirdi]

yes even i tried using older versions but nothing seems to work.
I have changed my whole pipelines so many times.

[AnishLushte07]

##11559

[nikhiltechahead]

Something is really broken here unexpectedly. Spent almost all day checking what I'm doing wrong.

[sathishkumar16]

#11560

[gskim]

i'm waiting....

[coder-pikachu]

We are also facing the same issue. The build was working until yesterday...

[daum1820]

Facing the same problem but I've also update "aws-sdk": "2.1262.0" which are also not working after downgrading the serveless againg to 3.24.1

[sbmvirdi]

but what i am thinking is how it was working until yesterday because 3.25.0 release was merged 7 days ago

[daum1820]

aws/aws-sdk-js#4286

[luksfarris]

Yeah, had to npm install --save-dev aws-sdk@2.1259.0. Thanks guys

[andrmoel]

Facing the same problem but I've also update "aws-sdk": "2.1262.0" which are also not working after downgrading the serveless againg to 3.24.1

@daum1820 This could be the issue. Serverless has a dependency to aws-sdk.

As I have written before. Version 3.24.1 is working for us. Please don't simply downgrade your package.json because it does not downgrade the aws-sdk dependency. You need to do a clean install!

[sbmvirdi]

yes serverless has bumped new version of aws js library and upgraded it. the serverless needs to downgrade the version back to old to make it working

[coder-pikachu]

Facing the same problem but I've also update "aws-sdk": "2.1262.0" which are also not working after downgrading the serveless againg to 3.24.1

@daum1820 This could be the issue. Serverless has a dependency to aws-sdk.

As I have written before. Version 3.24.1 is working for us. Please don't simply downgrade your package.json because it does not downgrade the aws-sdk dependency. You need to do a clean install!

Clean install meaning?

[andrmoel]

Clean install meaning?

yarn remove serverless
yarn add serverless@version

Be aware: If you have aws-sdk as dependency in your package.json, you also need to remove this one otherwise it won't get downgraded.

[sbmvirdi]

@andrmoel i am using serverless deploy in pipelines so it should not fetch the latest version for aws sdk js right ?

[AnishLushte07]

Deleting node modules worked for me. After serverless version 3.17

[andrmoel]

@sbmvirdi depends on your package.json and yarn.lock (Sorry, we only use yarn. I think for npm it is package-lock.json).

Your yarn.lock contains the state of truth and should be checked in version control. Just check if your yarn.lock contains the correct version of serverless and aws-sdk. Then you should be green.

[pankaj-torinit]

I am using serverless for laravel deployment using bref package. Getting the same error. Any solution?

[ziamler]

I'm also facing the same issue since morning. I have tried lots of solutions but don’t work. at last, I have removed node_modules and use the node 16 version. it's working fine for me.

[sbmvirdi]

@andrmoel but i am not pushing my lock file to github i am just pushing my package json file to repo and doing npm i in the ci. i tried using specific version for serverless in package json (3.23.0) but it was getting failed.

[andrmoel]

@sbmvirdi then you should also push your lock. It is best practice because you always want to have the same build on your machine that you have in production. In your case, the fix and minor versions differ. This can cause side effects.

But we are getting off-topic ;)

[brainframe-me]

Same issue here, with v2 of serverless. Even tried upgrading from 2.72.2 to 2.72.3 but does not change anything.
So a build that worked perfectly last week is now failing with "Request is missing Authentication Token".

Your Environment Information ---------------------------
Operating System: linux
Node Version: 16.13.1
Framework Version: 2.72.3
Plugin Version: 5.5.4
SDK Version: 4.3.2
Components Version: 3.18.2

[MehranJanfeshan]

@medikoo why did you close it? There is no clear solution yet; even downgrading doesn’t work.

[brainframe-me]

I finally got it to work from my side. Make sure to understand if you are using serverless npm installed globally or as a local project in your packages. When your serverless is installed through npm install -global, you can also fix the correct aws-sdk as following:

npm i -g serverless@2.72.2 aws-sdk@2.1259.0 (tested and finally working)
or
npm i -g serverless@3.25.0 aws-sdk@2.1259.0 (not tested, but assuming this also fixes the issue here)

[sbmvirdi]

Yes for me i just updated my package json to have these versions and instead of installing serverless globally used npx command to deploy which uses package from node_modules.
npx serverless deploy --stage someStage

I finally got it to work from my side. Make sure to understand if you are using serverless npm installed globally or as a local project in your packages. When your serverless is installed through npm install -global, you can also fix the correct aws-sdk as following:

npm i -g serverless@2.72.2 aws-sdk@2.1259.0 (tested and finally working) or npm i -g serverless@3.25.0 aws-sdk@2.1259.0 (not tested, but assuming this also fixes the issue here)

[MehranJanfeshan]

I finally got it to work from my side. Make sure to understand if you are using serverless npm installed globally or as a local project in your packages. When your serverless is installed through npm install -global, you can also fix the correct aws-sdk as following:

npm i -g serverless@2.72.2 aws-sdk@2.1259.0 (tested and finally working) or npm i -g serverless@3.25.0 aws-sdk@2.1259.0 (not tested, but assuming this also fixes the issue here)

this worked very well, thanks

[medikoo]

@medikoo why did you close it? There is no clear solution yet; even downgrading doesn’t work.

@MehranJanfeshan I didn't close it (it's other duplicate issue that was closed)

[RohitSaini15]

facing the same issue while deploying the aws lambda after using the serverless-python-requirements plugin. Without it, it 's working fine

[trivikr]

This is maintainer of AWS SDK for JavaScript, and I'm visiting from aws/aws-sdk-js#4286

If you can provide a minimal repro there, it would be helpful for us to triage, debug and provide a fix.

[medikoo]

Just released v3.25.1 version of the Framework, which fixes AWS SDK to the last working version. Please upgrade and the issue should be fixed

[trivikr]

Commits between v3.25.0...v3.25.1

This was fixed by pinning aws-sdk to 2.1259.0 in cb81085 right?

[trivikr]

Do you suspect the issue was introduced in aws-sdk 2.1262.0 or 2.1260.0?
Why do the recommendations pin aws-sdk version to 2.1259.0?

Can someone pin aws-sdk to 2.1261.0, and check if it works for you?

EDIT: Never mind. The versions 2.1260.0 and 2.1261.0 did not have custom changes aws/aws-sdk-js#4286 (comment)

If you can provide minimal repro in aws/aws-sdk-js#4286, so that we can compare it against different versions of aws-sdk.

[andrmoel]

@trivikr I have setup a simple project to reproduce the issue

https://github.com/andrmoel/serverless-aws-bug

Edit: Checkout the "working" branch to see the solution provided here #11561 (comment)
https://github.com/andrmoel/serverless-aws-bug/tree/working

[trivikr]

I have setup a simple project to reproduce the issue

andrmoel/serverless-aws-bug

Thank you @andrmoel

Do you have another repro without serverless?
i.e. a repro which just has dependency on aws-sdk in package.json and nothing else.

[andrmoel]

@trivikr sorry can't provide this.
But I have created also a working example using the old version numbers.
Just checkout the branch "working"

https://github.com/andrmoel/serverless-aws-bug/tree/working

[andrmoel]

@trivikr sorry. May there is a misunderstanding here.
The aws-sdk is a dependency of serverless. So it is part of the project

[trivikr]

The aws-sdk is a dependency of serverless. So it is part of the project

AWS SDK for JavaScript is maintained by AWS, but serverless is community maintained.

Looking at the current repro and fixes, the issue is likely with aws-sdk introduced in v2.1262.0.
We just need a repro to triage, debug and fix the issue in aws/aws-sdk-js#4286

[trivikr]

We have a fix available in aws/aws-sdk-js#4288 using the repro which @andrmoel provided.
We'll mostly merge and aim to release it with v2.1263.0 within the next 24 hours.

It will be helpful if any maintainer of serverless can provide a minimal repro just with aws-sdk so that we can understand why this issue is happening and post relevant comments in the SDK codebase.
We suspect it's either caused by setting AWS global configuration, or multiple client initializations at once, in the serverless code.

We are formalizing our plans to enter AWS SDK for JavaScript v2 into maintenance mode in 2023. For more details, check our README in Version 2.x Support.

[NGIamarino]

I finally got it to work from my side. Make sure to understand if you are using serverless npm installed globally or as a local project in your packages. When your serverless is installed through npm install -global, you can also fix the correct aws-sdk as following:

npm i -g serverless@2.72.2 aws-sdk@2.1259.0 (tested and finally working) or npm i -g serverless@3.25.0 aws-sdk@2.1259.0 (not tested, but assuming this also fixes the issue here)

This worked for me.
Thank you!

[trivikr]

We have a fix available in aws/aws-sdk-js#4288 using the repro which @andrmoel provided.

Can you merge bump to yarn modern in andrmoel/serverless-aws-bug#1?

I'll post a draft PR using yarn patch to verify our fix prior to release.

[trivikr]

Verified that fix from aws/aws-sdk-js#4288 fixes "Request is missing Authentication Token" issue.

Refer Pull request in https://github.com/trivikr/serverless-aws-bug/pull/1
We're aiming to release this fix in v2.1263.0 within the next 24 hours.

[medikoo]

@trivikr great thanks for fast action! 💯 🎉

[trivikr]

Release aws-sdk@v2.1263.0 is out, and I verified that "Request is missing Authentication Token" error is no longer thrown.

andrmoel/serverless-aws-bug#2

@medikoo Release v3.25.2 after reverting commit cb81085?

[han-fs]

I added serverless config and it works.

serverless config credentials --provider aws --key ... --secret ...

serverless version: 2.72.3, 3.25.1
aws sdk version: 2.x

[andrmoel]

@trivikr thank you for the quick response

[MehranJanfeshan]

I added serverless config and it works.

serverless config credentials --provider aws --key ... --secret ...

serverless version: 2.72.3, 3.25.1 aws sdk version: 2.x

@han-fs doesn’t work for pipelines that use assumed roles, and exposing keys and secrets in pipelines is not ideal.

[Scoppio]

AWS-SDK has released a new version which fix the issue.

[medikoo]

@medikoo Release v3.25.2 after reverting commit cb81085?

@trivikr yes, with the next version, we'll get back to the regular versioning of AWS-SDK

[BasilaryGroup]

Your fix works for us. We are now able to deploy everything that failed yesterday.

Thank you!