Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot find module 'ajv/dist/compile/codegen' #8727

Closed
lewgordon opened this issue Jan 7, 2021 · 30 comments
Closed

Cannot find module 'ajv/dist/compile/codegen' #8727

lewgordon opened this issue Jan 7, 2021 · 30 comments
Assignees
Labels
bug/external Bug on provider side

Comments

@lewgordon
Copy link
Contributor

Not sure if this is something that just can't be helped, but it seems that if a NodeJS project has a dependency on ajv^6 it'll cause the packaging step to fail. I believe this was caused by the bump to ajv^7. I know that many projects still depend on the earlier version, so it might break a lot of projects.

Steps to reproduce:

  1. Create a new project using sls init
  2. Select NodeJS project
  3. Install ajv^6 via npm i ajv@^6
  4. Execute sls package
serverless.yml
# Welcome to Serverless!
#
# This file is the main config file for your service.
# It's very minimal at this point and uses default values.
# You can always add more config options for more control.
# We've included some commented out config examples here.
# Just uncomment any of them to get that config option.
#
# For full config options, check the docs:
#    docs.serverless.com
#
# Happy Coding!

service: test
# app and org for use with dashboard.serverless.com
#app: your-app-name
#org: your-org-name

# You can pin your service to only deploy with a specific Serverless version
# Check out our docs for more details
frameworkVersion: '2'

provider:
  name: aws
  runtime: nodejs12.x

# you can overwrite defaults here
#  stage: dev
#  region: us-east-1

# you can add statements to the Lambda function's IAM Role here
#  iamRoleStatements:
#    - Effect: "Allow"
#      Action:
#        - "s3:ListBucket"
#      Resource: { "Fn::Join" : ["", ["arn:aws:s3:::", { "Ref" : "ServerlessDeploymentBucket" } ] ]  }
#    - Effect: "Allow"
#      Action:
#        - "s3:PutObject"
#      Resource:
#        Fn::Join:
#          - ""
#          - - "arn:aws:s3:::"
#            - "Ref" : "ServerlessDeploymentBucket"
#            - "/*"

# you can define service wide environment variables here
#  environment:
#    variable1: value1

# you can add packaging information here
#package:
#  include:
#    - include-me.js
#    - include-me-dir/**
#  exclude:
#    - exclude-me.js
#    - exclude-me-dir/**

functions:
  hello:
    handler: handler.hello
#    The following are a few example events you can configure
#    NOTE: Please make sure to change your handler code to work with those events
#    Check the event documentation for details
#    events:
#      - http:
#          path: users/create
#          method: get
#      - websocket: $connect
#      - s3: ${env:BUCKET}
#      - schedule: rate(10 minutes)
#      - sns: greeter-topic
#      - stream: arn:aws:dynamodb:region:XXXXXX:table/foo/stream/1970-01-01T00:00:00.000
#      - alexaSkill: amzn1.ask.skill.xx-xx-xx-xx
#      - alexaSmartHome: amzn1.ask.skill.xx-xx-xx-xx
#      - iot:
#          sql: "SELECT * FROM 'some_topic'"
#      - cloudwatchEvent:
#          event:
#            source:
#              - "aws.ec2"
#            detail-type:
#              - "EC2 Instance State-change Notification"
#            detail:
#              state:
#                - pending
#      - cloudwatchLog: '/aws/lambda/hello'
#      - cognitoUserPool:
#          pool: MyUserPool
#          trigger: PreSignUp
#      - alb:
#          listenerArn: arn:aws:elasticloadbalancing:us-east-1:XXXXXX:listener/app/my-load-balancer/50dc6c495c0c9188/
#          priority: 1
#          conditions:
#            host: example.com
#            path: /hello

#    Define function environment variables here
#    environment:
#      variable2: value2

# you can add CloudFormation resource templates here
#resources:
#  Resources:
#    NewResource:
#      Type: AWS::S3::Bucket
#      Properties:
#        BucketName: my-new-bucket
#  Outputs:
#     NewOutput:
#       Description: "Description for the output"
#       Value: "Some output value"
sls package output
Serverless: Running "serverless" installed locally (in service node_modules)
 
  Error --------------------------------------------------
 
  Error: Cannot find module 'ajv/dist/compile/codegen'
  Require stack:
  - /tmp/test/node_modules/ajv-keywords/dist/definitions/typeof.js
  - /tmp/test/node_modules/ajv-keywords/dist/keywords/typeof.js
  - /tmp/test/node_modules/ajv-keywords/dist/keywords/index.js
  - /tmp/test/node_modules/ajv-keywords/dist/index.js
  - /tmp/test/node_modules/serverless/lib/classes/ConfigSchemaHandler/index.js
  - /tmp/test/node_modules/serverless/lib/Serverless.js
  - /usr/local/lib/node_modules/serverless/lib/Serverless.js
  - /usr/local/lib/node_modules/serverless/scripts/serverless.js
  - /usr/local/lib/node_modules/serverless/bin/serverless.js
      at Function.Module._resolveFilename (internal/modules/cjs/loader.js:957:15)
      at Function.Module._load (internal/modules/cjs/loader.js:840:27)
      at Module.require (internal/modules/cjs/loader.js:1019:19)
      at require (internal/modules/cjs/helpers.js:77:18)
      at Object.<anonymous> (/tmp/test/node_modules/ajv-keywords/src/definitions/typeof.ts:2:1)
      at Module._compile (internal/modules/cjs/loader.js:1133:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
      at Module.load (internal/modules/cjs/loader.js:977:32)
      at Function.Module._load (internal/modules/cjs/loader.js:877:14)
      at Module.require (internal/modules/cjs/loader.js:1019:19)
      at require (internal/modules/cjs/helpers.js:77:18)
      at Object.<anonymous> (/tmp/test/node_modules/ajv-keywords/src/keywords/typeof.ts:2:1)
      at Module._compile (internal/modules/cjs/loader.js:1133:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
      at Module.load (internal/modules/cjs/loader.js:977:32)
      at Function.Module._load (internal/modules/cjs/loader.js:877:14)
      at Module.require (internal/modules/cjs/loader.js:1019:19)
      at require (internal/modules/cjs/helpers.js:77:18)
      at Object.<anonymous> (/tmp/test/node_modules/ajv-keywords/src/keywords/index.ts:2:1)
      at Module._compile (internal/modules/cjs/loader.js:1133:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
      at Module.load (internal/modules/cjs/loader.js:977:32)
      at Function.Module._load (internal/modules/cjs/loader.js:877:14)
      at Module.require (internal/modules/cjs/loader.js:1019:19)
      at require (internal/modules/cjs/helpers.js:77:18)
      at Object.<anonymous> (/tmp/test/node_modules/ajv-keywords/src/index.ts:3:1)
      at Module._compile (internal/modules/cjs/loader.js:1133:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
      at Module.load (internal/modules/cjs/loader.js:977:32)
      at Function.Module._load (internal/modules/cjs/loader.js:877:14)
      at Module.require (internal/modules/cjs/loader.js:1019:19)
      at require (internal/modules/cjs/helpers.js:77:18)
      at ConfigSchemaHandler.validateConfig (/tmp/test/node_modules/serverless/lib/classes/ConfigSchemaHandler/index.js:102:5)
      at Service.validate (/tmp/test/node_modules/serverless/lib/classes/Service.js:222:41)
      at /tmp/test/node_modules/serverless/lib/Serverless.js:200:49
      at tryCatcher (/tmp/test/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/tmp/test/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/tmp/test/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/tmp/test/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/tmp/test/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/tmp/test/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/tmp/test/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/tmp/test/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/tmp/test/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:456:21)
      at process.topLevelDomainCallback (domain.js:137:15)
 
     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              12.16.3
     Framework Version:         2.11.1 (local)
     Plugin Version:            4.1.2
     SDK Version:               2.3.2
     Components Version:        3.3.0

Installed version

Serverless: Running "serverless" installed locally (in service node_modules)
Framework Core: 2.18.0 (local)
Plugin: 4.4.2
SDK: 2.3.2
Components: 3.4.6
@srdone
Copy link

srdone commented Jan 7, 2021

I'm seeing this issue - we've had to specify 2.17.0 in order for our deploys to work.

@srdone
Copy link

srdone commented Jan 7, 2021

I should add - we don't have a direct dependency on ajv, but we're using other packages that likely have dependencies on ajv.

@medikoo
Copy link
Contributor

medikoo commented Jan 7, 2021

@lewgordon Thanks for report, Still this looks to me as issue in your setup, at least provided information shows that:

You use globally installed serverless v2.18.0 , which in context of /tmp/test fallbacks to it's locally installed serverless v2.11.1.

And it appears that local serverless installation in tmp/test/node_modules v2.11.1 is broken:

Note that v2.11.1 depends on ajv-keywords@^3.5.2 ->

"ajv-keywords": "^3.5.2",

and requires it in lib/class/ConfigSchemaHandler/index.js:
require('ajv-keywords')(ajv, 'regexp');

Still what it receives in your case is ajv-keywords v4, we can tell that by resolved require path ajv-keywords/dist/index.js exposed in your stack trace which is specific to v4 -> https://npmview.now.sh/ajv-keywords@4 while v3 looks as https://npmview.now.sh/ajv-keywords@3.

How did you install serverless locally? Is this referenced in package.json of /tmp/test?, because if it's not and then e.g. ajv-keywords@4 was installed in context of /tmp/test it possibly could break the unlisted serverless installation

@srdone
Copy link

srdone commented Jan 7, 2021

In our case, serverless is installed as a dev dependency of our project. We are not using ajv directly (though one or more of our dependencies are bringing it in on install).

@claylevering
Copy link

As a note, had the same issues starting in 2.18.0 - Our team uses the serverless library in our dev dependencies and of course we have some library dependencies on AJV but at least in the app we're working with now, the list of dependencies is fairly minimal:

  ...
  "devDependencies": {
    "jest": "^26.6.3",
    "serverless": "^2.17.0",
    "serverless-offline": "^6.8.0",
    "serverless-plugin-git-variables": "^5.1.0"
  },
  "dependencies": {
    "INTERNAL LIBRARY": "^1.0.0-alpha.6"
  }

2.17.0 works, 2.18.0 fails with the AJV errors listed above.

@claylevering
Copy link

I should note we never leverage a global serverless installation. We rely on nvm and per-repo version locking with either npx sls or npm scripts to call any serverless executions (e.g. npm run deploy:production ==> serverless deploy --stage production)

@medikoo
Copy link
Contributor

medikoo commented Jan 8, 2021

It'll be great to find out how this broken state happens.

At least in @lewgordon it looks as broken installation issue, which is not an issue in serverless library itself (as it doesn't install by itself, but via tools as npm or yarn)

Even if service on its own depends on ajv or ajv-keywords (or has dependencies that depend on it) at different versions than ones on which serverless depends. Then installer should ensure versions which serverless expects in its servelress/node_modules folder (that's how npm and yarn based installation works)

@FergusMcGlynn
Copy link

FergusMcGlynn commented Jan 8, 2021

I am running into this issue as well. I actually have two projects that use serverless as a dev dependency - I upgraded both to use version 2.18.0 yesterday in order to fix the axios security vulnerability - for one project there are no problems running serverless deploy, but for the other I get the same error as the original poster: Cannot find module 'ajv/dist/compile/codegen'.

Running npm ls ajv on both projects, I see that on the project where serverless deploy fails, I have an UNMET PEER DEPENDENCY:

$ npm ls ajv

+-- jest@26.6.3
| `-- @jest/core@26.6.3
|   `-- jest-config@26.6.3
|     `-- jest-environment-jsdom@26.6.2
|       `-- jsdom@16.4.0
|         `-- request@2.88.2
|           `-- har-validator@5.1.5
|             `-- ajv@6.12.6  deduped
+-- serverless@2.18.0
| +-- UNMET PEER DEPENDENCY ajv@7.0.3
| `-- ajv-formats@1.5.1
|   `-- ajv@7.0.3
`-- xo@0.33.1
  `-- eslint@7.17.0
    +-- @eslint/eslintrc@0.2.2
    | `-- ajv@6.12.6  deduped
    +-- ajv@6.12.6
    `-- table@6.0.7
      `-- ajv@7.0.3

But on the other project, where serverless deploy works fine, no peer dependency issues, even though the conditions regarding ajv look very similar:

$ npm ls ajv                                                          

+-- @percy/script@1.1.0                                               
| `-- @percy/agent@0.28.6                                             
|   `-- percy-client@3.8.0                                            
|     `-- request@2.88.2                                              
|       `-- har-validator@5.1.5                                       
|         `-- ajv@6.12.6  deduped                                     
+-- @university-of-york/esg-lib-pattern-library-react-components@6.1.4
| `-- babel-plugin-react-css-modules@5.2.6                            
|   `-- ajv@6.12.6                                                    
+-- next@10.0.5                                                       
| +-- sass-loader@10.0.5                                              
| | `-- schema-utils@3.0.0                                            
| |   `-- ajv@6.12.6  deduped                                         
| +-- schema-utils@2.7.1                                              
| | `-- ajv@6.12.6  deduped                                           
| `-- webpack@4.44.1                                                  
|   +-- ajv@6.12.6  deduped                                           
|   +-- schema-utils@1.0.0                                            
|   | `-- ajv@6.12.6  deduped                                         
|   `-- terser-webpack-plugin@1.4.5                                   
|     `-- schema-utils@1.0.0                                          
|       `-- ajv@6.12.6  deduped                                       
+-- serverless@2.18.0                                                 
| +-- ajv@7.0.3                                                       
| `-- ajv-formats@1.5.1                                               
|   `-- ajv@7.0.3                                                     
`-- xo@0.33.1                                                         
  `-- eslint@7.17.0                                                   
    +-- @eslint/eslintrc@0.2.2                                        
    | `-- ajv@6.12.6  deduped                                         
    +-- ajv@6.12.6  deduped                                           
    `-- table@6.0.7                                                   
      `-- ajv@7.0.3        

I don't understand why there would be an unmet peer dependency in one project and not the other. Neither explicitly includes ajv as a dependency. Also, it looks to me like serverless has ajv as an explicit dependency, so I'm confused by the peer dependency issue here.

@lewgordon
Copy link
Contributor Author

lewgordon commented Jan 8, 2021

Sorry about that @medikoo ! I agree there was something wrong in my setup. I reproduced the issue I was seeing again, though. I believe there might have been a typo in my original post. Here are the files for the most recent reproduction. Let me know if you need additional information or find anything I've missed.

This time I already ran it in the context of a container to isolate the filesystem. I used node with tag 12.

serverless.yml
# Welcome to Serverless!
#
# This file is the main config file for your service.
# It's very minimal at this point and uses default values.
# You can always add more config options for more control.
# We've included some commented out config examples here.
# Just uncomment any of them to get that config option.
#
# For full config options, check the docs:
#    docs.serverless.com
#
# Happy Coding!

service: test
# app and org for use with dashboard.serverless.com
#app: your-app-name
#org: your-org-name

# You can pin your service to only deploy with a specific Serverless version
# Check out our docs for more details
frameworkVersion: '2'

provider:
  name: aws
  runtime: nodejs12.x

# you can overwrite defaults here
#  stage: dev
#  region: us-east-1

# you can add statements to the Lambda function's IAM Role here
#  iamRoleStatements:
#    - Effect: "Allow"
#      Action:
#        - "s3:ListBucket"
#      Resource: { "Fn::Join" : ["", ["arn:aws:s3:::", { "Ref" : "ServerlessDeploymentBucket" } ] ]  }
#    - Effect: "Allow"
#      Action:
#        - "s3:PutObject"
#      Resource:
#        Fn::Join:
#          - ""
#          - - "arn:aws:s3:::"
#            - "Ref" : "ServerlessDeploymentBucket"
#            - "/*"

# you can define service wide environment variables here
#  environment:
#    variable1: value1

# you can add packaging information here
#package:
#  include:
#    - include-me.js
#    - include-me-dir/**
#  exclude:
#    - exclude-me.js
#    - exclude-me-dir/**

functions:
  hello:
    handler: handler.hello
#    The following are a few example events you can configure
#    NOTE: Please make sure to change your handler code to work with those events
#    Check the event documentation for details
#    events:
#      - http:
#          path: users/create
#          method: get
#      - websocket: 
#      - s3: 
#      - schedule: rate(10 minutes)
#      - sns: greeter-topic
#      - stream: arn:aws:dynamodb:region:XXXXXX:table/foo/stream/1970-01-01T00:00:00.000
#      - alexaSkill: amzn1.ask.skill.xx-xx-xx-xx
#      - alexaSmartHome: amzn1.ask.skill.xx-xx-xx-xx
#      - iot:
#          sql: "SELECT * FROM 'some_topic'"
#      - cloudwatchEvent:
#          event:
#            source:
#              - "aws.ec2"
#            detail-type:
#              - "EC2 Instance State-change Notification"
#            detail:
#              state:
#                - pending
#      - cloudwatchLog: '/aws/lambda/hello'
#      - cognitoUserPool:
#          pool: MyUserPool
#          trigger: PreSignUp
#      - alb:
#          listenerArn: arn:aws:elasticloadbalancing:us-east-1:XXXXXX:listener/app/my-load-balancer/50dc6c495c0c9188/
#          priority: 1
#          conditions:
#            host: example.com
#            path: /hello

#    Define function environment variables here
#    environment:
#      variable2: value2

# you can add CloudFormation resource templates here
#resources:
#  Resources:
#    NewResource:
#      Type: AWS::S3::Bucket
#      Properties:
#        BucketName: my-new-bucket
#  Outputs:
#     NewOutput:
#       Description: "Description for the output"
#       Value: "Some output value"
sls package output
 Error --------------------------------------------------

 Error: Cannot find module 'ajv/dist/compile/codegen'
 Require stack:
 - /git/node_modules/ajv-keywords/dist/definitions/typeof.js
 - /git/node_modules/ajv-keywords/dist/keywords/typeof.js
 - /git/node_modules/ajv-keywords/dist/keywords/index.js
 - /git/node_modules/ajv-keywords/dist/index.js
 - /git/node_modules/serverless/lib/classes/ConfigSchemaHandler/index.js
 - /git/node_modules/serverless/lib/Serverless.js
 - /git/node_modules/serverless/scripts/serverless.js
 - /git/node_modules/serverless/bin/serverless.js
     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
     at Function.Module._load (internal/modules/cjs/loader.js:667:27)
     at Module.require (internal/modules/cjs/loader.js:887:19)
     at require (internal/modules/cjs/helpers.js:74:18)
     at Object.<anonymous> (/git/node_modules/ajv-keywords/src/definitions/typeof.ts:2:1)
     at Module._compile (internal/modules/cjs/loader.js:999:30)
     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
     at Module.load (internal/modules/cjs/loader.js:863:32)
     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
     at Module.require (internal/modules/cjs/loader.js:887:19)
     at require (internal/modules/cjs/helpers.js:74:18)
     at Object.<anonymous> (/git/node_modules/ajv-keywords/src/keywords/typeof.ts:2:1)
     at Module._compile (internal/modules/cjs/loader.js:999:30)
     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
     at Module.load (internal/modules/cjs/loader.js:863:32)
     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
     at Module.require (internal/modules/cjs/loader.js:887:19)
     at require (internal/modules/cjs/helpers.js:74:18)
     at Object.<anonymous> (/git/node_modules/ajv-keywords/src/keywords/index.ts:2:1)
     at Module._compile (internal/modules/cjs/loader.js:999:30)
     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
     at Module.load (internal/modules/cjs/loader.js:863:32)
     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
     at Module.require (internal/modules/cjs/loader.js:887:19)
     at require (internal/modules/cjs/helpers.js:74:18)
     at Object.<anonymous> (/git/node_modules/ajv-keywords/src/index.ts:3:1)
     at Module._compile (internal/modules/cjs/loader.js:999:30)
     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
     at Module.load (internal/modules/cjs/loader.js:863:32)
     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
     at Module.require (internal/modules/cjs/loader.js:887:19)
     at require (internal/modules/cjs/helpers.js:74:18)
     at ConfigSchemaHandler.validateConfig (/git/node_modules/serverless/lib/classes/ConfigSchemaHandler/index.js:102:5)
     at Service.validate (/git/node_modules/serverless/lib/classes/Service.js:222:41)
     at /git/node_modules/serverless/lib/Serverless.js:200:49
     at tryCatcher (/git/node_modules/bluebird/js/release/util.js:16:23)
     at Promise._settlePromiseFromHandler (/git/node_modules/bluebird/js/release/promise.js:547:31)
     at Promise._settlePromise (/git/node_modules/bluebird/js/release/promise.js:604:18)
     at Promise._settlePromise0 (/git/node_modules/bluebird/js/release/promise.js:649:10)
     at Promise._settlePromises (/git/node_modules/bluebird/js/release/promise.js:729:18)
     at _drainQueueStep (/git/node_modules/bluebird/js/release/async.js:93:12)
     at _drainQueue (/git/node_modules/bluebird/js/release/async.js:86:9)
     at Async._drainQueues (/git/node_modules/bluebird/js/release/async.js:102:5)
     at Immediate.Async.drainQueues [as _onImmediate] (/git/node_modules/bluebird/js/release/async.js:15:14)
     at processImmediate (internal/timers.js:461:21)

    For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

 Get Support --------------------------------------------
    Docs:          docs.serverless.com
    Bugs:          github.com/serverless/serverless/issues
    Issues:        forum.serverless.com

 Your Environment Information ---------------------------
    Operating System:          linux
    Node Version:              12.20.1
    Framework Version:         2.18.0 (local)
    Plugin Version:            4.4.2
    SDK Version:               2.3.2
    Components Version:        3.4.6

Installed version

Framework Core: 2.18.0 (local)
Plugin: 4.4.2
SDK: 2.3.2
Components: 3.4.6
history output
    1  ls
    2  mkdir git
    3  cd git
    4  ls
    5  npm i serverless
    6  ls
    7  which serverlesss
    8  which serverless
    9  ls
   10  export PATH=/git/node_modules/.bin:$PATH
   11  sls
   12  ls
   13  vi
   14  cat <<EOF >> foo.sh
   15  ls
   16  cat foo.sh 
   17  rm foo.sh 
   18  cat <<EOF >> serverless.yml
   19  cat serverless.yml 
   20  sls package
   21  ls
   22  npm i ajv@^6
   23  sls package
   24  cat serverless.yml 
   25  sls --version

@lewgordon
Copy link
Contributor Author

Here's another example of probably a more realistic scenario, where there is a dependency on ajv@^6. Again using a node:12 container.

package.json
{
  "name": "test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "ajv": "^6.12.6"
  },
  "devDependencies": {
    "serverless": "^2.18.0"
  }
}
serverless.yml
# Welcome to Serverless!
#
# This file is the main config file for your service.
# It's very minimal at this point and uses default values.
# You can always add more config options for more control.
# We've included some commented out config examples here.
# Just uncomment any of them to get that config option.
#
# For full config options, check the docs:
#    docs.serverless.com
#
# Happy Coding!

service: test
# app and org for use with dashboard.serverless.com
#app: your-app-name
#org: your-org-name

# You can pin your service to only deploy with a specific Serverless version
# Check out our docs for more details
frameworkVersion: '2'

provider:
  name: aws
  runtime: nodejs12.x

# you can overwrite defaults here
#  stage: dev
#  region: us-east-1

# you can add statements to the Lambda function's IAM Role here
#  iamRoleStatements:
#    - Effect: "Allow"
#      Action:
#        - "s3:ListBucket"
#      Resource: { "Fn::Join" : ["", ["arn:aws:s3:::", { "Ref" : "ServerlessDeploymentBucket" } ] ]  }
#    - Effect: "Allow"
#      Action:
#        - "s3:PutObject"
#      Resource:
#        Fn::Join:
#          - ""
#          - - "arn:aws:s3:::"
#            - "Ref" : "ServerlessDeploymentBucket"
#            - "/*"

# you can define service wide environment variables here
#  environment:
#    variable1: value1

# you can add packaging information here
#package:
#  include:
#    - include-me.js
#    - include-me-dir/**
#  exclude:
#    - exclude-me.js
#    - exclude-me-dir/**

functions:
  hello:
    handler: handler.hello
#    The following are a few example events you can configure
#    NOTE: Please make sure to change your handler code to work with those events
#    Check the event documentation for details
#    events:
#      - http:
#          path: users/create
#          method: get
#      - websocket: 
#      - s3: 
#      - schedule: rate(10 minutes)
#      - sns: greeter-topic
#      - stream: arn:aws:dynamodb:region:XXXXXX:table/foo/stream/1970-01-01T00:00:00.000
#      - alexaSkill: amzn1.ask.skill.xx-xx-xx-xx
#      - alexaSmartHome: amzn1.ask.skill.xx-xx-xx-xx
#      - iot:
#          sql: "SELECT * FROM 'some_topic'"
#      - cloudwatchEvent:
#          event:
#            source:
#              - "aws.ec2"
#            detail-type:
#              - "EC2 Instance State-change Notification"
#            detail:
#              state:
#                - pending
#      - cloudwatchLog: '/aws/lambda/hello'
#      - cognitoUserPool:
#          pool: MyUserPool
#          trigger: PreSignUp
#      - alb:
#          listenerArn: arn:aws:elasticloadbalancing:us-east-1:XXXXXX:listener/app/my-load-balancer/50dc6c495c0c9188/
#          priority: 1
#          conditions:
#            host: example.com
#            path: /hello

#    Define function environment variables here
#    environment:
#      variable2: value2

# you can add CloudFormation resource templates here
#resources:
#  Resources:
#    NewResource:
#      Type: AWS::S3::Bucket
#      Properties:
#        BucketName: my-new-bucket
#  Outputs:
#     NewOutput:
#       Description: "Description for the output"
#       Value: "Some output value"
sls package output
  Error --------------------------------------------------
 
  Error: Cannot find module 'ajv/dist/compile/codegen'
  Require stack:
  - /git/node_modules/ajv-keywords/dist/definitions/typeof.js
  - /git/node_modules/ajv-keywords/dist/keywords/typeof.js
  - /git/node_modules/ajv-keywords/dist/keywords/index.js
  - /git/node_modules/ajv-keywords/dist/index.js
  - /git/node_modules/serverless/lib/classes/ConfigSchemaHandler/index.js
  - /git/node_modules/serverless/lib/Serverless.js
  - /git/node_modules/serverless/scripts/serverless.js
  - /git/node_modules/serverless/bin/serverless.js
      at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
      at Function.Module._load (internal/modules/cjs/loader.js:667:27)
      at Module.require (internal/modules/cjs/loader.js:887:19)
      at require (internal/modules/cjs/helpers.js:74:18)
      at Object.<anonymous> (/git/node_modules/ajv-keywords/src/definitions/typeof.ts:2:1)
      at Module._compile (internal/modules/cjs/loader.js:999:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
      at Module.load (internal/modules/cjs/loader.js:863:32)
      at Function.Module._load (internal/modules/cjs/loader.js:708:14)
      at Module.require (internal/modules/cjs/loader.js:887:19)
      at require (internal/modules/cjs/helpers.js:74:18)
      at Object.<anonymous> (/git/node_modules/ajv-keywords/src/keywords/typeof.ts:2:1)
      at Module._compile (internal/modules/cjs/loader.js:999:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
      at Module.load (internal/modules/cjs/loader.js:863:32)
      at Function.Module._load (internal/modules/cjs/loader.js:708:14)
      at Module.require (internal/modules/cjs/loader.js:887:19)
      at require (internal/modules/cjs/helpers.js:74:18)
      at Object.<anonymous> (/git/node_modules/ajv-keywords/src/keywords/index.ts:2:1)
      at Module._compile (internal/modules/cjs/loader.js:999:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
      at Module.load (internal/modules/cjs/loader.js:863:32)
      at Function.Module._load (internal/modules/cjs/loader.js:708:14)
      at Module.require (internal/modules/cjs/loader.js:887:19)
      at require (internal/modules/cjs/helpers.js:74:18)
      at Object.<anonymous> (/git/node_modules/ajv-keywords/src/index.ts:3:1)
      at Module._compile (internal/modules/cjs/loader.js:999:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
      at Module.load (internal/modules/cjs/loader.js:863:32)
      at Function.Module._load (internal/modules/cjs/loader.js:708:14)
      at Module.require (internal/modules/cjs/loader.js:887:19)
      at require (internal/modules/cjs/helpers.js:74:18)
      at ConfigSchemaHandler.validateConfig (/git/node_modules/serverless/lib/classes/ConfigSchemaHandler/index.js:102:5)
      at Service.validate (/git/node_modules/serverless/lib/classes/Service.js:222:41)
      at /git/node_modules/serverless/lib/Serverless.js:200:49
      at tryCatcher (/git/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/git/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/git/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/git/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/git/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/git/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/git/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/git/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/git/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:461:21)
 
     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              12.20.1
     Framework Version:         2.18.0 (local)
     Plugin Version:            4.4.2
     SDK Version:               2.3.2
     Components Version:        3.4.6
history output
root@a78b7ee06b20:/git# history
    1  mkdir git
    2  cd git
    3  cat <<EOF >> package.json
...
EOF

    4  npm i
    5  cat <<EOF >> serverless.yml
...
EOF

    6  sls package
    7  export PATH=/git/node_modules/.bin:$PATH
    8  sls package

@medikoo
Copy link
Contributor

medikoo commented Jan 8, 2021

@lewgordon From what I see it's an installation error. How do you install the project ? Is it simply npm install ? If so, what's the version of npm?

In your output we can see that ajv-keyword@4 which has a peer dependency set to ajv@^7.0.0 is installed top-level aside of ajv at v6.

It looks as a clear bug of installer, as with your configuration, ajv-keyword should be installed in serverless/node_modules aside of its ajv@7

@lewgordon
Copy link
Contributor Author

Hey @medikoo, it's just a npm install. npm version is 6.14.10.

@medikoo medikoo added bug/external Bug on provider side and removed question labels Jan 8, 2021
@medikoo
Copy link
Contributor

medikoo commented Jan 8, 2021

@lewgordon I've just tested it locally, and indeed it happens with npm v6, but not with npm v7 which installs project as expected

It's a clear bug in npm v6 (it's probably unlikely they'll fix it in that branch)

@FergusMcGlynn
Copy link

FergusMcGlynn commented Jan 8, 2021

npm v7 auto installs peer dependencies, whereas npm v6 does not. It might be this feature that makes it work under npm v7, rather than a bug fix for some behaviour in npm v6?

@medikoo
Copy link
Contributor

medikoo commented Jan 8, 2021

Issue is not that peer dependency is not installed, but it's because dependency of serverless is installed in wrong location against its peer dependency which is also marked as dependency of serverless

@medikoo
Copy link
Contributor

medikoo commented Jan 8, 2021

But it might be that with npm v6 support for peer dependencies was just informative, and in light of that, it's no wonder it cannot handle such setup properly.

@lewgordon workaround on your side could be to define ajv-keywords@3 dependency aside ajv@6. It'll ensure that ajv-keywords@4 will get installed in serverless/node_modules

@srdone
Copy link

srdone commented Jan 8, 2021

Seeing as npm v6 is the current LTS version, and npm v7 is the development release, I believe serverless should not break under npm v6 just because the project where it is used happens to have ajv 6 as a dependency. Especially considering that the previous minor release does not break in this scenario.

That's assuming I understand the problem correctly.

Is there not a way the change to serverless that broke this scenario could be modified so that it does not break in these circumstances?

@medikoo
Copy link
Contributor

medikoo commented Jan 8, 2021

@srdone this problem is not specific to latest changes.

Same would happen if you'd try to install ajv at v5 (or at v7) with previous version of serverless

The only "complete fix" could be to not use in serverless packages that declare peer dependencies at all, and imo that's not really viable.

I believe right approach is to acknowledge the specificity of a setup and find a way to install it properly (either use package manager that handles peer dependencies, or workaround it as I proposed above)

@jakoritarleite
Copy link

jakoritarleite commented Jan 8, 2021

I was facing the same issue with, I temporarily solved it by installing ajv@7.0.3 and now it's added as dependency on my package.json:

  ...
  "devDependencies": {
    "serverless-step-functions": "^2.27.1"
  },
  "dependencies": {
    "ajv": "^7.0.3",
    "serverless": "^2.18.0",
    "serverless-pseudo-parameters": "^2.5.0",
    "serverless-python-requirements": "^5.1.0"
  }

It's just a "workaround" for those who need to deploy their applications, and problably will be fixed asap. By the way I'm running the serverless v2.18.0, npm v6.14.10 and node v10.19.0.

@srdone
Copy link

srdone commented Jan 8, 2021

@medikoo - we are currently getting around this issue by pinning our dev dependency on serverless at v.2.17.0, so the problem does not exist there.

Note that we do not have any direct dependency on ajv - but rather another package we are using has it as a dependency.

@FergusMcGlynn
Copy link

I think there is something funny about version 2.18.0 of serverless. 3 separate users have posted here that they have run into the error Cannot find module 'ajv/dist/compile/codegen' since upgrading to 2.18.0, and they didn't have any errors with 2.17.0. Also, in the example I described in an earlier post, upgrading to 2.18.0 caused the error in one project but not in the other. If this were a straight up bug in npm v6, wouldn't both of them fail?

My projects don't have a direct dependency on ajv either.

@medikoo
Copy link
Contributor

medikoo commented Jan 8, 2021

My projects don't have a direct dependency on ajv either.

I believe some other dependency in your setup relies on ajv at other version than serverless.

Note that this issue will happen if two dependencies rely on different version of ajv (and aside in any of the dependencies there's a dependency that marks ajv as peer).

We may downgrade back ajv to v6, but if your other dependency upgrades ajv internally to v7, again you will have same issue

@FergusMcGlynn
Copy link

In the project where upgrading to version 2.18.0 worked fine, there were two dependencies relying on different versions of ajv:

$ npm ls ajv                                                          

+-- @percy/script@1.1.0                                               
| `-- @percy/agent@0.28.6                                             
|   `-- percy-client@3.8.0                                            
|     `-- request@2.88.2                                              
|       `-- har-validator@5.1.5                                       
|         `-- ajv@6.12.6  deduped                                     
+-- @university-of-york/esg-lib-pattern-library-react-components@6.1.4
| `-- babel-plugin-react-css-modules@5.2.6                            
|   `-- ajv@6.12.6                                                    
+-- next@10.0.5                                                       
| +-- sass-loader@10.0.5                                              
| | `-- schema-utils@3.0.0                                            
| |   `-- ajv@6.12.6  deduped                                         
| +-- schema-utils@2.7.1                                              
| | `-- ajv@6.12.6  deduped                                           
| `-- webpack@4.44.1                                                  
|   +-- ajv@6.12.6  deduped                                           
|   +-- schema-utils@1.0.0                                            
|   | `-- ajv@6.12.6  deduped                                         
|   `-- terser-webpack-plugin@1.4.5                                   
|     `-- schema-utils@1.0.0                                          
|       `-- ajv@6.12.6  deduped                                       
+-- serverless@2.18.0                                                 
| +-- ajv@7.0.3                                                       
| `-- ajv-formats@1.5.1                                               
|   `-- ajv@7.0.3                                                     
`-- xo@0.33.1                                                         
  `-- eslint@7.17.0                                                   
    +-- @eslint/eslintrc@0.2.2                                        
    | `-- ajv@6.12.6  deduped                                         
    +-- ajv@6.12.6  deduped                                           
    `-- table@6.0.7                                                   
      `-- ajv@7.0.3        

As I say, this one works and does not run into the error Cannot find module 'ajv/dist/compile/codegen'.

@srdone
Copy link

srdone commented Jan 8, 2021

In our project, before upgrading to 2.18.0, this is the result on running npm ls ajv:

├─┬ objection@2.2.5
│ └── ajv@6.12.6 
├─┬ redoc-cli@0.9.7
│ └─┬ redoc@2.0.0-rc.24
│   └─┬ swagger2openapi@5.4.0
│     └─┬ oas-validator@3.4.0
│       └── ajv@5.5.2 
├─┬ request@2.88.2
│ └─┬ har-validator@5.1.5
│   └── ajv@6.12.6  deduped
├─┬ serverless@2.17.0
│ └── ajv@6.12.6  deduped
└─┬ standard@14.3.4
  └─┬ eslint@6.8.0
    ├── ajv@6.12.6  deduped
    └─┬ table@5.4.6
      └── ajv@6.12.6  deduped

After upgrading to 2.18.0, the result of npm ls ajv becomes:

├─┬ objection@2.2.5
│ └── ajv@6.12.6 
├─┬ redoc-cli@0.9.7
│ └─┬ redoc@2.0.0-rc.24
│   └─┬ swagger2openapi@5.4.0
│     └─┬ oas-validator@3.4.0
│       └── ajv@5.5.2 
├─┬ request@2.88.2
│ └─┬ har-validator@5.1.5
│   └── ajv@6.12.6  deduped
├─┬ serverless@2.18.0
│ ├── UNMET PEER DEPENDENCY ajv@7.0.3 
│ └─┬ ajv-formats@1.5.1
│   └── ajv@7.0.3 
└─┬ standard@14.3.4
  └─┬ eslint@6.8.0
    ├── ajv@6.12.6  deduped
    └─┬ table@5.4.6
      └── ajv@6.12.6  deduped

This suggests to me that serverless expects ajv@7.x to be installed in any project where it is used. That's how peer dependencies are expected to work in npm - if a package is declared as a peer dependency, npm does not automatically install it but instead warn the user of the package that it is missing a peer dependency. In this case, I get the warning:

npm WARN ajv-keywords@4.0.0 requires a peer of ajv@^7.0.0 but none is installed. You must install peer dependencies yourself.

The user is then expected to install the missing peer dependency in their project where the package requiring the peer dependency is used.

I can, of course, install ajv@7.0.0 as a dev dependency alongside serverless to fix the issue I am experiencing (which is the standard solution in this scenario). However, it seems to me that since serverless is intended to run independently of the project and it requires another package to operate properly, it should not declare that dependency as a peer dependency - it should declare it as a dependency.

@medikoo
Copy link
Contributor

medikoo commented Jan 10, 2021

it should not declare that dependency as a peer dependency - it should declare it as a dependency.

@srdone serverless declares ajv@7 as dependency:

"ajv": "^7.0.3",

If in your case installedserverless has no visiblity of ajv it means that installation is broken (and installer is to blame not serverless)

@SamuelHaleMN
Copy link

Just to chime in... this odd issue also threw our CI/CD build pipeline for a loop.

Clearly, whether this is "intended behavior" or not, this change to the ajv dependency has caused quite a few issues.

@medikoo
Copy link
Contributor

medikoo commented Jan 11, 2021

Clearly, whether this is "intended behavior" or not, this change to the ajv dependency has caused quite a few issues.

I believe it caused issues to cases where (1) service depends directly on ajv and (2) service was installed with installer that has limited support for handling peer dependencies

Note that all we did is upgrading ajv, downgrading it back won't ensure that same issue cannot happen.

@FergusMcGlynn
Copy link

FergusMcGlynn commented Jan 11, 2021

I think @medikoo is correct that this issue has been caused by serverless version 2.18.0 upgrading its dependencies. Specifically, I think it's the upgrade of ajv-keywords to version ^4.0.0 that's brought this problem to the surface - that's the package that has the peer dependency on ajv version ^7.0.3.

This looks to be an example of a bug in npm that's been around for a long time. That issue is almost 2 years old and was never resolved by the look of things. It might have been resolved by npm version 7, but I can't find confirmation of that anywhere. The person who raised that issue did a great job of reproducing the problem, and you can see that the circumstances in their demo project are very similar to the case that we're seeing here. Just replace webpack in their example with serverless in ours and it looks like the very same situation.

In which case, I suppose the options are to do one of the following:

FergusMcGlynn added a commit to university-of-york/uoy-api-courses that referenced this issue Jan 13, 2021
See serverless/serverless#8712 for
details of the security vulnerability.

This update to Serverless has necessitated us installing "ajv"
as a dev dependency. "ajv" is a peer dependency of one of Serverless'
dependencies, and shouldn't normally require us to install it
(because Serverless itself does), however there is a long-standing
bug in npm that leads to npm being unable to resolve peer
dependencies in certain circumstances, this being one of them.
More details at
serverless/serverless#8727
and
npm/npm#19877
@medikoo
Copy link
Contributor

medikoo commented Jan 15, 2021

AJV will be reverted to v6 starting with next release of Framework, it's due to performance issues with observed which appear to not be easily fixable, for more info check:

This means that this problem will be fixed for you, unless you've upgraded your services to rely on AJV v7, as then you'll face the same problem with same installer when serverless depends on AJV v6

@claylevering
Copy link

This seems fixed as far as I can tell, I think we can close it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug/external Bug on provider side
Projects
None yet
Development

No branches or pull requests

7 participants