Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

operator out of memory - simple TS project with azure-native #1562

Closed
lghinet opened this issue Mar 7, 2022 · 6 comments
Closed

operator out of memory - simple TS project with azure-native #1562

lghinet opened this issue Mar 7, 2022 · 6 comments
Assignees
@danielrbradley
Labels
kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed

Comments

@lghinet
Copy link

lghinet commented Mar 7, 2022

Hello,
I am trying to run this demo stack

apiVersion: pulumi.com/v1
kind: Stack
metadata:
  finalizers:
    - finalizer.stack.pulumi.com
  name: pulumi-demo-stack
  namespace: pulumi
status:
  lastUpdate:
    lastAttemptedCommit: c5f2195d170820b2d1646a3de68982f1d3654584
    lastResyncTime: '2022-03-07T04:46:37Z'
    lastSuccessfulCommit: b113f2c4c0425d6d0e9d346537f86a510e4d90ba
    state: failed
spec:
  branch: refs/heads/main
  config:
    azure-native:clientId: xxxx
    azure-native:location: West Europe
    azure-native:subscriptionId: xxx
    azure-native:tenantId: xxx
  destroyOnFinalize: true
  envRefs:
    ARM_CLIENT_SECRET:
      secret:
        key: ARM_CLIENT_SECRET
        name: azure-secret
        namespace: pulumi
      type: Secret
    PULUMI_ACCESS_TOKEN:
      secret:
        key: accessToken
        name: pulumi-api-secret
        namespace: pulumi
      type: Secret
  projectRepo: https://github.com/lghinet/pulumi_demo
  stack: totalsoft/nginx/dev3

with pulumi operator but I get an OOM exception
here are the logs

{"level":"info","ts":"2022-03-07T05:26:55.513Z","logger":"cmd","msg":"Operator Version: v1.4.0"}
{"level":"info","ts":"2022-03-07T05:26:55.515Z","logger":"cmd","msg":"Go Version: go1.16.2"}
{"level":"info","ts":"2022-03-07T05:26:55.515Z","logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":"2022-03-07T05:26:55.515Z","logger":"cmd","msg":"Version of operator-sdk: v0.19.4"}
{"level":"info","ts":"2022-03-07T05:26:55.515Z","logger":"cmd","msg":"Graceful shutdown","timeout":300}
{"level":"info","ts":"2022-03-07T05:26:57.274Z","logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
{"level":"info","ts":"2022-03-07T05:26:57.275Z","logger":"cmd","msg":"Registering Components."}
{"level":"debug","ts":"2022-03-07T05:26:57.276Z","logger":"k8sutil","msg":"Found namespace","Namespace":"pulumi"}
{"level":"debug","ts":"2022-03-07T05:26:57.276Z","logger":"kubemetrics","msg":"Starting collecting operator types"}
{"level":"debug","ts":"2022-03-07T05:26:57.276Z","logger":"kubemetrics","msg":"Generating metric families","apiVersion":"pulumi.com/v1","kind":"Stack"}
{"level":"debug","ts":"2022-03-07T05:26:59.041Z","logger":"kubemetrics","msg":"Generating metric families","apiVersion":"pulumi.com/v1alpha1","kind":"Stack"}
{"level":"debug","ts":"2022-03-07T05:27:00.797Z","logger":"kubemetrics","msg":"Starting serving custom resource metrics"}
{"level":"debug","ts":"2022-03-07T05:27:02.550Z","logger":"k8sutil","msg":"Found namespace","Namespace":"pulumi"}
{"level":"debug","ts":"2022-03-07T05:27:02.550Z","logger":"k8sutil","msg":"Found podname","Pod.Name":"pulumi-kubernetes-operator-689d6b7d9b-gckzt"}
{"level":"debug","ts":"2022-03-07T05:27:02.556Z","logger":"k8sutil","msg":"Found Pod","Pod.Namespace":"pulumi","Pod.Name":"pulumi-kubernetes-operator-689d6b7d9b-gckzt"}
{"level":"debug","ts":"2022-03-07T05:27:02.592Z","logger":"metrics","msg":"Pods owner found","Kind":"Deployment","Name":"pulumi-kubernetes-operator","Namespace":"pulumi"}
{"level":"info","ts":"2022-03-07T05:27:02.630Z","logger":"metrics","msg":"Metrics Service object updated","Service.Name":"pulumi-kubernetes-operator-metrics","Service.Namespace":"pulumi"}
{"level":"info","ts":"2022-03-07T05:27:04.403Z","logger":"cmd","msg":"Could not create ServiceMonitor object","error":"servicemonitors.monitoring.coreos.com \"pulumi-kubernetes-operator-metrics\" already exists"}
{"level":"info","ts":"2022-03-07T05:27:04.403Z","logger":"cmd","msg":"Starting the Cmd."}
{"level":"info","ts":"2022-03-07T05:27:04.403Z","logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
{"level":"debug","ts":"2022-03-07T05:27:20.728Z","logger":"controller-runtime.manager.events","msg":"Normal","object":{"kind":"ConfigMap","namespace":"pulumi","name":"pulumi-kubernetes-operator-lock","uid":"e4b9f2e0-7daf-4642-af37-7b358f660b24","apiVersion":"v1","resourceVersion":"172758597"},"reason":"LeaderElection","message":"pulumi-kubernetes-operator-689d6b7d9b-gckzt_80f427c0-ef45-4c46-8203-9f4f88b1c66d became leader"}
{"level":"debug","ts":"2022-03-07T05:27:20.733Z","logger":"controller-runtime.manager.events","msg":"Normal","object":{"kind":"Lease","namespace":"pulumi","name":"pulumi-kubernetes-operator-lock","uid":"349ef4ca-eaf3-4288-82ec-a6dc48431fc1","apiVersion":"coordination.k8s.io/v1","resourceVersion":"172758598"},"reason":"LeaderElection","message":"pulumi-kubernetes-operator-689d6b7d9b-gckzt_80f427c0-ef45-4c46-8203-9f4f88b1c66d became leader"}
{"level":"info","ts":"2022-03-07T05:27:20.733Z","logger":"controller-runtime.manager.controller.stack-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":"2022-03-07T05:27:20.733Z","logger":"controller-runtime.manager.controller.stack-controller","msg":"Starting Controller"}
{"level":"info","ts":"2022-03-07T05:27:20.733Z","logger":"controller-runtime.manager.controller.stack-controller","msg":"Starting workers","worker count":10}
{"level":"info","ts":"2022-03-07T05:27:20.734Z","logger":"controller_stack","msg":"Reconciling Stack","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack"}
{"level":"debug","ts":"2022-03-07T05:27:20.734Z","logger":"controller_stack","msg":"Setting up pulumi workdir for stack","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","stack":{"envRefs":{"ARM_CLIENT_SECRET":{"type":"Secret","secret":{"namespace":"pulumi","name":"azure-secret","key":"ARM_CLIENT_SECRET"}},"PULUMI_ACCESS_TOKEN":{"type":"Secret","secret":{"namespace":"pulumi","name":"pulumi-api-secret","key":"accessToken"}}},"stack":"totalsoft/nginx/dev3","config":{"azure9eb9-31a84ee91665","azure-native:location":"West Europe","azure-native:subscriptionId":"xxxxxx"},"projectRepo":"https://github.com/lghinet/pulumi_demo","branch":"refs/heads/main","destroyOnFinalize":true}}
{"level":"info","ts":"2022-03-07T05:27:22.076Z","logger":"controller_stack","msg":"Upserting stack","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","stack":"totalsoft/nginx/dev3","workspace":{}}
{"level":"debug","ts":"2022-03-07T05:27:24.072Z","logger":"controller_stack","msg":"Setting autostack","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","autostack":{}}
{"level":"debug","ts":"2022-03-07T05:27:25.122Z","logger":"controller_stack","msg":"Initial autostack config","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","config":{}}
{"level":"info","ts":"2022-03-07T05:27:25.123Z","logger":"controller_stack","msg":"Missing stack config file. Will assume no stack config checked-in.","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Cause":"unable to find stack settings in workspace for totalsoft/nginx/dev3","CauseVerbose":"unable to find stack settings in workspace for totalsoft/nginx/dev3\ngithub.com/pulumi/pulumi/sdk/v3/go/auto.(*LocalWorkspace).StackSettings\n\t/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.23.2/go/auto/local_workspace.go:90\ngithub.com/pulumi/pulumi-kubernetes-operator/pkg/controller/stack.(*reconcileStackSession).ensureStackSettings\n\t/home/runner/work/pulumi-kubernetes-operator/pulumi-kubernetes-operator/pkg/controller/stack/stack_controller.go:761\ngithub.com/pulumi/pulumi-kubernetes-operator/pkg/controller/stack.(*reconcileStackSession).SetupPulumiWorkdir\n\t/home/runner/work/pulumi-kubernetes-operator/pulumi-kubernetes-operator/pkg/controller/stack/stack_controller.go:738\ngithub.com/pulumi/pulumi-kubernetes-operator/pkg/controller/stack.(*ReconcileStack).Reconcile\n\t/home/runner/work/pulumi-kubernetes-operator/pulumi-kubernetes-operator/pkg/controller/stack/stack_controller.go:206\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:214\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.16.2/x64/src/runtime/asm_amd64.s:1371"}
{"level":"debug","ts":"2022-03-07T05:27:25.123Z","logger":"controller_stack","msg":"stackConfig loaded","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","stack":{},"stackConfig":{}}
{"level":"debug","ts":"2022-03-07T05:27:25.993Z","logger":"controller_stack","msg":"Updated stack config","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stack.Name":"totalsoft/nginx/dev3","config":{"azure-native:clientId":{"Value":"xxxxxxx","Secret":false}}}
{"level":"debug","ts":"2022-03-07T05:27:25.993Z","logger":"controller_stack","msg":"InstallProjectDependencies","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","workspace":"/tmp/pulumi_auto1854221909"}
{"level":"debug","ts":"2022-03-07T05:27:26.878Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!"}
{"level":"debug","ts":"2022-03-07T05:31:11.265Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:11.266Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"> @pulumi/azure-native@1.60.0 install /tmp/pulumi_auto1854221909/node_modules/@pulumi/azure-native"}
{"level":"debug","ts":"2022-03-07T05:31:11.266Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"> node scripts/install-pulumi-plugin.js resource azure-native v1.60.0"}
{"level":"debug","ts":"2022-03-07T05:31:11.266Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:11.486Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"[resource plugin azure-native-1.60.0] installing"}
{"level":"debug","ts":"2022-03-07T05:31:12.405Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"\rDownloading plugin: 0 B / 25.10 MiB    0.00%\rDownloading plugin: 3.62 MiB / 25.10 MiB   14.44% 1s\rDownloading plugin: 8.72 MiB / 25.10 MiB   34.75%\rDownloading plugin: 17.37 MiB / 25.10 MiB   69.20%\rDownloading plugin: 25.10 MiB / 25.10 MiB  100.00% 0s"}
{"level":"debug","ts":"2022-03-07T05:31:12.405Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"warning: A new version of Pulumi is available. To upgrade from version '3.23.2' to '3.25.1', visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes."}
{"level":"debug","ts":"2022-03-07T05:31:12.416Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:12.416Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"> @pulumi/kubernetes@3.16.0 install /tmp/pulumi_auto1854221909/node_modules/@pulumi/kubernetes"}
{"level":"debug","ts":"2022-03-07T05:31:12.416Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"> node scripts/install-pulumi-plugin.js resource kubernetes v3.16.0"}
{"level":"debug","ts":"2022-03-07T05:31:12.416Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:12.602Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"[resource plugin kubernetes-3.16.0] installing"}
{"level":"debug","ts":"2022-03-07T05:31:13.636Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"\rDownloading plugin: 0 B / 30.27 MiB    0.00%\rDownloading plugin: 3.82 MiB / 30.27 MiB   12.63% 1s\rDownloading plugin: 8.64 MiB / 30.27 MiB   28.56%\rDownloading plugin: 13.08 MiB / 30.27 MiB   43.22%\rDownloading plugin: 25.32 MiB / 30.27 MiB   83.64%\rDownloading plugin: 30.27 MiB / 30.27 MiB  100.00% 0s"}
{"level":"debug","ts":"2022-03-07T05:31:13.636Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"warning: A new version of Pulumi is available. To upgrade from version '3.23.2' to '3.25.1', visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes."}
{"level":"debug","ts":"2022-03-07T05:31:13.666Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:13.666Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"> protobufjs@6.11.2 postinstall /tmp/pulumi_auto1854221909/node_modules/protobufjs"}
{"level":"debug","ts":"2022-03-07T05:31:13.666Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"> node scripts/postinstall"}
{"level":"debug","ts":"2022-03-07T05:31:13.666Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:14.487Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"npm WARN nginx@ No repository field."}
{"level":"debug","ts":"2022-03-07T05:31:14.497Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":"npm WARN nginx@ No license field."}
{"level":"debug","ts":"2022-03-07T05:31:14.497Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Text":""}
{"level":"debug","ts":"2022-03-07T05:31:14.500Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"added 124 packages from 162 contributors and audited 125 packages in 227.675s"}
{"level":"debug","ts":"2022-03-07T05:31:14.576Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:14.576Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"31 packages are looking for funding"}
{"level":"debug","ts":"2022-03-07T05:31:14.576Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"  run `npm fund` for details"}
{"level":"debug","ts":"2022-03-07T05:31:14.576Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:14.577Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":"found 0 vulnerabilities"}
{"level":"debug","ts":"2022-03-07T05:31:14.577Z","logger":"controller_stack","msg":"NPM/Yarn","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Dir":"/tmp/pulumi_auto1854221909","Path":"/usr/bin/npm","Args":["/usr/bin/npm","install"],"Stdout":""}
{"level":"info","ts":"2022-03-07T05:31:14.621Z","logger":"controller_stack","msg":"Checking current HEAD commit hash","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Current commit":"c5f2195d170820b2d1646a3de68982f1d3654584"}
{"level":"info","ts":"2022-03-07T05:31:14.621Z","logger":"controller_stack","msg":"New commit hash found","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Current commit":"c5f2195d170820b2d1646a3de68982f1d3654584","Last commit":"b113f2c4c0425d6d0e9d346537f86a510e4d90ba"}
{"level":"debug","ts":"2022-03-07T05:31:14.621Z","logger":"controller-runtime.manager.events","msg":"Normal","object":{"kind":"Stack","namespace":"pulumi","name":"pulumi-demo-stack","uid":"3ea90ad6-9090-4fff-b090-03ce5a6b11f2","apiVersion":"pulumi.com/v1","resourceVersion":"172743742"},"reason":"StackUpdateDetected","message":"New commit detected: \"c5f2195d170820b2d1646a3de68982f1d3654584\"."}
{"level":"debug","ts":"2022-03-07T05:31:15.490Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":"Updating (dev3)"}
{"level":"debug","ts":"2022-03-07T05:31:15.490Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:15.931Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":"View Live: https://app.pulumi.com/totalsoft/nginx/dev3/updates/1"}
{"level":"debug","ts":"2022-03-07T05:31:15.931Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:16.893Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":""}
{"level":"debug","ts":"2022-03-07T05:31:17.773Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":" +  pulumi:pulumi:Stack nginx-dev3 creating "}
{"level":"debug","ts":"2022-03-07T05:31:38.866Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":" +  azure-native:resources:ResourceGroup pulumi_qa_RG creating "}
{"level":"debug","ts":"2022-03-07T05:31:39.573Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":" +  azure-native:resources:ResourceGroup pulumi_qa_RG created "}
{"level":"debug","ts":"2022-03-07T05:31:40.042Z","logger":"controller_stack","msg":"Pulumi Update","Request.Namespace":"pulumi","Request.Name":"pulumi-demo-stack","Stdout":" +  azure-native:sql:Server qa-sqlserver creating "}
rpc error: code = NotFound desc = an error occurred when try to find container "071e11f988c89b66649d3d469477cd4a1bf4e1fa24e89620b21cf8710dea3ee5": not found

everything seems ok, it even creates a resourceGroup in azure, but when creating the sqlServer resource .. it crashes serveral times with podEvicted from kube, because it uses all the node memory .. which is around 10 GB

i also want to mention that npm install .. takes around 3 minutes, which is quite long ..

any ideas .. ?

thanks
@lghinet lghinet added the kind/bug Some behavior is incorrect or out of spec label Mar 7, 2022
@viveklak
Copy link
Contributor

Just double checking you are not setting a memory resource limit on the operator pod? I will transfer this to the azure-native repo since its more relevant to its footprint.

@viveklak viveklak transferred this issue from pulumi/pulumi-kubernetes-operator Mar 10, 2022
@viveklak
Copy link
Contributor

Could you also post the stack code you have?

@lghinet
Copy link
Author

lghinet commented Mar 11, 2022 •
edited

Could you also post the stack code you have?

hello
the stack is public
https://github.com/lghinet/pulumi_demo
https://github.com/lghinet/pulumi_demo/blob/main/index.ts

@lghinet
Copy link
Author

lghinet commented Mar 18, 2022

OOMs only in docker (pulumi-operator)

@joeskeen
Copy link

joeskeen commented Feb 6, 2023

duplicate of #1997

@danielrbradley danielrbradley added the resolution/fixed This issue was fixed label Sep 12, 2023
@danielrbradley
Copy link
Member

I'm going to close this as it appears similar to #1997 and so should have also been resolved by the recent v2 release which significantly reduces the size of the node.js SDK. Please do open a new issue if this can still be reproduced against the latest version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@danielrbradley danielrbradley

Labels
kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@danielrbradley @viveklak @joeskeen @lghinet