Chrome process memory is increasing with long running NewRemoteAllocator with periodic Evaluate (js)

[lundin]

Hi,

I have a question for best practice with running a single chrome instance with one tab. First i do chromedp.Run and navigate to a website (that creates a browser/tab). Then i have a message subscriber with nats,.io that can get incoming messages. This subscriber do a chromedp.Evaluate (i think this chromedp.Run is running in the same tab) and send back the output of the js/evaluate. However i can see that the chrome process using top is increasing until it crash. Is this expected that running many console/js in same tab is increasing memory ?

What versions are you running?

$ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.7.8

$ google-chrome --version
Google Chrome 99.0.4844.84 

$ go version
go version go1.17.7 linux/amd64

What did you do? Include clear steps.

//connect to remote instance
devtoolsWsURL := flag.String("devtools-ws-url", "", "devtools url")
        flag.Parse()
        if *devtoolsWsURL == "" {
                log.Fatal("must specify -devtools-ws-url")
        }
        allocatorContext, Alloccancel := chromedp.NewRemoteAllocator(context.Background(), *devtoolsWsURL)
        defer Alloccancel()
   chromedepCtx, cancel := chromedp.NewContext(allocatorContext)
        defer cancel()

//open first tab with run
err = chromedp.Run(chromedepCtx,

chromedp.Navigate("website here"),

chromedp.Evaluate(jsRequest, &res, func(p *runtime.EvaluateParams) *runtime.EvaluateParams {
	return p.WithAwaitPromise(true)
}),
); 

if err != nil {
		panic(err)
	}


//Nats subscriber that runs js commands many times (incoming msg)
nc.Subscribe("time", func(m *nats.Msg) {
  var res interface{}
  err:= chromedp.Run(chromedepCtx,
    chromedp.Evaluate(jsRequest, &res, func(p *runtime.EvaluateParams) *runtime.EvaluateParams {
      return p.WithAwaitPromise(true)
    }),
   ,
  );
  if err != nil {
		panic(err)
	}
 
  log.Printf("chromedp out: %v\n", res)
  
  nc.Publish(m.Reply, []byte(res.(string)) )
  })

What did you expect to see?

More or less stable memory ?

What did you see instead?

Increasing memory

[ZekeLu]

Two things to check based on the current information:

1. What is returned as res? Maybe you should call runtime.ReleaseObject or runtime.ReleaseObjectGroup to release the remote object. See

chromedp/eval.go

Lines 15 to 37 in b88710e

	// Evaluate is an action to evaluate the Javascript expression, unmarshaling
	// the result of the script evaluation to res.
	//
	// When res is nil, the script result will be ignored.
	//
	// When res is a *[]byte, the raw JSON-encoded value of the script
	// result will be placed in res.
	//
	// When res is a **runtime.RemoteObject, res will be set to the low-level
	// protocol type, and no attempt will be made to convert the result.
	// Original objects are maintained in memory until the page navigated or closed,
	// unless they are either explicitly released or are released along with the
	// other objects in their object group. runtime.ReleaseObject or
	// runtime.ReleaseObjectGroup can be used to ask the browser to release
	// original objects.
	//
	// For all other cases, the result of the script will be returned "by value" (ie,
	// JSON-encoded), and subsequently an attempt will be made to json.Unmarshal
	// the script result to res. It returns an error if the script result is
	// "undefined" in this case.
	//
	// Note: any exception encountered will be returned as an error.
	func Evaluate(expression string, res interface{}, opts ...EvaluateOption) EvaluateAction {

2. Try to do the same thing without chromedp to figure out whether it's caused by the usage of chromedp. You can choose one of (or all of) the following methods to do the same thing:
   • Open the target URL in the browser, and keep running your js code in the console of the DevTools
   • Open the target URL in the browser, and issue the Runtime.evaluate command using Protocol Monitor
   • Implement the same logic with https://github.com/puppeteer/puppeteer

[lundin]

Hello again,
Thanks for your input!
I tried manually with the browser and execute the promise/js code in the console. It did not affect the memory.

But res was made an *runtime.RemoteObject (even though i know it is a string) and i freed it with your suggested runtime.ReleaseObject(res.ObjectID) and this actually seems to keep the memory stable after lots of eval execs. So i would say this is the way to go for a long running instance with evals (if that make sense, atleast it seems to work in this case).

[ZekeLu]

@lundin Thanks for the feedback!

If you know that it is a string, maybe you should use a string instead of *runtime.RemoteObject. I think when the type of the variable is a string, you don't need to call runtime.ReleaseObject. Would you like to have a test?

Note: when the type of the variable is string, make sure the result of the js expression is not undefined; otherwise, it will result in an error. See #526