How to use chromedp to drive an Electron application

[chinenual]

What versions are you running?

$ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.5.3
$ google-chrome --version
N/A

I'm running a go-based Electron app via:
github.com/asticode/go-astilectron v0.14.2

Underlying electron is version 8.2.5

$ go version
go version go1.14.1 darwin/amd64

What did you do? Include clear steps.

I've enabled dev-tools on my application via

				"remote-debugging-port", "8315",
				"host-rules", "MAP * 127.0.0.1",

I've manually started the application; it's running and listening on the debug port --
I can browse to https://localhost:8315 and see interactive dev tools. However, if I try to drive the app via chromedp (e.g. with the "remote" example), I get a 404 error. Eg.:

~/go/src/github.com/chromedp/examples/remote> go run main.go -devtools-ws-url ws://localhost:8315
2020/05/19 15:55:25 Failed getting body of duckduckgo.com: could not dial "ws://127.0.0.1:8315": unexpected HTTP response status: 404
exit status 1

I've debugged this and determined that the first call to Run(ctxt) is what is failing.

If I change the URL to include a page id that I can see in the /json response (http://localhost:8315/json), the error changes to "Not Supported (-32000)". Not sure if that's an improvement, or just that I am doing something even more wrong.

~/go/src/github.com/chromedp/examples/remote> go run main.go -devtools-ws-url ws://localhost:8315/devtools/page/396803C460A3CEF97A38A5E67EACBAE2
2020/05/19 16:18:07 Failed getting body of duckduckgo.com: Not supported (-32000)
exit status 1

Can anyone advise how to connect chromedp to an Electron application? Perhaps the WS URL needs more context (not just host and port?)

[chinenual]

Bump...

I've determined that the ws://localhost:8315/devtools/page/396803C460A3CEF97A38A5E67EACBAE2 form of the URL is in fact "correct" -- this sort of URL works when I drive an instance of Chrome proper with the debugging url enabled. I derive that URL from http://localhost:8315/json/version -- which returns the value on the "webSocketDebuggerUrl" attribute. So the Electron app is in fact listening on 8315 and serving up the debug API.

But when I try to debug my electron app, I still get the "Not supported (-32000)" error from even a "raw" Run() command without any Navigate, etc.:

package main

import (
	"context"
	"flag"
	"log"

	"github.com/chromedp/chromedp"
)

var flagDevToolWsUrl = flag.String("devtools-ws-url", "", "DevTools WebSsocket URL")

func main() {
	flag.Parse()
	if *flagDevToolWsUrl == "" {
		log.Fatal("must specify -devtools-ws-url")
	}

	// create allocator context for use with creating a browser context later
	allocatorContext, cancel := chromedp.NewRemoteAllocator(context.Background(), *flagDevToolWsUrl)
	defer cancel()
	log.Printf("allocatorContext: %#v", allocatorContext)

	// create context
	ctxt, cancel := chromedp.NewContext(allocatorContext)
	defer cancel()
	log.Printf("ctxt: %#v", ctxt)

	// run task list
	var body string
	if err := chromedp.Run(ctxt); err != nil {
		log.Fatalf("raw Run failed: %v", err)
	}
}

[mvdan]

Could you provide a way to reproduce that error? I don't have an Electron app I can test with.

[chinenual]

My electron app is based on go-astilectron. I've managed to find a way to test it with Spectron (via nodejs) and can send you a sample application (based on the astilectron demo). If you want to build it yourself, see my pull request at asticode/go-astilectron-demo#62. If that's not convenient, I can create a tarball (but it will be very large...). Let me know.

[ZekeLu]

I believe the URL should be .../devtools/browser/... instead of .../devtools/page/.... Please check the discussions in #438.

chromedp/allocate.go

Lines 458 to 461 in 9191ea2

	// NewRemoteAllocator creates a new context set up with a RemoteAllocator,
	// suitable for use with NewContext. The url should point to the browser's
	// websocket address, such as "ws://127.0.0.1:$PORT/devtools/browser/...".
	func NewRemoteAllocator(parent context.Context, url string) (context.Context, context.CancelFunc) {

@chinenual Are you still working on this? Do you want to give it a shot?

[ZekeLu]

I have tested with Electron (Sine astilectron is powered by Electron, I assume the conclusion is valid for both of them), and it works.

The app I used is https://github.com/electron/electron-quick-start. First, start it with the following command:

$ yarn start --remote-debugging-port=9222

And here is the output (we need the WebSocket debug URL later):

yarn run v1.22.5
$ electron . --remote-debugging-port=9222

DevTools listening on ws://127.0.0.1:9222/devtools/browser/f2129e98-5476-435d-9803-de8ba99e2032

Here is a simple demo to automate the app:

package main

import (
	"context"
	"log"

	"github.com/chromedp/chromedp"
)

func main() {
	ctx, cancel := chromedp.NewRemoteAllocator(context.Background(), "ws://127.0.0.1:9222/devtools/browser/f2129e98-5476-435d-9803-de8ba99e2032")
	defer cancel()

	ctx, cancel = chromedp.NewContext(ctx,
		chromedp.WithDebugf(log.Printf),
	)
	defer cancel()

	// go get the current targets in the Electron app.
	ts, err := chromedp.Targets(ctx)
	if err != nil {
		log.Fatal(err)
	}

	if len(ts) == 0 {
		log.Fatal("targets not found.")
	}

	ctx, cancel = chromedp.NewContext(ctx,
		// use the first target
		chromedp.WithTargetID(ts[0].TargetID),
	)
	// if we cancel the target, it will close the Electron app.
	//defer cancel()

	var body string
	if err := chromedp.Run(ctx,
		chromedp.OuterHTML("body", &body, chromedp.ByQuery),
		chromedp.Navigate("https://www.google.com"),
	); err != nil {
		log.Fatal(err)
	}

	log.Println(body)
}

Notes:

1. creating new target is not supported, that's why we list the targets first and then connect to the existing target.

   2021/05/20 14:32:29 -> {"id":1,"method":"Target.createTarget","params":{"url":"about:blank"}}
   2021/05/20 14:32:29 <- {"id":1,"error":{"code":-32000,"message":"Not supported"}}
   

2. cancelling the target context will close the attached target, which will close the Electron app.

3. with PR make NewRemoteAllocator accept url without devtools/browser/... #817, we can create the RemoteAllocator like this:

   ctx, cancel := chromedp.NewRemoteAllocator(context.Background(), "ws://127.0.0.1:9222/")

4. maybe the shipped Chromium/V8 is not compatible with the github.com/chromedp/cdproto package.

   $ curl http://127.0.0.1:9222/json/version
   {
      "Browser": "Chrome/89.0.4389.128",
      "Protocol-Version": "1.3",
      "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) electron-quick-start/1.0.0 Chrome/89.0.4389.128 Electron/12.0.9 Safari/537.36",
      "V8-Version": "8.9.255.25",
      "WebKit-Version": "537.36 (@1510b4a1988e383eb6b3c47622e69ebbbfe09826)",
      "webSocketDebuggerUrl": "ws://127.0.0.1:9222/devtools/browser/f2129e98-5476-435d-9803-de8ba99e2032"
   }

[batara666]

allocatorContext, cancel := chromedp.NewRemoteAllocator(context.Background(), "ws://localhost:8202/")
defer cancel()

// create context
ctx, cancel := chromedp.NewContext(allocatorContext)
defer cancel()

var body string
err := chromedp.Run(ctx,
chromedp.OuterHTML("html", &body),
)
fmt.Println(err)
fmt.Println(body)

Output:

> Not supported (-32000)
> 

[batara666]

but this library works

https://github.com/mkenney/go-chrome

[ZekeLu]

1. creating new target is not supported, that's why we list the targets first and then connect to the existing target.

   2021/05/20 14:32:29 -> {"id":1,"method":"Target.createTarget","params":{"url":"about:blank"}}
   2021/05/20 14:32:29 <- {"id":1,"error":{"code":-32000,"message":"Not supported"}}