-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use deno 1.43: binaries ~25% smaller, ~5% faster #978
base: main
Are you sure you want to change the base?
Conversation
Yeah totes want it. We’re having trouble building 1.41 over at the pantry, and couldn't bump to 1.40 due to deprecated API with no replacements at that time. Once 1.41 builds we can figure it all out. |
Awesome! |
FYI if you strip deno first it results in smaller binaries too. |
Right, which can be done even with deno 1.39. But I don't think is so urgent. :) |
1.41 likely in an hour or so. or two. it's not exactly fast to build. |
1.41.0 not ready for linux/aarch64. I pushed fixes for the other 3, but it uses their build of denort, which uses ubuntu22's glibc. You can see the outstanding issue as a comment in the package.yml. |
Right, and apparently, they don't allow to specify your own denort binary as well. Thanks for following it up. |
So main remaining issue is libpkgx uses And At least that was true with deno 1.40 |
The binary I compiled is not throwing any warnings when invoked. Is there some specific command to trigger it? Code_SbayEe6Xw8.mp4 |
it is for building the same deno across all environments for the time being. per their notes in the PR: # https://github.com/denoland/deno/pull/22298
# deno.land 1.41.0 will currently _not_ run deno compile on linux/aarch64
# for their first official release, they're using ubuntu 22.04, which means
# a newer glibc. Patching via the https://github.com/LukeChannings/deno-arm64
# repo may be possible, but lets not delay the three usable arches for the rare
# one. Revist this. the problem is, otherwise, we could put in build logic that uses newer |
Ok. I see now. Here's a quick reproduction of the problem btw (I used it to verify whether 1.41.2 had solved it or not -- it didn't). FROM denoland/deno:1.41.2 AS build
RUN apt-get update && apt-get install unzip -y
RUN deno compile https://docs.deno.com/examples/welcome.ts
FROM ubuntu:18.04
COPY --from=build /welcome /
ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache
RUN /welcome # install arm64 emulator
$ docker run --privileged --rm --pull=always tonistiigi/binfmt --install arm64
$ docker build . --platform=amd64
[...]
Welcome to Deno!
$ docker build . --platform=arm64
[...]
/welcome: /lib/aarch64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /welcome) I'll let you know if I find out something. |
Which "our" compiled denort? It's not in the pantry from what I can tell. That's the easiest path forward I can think of as well. And I don't think it should be considered a workaround either, as I think it would be desirable to have control over the libs used to compile denort and therefore pkgx itself. I think what it would take is:
The best possible solution IMO would be if deno could self-package the denort in runtime (after all it's a sub-set of the full deno binary). |
current builds of |
Understood! Will look into it. Thanks. |
Checking the denort binary beside deno may not be the best option because deno is capable of cross compiling it too (by downloading the extra denort binaries). |
But it's ok in pkgx's case because cross compilation is not used. @jhheider take a look at my latest commit, I think it's good enough now. :D The bad news however is: ❯ ls -lah pkgx*
-rwxrwxrwx 1 felipecrs felipecrs 88M Mar 8 14:45 pkgx.deno-denort
-rwxrwxrwx 1 felipecrs felipecrs 106M Mar 8 14:44 pkgx.pkgx-denort But it's an improvement anyway over the previous 118MB. We can probably trim this size down by compiling denort in the pipeline with some different flags. We can potentially make the final denort size even smaller than deno's own denort. |
Good find! @mxcl what do you think? I'm hesitant to runtime.env this, since it'll break cross compiling, but this should fix our builds. |
Bumping Deno to 1.42, the compiled binary is now 100MB, compared to 106MB from Deno 1.41 and 118MB from Deno 1.39. Deno 1.42 also claims to improve startup time, which should greatly help pkgx. I'm yet to benchmark it though. |
I did a quick and dirty benchmark file which I uploaded to the repo. Here are the results: ❯ pkgx deno@1.40 task benchmark && pkgx deno@1.41 task benchmark && pkgx deno@1.42 task benchmark
Task benchmark deno task compile &>/dev/null && deno bench --allow-run --allow-env --seed=250 benchmarks
Check file:///home/felipecrs/repos/pkgx/benchmarks/startup_bench.ts
cpu: AMD Ryzen 7 5800X 8-Core Processor
runtime: deno 1.40.5 (x86_64-unknown-linux-gnu)
file:///home/felipecrs/repos/pkgx/benchmarks/startup_bench.ts
benchmark time (avg) iter/s (min … max) p75 p99 p995
------------------------------------------------------------------------ -----------------------------
group startup
git --version 4.32 ms/iter 231.4 (3.98 ms … 4.92 ms) 4.42 ms 4.83 ms 4.92 ms
pkgx git --version 528.89 ms/iter 1.9 (517.75 ms … 539.56 ms) 537.1 ms 539.56 ms 539.56 ms
summary
git --version
122.4x faster than pkgx git --version
Task benchmark deno task compile &>/dev/null && deno bench --allow-run --allow-env --seed=250 benchmarks
Check file:///home/felipecrs/repos/pkgx/benchmarks/startup_bench.ts
cpu: AMD Ryzen 7 5800X 8-Core Processor
runtime: deno 1.41.3 (x86_64-unknown-linux-gnu)
file:///home/felipecrs/repos/pkgx/benchmarks/startup_bench.ts
benchmark time (avg) iter/s (min … max) p75 p99 p995
------------------------------------------------------------------------ -----------------------------
group startup
git --version 4.39 ms/iter 227.8 (4.04 ms … 5.03 ms) 4.5 ms 4.97 ms 5.03 ms
pkgx git --version 525.04 ms/iter 1.9 (503.98 ms … 573.2 ms) 529.41 ms 573.2 ms 573.2 ms
summary
git --version
119.61x faster than pkgx git --version
Task benchmark deno task compile &>/dev/null && deno bench --allow-run --allow-env --seed=250 benchmarks
Check file:///home/felipecrs/repos/pkgx/benchmarks/startup_bench.ts
cpu: AMD Ryzen 7 5800X 8-Core Processor
runtime: deno 1.42.0 (x86_64-unknown-linux-gnu)
file:///home/felipecrs/repos/pkgx/benchmarks/startup_bench.ts
benchmark time (avg) iter/s (min … max) p75 p99 p995
------------------------------------------------------------------------ -----------------------------
group startup
git --version 4.32 ms/iter 231.7 (4.01 ms … 5.16 ms) 4.37 ms 5.06 ms 5.16 ms
pkgx git --version 499.3 ms/iter 2.0 (490.89 ms … 516.15 ms) 502.55 ms 516.15 ms 516.15 ms
summary
git --version
115.67x faster than pkgx git --version However, I ran it several times. The results aren't consistent at all. Maybe |
Ok, I made some changes and now the results are a bit more consistent: ❯ scripts/benchmark.sh
Compiling with deno@1.40...
Benchmarking...
113.24x faster than pkgx git --version
114.81x faster than pkgx git --version
115.91x faster than pkgx git --version
114.68x faster than pkgx git --version
115.35x faster than pkgx git --version
114.02x faster than pkgx git --version
114.56x faster than pkgx git --version
114.52x faster than pkgx git --version
115.32x faster than pkgx git --version
115.06x faster than pkgx git --version
Compiling with deno@1.41...
Benchmarking...
114.02x faster than pkgx git --version
112.7x faster than pkgx git --version
116.1x faster than pkgx git --version
111.63x faster than pkgx git --version
112.52x faster than pkgx git --version
111.36x faster than pkgx git --version
112.66x faster than pkgx git --version
111.69x faster than pkgx git --version
113.51x faster than pkgx git --version
113.86x faster than pkgx git --version
Compiling with deno@1.42...
Benchmarking...
111.43x faster than pkgx git --version
110.5x faster than pkgx git --version
113.28x faster than pkgx git --version
111.98x faster than pkgx git --version
111.71x faster than pkgx git --version
113.25x faster than pkgx git --version
124.55x faster than pkgx git --version
112.06x faster than pkgx git --version
113.39x faster than pkgx git --version
112.52x faster than pkgx git --version The lower the better. I can see a clear improvement of 1.41 over 1.40, but 1.42 over 1.41 didn't produce any meaningful changes. The highest is higher than 1.41, but the lowest is also lower than 1.41. |
When
Probably this is a hint on how we can make the binaries smaller. |
Ah ok, now I realize. Compiling with glibc 2.35+ would mean the compiled binaries would only work in newer systems. This in turn would also restrict pkgx itself to only run in newer systems. That's obviously undesired. |
I would say this PR is ready to be merged (but I'd recommend merging #959 first so that this one becomes lighter). Here are up to date stats: I.e:
|
9038067
to
342373c
Compare
Deno 1.43.0 has some small improvements regarding startup time: https://deno.com/blog/v1.43#faster-es-and-commonjs-module-loading |
Note: this description is outdated.
This PR builds on #959, and bumps deno to 1.41.
With this change, the pkgx binaries size were decrease by 40MB in Linux AMD64:
PS: you can close this PR. I just wanted to demonstrate the benefits of upgrading to deno 1.41.
You guys know best what to do. Probably some changes in libpkgx are needed as well.
Closes #959