Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache locally-built Homebrew bottles in OSX builds #144

Merged
merged 6 commits into from Nov 25, 2018
Merged

Cache locally-built Homebrew bottles in OSX builds #144

merged 6 commits into from Nov 25, 2018

Conversation

native-api
Copy link
Contributor

@native-api native-api commented Nov 15, 2018

Closes #139.

It saves brew dependencies as they are built, measures elapsed time, and once it's high enough, fails the build but arranges for Travis cache to be saved. Then reuses the bottles on the next run. It doesn't seem that any solution currently exists, so rolled my own. Now, it does!
It also correctly detects updates and rebuilds bottles for new versions as needed.

So, basically, you need to restart jobs that print a message like "Building dependencies took too long" in Travis UI until they finish. As things are now, it takes 2 restarts to get a job to finish for the first time.
I'm not sure if Travis UI allows to restart jobs in a PR's automatic check -- in this PR, you'll be able to test this!


In other news: Other changes:

  • Cached the Homebrew installation (but not the cellar) as well so it's not updated all the way each time.
  • Set up ccache cache. This brings recompile times down from 20 to 8(!) minutes. Dunno how much it will help in the normal course of action 'cuz the effect depends on how many source files are unchanged between builds.
  • Added -j<...> to make args since all Travis builders are 2-core. This only affects the project itself (brew uses its own make args which already include -j<...>, see brew --env output).

@skvark
Copy link
Member

skvark commented Nov 15, 2018

Nice! I will check this during weekend, I'm too busy today/tomorrow.

@native-api
Copy link
Contributor Author

native-api commented Nov 18, 2018

Didn't manage to enable ccache on both Linux and OSX: in Linux, creating /usr/local/Homebrew fails with EACCES, terminating caching logic. The only way is to replicate cache: directories: $HOME/.ccache in every Linux job entry which I presume would inconvenience you too much.

@skvark
Copy link
Member

skvark commented Nov 21, 2018

I will check this and the other pull requests / issues asap during this week, didn't found the time last weekend.

@native-api
Copy link
Contributor Author

native-api commented Nov 21, 2018

which I presume would inconvenience you too much.

On second thought, if - BDIST_PARAMS="-- -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.7" in every OSX entry (that really belongs in setup.py instead since compilation fails without it) don't inconvenience you, these probably won't, either.

@skvark
Copy link
Member

skvark commented Nov 21, 2018

About that deployment target param: it was a hack due to a bug in scikit-build (#110 (comment)). I think it can be moved safely to setup.py now.

@skvark
Copy link
Member

skvark commented Nov 25, 2018

I'm a bit afraid of the maintenance burden this might introduce, but I think there's currently no better way to do this. And the build times are so much better with this approach.

@skvark skvark merged commit 3d8a1e9 into opencv:master Nov 25, 2018
@native-api
Copy link
Contributor Author

native-api commented Nov 25, 2018

This PR doesn't fix #142

Shucks, Github detected the "fix" word, too, and closed #142 as well when you merged this!
Fortunately, it doesn't matter in this case :)

@skvark
Copy link
Member

skvark commented Nov 25, 2018

Github is trying to be too smart 😄

@native-api native-api deleted the remove_ffmpeg_options branch November 25, 2018 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OSX FFmpeg can no longer be built without GPL libs Fix ffmpeg build timeout in OSX builds
2 participants