Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout support #491

Merged
merged 28 commits into from Aug 14, 2022
Merged

Conversation

TApplencourt
Copy link
Contributor

@TApplencourt TApplencourt commented Aug 24, 2021

Hi,

The PR tries to fix #396. It adds a global timeout options (-E <second>) to ./bin/bats.
This option adds a timeout to each test. If a test takes longer than the timeout, it will be killed and will exit with an error code set at 141. In the case of the --tap formater, a # timetout will be printed in the terminal (similar to the #skip behavior)

Technical detail

The implementation is inspired by the great work of https://github.com/MountainField/bash-timeout.
I did tweak it a little bit so it works on my SUSE system. The main idea is to spawn a subprocess who will sleep and kill himself and his parents when waking up.

Summary

I'm not a bash expert, nor a bats one. But I implemented a version of timeout that worked for my use case, and maybe also useful for other people.

Please don't hesitate if you have any feedback, I will be happy to do any modification to the PR to make it more "professional",

Hope this help,
Regards@

@TApplencourt TApplencourt requested a review from a team as a code owner August 24, 2021 22:37
@TApplencourt
Copy link
Contributor Author

TApplencourt commented Aug 26, 2021

Fixed the errors reported by the CI. Sorry about that. All the 178 Tests are now passing on my machine.

I don't think I fixed the contribution error, but I'm not sure how to do it to be honest

EDIT: Test didn't pass on Ubuntu / Windows, I have no idea why... Any help welcomed

@TApplencourt TApplencourt marked this pull request as draft August 31, 2021 21:23
@martin-schulze-vireso martin-schulze-vireso added this to the 1.8.0 milestone May 17, 2022
@martin-schulze-vireso
Copy link
Member

martin-schulze-vireso commented Jul 25, 2022

Using SIGPIPE might interfere with some user scripts. I would propose to use SIGALRM or SIGUSR1/2. Furthermore, I think we should name the variable BATS_TEST_TIMEOUT to allow for other kinds of timeouts later on (like setup/setup_file,....)

@TApplencourt
Copy link
Contributor Author

TApplencourt commented Jul 25, 2022

Thanks a lot for the review.

Using SIGPIPE might interfere with some user scripts. I would propose to use SIGALRM or SIGUSR1/2

I spent a few hours today trying to do so but wasn't successful. My trap was either never called or without a trap, I got some error message corresponding to the trap call. Will potentially invest more time later :)

Furthermore, I think we should name the variable BATS_TEST_TIMEOUT to allow for other kinds of timeouts later on (like setup/setup_file,....)

Good idea will do!

@martin-schulze-vireso
Copy link
Member

martin-schulze-vireso commented Jul 26, 2022

I already have some local changes for that. I am not sure how to proceed with the traps but will try my own solution

@martin-schulze-vireso
Copy link
Member

Sigh, Windows does not support SIGUSR1/2. SIGALRM interferes with sleep. I chose SIGABRT now in the hope of not needing it in the test itself. It may still be interfered with from user code sending it...

@martin-schulze-vireso martin-schulze-vireso merged commit 44e2da6 into bats-core:master Aug 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add optional timeout for setup functions, files, and tests
2 participants