Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add global hooks #102

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from
Draft

add global hooks #102

wants to merge 18 commits into from

Conversation

davidjgoss
Copy link
Contributor

@davidjgoss davidjgoss commented Nov 12, 2022

🤔 What's changed?

  • Add new messages for global hooks starting and finishing
  • Add id to TestRunStarted so that the corresponding finish event, test cases and global hooks can be tied back to it
  • Split out TestStepResult into its own file

⚡️ What's your motivation?

Per #66 we want to represent global hooks in the messages, because currently if they fail (thus causing the test run to fail) they aren't visible in reporting.

Putting an id on TestRunStarted is an approach we've discussed before and roughly agreed on in the context of global attachments per cucumber/cucumber-js#1394 (comment) - this PR doesn't address the attachments problem directly, but does lay the groundwork for it, and also means it starts to look more possible to have multiple test runs in the same stream of messages and be able to differentiate them. Note that Gherkin messages and support code aren't linked to a test run because they can be reused - it's TestCase that has the link.

🏷️ What kind of change is this?

  • 💥 Breaking change (incompatible changes to the API) (because id is required on TestRunStarted)

📋 Checklist:

  • I agree to respect and uphold the Cucumber Community Code of Conduct
  • I've changed the behaviour of the code
    • I have added/updated tests to cover my changes.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly.
  • Users should know about my change
    • I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

This text was originally generated from a template, then edited by hand. You can modify the template here.

@davidjgoss
Copy link
Contributor Author

A bit of a chicken and egg problem with the perl tests here - they rely on the current published version of the CCK, whose messages don't include the new field, which is causing the failure. Any thoughts on how to address this @ehuelsmann?

go/messages.go Outdated Show resolved Hide resolved
@mpkorstanje
Copy link
Contributor

mpkorstanje commented Nov 17, 2022

A bit of a chicken and egg problem with the perl tests here - they rely on the current published version of the CCK, whose messages don't include the new field, which is causing the failure. Any thoughts on how to address this @ehuelsmann?

@ciaranmcnulty looks like the acceptance tests against the CCK are not invoked for PHP:

There were 2 skipped tests:

1) AcceptanceTest::testAllNdJsonSurvivesDecodingThenEncoding
Test for AcceptanceTest::testAllNdJsonSurvivesDecodingThenEncoding skipped by data provider
PHPUnit\Framework\SkippedTestError: <no message>

2) AcceptanceTest::testAllFileStreamsSurviveDecodingThenEncoding
Test for AcceptanceTest::testAllFileStreamsSurviveDecodingThenEncoding skipped by data provider
PHPUnit\Framework\SkippedTestError: <no message>

@mpkorstanje
Copy link
Contributor

mpkorstanje commented Nov 17, 2022

A bit of a chicken and egg problem with the perl tests here - they rely on the current published version of the CCK, whose messages don't include the new field, which is causing the failure. Any thoughts on how to address this @ehuelsmann?

@davidjgoss looking at the test I think we can change them a little so. So instead of comparing the round trip ndjson --> object --> ndjson and comparing the result we can do object --> ndjson --> object and compare that result. That way we don't need the CCK at all and the code will change along with that specific message object.

Though I think it would be even better to include a testdata folder with (for now) a single message that all languages try to round trip. That way we won't have to update N implementations manually.

@@ -36,6 +39,10 @@ public java.util.List<TestStep> getTestSteps() {
return testSteps;
}

public Optional<String> getTestRunStartedId() {
return Optional.ofNullable(testRunStartedId);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the rationale for making the field optional? I don't think there is a sensible behavior for a consumer that needs this field when it is absent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we make it required now, the new code will be immediately incompatible with messages generated from the old code. This means e.g. we'd need to wait for all implementations to add this before we could use the new messages version in formatters.

If we're fine with that then I'm happy to change. But it seemed better to add it as optional, and maybe circle back and make it required down the line once it's established everywhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that's acceptable. Can you make a ticket so we won't forget to make it required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done #113

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants