Handle tests suites exiting with an exception #54
Conversation
|
Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests |
|
You're right, this should be an 'error'. I've added support for 'error' elements to the JUnit outputter and made it output them. I'm not sure about parsing the time: the rest of the the time parsers just grab the time the test took, whereas we only have the wallclock time of the box, so we could probably work it out but since test crashed anyway, I think it's unlikely to ever be that useful. |
|
Are we missing the updated XCodeBuildOutputParser in your latest commit/push ? |
|
As for the time, I agree that it's not critical. I am not even sure this schema is official and I suspect parsers have learned to be more resistant :) Jenkins doesn't seem to complain without it today. Note: make sure that the TestSuite is probably updated to count the errors. Ideally they would 'bubble up' all the way to the 'All tests' one I suspect. |
|
Oops, yes, sorry: I forgot to git add -u. I added addError() to the TestSuite to match addFailure() and called it when it detects an exception so hopefully this should be okay. |
XCTest output parsing: handle tests suites exiting with an error
|
Thanks for the Pull Request. Merged. |
I think ideally the parser should notice whenever it's got to the end of the output and mark and outstanding tests as failed, which would catch this situation in the more general case. Unfortunately there's no clean way for the parser to know when it's reached the end (close() is specifically blocked although it's not clear why).
However for the specific case of exiting with an exception, this patch makes the output 'unstable' rather than succeeding or failing outright (depending on whether the 'allow failing build results' is enabled).