-
-
Notifications
You must be signed in to change notification settings - Fork 380
Add OpenClover XML reporter #1080
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov ReportAttention: Patch coverage is
✅ All tests successful. No failed tests found.
Additional details and impacted files@@ Coverage Diff @@
## main #1080 +/- ##
============================================
+ Coverage 75.25% 75.78% +0.53%
- Complexity 1375 1400 +25
============================================
Files 96 97 +1
Lines 4708 4865 +157
============================================
+ Hits 3543 3687 +144
- Misses 1165 1178 +13 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
@marek-parfianowicz Sorry for mentioning you out of the blue, but can you help me by pointing me to examples of valid Clover XML file for (small) Java projects so that I may look at them for reference? This is sometimes easier for me than just looking at the XSD. Thank you! |
Sure! git clone https://github.com/openclover/clover-examples.git
Here's the content of the clover.xml file generated: <?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1747839983472" clover="4.4.1">
<project name="Spock Framework - Example Project 1.0" timestamp="1747839978763">
<metrics coveredelements="0" complexity="0" loc="3" methods="0" classes="1" statements="0" packages="1" coveredconditionals="0" coveredmethods="0" elements="0" ncloc="0" files="1" conditionals="0" coveredstatements="0"/>
<package name="default-pkg">
<metrics coveredelements="0" complexity="0" loc="3" methods="0" classes="1" statements="0" coveredconditionals="0" coveredmethods="0" elements="0" ncloc="0" files="1" conditionals="0" coveredstatements="0"/>
<file path="/Volumes/DATA/_PROJEKTY/github/clover-examples/spock-example/src/main/groovy/Foo.groovy" name="Foo.groovy">
<metrics coveredelements="0" coveredconditionals="0" complexity="0" loc="3" coveredmethods="0" methods="0" elements="0" classes="1" ncloc="0" statements="0" conditionals="0" coveredstatements="0"/>
<class name="Foo">
<metrics coveredelements="0" coveredconditionals="0" complexity="0" coveredmethods="0" methods="0" elements="0" statements="0" conditionals="0" coveredstatements="0"/>
</class>
</file>
</package>
</project>
<testproject name="Spock Framework - Example Project 1.0" timestamp="1747839978763">
<metrics coveredelements="83" complexity="31" loc="298" methods="29" classes="10" statements="52" packages="1" coveredconditionals="2" coveredmethods="29" elements="83" ncloc="0" files="6" conditionals="2" coveredstatements="52"/>
<package name="default-pkg">
<metrics coveredelements="83" complexity="31" loc="298" methods="29" classes="10" statements="52" coveredconditionals="2" coveredmethods="29" elements="83" ncloc="0" files="6" conditionals="2" coveredstatements="52"/>
<file path="/Volumes/DATA/_PROJEKTY/github/clover-examples/spock-example/src/test/groovy/UnrollWithSeqNumber.groovy" name="UnrollWithSeqNumber.groovy">
<metrics coveredelements="3" coveredconditionals="0" complexity="1" loc="33" coveredmethods="1" methods="1" elements="3" classes="1" ncloc="0" statements="2" conditionals="0" coveredstatements="2"/>
<class name="UnrollWithSeqNumber">
<metrics coveredelements="3" complexity="1" methods="1" testruns="3" statements="2" coveredconditionals="0" coveredmethods="1" elements="3" testfailures="0" testduration="0.002" conditionals="0" coveredstatements="2" testpasses="3"/>
</class>
<line complexity="1" visibility="public" signature="$spock_feature_0_0(def,def,def) : void" num="22" count="3" type="method"/>
<line num="24" count="3" type="stmt"/>
<line num="25" count="3" type="stmt"/>
</file>
<file path="/Volumes/DATA/_PROJEKTY/github/clover-examples/spock-example/src/test/groovy/UnrollWithVarsWithSelectors.groovy" name="UnrollWithVarsWithSelectors.groovy">
<metrics coveredelements="7" coveredconditionals="2" complexity="4" loc="39" coveredmethods="2" methods="2" elements="7" classes="2" ncloc="0" statements="3" conditionals="2" coveredstatements="3"/>
<class name="UnrollWithVarsWithSelectors">
<metrics coveredelements="3" complexity="1" methods="1" testruns="2" statements="2" coveredconditionals="0" coveredmethods="1" elements="3" testfailures="0" testduration="0.003" conditionals="0" coveredstatements="2" testpasses="2"/>
</class>
<class name="UnrollWithVarsWithSelectors$Person">
<metrics coveredelements="4" coveredconditionals="2" complexity="3" coveredmethods="1" methods="1" elements="4" statements="1" conditionals="2" coveredstatements="1"/>
</class>
<line complexity="3" visibility="public" signature="getSex() : String" num="24" count="4" type="method"/>
<line num="25" count="4" type="stmt"/>
<line falsecount="2" truecount="2" num="25" type="cond"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_0(def,def) : void" num="29" count="2" type="method"/>
<line num="31" count="2" type="stmt"/>
<line num="32" count="2" type="stmt"/>
</file>
<file path="/Volumes/DATA/_PROJEKTY/github/clover-examples/spock-example/src/test/groovy/UnrollWithSimpleVar.groovy" name="UnrollWithSimpleVar.groovy">
<metrics coveredelements="3" coveredconditionals="0" complexity="1" loc="34" coveredmethods="1" methods="1" elements="3" classes="1" ncloc="0" statements="2" conditionals="0" coveredstatements="2"/>
<class name="UnrollWithSimpleVar">
<metrics coveredelements="3" complexity="1" methods="1" testruns="3" statements="2" coveredconditionals="0" coveredmethods="1" elements="3" testfailures="0" testduration="0.009" conditionals="0" coveredstatements="2" testpasses="3"/>
</class>
<line complexity="1" visibility="public" signature="$spock_feature_0_0(def,def,def) : void" num="22" count="3" type="method"/>
<line num="24" count="3" type="stmt"/>
<line num="25" count="3" type="stmt"/>
</file>
<file path="/Volumes/DATA/_PROJEKTY/github/clover-examples/spock-example/src/test/groovy/Stacks.groovy" name="Stacks.groovy">
<metrics coveredelements="47" coveredconditionals="0" complexity="14" loc="120" coveredmethods="14" methods="14" elements="47" classes="3" ncloc="0" statements="33" conditionals="0" coveredstatements="33"/>
<class name="EmptyStack">
<metrics coveredelements="12" complexity="4" methods="4" testruns="4" statements="8" coveredconditionals="0" coveredmethods="4" elements="12" testfailures="0" testduration="0.013999999999999999" conditionals="0" coveredstatements="8" testpasses="4"/>
</class>
<class name="StackWithOneElement">
<metrics coveredelements="16" complexity="5" methods="5" testruns="4" statements="11" coveredconditionals="0" coveredmethods="5" elements="16" testfailures="0" testduration="0.001" conditionals="0" coveredstatements="11" testpasses="4"/>
</class>
<class name="StackWithThreeElements">
<metrics coveredelements="19" complexity="5" methods="5" testruns="4" statements="14" coveredconditionals="0" coveredmethods="5" elements="19" testfailures="0" testduration="0.002" conditionals="0" coveredstatements="14" testpasses="4"/>
</class>
<line complexity="1" visibility="public" signature="$spock_feature_0_0() : void" num="22" count="1" type="method"/>
<line num="23" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_1() : void" num="26" count="1" type="method"/>
<line num="27" count="1" type="stmt"/>
<line num="28" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_2() : void" num="31" count="1" type="method"/>
<line num="32" count="1" type="stmt"/>
<line num="33" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_3() : void" num="36" count="1" type="method"/>
<line num="38" count="1" type="stmt"/>
<line num="41" count="1" type="stmt"/>
<line num="42" count="1" type="stmt"/>
<line complexity="1" visibility="private" signature="setup() : def" num="49" count="4" type="method"/>
<line num="50" count="4" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_0() : void" num="53" count="1" type="method"/>
<line num="54" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_1() : void" num="57" count="1" type="method"/>
<line num="59" count="1" type="stmt"/>
<line num="62" count="1" type="stmt"/>
<line num="63" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_2() : void" num="66" count="1" type="method"/>
<line num="68" count="1" type="stmt"/>
<line num="71" count="1" type="stmt"/>
<line num="72" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_3() : void" num="75" count="1" type="method"/>
<line num="77" count="1" type="stmt"/>
<line num="80" count="1" type="stmt"/>
<line num="81" count="1" type="stmt"/>
<line complexity="1" visibility="private" signature="setup() : def" num="88" count="4" type="method"/>
<line num="89" count="12" type="stmt"/>
<line num="89" count="4" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_0() : void" num="92" count="1" type="method"/>
<line num="93" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_1() : void" num="96" count="1" type="method"/>
<line num="98" count="1" type="stmt"/>
<line num="99" count="1" type="stmt"/>
<line num="100" count="1" type="stmt"/>
<line num="101" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_2() : void" num="104" count="1" type="method"/>
<line num="106" count="1" type="stmt"/>
<line num="107" count="1" type="stmt"/>
<line num="108" count="1" type="stmt"/>
<line num="109" count="1" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_3() : void" num="112" count="1" type="method"/>
<line num="114" count="1" type="stmt"/>
<line num="117" count="1" type="stmt"/>
<line num="118" count="1" type="stmt"/>
</file>
<file path="/Volumes/DATA/_PROJEKTY/github/clover-examples/spock-example/src/test/groovy/HelloSpock.groovy" name="HelloSpock.groovy">
<metrics coveredelements="3" coveredconditionals="0" complexity="1" loc="31" coveredmethods="1" methods="1" elements="3" classes="1" ncloc="0" statements="2" conditionals="0" coveredstatements="2"/>
<class name="HelloSpock">
<metrics coveredelements="3" complexity="1" methods="1" testruns="3" statements="2" coveredconditionals="0" coveredmethods="1" elements="3" testfailures="0" testduration="0.019" conditionals="0" coveredstatements="2" testpasses="3"/>
</class>
<line complexity="1" visibility="public" signature="$spock_feature_0_0(def,def) : void" num="20" count="3" type="method"/>
<line num="22" count="3" type="stmt"/>
<line num="23" count="3" type="stmt"/>
</file>
<file path="/Volumes/DATA/_PROJEKTY/github/clover-examples/spock-example/src/test/groovy/SpecInheritance.groovy" name="SpecInheritance.groovy">
<metrics coveredelements="20" coveredconditionals="0" complexity="10" loc="41" coveredmethods="10" methods="10" elements="20" classes="2" ncloc="0" statements="10" conditionals="0" coveredstatements="10"/>
<class name="BaseSpec">
<metrics coveredelements="10" coveredconditionals="0" complexity="5" coveredmethods="5" methods="5" elements="10" statements="5" conditionals="0" coveredstatements="5"/>
</class>
<class name="DerivedSpec">
<metrics coveredelements="10" complexity="5" methods="5" testruns="2" statements="5" coveredconditionals="0" coveredmethods="5" elements="10" testfailures="0" testduration="0.002" conditionals="0" coveredstatements="5" testpasses="2"/>
</class>
<line complexity="1" visibility="private" signature="setupSpec() : def" num="22" count="1" type="method"/>
<line num="22" count="1" type="stmt"/>
<line complexity="1" visibility="private" signature="cleanupSpec() : def" num="23" count="1" type="method"/>
<line num="23" count="1" type="stmt"/>
<line complexity="1" visibility="private" signature="setup() : def" num="25" count="2" type="method"/>
<line num="25" count="2" type="stmt"/>
<line complexity="1" visibility="private" signature="cleanup() : def" num="26" count="2" type="method"/>
<line num="26" count="2" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_0_0() : void" num="28" count="1" type="method"/>
<line num="28" count="1" type="stmt"/>
<line complexity="1" visibility="private" signature="setupSpec() : def" num="34" count="1" type="method"/>
<line num="34" count="1" type="stmt"/>
<line complexity="1" visibility="private" signature="cleanupSpec() : def" num="35" count="1" type="method"/>
<line num="35" count="1" type="stmt"/>
<line complexity="1" visibility="private" signature="setup() : def" num="37" count="2" type="method"/>
<line num="37" count="2" type="stmt"/>
<line complexity="1" visibility="private" signature="cleanup() : def" num="38" count="2" type="method"/>
<line num="38" count="2" type="stmt"/>
<line complexity="1" visibility="public" signature="$spock_feature_1_0() : void" num="40" count="1" type="method"/>
<line num="40" count="1" type="stmt"/>
</file>
</package>
</testproject>
</coverage> |
Thank you so much! |
@marek-parfianowicz I do not understand the purpose of the |
Hi Sebastian! Yes, in OpenClover you can separate code into application and test. The tag contains coverage information for application code, the tag for test. Collecting coverage information for test code is very useful in OpenClover, because then it instruments tests allowing to measure per-test code coverage as well as record test execution time and results. Which is later being used in reports, especially the HTML one. |
With the current state of the @RykHawthorn @jimdelois @vnrmc @amottier @ravage84 Sorry to bother you with a mass-ping, but you commented on #578 and/or #1079 and I would like to ask for your help and advice. Can you please have a look at the Clover XML that is generated on the |
Hint: Atlassian Bamboo offers integration with Clover/OpenClover. If you select the manual clover integration option, it should be possible to trick Bamboo to show a report generated by CloverPHP on the build results tab. This would be a good test, in my opinion. Also, the Jenkins has CloverPHP plugin as far as I recall. It could be worth checking it too. https://openclover.org/doc/manual/latest/hacking--using-openclover-for-php.html https://confluence.atlassian.com/bamboo/enabling-clover-for-bamboo-289277268.html It should be possible to see something like this (in Bamboo): |
Here's an idea: we could keep the existing In the context of PHPUnit, we could then introduce a configuration option for opting into using |
Clover's schema didn't change for years. I doubt that there's anyone using older version of Clover than 4.5. |
The question is: how many years? :) However, the code that generates code coverage information in Clover XML in the php-code-coverage dependency of PHPUnit is almost 20 years old. Over the years, it has only been adapted to the changing internals of PHPUnit / php-code-coverage. Maybe it never generated valid Clover XML, maybe the schema evolved, I do not know. I am happy, though, that we are (hopefully) getting out of this messy situation now. |
Yeah, that code goes back to 14.11.2009. The implementation back then hints at that Clover back then didn't have any official schema documentation.
0c8db36#diff-0e105740559ef4553f3ff8c3e5e6dc0db0eb9df4e5b8142a36b5aa958f372b81R49-R51
Since it isn't |
I'm in touch with the maintainer of the Jenkins plugin and use that plugin myself. |
Even better, thank you for this idea. I have updated this PR to keep the old |
…phpunit/php-code-coverage
- Set complexity attribute on the <metrics> element under <file> - Generate <metrics> element as first child of <package> element
…rter that generates Clover XML that validates against OpenClover's XML schema definition
This has been released in Support for this has been implemented for PHPUnit 12.2 which will be released on June 6, 2025. |
As discussed in #578 and #1079, the Clover XML generated by php-code-coverage does not validate against Clover's XML schema definition.
We should change the generated Clover XML so that it does validate against the Clover XSD.
clover
attribute of<coverage>
element to version identifier for phpunit/php-code-coverage<metrics>
element the first child of<project>
and<file>
name
andpath
attributes on the<file>
element<package>
element for global namespacecomplexity
attribute on the<metrics>
element under<project>
complexity
attribute on the<metrics>
element under<file>
name
of<class>
element to unqualified class namenamespace
attribute from<class>
elementsignature
attribute on the<line>
elementname
andcrap
attributes from<line>
element<testproject>
element (see Add OpenClover XML reporter #1080 (comment))Here is an example of a Clover XML file genereated using the changes proposed by this PR: https://gist.github.com/sebastianbergmann/2373d029e8972e038929f717c5ffa868