HTML formatter

(no need for Samba for this to be in perl btw)

What would it look like? Whipping up something trivial in simpletal should be a 10 minute job, if you can sketch out what you want the output to look like. Or possibly better yet, write a separate project formatter like I did in python-junitxml (and I'd be happy to write that for you).

http://tungwaiyip.info/software/sample_test_report.html is an example output from http://tungwaiyip.info/software/HTMLTestRunner.html, which sadly doesn't obey the unittest protocol - it fiddles stdout/stderr rather than depending on the TestResult protocol.

I'm dropping a mail to TIP in case someone there has an existing result object that would suite.

Another option methoned on the list TIP list is to use an xslt transform + subunit2junitxml.

e.g.
make check | subunit2junitxml -o results.xml
xslt --whatever < results.xml > foo.html

Oh, and another conceptually cool way would be to just write a javascript parser for subunit; then render just-in-time.

Actually, the output from http://tungwaiyip.info/software/HTMLTestRunner.html would be sortof the thing we're looking for.

What I would like to replicate is something that works at least as well as this:

http://build.samba.org/?function=View+Build;host=bnhtest;tree=samba_4_0_test;compiler=gcc

but preferably something nicer and with more overview.

On Sun, 2009-09-27 at 14:22 +0000, Jelmer Vernooij wrote:
> Actually, the output from
> http://tungwaiyip.info/software/HTMLTestRunner.html would be sortof the
> thing we're looking for.
>
> What I would like to replicate is something that works at least as well
> as this:
>
> http://build.samba.org/?function=View+Build;host=bnhtest;tree=samba_4_0_test;compiler=gcc
>
> but preferably something nicer and with more overview.

So, basically this looks like:
 - pick a template engine
 - pick a JS UI library
 - write your page

The glue code required is quite trivial - you can see how small
pyjunitxml is.

-Rob

On Sun, 2009-09-27 at 22:37 +0000, Robert Collins wrote:
> On Sun, 2009-09-27 at 14:22 +0000, Jelmer Vernooij wrote:
> > Actually, the output from
> > http://tungwaiyip.info/software/HTMLTestRunner.html would be sortof the
> > thing we're looking for.
> >
> > What I would like to replicate is something that works at least as well
> > as this:
> >
> > http://build.samba.org/?function=View+Build;host=bnhtest;tree=samba_4_0_test;compiler=gcc
> >
> > but preferably something nicer and with more overview.
>
> So, basically this looks like:
> - pick a template engine
> - pick a JS UI library
> - write your page
>
> The glue code required is quite trivial - you can see how small
> pyjunitxml is.
I'll have a look at this next time I play with the Samba buildfarm.

One of the next steps would be to display HTML with statistics about
these various tests over different test runs. This might need a helper
Python module that's not specific to HTML though.

Some examples of things this could display would be: what tests are
flapping, what tests fail across all buildfarm hosts (as opposed to one
that might be broken), what tests break regularly (and might be good to
run first or as part of "make quicktest"), etc.

Cheers,

Jelmer

--
Jelmer Vernooij <email address hidden> - http://samba.org/~jelmer/
Jabber: <email address hidden>

So to do flapping needs historical view - thats more a test repository problem than simply 'format a single test result'.

I agree that that is good to have, and one advantage of subunit is the ability to use a stock python based test repository. I'm open to writing a dedicated one if thats what you need, but I suspect that it isn't a subunit problem, per se.

So, I've written a stock python based test repository. Someone needs to write/suck in the HTML formatting.

OpenStack has one of these, would it perhaps be an option to just ship that with subunit?

https://docs.openstack.org/developer/os-testr/subunit2html.html