Re: [Tails-project] Live Demo proposal: Blackbox automated t…

Delete this message

Reply to this message
Author: bertagaz
Date:  
To: Public mailing list about the Tails project
Subject: Re: [Tails-project] Live Demo proposal: Blackbox automated testing of operating systems: how Tails does it
Hi,

On Sat, Aug 08, 2015 at 11:42:20PM +0200, intrigeri wrote:
>
> bertagaz wrote (08 Aug 2015 19:08:01 GMT) :
> > the dpkg trigger bug could have been found far more sooner with such
> > a tool,
>
> In passing, piuparts is way better suited for this task, and
> https://piuparts.debian.org/ is amazing.
>
> Still in passing: in my experience, such bugs, that happen only
> occasionally / in specific contexts, are hard to identify with an
> automated test suite, for a number of reasons including the fact that
> it's always tempting to call it a false positive, transient error, you
> name it; and the fact that test suites run in controlled environment
> (which is usually seen as a good thing) tend to not expose crazy race
> conditions that only happen under load, or with a specific combination
> of packages, etc. The fact that we currently run our own test suite on
> various systems, some of them being "production" environments as well,
> helps us on the latter. The fact that it's been historically brittle
> and prone to transient errors hasn't been helping us on the former...
> but thankfully, this has changed a lot recently :)
>
> > as well as having functional graphical upgrade in Debian...
>
> ACK!


Sure, I think I melt down the two problems because the first one was the
one that prevented the second to be possible.

bert.