Re: [Tails-dev] Automated tests specification

On 09/02/2015 01:12 PM, intrigeri wrote:
> hi,
>
> anonym wrote (02 Sep 2015 09:11:43 GMT) :
>> I got a video that was 309 MiB large [...]
>
> Assuming (very wild guess) that the size gain brought by the proposed
> changes from #10001 can be extrapolated, this could become 117 MiB.
> If that's mostly correct, then with an aggressive enough cleaning
> policy (e.g. delete those that are more than 7 days old), I *guess*
> that we can very well afford storing those. I guess that our existing
> artifacts cleanup script could easily be adjusted to support expiring
> videos as well (note that we may need different expiration policies
> for videos and for other test suite artifacts, that we may want to
> keep longer; not sure yet; needs to be refined).

Great!

>> It's interesting to note that this runtime is not different from what I
>> normally get when running this branch on the isotesters (I said "~350
>> minutes" earlier in this thread),
>
> Excellent news: this means that the runtime increase concern is not
> valid on lizard -- one less blocker for archiving videos.
> bertagaz also confirmed that archiving them is not much more work for
> him, so we should be good.

I just updated the ticket with more thorough test results. It seems it
*does* increase the runtime with 17% (old video compression) vs 5.8% (new).

The run I talked about above was just one data point, after all, so it's
not entirely surprising if I got outlying results. I'm running another
run without --capture now for comparison. However, there are so many
other variables that can make the runtime differ (transient errors, Tor
bootstrapping issues that make it take a longer time (which seems to
cluster in time for some reasons)) so I'm not sure what that will be run.

Any way, I trust my test results on #10001.

>> which makes me wonder if the main concern of #10001, that --capture
>> increases the runtime (even when enough CPU is available), is
>> actually valid. I'm gonna investigate that now...
>
> Well, it *is* valid at least on my system. However, it could depend on
> hardware, and/or on the version of the video codec libraries (I did my
> tests on sid, which is useful info because that gives us an idea of
> what software we'll run on our infra in ~2 years).

Right. I guess we need longterm statistics on the isotesters to be able
to see any meaningful numbers.

Cheers!