Testing PVR firmware

At the moment, I’m working on a roundup of Freeview HD PVRs; while I do that, I thought I’d share some information about Topfield PVRs. The TF5800 was one of the first twin tuner Freeview recorders in the UK, and when it launched in 2005, I had one of the first models.

The amount of interest that built up around it was quite considerable – enough for me to set up the Toppy.org.uk web site, and through that a nascent interest in digital television grew much stronger.

Those who’ve persisted with their Toppy since the early days will remember some of the truly, shockingly, awful firmware updates that we had – though to be fair to Topfield’s engineers, they’re far from unique in shoving out PVR firmware that has serious problems. Some of the current crop of Freeview HD kit has arrived on store shelves with sluggish response, inability to play back sound on some recordings, and so on.

So, one of the things that we did on the Toppy web site was set up an extended beta testing scheme, working with Turbosat, the UK distributor, to do our own testing of how the units performed, both to check some of the bugs that we’d experienced in previous firmwares hadn’t reappeared, and to verify some of the things in operation that, frankly, it didn’t look like the Topfield tests included.

Test documents

For those who are interested in these things then, the two PDF documents attached to this posting are part of the unofficial test scheme that we devised. Before going any further, you should note that they’ve not been updated since 2008, and some of the tests will be applicable only to the Topfield models, though the bulk are worth checking on any twin tuner recorder, to see how well it performs.

channelmap2008

This first document, the ‘channel map’ is simply to help people select the appropriate channel needed for testing. It’s not terribly up to date, but you’ll get the general idea; I’ll try to find time to do a revised version when I can.

The second document is the test scheme itself; testers work through this – and it can take hours to do in full – and record the results on the web site; we can then see which things are always failing, which might be intermittent, and so on, and pass more feedback to the engineers. Some of it may look dull and tedious, but if you want to be sure that a PVR is working properly you really do need to check these things.

It’s worth bearing in mind, too, that the scheme doesn’t include all the possible tests, especially with the launch of Freeview+ and things such as series link, which would require a whole new set of tests, to determine how well automatic recordings and scheduling work.

If you think testing a digital TV recorder thoroughly is a matter of just setting a few timers, and sitting back, think again. And if you have a recorder of your own, and fancy working through some of the tests below – the notes should help you – let me know how you get on. If there’s enough interest, perhaps I’ll create a generic version of these PVR tests that people can use to score any brand of recorder.

TestScheme-0.4

Notes on the tests

To help you understand the tests, here are some notes which explain some of the Topfield-specific elements.

• 1.3 On the TF5800, you can key in a percentage on the remote to jump to any part of the recording
• 1.4 An early bug caused the memory to fill up as the unit reloaded the BBC interactive application
• 2.3 If you use the remote when the TF5800 has woken up to record, it should stay on afterwards, as it thinks you’re watching TV
• 2.7,2.8 Unlike some recorders, the TF5800 is very flexible about which channels you can watch, even with two recordings running
• 2.9 The bug mentioned here would cause a recording to change channel part way through, as the TF5800 ‘stole’ a tuner to allow you to watch live TV
• 2.10 When a channel is off air, it displays a static text screen; sometimes, recordings that start then fail to pick up the video stream correctly
• 2.14-17 A bug in tuner allocation on the TF5800 in some firmwares meant that, even if there was a spare tuner, sometimes the box would change channels on the one you were watching, or interrupt a playback unnecessarily
• 2.24-27 The Topfield units can save a Pay TV channel in unencrypted form, either directly, or by using the ‘copy’ function
• 6.1 A bug in some firmwares meant that identically named files, in different folders, would be mistaken for each other
• 6.5 The TF5800 includes some games in the firmware
• 6.6 Some firmwares would forget your selection of favourites when switching between TV and radio modes
• 6.7-9 Some firmwares would not play back TV programmes correctly when in radio mode