Old friend ISO Testing Tracker
This post is going to be all about the ISO Testing Tracker. You might know it already, it’s been with us for a while. But, like old friends, sometimes you need a reminder that they are there and give them a hug. But before we continue with the tracker, let’s start with the basics:
What’s ISO Testing?
If you are reading this, you would probably know that Ubuntu has a 6-month release cycle, which means that every 6 months we have a new Ubuntu release, ready to install.
The good thing about software fixed cycles is that you can schedule anything you want. In fact, for Ubuntu, we schedule just at the beginning of the cycle up to 8 development releases. You can check at the Karmic schedule that there was an Alpha 1, Alpha 2, Alpha 3, Alpha 4, Alpha 5 and Alpha 6. And that there’s going to be a Beta and a Release Candidate before the “real” Ubuntu 9.10.
Those milestones need to be tested properly, because we release CDs for people to test on their hardware, and we have to make sure that everything is more or less OK. (with “more or less” I mean that these milestones are still software in development, so you have to be careful, anyway).
How do we track milestone testing?
So, here it is how the tracker appears in the show. Milestone releases happen normally on Thursdays. On Monday, in scheduled milestone weeks, testers start looking at the tracker looking for the candidate images: the ISOs that might be announced as the “Karmic Alpha something”. But these images need to be tested before they get announced.
How is the test tracker structured?
At Ubuntu, we produce builds for many different *buntu flavours. In the ISO Tracker we have a category for each of them. You can filter by category using the “Filter” menu in the bottom left of the page. That would allow you to concentrate in the flavour you want to test.
Once you select a category, you will be able to select an image to download for that category. A category like “Kubuntu” may have different images: i386 Live CD, amd64 alternate, etc., and all of them need to be tested before releasing a milestone.
If you click now in any of those images, you will have access to a direct link of the ISO to download and a selection of tests that you may run for that image. For every test, there is a link to the test case description in the test case wiki. The test cases are easy to follow when running any of them.
So, to sum up, every category contains a set of ISO images and every image contains at least one test case.
How do I help?
As I wrote before, the week of the release of a milestone we need a lot of help testing the candidate images. If we want full coverage we need to run more than 150 test cases. If you have a spare machine or a virtual machine installed, you can help making Ubuntu better by running the untested test cases during the week of the release of a milestone.
First thing, you will need an account in the ISO tracker. Unfortunately, your Launchpad Open ID won’t work for this site (indeed something that needs to be improved). Click on “Log in” and fill the requested data. As the page explains, if you already have an account in Ubuntu Brainstorm, you can use it for the ISO Testing Tracker.
Once you have an account just select the image you want to test, choose a test case for that image, start running the installation in a virtual machine or a spare machine and report any bugs you may find in Launchpad.
Once finished, you have to report back in the ISO Testing Tracker. Click on the test case you have run and report Pass/Fail depending on your success. You can also add the bug numbers of the bugs you found. They will be linked to the Launchpad bug and it will help the release team to track them.
The time is NOW!
Next week, on Thursday 1st of October, we are releasing Ubuntu Karmic Beta, which eventually will become Ubuntu 9.10. This is a very important milestone, because many people upgrade during the beta, so it is very important to release something very stable at this point. From Monday, 27th of September, we will be testing the candidate images at #ubuntu-testing. I will send reminders to the different mailing lists later this week and at the beginning of next week, but you can start right now creating your account and making yourself familiar with the ISO Testing Tracker.
Help us releasing Ubuntu better and better every 6 months! And remember that we are always at #ubuntu-testing if you need to ask any question!
Thanks for explaining this. It is very informative for those who want to help.
Victor
September 21, 2009 at 5:00 pm
[…] more from the original source: Ara Pulido: Old friend ISO Testing Tracker Share and […]
Ara Pulido: Old friend ISO Testing Tracker | TuxWire : The Linux Blog
September 22, 2009 at 4:13 pm
Can I suggest a couple of new tests for the livecd? These were issues I found running the Karmic alpha 6.
‘text’ mode no longer works : https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/434769
‘integrity-check’ doesn’t present an interface any more : https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/434769
Michael Fletcher
September 23, 2009 at 2:13 pm
[…] don’t know how to use it? It is pretty easy! […]
Karmic Beta Testing « Let’s test Ubuntu NOW!
September 29, 2009 at 11:18 am
the test tracker does not filter the images when i select ‘desktop’ – it still shows all the images
not important, but probably easy to fix
andrew u frank
October 1, 2009 at 3:49 pm
Thanks for the feedback!!
Can you file a bug in the project page? https://bugs.edge.launchpad.net/ubuntu-qa-website/+bugs
Ara Pulido
October 6, 2009 at 10:19 am
may i also suggest some test cases:
it is a goal that ubuntu installs on tablet pc – this should be included as a test case (and is hopefully tested by users with a variety of hardware).
i will do it (for two different tablets) – and would love to report about it.
andrew u frank
October 1, 2009 at 3:53 pm
[…] with instructions to follow, in the ISO tracker. If you don’t know how to use the tracker, this blog post will serve as starting […]
Let’s test Karmic RC! « Let’s test Ubuntu NOW!
October 19, 2009 at 9:02 am
[…] will be spending next week testing the ISOs and coordinating efforts in #ubuntu-testing Freenode IRC […]
Lucid Alpha 1 is coming to town! « Let’s test Ubuntu NOW!
December 4, 2009 at 12:58 pm
[…] feeling like you want to contribute, there is something you can do right this minute – help test the ISOs before the […]
Ubuntu Lucid Lynx Alpha 1 released 10 December « Interesting
December 4, 2009 at 11:41 pm
[…] will be spending next week testing the ISOs and coordinating efforts in #ubuntu-testing Freenode IRC […]
Lucid Alpha 1 is coming to town! :Addz It Now
December 7, 2009 at 3:09 am
[…] first thing you’ll need is an account at the ISO Tracker, which will allow you to record your tests and their […]
Help localization testing with the ISO tracker « Benvinguts, passeu passeu
February 24, 2010 at 9:10 am
[…] Testing and Reporting ISO’s on QATracker: https://ubuntutesting.wordpress.com/2009/09/21/old-friend-iso-testing-tracker/ […]
Open Source Musician Podcast Episode #38 – Interview with Scott Lavender of Ubuntu Studio « Guitar Emporium Online
May 15, 2010 at 7:23 pm