Ubuntu
Chris Cheney posed an interesting problem - how to get a listing of currently opened Firefox windows.
You can get this info from sessionstore.js in your firefox profile, but it's a jumble of javascript. I wrote a script ff-pages to display them in a more useful way.
Example usage:
$ ff-pages .mozilla/firefox/*.default/sessionstore.js
Window 0:
http://shoutcast.net/directory/index.phtml
http://www.radioparadise.com
http://people.ubuntu.com/~bryce/Plots/
file:///var/www/X/Graphs
Window 1:
https://bugs.launchpad.net/ubuntu/+bug/379797
https://bugs.freedesktop.org/show_bug.cgi?id=21315
Window 2:
https://bugs.launchpad.net/ubuntu/+bug/360319
https://bugs.launchpad.net/ubuntu/+bug/376092
Window 3:
https://bugs.freedesktop.org/show_bug.cgi?id=21756
https://bugs.freedesktop.org/show_bug.cgi?id=16702
...
Posted in Ubuntu Submitted by bryce on Thu, 2009-07-02 02:22.
bryce's blog | 5 comments
Yesterday I wrote about me-too storms, including how they develop and some of the problems they can cause (most notably - delaying the bug's solution).
Since this is a bad problem there's been quite a few ideas proposed.
FIXING the bugs faster would be the best thing of course. That's a whole other topic... but does deserves first mention.
Launchpad already has a solution in the form of the "This bug doesn't affect me (change)" text and link on every bug.
Presumably this is serving as a lightning rod saving us from heaps of me-too comments. And presumably the data is being stored somewhere to help developers identify highly popular bugs, but I don't know how to access that info. I would imagine if developers could filter/sort/view this data it'd give added incentive for people to use that and it would be a more effective tool.
Another issue is that the link just isn't that noticeable. I'll leave it to usability experts to work out how best to improve it, but it definitely needs a re-think so it's a bit more obvious, especially for casual launchpad visitors.
Educating bug reporters is another oft-suggested solution. I think there's some truism about documentation being the worst way to explain how software works, because no one will read it. Actually there's been reams and reams of discussion and documentation about how to write and behave on bug reports, and indeed I think this is really helpful as it can turn bad bug reporters into really good bug reporters. That said, many people are just too busy, or too lazy, or don't even know they don't know how to report bugs.
Hiding comments or otherwise emphasizing/de-emphasizing comments by various criteria has been suggested and I believe is being considered by the launchpad crew. The trick is to do this in a way that doesn't end up actually consuming more time to manage than it saves. I think a lot of useful experimentation could be done on this idea using greasemonkey, as Brian Murray has done.
Splitting bug reports would be pretty cool feature, at least in theory. If we realize that John and Bob have different issues, it'd be slick to be able to push a button and have all of Bob's comments moved to its own bug report. But in practice I suspect this could result in creating more confusion by losing context. In any case, I don't think it'd help with me-too storms since most of the low value comments really aren't worth setting up a separate bug for; better to just have the people file new bugs from scratch.
Locking bug reports, is sometimes suggested by those familiar with similar functionality provided by forums software.
Minimum karma requirements to post onto other people's bugs is an idea I've been kicking around for a while. I've noticed (thanks to Kees Cook's lp_karma_suffix greasemonkey script) a very strong correlation that the worst me-too'ers also tend to have the lowest karma. People with the most helpful comments usually have higher karma.
So why not make use of that by requiring posters to have a certain karma level before being able to comment on someone else's bug report? From what I can tell, it needn't be high; 50 points would be enough to weed out the vast majority of the noise, but still be low enough that anyone with a legitimate interest in helping on bug reports could earn with a fairly minimal amount of work, such as helping on answers.ubuntu.com or going through the full bug process following up on their own bug reports.
Perhaps it could be set up to kick in only after there are already a few commenters, so we don't get in the way of random helpers on bugs that would otherwise be neglected. In a way, this would be akin to the "locking" idea above, but a bit more targeted.
Other thoughts?
I deal with a lot of X.org bugs in launchpad, and I often run into an intriguing phenomenon, the so-called "me-too storm".
Essentially, a me-too storm is a bug report which accumulates a large number of confirmation statements from different people, to the point that it actually hinders progress.
Knowing that a bug affects additional people does have some value to it. I particularly appreciate it when someone of a technical bent is able to reproduce a problem, because I can have confidence that they can test out patches or at least provide insightful technical analysis.
However, past maybe half a dozen comments, the value of an additional confirmation drops off to zero. So, comments in a healthy bug report would shift from confirmatory statements towards comparing data, discussing workarounds, identifying test cases, proposing patches, and so on.
Unfortunately, with a me-too storm, the confirmation statements come in at a faster clip, and their quality often drops further. Commonly, the confirmer will not provide log files or other proof that they do indeed have the bug. This is a particular problem with X because often there are different unrelated bugs that have identical symptoms (examples include X freezes, black screens, performance degradation), and so they might have a different bug; by "me-too'ing" their bug instead of creating their own bug report, it means their issue probably won't be investigated, or it can cause massive confusion or even sidetrack the discussion away from the original bug.
The bug suffers from a bad feedback loop at this point. The more comments a bug has, the more "important" it looks to launchpad's search engine, so it gets suggested to anyone with vaguely similar symptoms. New commenters notice that some old commenters provided data already so they don't bother putting it in.
Indeed, after a while the bug report can start accumulating negative comments, that have a value below zero - they actually detract from the discussion. These range from demanding "is it fixed yet?" questions to rude, inflammatory or borderline threatening comments "fix it now or I go back to windows, you insensitive clod!!!1!" These predictably stir up a variety of 0-value follow on comments, "No, it's not fixed yet, please be patient," "yeah +1 for fixing this soon," "did you try rebooting? it helped me," "plz unsub me from this list, to many emailz," "'doze sucks, get a mac," and so on.
Now, no one would argue that "me-too storm" bugs are not important. Obviously with so many people commenting, there must be some real problem that needs dealt with. It could be a real bug, or a class of bugs with similar symptoms, or a usability issue, or even just poorly set expectations...
However, the stormy nature of these high comment bugs tends to work against themselves, for a few reasons. First, as a developer it's just plain time consuming to read through a gazillion comments. Second, these bugs can be hard to summarize and send upstream, particularly if analysis data is coming from different people (with perhaps differing hardware). Third, if you post a proposed solution, you often get feedback from people having unrelated bugs that leads you to make erroneous conclusions about the fix.
The "me-too storm" is a fascinating phenomenon, but since it hinders bug solution it is interesting to consider ways that this could be prevented or mitigated in launchpad. I'll share my own thoughts in a future post. Meanwhile, I'd love to hear other's ideas.
The xorg-edgers Personal Package Archive (PPA) has been proving itself in battle for Karmic.
PPAs can be thought of as basically a "build service", however they also include dependency info that lets you deploy a set of interdependent packages. This feature is extremely handy for X.org since it is not unusual for a package to require a new mesa, libdrm, or kernel update. PPAs provide a package-system compatible way to let users install a package and all its dependencies in one go.
xorg-edgers was originally set up by Tormod Volden a couple years ago to make it easier for bug reporters to test git snapshots of the -ati and -radeonhd drivers. These drivers were not putting out regular releases, but were very active at committing fixes for problems we reported, so xorg-edgers gave a way to both enable users to easily validate the fixes, and as a way to test-drive a particular snapshot before uploading it to Ubuntu proper.
Several of us noticed how useful this approach was, and assisted Tormod. One key was to turn the procedure of packaging new git snapshots into a script, and document the procedure so others could participate more easily.
Over time, xorg-edgers expanded to include more than just ATI drivers. Today it encompasses the major video drivers, the x-server itself, and all manner of various dependency libraries.
Since xorg-edgers is set up as a team PPA, the overhead of getting access to uploading to it is much lower than to upload X.org bits to Ubuntu main. This has paid off recently as the team gained a new member, Robert 'Sarvatt' Hooker, who has dug into the tough work of sorting out KMS packaging issues with Tormod and myself, and has been using the git snapshot scripts to keep packages up to date.
One thing I really like about how xorg-edgers is set up is that it is strongly community-owned and driven. It gives freedom to experiment beyond what would be suitable in the distro proper, and it engages contributions that might not have been made otherwise. This bleeding edge packaging work can be an exciting challenge, especially when it results in you being the first to see some sexy new feature.
Yet I also see xorg-edgers as strategically important for tackling the types of problems we've run into with X.org in the past. First, it should help us more reliably detect major regressions in upstream versions prior to upload by engaging a larger pool of testers than we've had in the past. Second, since upstream prefers reported bugs to be tested against their recent work, it makes it easier to ensure our upstreamed bugs are maximally relevant to upstream. Third, it gives users with bugs in the current version an alternative to try.
Another reason I strongly support xorg-edgers is to help upstream. Without something like xorg-edgers there is a trade-off between including newly released versions in Ubuntu, which helps upstream by giving them more testers but risks regressions, and sticking with older but proven-stable versions. upstream can simply recommend bug reporters test against xorg-edgers, rather than walk them through all the steps of compiling all the various bits from git (which can sometimes overwhelm non-technical folks!) Ubuntu benefits as well in that the bugs and their fixes are tested in a Ubuntu environment, so when we do update to the new version we can be assured that the reported problem will be fixed.
We've gotten good feedback from upstream about xorg-edgers, such as in a recent blog post by Carl Worth. I know that other Linux distributions are often considered more "developer-oriented" because they carry newer (if unstable) versions of packages, but I hope that techniques such as xorg-edgers provide a better solution to the problem in the eyes of upstreams, that gives a stable Ubuntu base to the majority of users, and testing "overlays" that users can opt-in to as needed for doing testing/development of particular subsystems.
Posted in Ubuntu Submitted by bryce on Mon, 2009-06-22 22:22.
bryce's blog | 2 comments
X.org bugs are often very specific to the graphics chipset, but many times bug reporters don't think to mention their chip, so a triager has to ask them for this information explicitly. This is a tedious task.
Instead, now I use a launchpadlib script that automatically scans bug reports for text that looks like lspci output. If it doesn't find a match, the script automatically asks the user to attach their 'lspci -vvnn' output.
Another script scans through bugs looking for lspci lines (whether uploaded in response to the above or separately), parses them, and places the text into the description. I list both the chip pci-id, and the bus id, since sometimes we need both bits (such as for crafting quirks). Along with that, the bug's title is updated to specify the chip family, e.g. "[R500]", "[i965]", etc.
Thanks to this, we now have really, really good indication of the chipset for most X bugs.
Separately, I made a report generator that lists bugs by chip and by symptom, for -intel and -ati. Still a WIP (you'll notice a few titles missing in the ati report - probably some wayward regex or something.)
These reports allow easy review of all the issues affecting a specific chip family, or to look for a bug symptom that is affecting a range of chips (or just one chipset in particular).
Usually when I'm doing bug work, I focus on a particular type of bug, like freezes, so this report should help to find other bugs with that same symptom, so I can use the same troubleshooting approach, and so I can identify potential dupes more readily.
The symptom tagging idea is great, and was originally suggested by a couple of the Ubuntu-X team members. We've established a semi-official list of symptom tags, and applied the tags to a wide array of bugs (mostly -intel so far).
Posted in Ubuntu Submitted by bryce on Thu, 2009-05-14 07:21.
bryce's blog | 3 comments
Pedro is organizing a Bug Day for X.org to focus on xserver and -intel bugs, this Thursday, April 2nd. Come to #ubuntu-bugs on FreeNode, and/or see the wiki page at https://wiki.ubuntu.com/UbuntuBugDay/20090402
I can't stress enough the importance of helping with triaging the -intel bugs. Since Hardy we've seen an increase in total bug reports from 100 to over 250 presently, many quite serious.
This bug day we also hope to smoosh a lot of the xserver crash bugs. We've had good luck in the week since beta at solving several really bad crash bugs. The more of these we can solve in the remaining time before release, the more robust Jaunty will feel for users.
Previously I showed a graph of total X bugs filed against ubuntu.
After uploading today's new -fglrx to Jaunty I got curious of the breakdown proportion of our bugs that are with proprietary code, vs. open:
The black line is the demarcation between open and proprietary, with the proprietary package bug reports laying below that, shown by the yellow/red section of the chart. As you can see, roughly a quarter to a third of Ubuntu's X.org bug reports are in proprietary stuff. (Actually somewhat inflated - linux-restricted-modules also includes proprietary wireless driver bugs too.)
Posted in Ubuntu Submitted by bryce on Wed, 2009-03-18 03:53.
bryce's blog | 4 comments
Last year the influx of X.org bug reports to Ubuntu was pretty crazy. So far this year, myself and a band of other community members have made a good dent in the volume of bugs and gotten the influx under control; see this graph: X Bug Triaging Activity (2008-2009).
A solid momentum has built up, which is good since X.org is such a key piece of Desktop Linux, and the more solid it is, the more solid the whole Ubuntu experience will be.
Since I think a lot of the basic bug triage is pretty well understood now, I'll be focusing less time on that going forward. I've documented many of the processes at http://wiki.ubuntu.com/X, and I'd like to encourage others to jump in.
Here's some pretty straightforward tasks/projects that would help clean up the bug tracker and move more of these bugs towards a solution:
1. Review packages with small numbers of bugs. A lot of the packages with Triaging common classes of issues. Certain types of bugs come up again and again, and while the bugs themselves are different, the symptoms and troubleshooting procedures are very similar. I've written several guides for common issues at https://wiki.ubuntu.com/X/Troubleshooting; pick one, and search launchpad for matching bugs.
2. Request re-testing against Jaunty. Especially for -intel and -ati bugs, we can't upstream bugs unless we know they still exist in the latest version of their code. So having them re-test Jaunty is an important step towards getting their bug upstreamed. If you ask someone to test Jaunty, please make sure to follow up by then forwarding their bug upstream once they've confirmed.
3. Forward triaged bugs upstream. While we fix some X bugs in Ubuntu, the vast majority actually need upstream's help. Your help in forwarding bugs upstream to bugzilla.freedesktop.org can go a LONG way towards getting these bugs finally figured out and solved.
Posted in Ubuntu Submitted by bryce on Sun, 2009-03-15 03:07.
bryce's blog | 3 comments
Often I get asked for a high level explanation of what exactly X.org is, and how the myriad of bits and pieces fit together, etc.
Not seeing something better out there, I drafted a short high level overview here:
https://wiki.ubuntu.com/X/Architecture
In my last post I outlined changes X is going through, and wanted to address the more philosophical question of why we're tracking this work, rather than holding off until all the changes have come through.
As a distribution, Ubuntu has the duty to pick and choose components that provide the end user with the best experience, balancing stability, performance, features, and functionality. In theory, we'd hold off until we saw all benefit and no regression for a given change. In practice, there's trade-offs.
Further, especially with open source, there's a fifth consideration to take into account, that of staying reasonably current with upstream. There are three reasons for this.
First, obviously, we get the latest and greatest new features.
Second, upstream tends to focus mostly on their current development tree, and to a lesser extent on the most recent stable release. Any releases before that get little or no attention.
So, in order for us to easily get bug fixes, new hardware enablement, and technical support, it is beneficial not to stay too far behind upstream. You can backport some fixes, but generally the more time between your version and the development version, the more work will be involved in doing the backport.
Third is the inverse of the first. The closer we are to the upstream development tree, the easier it is for us to pass fixes upstream, and the more relevant our bug reports and testing results will be for them. By not lagging behind upstream too much, Ubuntu acts as a good community citizen.
I think a lot of people assume we update mainly for the first reason - to get the PR benefit of having the new features. But at least for me personally, the second and third reasons are the bigger motivations.
