I want to echo what Brice has said so well already. X is full of interesting problems and important things to work on. When you consider the free software that's out there, very little of it comes close to the importance of X in your day to day life. When we've done our job well, you don't even think about it because it just works, but when X fails it's like a minor catastrophe. Working on the kernel is largely considered to be something glorious and grand, but people tend to ignore X even though it plays very much in the same domain as the kernel. So working on X you not only get to play with interesting problems that are similar to the ones that the kernel folks get to play with, but you also get to work on software that's critically important and makes a significant difference in people's day to day lives.
Furthermore, X.org is a great organization to work in. The number of core contributers is very few (maybe 20 all told) so everyone knows who everyone else is and people are willing to help others out by answering questions about how things work. Reflecting this, the XSF has turned in to a fantastic team over the past few years, and it's one I couldn't be prouder to be a part of. Julien and Brice are doing insanely fantastic work, and we're constantly pushing to do a better job at just about everything, from collaborating with Ubuntu to using the best cutting edge tools out there for our work.
Despite all this though, as Brice said, we're overwhelmed by everything. We're still carrying hundreds of bugs in the Debian BTS alone, most of which we'll never have a chance to look at again. As mentioned elsewhere on planet upstream is totally swamped as well. There's just too many critical projects that need doing (new drivers, fixed drivers, adding capabilities to the server, and that's ignoring day to day needs like documentation and patch review) and too few people to do them effectively. We have a wonderful set of things to do and a great group of people to mentor motivated people to do them, and yet we're still lacking contributers to this critically important set of software.
So as Brice said, if you want to get involved in something that's essentially important to Linux and Free Software, if you want to get involved in something with great problems and really fantastic opportunities to make a difference, you can't do much better than getting involved in X. Brice's method of getting involved in processing bug reports is a perfect way to start. Another is to write much needed documentation. Another is to simply help the XSF maintain some small piece of the Xorg stack, be it a driver for some hardware that you own (the XSF only really runs two or three video drivers collectively so we badly need driver maintainers), or just start poking at something you're curious but mystified about like how mesa or the X server work. There's no magic to any of it, and while it can get complicated I promise that you will become a better coder or maintainer by working on something as challenging as X. You don't have to understand it all from the start, just picking a small thing to work on will be greatly appreciated. So drop us a note on the debian-x list or drop in to #debian-x on oftc and we'll help you get going.