I've tried OpenSolaris in various incarnations since before the Oracle acquisition. After the acquisition, Oracle decided to kill OpenSolaris, which means they stopped providing code drops under the CDDL for all of the technologies that made OpenSolaris interesting (like ZFS), not to mention the kernel and other OS components. So ZFS has continued to be developed behind closed doors with all kinds of neat features (like transparent encryption), while the open ZFS effort has kinda stagnated.
I tried OpenIndiana again about 8 months ago, using it as my daily driver for a few weeks, and I really don't want to knock it. Considering how small the community is that has taken the CDDL OpenSolaris code and run with it under the Illumos/OpenIndiana projects, they're really doing an amazing job. They've continued to develop the technologies, port software, and they had finally gotten the whole codebase to compile under GCC instead of Sun's compilers, with the exception of (I think) a few utilities. But, in spite of their talent and hard work, it is a small community, with all of the manpower problems that entails.
In the end, OpenIndiana wasn't different enough in some ways from Linux or the BSDs to be really compelling (yup, it's another Unix-descendent/Unix-like), and was different enough in other ways (I couldn't figure out how to harden it and encrypt my drives as I'm accustomed to on Linux and FreeBSD) that I wound up switching away again. Also, while I'm sure that ZFS and DTrace and all of those fun technologies are well-documented, in a fashion, I've never found user-friendly documentation that I was able to use to get up and running with them. Because of the small size of the community, I was concerned about security, and bugs do seem to get fixed at a pretty slow rate.
If you want to get into OS development, I'd definitely check it out. They could use the help, they seem pretty friendly, and it could be a neat opportunity to be a big fish in a small pond. I mean, thousands of people contribute to the Linux kernel. Big fuckin' deal. But there are few enough people working on Illumos/OpenIndiana that you could get some badass hacker cred if you made a significant contribution.
Oh, and this is from a desktop/development platform perspective, of course. As user mentioned, if you're interested in it as a server platform, that's a whole different thing, and it may be just what you want.