You are hereFeed aggregator
Feed aggregator
Hello Joyent!
Planet Illumos - 16 min 49 sec ago
As I mentioned in my farewell to Sun, I am excited by the future; as you may have seen, that future is joining Joyent as their VP of Engineering. So, why Joyent? I have known Joyeurs like Jason, Dave, Mark and Ben since back when the “cloud” was still just something that you drew up [...]
Categories: Technology
Happy SysAdmins Day
Planet Illumos - 16 min 49 sec ago
Its that time of the year again. Happy SysAdmin Day everyone. If today is dragging, might want to refresh your memory of the great OddTodd... always a pick-me-up. 2010-07-30T18:25:00Z
Categories: Technology
Revelations: 145
Planet Illumos - 16 min 49 sec ago
Phoronix published a sensationalist article last week claiming that my regular e-mail updates of our biweekly builds somehow signified some out of the ordinary newsworthy event, without bothering to do even the most basic of fact checking. While I pointed this out in their forums within hours of publication, I'm still seeing it cited by other web magazines that don't bother to fact check, as well as in various e-mails and blogs, so am publishing a more complete explanation here of why it really is a non-event. The article claimed: As the first email of its kind in months, Alan Coopersmith who is a known X.Org contributor and longtime Sun Microsystems employee now working for Oracle, has written a new email entitled "IPS distro-import changes needed for X packages for nv_145." Alan immediately began this public email by saying, "Just when you thought you'd never see another one of these biweekly mails..." Sadly, all they needed to do to disprove the claim that it was the “first of its kind in months” was simply follow the links from the e-mail archive page they linked to, to see that I had sent a similar message two weeks earlier for the previous biweekly build nv_144. In fact, if they checked the archives for previous months, they would have found that, except for missing build 143 (a mistake on my part), I've sent these approximately every two weeks for every biweekly build for a very long time. Perhaps I'd confused the article's author with the offhand comment he seems to have misinterpreted, but explaining that requires a bit of background explaining what these e-mails are and why I send them in the first place. As many OpenSolaris users know, over the course of the last couple of years, we've been transitioning from the old SVR4 package system used in Solaris 2.0 through Solaris 10 to the new Image Packaging System (also sometimes known as “IPS” or “pkg(5)”) being developed by a team of Solaris engineers and community members. Initially, we maintained two parallel distros, Solaris Express: Community Edition (SXCE), which was built using the SVR4 packages and the old Solaris installer that worked with them, and OpenSolaris (originally codenamed “Project Indiana”), which was built using the IPS packages and the new Caiman install software designed to work with them. The teams providing the package contents continued to deliver SVR4 packages, and the OpenSolaris distro team converted those to IPS. One of the goals of the OpenSolaris distro was to provide a set of ISO install images and package repository that was completely, freely redistributable, so that it could be easily mirrored, copied, and downloaded without having to deal with the various encumbrances required by some of the third-party licenses in the traditional Solaris and SXCE packages. Unfortunately, at that time, we had not yet finished separating the encumbered code from the open source code in our X packages so that they could be included, since when Sun made its proprietary fork of X11R6 in the early 90's, the engineers never figured we'd be open sourcing Solaris a decade later and need to easily separate out the encumbered bits they were merging into the main code base. The initial Developer Preview releases of the OpenSolaris distro thus included a set of X packages that Moinak Ghosh built from the Fully Open X (FOX) project work we'd done to rebuild our source trees from the ground up using the open source code from the X11R7 modular releases in order to ensure everything was either open source or cleanly separated out. Over the first few months, we migrated from those to the packages our team delivered to the OS as we integrated the FOX project work into our main source tree. Because these changes weren't always obvious to the external observer, I started sending notes for each biweekly build to let the team maintaining the SVR4 to IPS conversion tables know which parts of our packages they could now include, as well as any other changes they needed to know about, such as version number changes. (Our SVR4 X packages all used the same version number, a holdover from the monolithic X source days, but we've migrated to using the upstream version numbers as much as possible in the new X IPS packages.) I did this not only to try to help the distro building team, but also to help myself keep on top of the changes they made in converting our packages, so that we could understand issues users hit and know how to help them, and to learn better what we'd need to do when it came time for us to start building the IPS packages ourselves. Last year, Sun announced that the time had finally come to start converting the builds to generate IPS packages directly, taking the next step in transitioning off SVR4 and ending the production of the SXCE distro, since the SVR4 packages used to build it would no longer be made. The ON consolidation went first, converting the packages containing the kernel, drivers and core utilities in build snv_136. The Install consolidation went next, in build snv_143, and X followed in build snv_144. So, when I wrote “Just when you thought you'd never see another one of these biweekly mails...”, I simply meant that after build 144, all the X packages are already delivered in IPS format, so there are no SVR4 to IPS conversion files to update for them any more, so I won't need to send those - except in cases like we had in 145, where I relocated one of the files that was listed as a dependency in one of the other packages still being converted by the distro builders, so they needed to update the dependency statement for it to list the new path to the file. Despite what Phoronix seems to have assumed, I was not in any way referring to the limbo state the OpenSolaris distro is currently in (and unfortunately, as much as I'd like to explain that, I can't), nor stating anything about build 145 that is fundamentally different than the previous builds. It should come as no surprise to anyone that while build 134 was the last build to be publicly released, we have continued work on the Nevada builds after that - after all as we've said since 2005, Nevada is the code name for the development branch in which we're working towards the next full release of Solaris (i.e. not another Solaris 10 update release, but the one we may someday call Solaris 11) - while that's been released under various forms, as the OpenSolaris open source code, or via the binary releases of the original Solaris Express, Solaris Express: Community Edition, Solaris Express: Developer Edition, or the OpenSolaris 2008.05, 2008.11, and 2009.06 distros, we've always kept driving towards the same goal, with biweekly builds assembled to test the current progress. My mail is hardly the only externally visible sign of this - you can see changelogs for the major consolidations (ON, SFW, X, and Desktop/JDS) for build 144 (the last fully finished build, as 145 is just finishing pre-integration testing now, with a delivery deadline of Monday for packages to be included, and the distro build process starting after that. Of course, the sources are also available, and there's plenty of activity on the various commit notification and discussion mailing lists showing that we're continuing to work away on Nevada. So unfortunately, Phoronix succeeded in making a mountain out of a molehill, confusing their readers and fellow webzine authors, but likely meeting their goal of driving more traffic to their site to generate page views for their ad revenue as people passed the link around twitter, IRC & email or linked to it from their blogs & articles. As others have pointed out, checking the facts or contacting the developers to find out the story is less juicy than it seems doesn't play well with that business model (and that's not just true for Phoronix - look at any number of the columnists for other web-based "trade publications" that generate traffic via controversial posts, and the more outraged the community gets over them and angrily passes them around to denounce, the better their numbers are - you can just imagine how many of their articles are designed to bait Groklaw or Slashdot readers). 2010-07-26T05:53:56Z Alan Coopersmith
Categories: Technology
Good-bye, Sun
Planet Illumos - 16 min 49 sec ago
In Februrary 1996, I came out to Sun Microsystems to interview for a job knowing only two things: that I wanted to do operating systems kernel development — and that I didn’t particularly want to work for Sun. I was right on the first count, but knew I was wrong on the second just moments [...]
Categories: Technology
Diversion: Ode to Lego Technic
Planet Illumos - 16 min 49 sec ago
Nova, my first daughter, is now 6 and Glenn, my first son, is now 5. As a GeekDad I ensure to bathe them in geeky goodness. I've been thankful that Glenn is obsessed with Lego. The kool thing about it is that of course, I get to help him, so its just a great time. Here was last nights project: Teaching him has gotten me thinking back to my own youth. I had a box of Lego's but not a lot of sets. The one that I did get was in 1988, when my parents got me perhaps my favorite (but forgotten until recently) toy of youth: the Lego Technic 8865 "Test Car". That set was amazing. I proudly displayed it on my shelf in my room, both because of my pride in building it as well as just how outright kool it is. Since that time Technic has grown up as much as I have. Take a look at the Technic Lego 8421 Mobile Crane: So tempted to buy that. I already have the Ferrari F1 set, which Tamarah bought for my birthday several years ago. But most fun of all... this week Glenn is in a one-week Lego Pre-Engineering class. For 3 hours a day they geek out and build all manner of fun stuff. One thing I'll throw out there for Dad's... Lego has an Education dept: Lego Education. Of particular interest to Tamarah and I, is that they have a complete Homeschool Curriculum and various kits, including robotics kits, for education. A really amazing resource for parents. 2010-07-22T19:43:00Z
Categories: Technology
Devops in Practice: How They Do It
Planet Illumos - 16 min 49 sec ago
Damon Edwards (DTO Solutions) & John Willis (Opscode) are the two guys really pumping out the "good news" of devops. They started a new podcast, Devops Cafe several weeks ago. Already on episode 8, having featured guests such as John Allspaw, R.I. Pienaar, Andrew Shafer, and more. Highly recommended. Whats interesting is that John & Damon really aware of an outcry from the community, that is: "How do all these devops shops do it!!" We want to emulate them, know what tools they have, how they use them, what works, what doesn't, etc. So to facilitate just that, they started a videocast sub-series called: Open Mic. Open Mic 1: DevOps Metrics and Dashboards at Shopzilla from dev2ops.org on Vimeo. In the first episode, they take us into Shopzilla, where Juan Paul Ramirez shows us their tools, metrics, and talks extensively about how they got to where they are. Excellent content! If you haven't already seen, perhaps the most popular talk this year at Velocity, "A Day in the Life of Facebook", in which the Facebook Ops team introduces us to their tools and organization. Whats really great here is that we're not share deeper information about how we're doing things, such that we can be a community of organizations. In the past, only a handful would really share and they were always far removed from useful pratice. I really hope this trend continues. Big thanks to John and Damon for helping fuel that fire! 2010-07-22T16:37:00Z
Categories: Technology
What is RAID-Z?
Planet Illumos - 16 min 49 sec ago
The mission of ZFS was to simplify storage and to construct an enterprise level of quality from volume components by building smarter software — indeed that notion is at the heart of the 7000 series. An important piece of that puzzle was eliminating the expensive RAID card used in traditional storage and replacing it with [...]
Categories: Technology
A Logzilla for your ZFS box
Planet Illumos - 16 min 49 sec ago
A key component of the ZFS Hybrid Storage Pool is Logzilla, a very fast device to accelerate synchronous writes. This component hides the write latency of disks to enable the use of economical, high-capacity drives. In the Sun Storage 7000 series, we use some very fast SAS and SATA SSDs from STEC as our Logzilla [...]
Categories: Technology
On generation matters
Planet Illumos - 16 min 49 sec ago
This article provided some food for thought on a recent discussion at home, so I wish to share it. A few questions raised after one hour of discussion (with no intent of reaching a 100% right answer). As a generation, why are we so afraid to be attached to anything ? The “don’t put a [...]
Categories: Technology
Please Be Patient
Planet Illumos - 16 min 49 sec ago
With all the ruckus surrounding Oracle's apparent abandonment of the community, and OGB's stated intention to suicide, the community uproar has been crazy.Without giving any details, let me say that a few of us are quietly but diligently working on solutions to the critical problems, and I expect we'll be able to talk much more freely about the solutions we will be offering in early August, which is coming up very soon now. So, I'm going to humbly ask folks to be patient -- hold your comments, complaints, and flames about Oracle and OpenSolaris and OGB in check please. If you can wait just a little bit longer, then I believe we'll be able to offer a more constructive outlet for your frustration and energies.Thanks.
Categories: Technology
In NYC for DebConf10
Planet Illumos - 16 min 49 sec ago
I'll be attending DebConf10 (the Debian developer's conference) in NYC this year. Nexenta will be presenting information about our distribution. Its my hope that we can use this to generate more interest in OpenSolaris technology. If you're in NYC, and want to meet during the first week of August, let me know!
Categories: Technology
OGB Threatens to Shoot Itself In The Head
Planet Illumos - 16 min 49 sec ago
This morning, at the 8AM (Pacific) OpenSolaris Governing Board (OGB) meeting, the following was proposed and unanimously resolved: The OGB is keen to promote the uptake and open development of OpenSolaris and to work on behalf of the community with Oracle, as such the OGB needs Oracle to appoint a liaison by August 16, 2010, who has the the authority to talk about the future of OpenSolaris and its interaction with the OpenSolaris community otherwise the OGB will take action at the August 23 meeting to trigger the clause in the OGB charter that will return control of the community to Oracle. That is to say, "start talking to us or we'll just shot ourselves in the head." I made my opinion very clear via the IRC back-channel during the call. At least my call for a liaison was added into the resolution, but I am extremely opposed to this cowardly act. What exactly do we have to gain or Oracle to loose? All Oracle does is runs out the clock, the entire OGB resigns, and then the one little bit of control the community has is gone. What motive, other than a benevolent act to garner press attention, does Oracle have to comply? We've just made their job easier. I once advocated this kind of self-implosion tactic back in the Sun days. The reason was to re-organize the OpenSolaris leadership to be more engaged and industry focused. That was a good idea back in the days when I had faith that Sun would "do the right thing". However, those times have past. Oracle has made it clear that it either controls things or it doesn't... there is no give and take. I don't think we can demolish the structure and believe that Oracle will re-organize in such a way as to give the community more power. It was a long shot with Sun anyway. Frankly, imho, this is just the OGB throwing its hands in the air. The body has been useless for a long time, but only because it has chosen to be. The majority of the OGB's life its wasted by trying to restrict its own authority by endlessly debating and re-writting the constitution. Its never lead anything, and it isn't now. But the fact that its a wet rag doesn't mean we should simply throw in the towel. A weak seat of power is better than no seat at the table. So where do we go from here? Who knows. At this point the die is cast and OGB is putting up their last stand. Maybe Oracle gets serious and does something, but I really doubt it. Not because they can't, but because its not in their best interest. Why kill something intent on killing itself. My only concern as this point is to not loose regular code updates and access to the bug database. Yes, the existing code is "out there", but Oracle is still the biggest contributor, 99.999 to 1. Anyone can fork at any time right now, as is, so if your going to do that why would you risk cutting off the huge contributions continuously made by Oracle? We're in no worse a position right now than we were during the Sun days. They didn't communicate, we had no visibility or impact on the OpenSolaris distribution, etc. Don't fall into the lie that things are now "worse" than they were... they aren't. Its status quo. The difference is that the OGB is no longer composed of Sun insiders who can get a sense of control from hallway conversations and are now as blind and weak as those of us in the community always have been. The request for a liaison is a good one... I support it. But damnit, put the gun down. We don't need to act like irrational children having a tantrum. Ultimatums rarely workout the way you hope. The bar is lower than the original resolution was, so we'll hope for the best and see. UPDATE: OGB Member Peter Tribble has written a blog entry about this action, recommended reading. While I disagree with the action, Peter is a great guy whom I greatly respect. 2010-07-12T19:31:00Z
Categories: Technology
Unpublished PSARC cases
Planet Illumos - 16 min 49 sec ago
There has been a sharp increase in the number of PSARC cases that are in “Unpublished” state since Oracle completed the Sun acquisition. It’s sad that we have to guess OpenSolaris’s future by picking bread crumbs here and there.
Categories: Technology
Planet Solaris Dies the Death
Planet Illumos - 16 min 50 sec ago
A very sad day indeed... Planet Solaris is dead. Just another in a long line of bad signs. Please use Planet.OpenSolaris.org instead. A big thanks to David Edmondson for running planetsolaris.org for so long. I am partly responsible. Sorry to everyone that the blog has been so quiet lately. Given that state of Solaris right now, its unclear what is dead and what is alive. It feels futile to blog about features that may never really be viable. Couple that with OpenSolaris which still hasn't delivered and the fact that many of the features that need documenting are really pretty uninteresting to me (ie: IPS/AI). The exodus still continues. Lots of engineers have left Sun and many more are considering leaving. I'm told by folks that its not a huge problem because while the big name guys are leaving, the real down in the trench do-ers are still there and working away. But it certainly is disheartening. The most recent news out there was that Oracle yanked HP's OEM License, so if you run Solaris on HP Prolient servers, your hosed. See? Not a lot of positive stuff for me to blog about. Personally, I've been more interested recently with the growing 'Devops' movement and IT standards. I've spent a lot of time in ITILv3, ISO 20K/27K, CobiT 4.1, COSO, NIST SP800-53, etc, etc, etc. A whole new and interesting world to me because I came to it instead of a company hoisting it on me against my will. I have several articles to get out for SearchDataCenter which I'll plug here, and then will start rolling new content out here in a bit. 2010-07-08T22:06:00Z
Categories: Technology
pymegacli
Planet Illumos - 16 min 50 sec ago
pymegacli is a very thin abstraction layer on top of the LSI MegaCli command-line tool. This is the first code revision and it’s lacking a lot of error checking and more advanced options. The goal was to get something up and running fast. There are four classes representing the major objects: Adapter, Enclosure, PhysicalDevice, VirtualDrive. [...]
Categories: Technology
ZFS disk monitoring...
Planet Illumos - 16 min 50 sec ago
So I've posted this on zfs-discuss at opensolaris dot org, but its been suggested I mention it here too.It turns out that the ZFS/FMA integration doesn't pick up on drive removals for most disk devices until the filesystem attempts to perform some I/O to the drive. This is rather unfortunate, because if a file system is not busy, you might suffer a loss of redundancy and not find out about it until too late. It also means that you won't know about failures of hot spare devices until you need to put them into service, since by definition they are idle. (Note: as an exception running periodic scrubs should detect this too, although scrubs are highly intrusive to the overall I/O load on the system and probably should not be performed too often as a result.)I'm told the Oracle 7000 series appliances have a solution for this problem, but of course the source for that is not in OpenSolaris. (Apparently there are quite a few differences in the core OS between the 7000 series and vanilla OpenSolaris -- unfortunately we can't know because -- unlike with NexentaStor -- we don't have access to the kernel source tree!)This is not good for folks who use ZFS with ordinary Solaris 10 or OpenSolaris... or with derivatives such as NexentaStor.To address that problem, I've developed a some code called "zfs-monitor" that periodically monitors the health of any physical vdev (disk) that is part of a ZFS pool (hot spare, log, or real device). This code is implemented as an FMA module. When a disk goes offline, zfs-monitor detects it, and triggers an FMA event, which allows ZFS to do the right thing. This means if a disk goes away, even if it isn't in use, whatever action is appropriate will be performed. (Logged in FMA fault logs, and if appropriate, a hot spare will be recruited to replace the failed or offline device.)This code is part of NexentaStor 3.0.3. As there are some semantic differences of opinion (what constitutes device failure versus intentional removal by an administrator), the code is unlikely to be pushed into ON without further change. (At the same time, I've fixed a different problem in the ZFS FMRI parsing code, and I've submitted a request to get that fix integrated -- but I've not heard back from anyone at Oracle who is willing to sponsor the change yet.)I'm happy to share the code for zfs-monitor to anyone who requests it. (In fact, you can examine the code in our open Mercurial repository directly!) Note that for it to work properly, you also will need the fix for the ZFS FMRI parsing bug just mentioned.At Nexenta, we're committed to innovating and improving upon the great foundation of ZFS and OpenSolaris, and to the reasonable extent possible, we want to share those innovations with the greater OpenSolaris community. Hopefully changes like this demonstrate this commitment in a tangible fashion.
Categories: Technology
The Open Core Debate
Planet Illumos - 16 min 50 sec ago
I have strong feelings regarding the open core business model and most of them are very negative. In my opinion, open core is a subversion of the open source philosophy and companies using it are doomed to fail in varying levels. Be it not reaping almost any of the benefits of open source to frustrating [...]
Categories: Technology
Looking for CIFS/AD expertise
Planet Illumos - 16 min 50 sec ago
(I know its probably questionable using my blog for this, but I thought I'd post it here anyway. My apologies if anyone finds this offensive. I'll keep it brief in any case.)I'm looking for a high-caliber developer, preferably with some kernel and/or OpenSolaris expertise, who's also got extensive knowledge of ActiveDirectory and CIFS. If that's you, or you know someone who fits that description, please contact me -- garrett at nexenta dot com. (No recruiters or agents please.)
Categories: Technology
LinuxCon Brazil 2010
Planet Illumos - 16 min 50 sec ago
Last week I bought my ticket to the LinuxCon Brazil 2010 conference that’s happening on August 31st and September 1st at the WTC Convention Center in São Paulo. Of course everybody is excited to listen to Linus Torvalds and so am I. But out of the keynote speakers confirmed to be speaking at the event, [...]
Categories: Technology
O_SYNC behavior not honored
Planet Illumos - 16 min 50 sec ago
UPDATE (6/21/2010): This problem is apparently solved in b142. Probably other builds as well. But I was unable to reproduce this problem with real hardware on b142. Note that VMware does not honor cache flushing, so VMware (and possibly other v12n users) will potentially still see this issue.So, it turns out that ZFS in recent (somewhere after build 134 apparently) builds has a critical bug ... O_SYNC writes are not really synchronous. This leads to potential data loss.I've not yet figured out which change introduced the bug, but I hope to work on it next week.In the meantime, I would strongly discourage use of post-134 binaries for anything where data integrity is important. I've filed a P1 bug with Oracle for this issue. I'll be trying to nail it down further next week; if I'm able to fix it before Oracle can, I'll offer up my fix. I'll post the CR number when I receive the number back.I imagine that this bug, which is trivially reproducible, will be getting top priority from the ZFS engineers next week.UPDATE: CR number is 6958848The link to access it isn't available yet.
Categories: Technology