Restart (By anthony Last mod: Jun 15 15:43 )
Wow, that gap's even long than I'd realized. Sorry about that. I've been working on my first external contract project since starting Strand 1. It's a pretty exciting social media sharing site. Well, okay, the idea's more exciting than the what's currently up, since we're still a few months away from being ready to show anything, but still. It's also been taking up most of my work time, and isn't directly related to core Strand 1 activities (although there are slightly tangential connections), so work on our own projects was pretty minimal for the past few months. Now that the external project is in more of a groove, I'm back to working on Strand 1 things.
The first Strand 1 release of pq is nearly ready. The starting point is the distribution available in Plan 9, which is already a reasonable and useful release, but here's a summary of the additions:
- Cross-platform
- Should work on Plan 9 and anything Plan 9 from User Space supports.
- Improved zap module
- Zap any attribute, not just ssn or pin.
- More documentation
- Not the full set, but some more module documentation.
- New versioning scheme
- Okay, it's small, but had to be done
That will almost certainly be available before the end of the month, probably by the end of next week (my own target, since I'll be heading back to NJ for a week for my sister's college graduation and picking up a sometimes-housemate from college).
I haven't yet gotten back to Minerva, really. I've decided that, given my less-than-stellar performance to date actually getting things out there, that I'd do better for now getting a first pq out rather than dividing my attention. Once this first pq package is out, I'll give some attention back to Minerva (although, honestly, pq will remain the priority for now, since I've actually got people interested in using my results there).
Finally, I'm going to try and make updates here more regular. Now that I'm spending some time on our own projects again (and thus things I can talk about), I'll hopefully have more to say about them. I might also start adding in a few more general notes on the science and industry of what we do. I'm not going to promise any particular schedule, but my mental target is at least weekly.
New Years Day (By anthony Last mod: Jun 15 15:43 )
Happy New Year, everyone.
Well, it turns out I vastly underestimated the impact of the holidays on my work schedule. Once the string of holiday guests started (most of one of my housemates' families), technical work dropped to about an hour a day on average. That puts me about two to three weeks behind where I was expecting to be at this point. Since not all the holiday festivities are ended (after all, it's only the 8th day of Christmas) and I've got a few travel days in the middle of the month, I'm building in some more time and making the new target for the first pq and Minerva releases the end of this month.
The happier news is that we've begun work on our first external development project. I'm working with a few other folks (half of which I've worked with before) on a pretty exciting social media project. No further details for public consumption until we've got our first prototype off the ground, which should be in the first quarter of the year.
The other exciting news is the unofficial Strand 1 Christmas present: a pair of XO laptops purchased through the One Laptop Per Child's Give One Get One program. I'm a big supporter of the project in the abstract, and think they've done a very nice job with the XO and sugar platform. I'd have liked to see the software based on something more innovative and better for education than Linux, but they've done very well considering that starting point. Eric Van Hensbergen has built Inferno for it, which enables me to work with all the tools in that environment. No specific plans at the moment, but it's been fun to play around with.
Post-IWP9 (By anthony Last mod: Jun 15 15:43 )
IWP9 is over and I'm back in lovely Cleveland. The workshop content was great. I found the Octopus-related bits the most interesting personally; the work various folks are doing with Blue Gene at IBM is very interesting (and may get significant amounts of cash thrown Plan 9's way), and some folks out of Japan are doing interesting things with natural language processing and using Acme as a translation environment. Content aside, hearing Dennis Ritchie present pretty much anything is a kick, too.
Most of today was spent re-wiring the house. All the Strand 1 facilities have been relocated from the garage to the much warmer basement. The ambient temperature was getting to less than 10°F from the same operational floor, and winter's just getting started here. We're now on UPS, which makes me feel much better. The wifi link between the external network and the Strand 1 network is stronger now, too, which makes drawterm much nicer.
I wasn't able to make any progress on getting the pq distribution updated while I was out at the Labs, which is a bit disapointing. Still working with Geoff on it. I've got my first request for an update, which I'm hoping to get stable this week (before the holidays - assorted families - descend upon us.
Coastal for IWP9 2007 (By anthony Last mod: Jun 15 15:43 )
I'm on the coast for the Second International Workshop on Plan 9 (IWP9 2007). I spent today visiting with friends, and tomorrow I head up to NJ, towards Murray Hill, for the conference. I'll be in town for a day or two afterwards to try and see some former coworkers; if anyone's interested in getting together, give me a call (email may be unreliable while traveling). The draft of the conference program has been published, and it looks pretty good to me. There's going to be time for a few Work in Progress sessions. It's a little frustrating that they weren't announced earlier; even just a few extra days would've given me time to get something ready on either pq or Minerva. Still, I'm eager to see what other folks are working on.
Oh, on Minerva and pq: the objective is to have a release of both by the end of the year. The pq release will be the package as it stands now, plus a small number of additional modules and improved documentations (including examples). I'm talking with Geoff about the best way to manage the distribution so that we don't have an outdated version sticking around on sources and don't put too much load on him as the "extras" maintainer. The Minerva release will be stand-alone, at first a packaging of the work Paul Lustgarten did when he was last working on it (before his time with us at Cibernet), plus some documentation work. The pq release aims to be production ready, but as Minerva was originally written for Paul's specific needs and we're not spending much time on coding for Minerva for this release, it's likely to be alpha quality (and certainly won't be further along than beta). After this initial set, pq and Minerva releases should come more often, and I'm working on ways to deliver snapshots from our development tree.
Root up (By anthony Last mod: Jun 15 15:43 )
Our new cpu server, root, is now up and running. The SATA controller problem took a bit longer to get resolved, due mostly to Thanksgiving, but thanks largely to Erik Quanstrom over at Coraid, we've got good service. A bit of config file tweaking in the morning and we'll be back to "real" work.
Erik also sent me a note extolling the virtues of Ken's file server, fs(4) in Plan 9, which was the standard in that environment until Venti and Fossil came along and (mostly) displaced it. KenFS was odd in that it was a stand-alone kernel, not a user-mode program running atop a more general-purpose program. This makes it a bit more of an investment, but has a number of significant wins. Reliability and simplicity of administration are the big two: except for hardware failure, the thing really just sits there and runs. You get to treat your file server like an appliance: plug it in, set it how you like it, and forget about it. This is the same theory people like NetApp try to sell their products under (and it's equally valid there; NetApp makes some decent stuff).
There's two reasons we're not running KenFS at Strand 1. First, and by far the bigger: cost. We're on a budget here while things are getting started up. We bought the new cpu server to run Strand 1 services; we would've had to buy another box to serve as a good host for a KenFS box. Second: generality. KenFS provides a Plan 9 file system, which cpu servers (and terminals) can get at, manipulate, and offer up in different ways. Venti, however, provides block-level storage. As network-accessible block-level storage systems go, Venti has quite a bit going for it, and we'll be using it for a few things here. Given that we'll be using it anyway, the administrative costs between setting it up for use as a file service (with Fossil) and setting up KenFS are significantly reduced. We can't really do things like set up our unix backup system to dump to KenFS in the same way we can Venti.
Since its official retirement, KenFS has been ported to user-mode in the form of cwfs(4) in the Plan 9 distribution. This allows reuse of existing KenFS systems, but misses out on the appliance-like benefits. It's really only intended as a transitional tool. The original is still quite usable, and Erik and the folks at Coraid have been keeping it up.
New server (By anthony Last mod: Jun 15 15:43 )
We've gotten a new server. It's a good box: a 3Ghz dual-core AMD chip with 4GB of RAM and a whopping 1.5TB of disk. I'm working through setup, but have been delayed by some problems with Plan 9's support of the SATA controller. The folks on 9fans, particularly Erik, have been quite helpful. I'm hoping to have this resolved by the end of the week (I'd say tomorrow, but I'll be somewhat busy making and consuming too much food).
The first big task (after having verified that the parts all work on friday) was figuring out how to partition all that disk. Just under a third of it will be mirrored venti arenas, which should last for a good while, but looking at venti's performance characteristics took some time. I think I've got it worked out such that I've got all the space we'll need for the near future, as well as upgrade paths for both space and performance.
I'm still slightly amazed that disk is now under 25¢/GB and has more RAM as cache than my first PC did total. My entire last file server fits in just about a quarter of one of these 500GB disks, even assuming 100% utilization. I'll be keeping the old disks running cwfs(4) to serve the old file system for a while, but with these numbers it'll probably just make sense to copy the entire contents onto the new venti. I thought about reusing the disks from that for the new venti's index, but keeping external disk array up and running probably isn't worth the cost, at least for now.
Hello World (By anthony Last mod: Jun 15 15:43 )
I've finally gotten a new Strand 1 site up and running. Things aren't yet 100% (in particular, the rss feeds don't yet work everywhere and the directory is, well, "sparse"); at least the first of those two should be fixed this week.
The technical blog not tied to a particular project is something of a new thing for me, and I'm still getting the hang of how to deal with this new soapbox. The content here will be more-or-less directly related to Strand 1 for the most part, so you won't have to wade through my experiments with tempeh or riding off mountains with my bike to get to interesting notes about new pq modules or Minerva applications. And, of course, all our significant releases will still hit the News page, if you just want the highlights.
Particular thanks is due to Uriel for significant help getting werc up and running last week. It's good to have more than a few lines of placeholder back up for a change. Looking forward to getting some more content up there, too.
First pq release scheduled (By anthony Last mod: Jun 15 15:43 )
I'm happy to announce our first actual scheduled software delivery: the version of pq published by Strand 1 will be out by May 30. The first release will be an incremental update to the existing Plan 9 version; the big news is that it'll be cross-platform, with the help of Plan 9 from User Space. A few more details (only a few for now, more soon) are up in my Strand 1 blog.
Reading down the page back in history a bit, one might be inclined to ask about that "before the end of the month" comment back in november. Well, it's not a very good excuse, but that wasn't really intended at a release date, more of a "statement of intent". There wasn't really any internal schedule attached (to much of anything, at that point!). Things're a bit more organized now, and there's been more-or-less consistent technical work being done on a handful of projects, so we've got a much better feel for how much work is involved.
So, really: pq by May 30. Promise.
Strand 1 is live (By anthony Last mod: Jun 15 15:43 )
It is with great pleasure (and a good bit of pride (and quite a lot of relief)) that I introduce the all-new Strand 1 web site. Work's been going on here since the summer, but with all the fun getting things running we only just now built ourselves a real website to replace the placeholder pages that've been sitting around since things went on hiatus.
We've been making some really good progress on some of our projects, particularly Minerva (our web server) and some updates to the pq database. Look for a release on each of those before the end of the month.
The web engine is based on some very fine work by Kris Maglione and Uriel Mangado called "werc". The site will continue to evolve, particularly over the next week or so, but it's nice to have something functional for people to browse.