Stuff and stuff

Test

2011-07-03T10:41:00.001-04:00

Test

Miscellaneous things on my mind today

2010-05-18T08:49:00.001-04:00

Thinking some more about that markov->ANN mapping, and wondering if
there'd be a better way to represent the signals from the inputs to
the first layer. Ideally, that'd involve getting the symbols to
correspond to a value on a meaningful range, rather than a set of
discrete values. Let's say I wanted to make that integer range
[0,2

Miscellaneous things on my mind today

2010-05-10T16:45:00.001-04:00

It occurs to me two fic geeks might find reading through the TVTropes pages for their favorite pieces of fiction a romantic way to spend an afternoon.

--

Pondering a message passing communication system tuned for bulk data (parcel) transfer. Use an addressing system like email's: To:username@example.com

Sender, relay and recipient nodes operate on a load-sharing basis similar to BitTorrent. Where with email, a sender has a single SMTP target, this system might have multiple such targets, each of which serve as feeders for the destination recipient(s). Relaying nodes may cease feeding once either all recipients' nodes have full copies of the parcel, or once a TTL metric has been met.

Sender-signing should be explicitly supported, both as a means of ham/spam separation and as a means of authentication/accounting where necessary.

Parcel bodies should be compressed and encrypted by default.

--

Just got the geek necessities
The simple geek necessities
Don't worry 'bout your food or your hygiene...

--

If you're standing on Four Corners, and you do something that's a crime under each state's law, which state's prosecution takes priority?

--

I don't think backprop is relevant to the HMM->ANN mapping, because ANN construction is done as a component of its training, and weight adjustments are done incrementally as each node gains experience, by the node itself. No final "result fitness" observation is done, because the from-map ANN inherently evolves the same way the HMM would.

Actually, looking at it further, what I have in mind could be considered a variation of a "cascading neural network", except the weights aren't fixed on node creation. Also, I don't see reference to any types of networks that allow for deconstruction of nodes.

--

I like this way of posting my thoughts; it keeps me from being the uber-spammer on the social networks I'm in. Plus, it's like a serialization of free association in a palatable form.

--

Hm. German. shadenfrau. Heh.

--

Need to start carrying my 3e/PF books around. Or PDFs thereof.

--

Barcamp presentation ideas: 1) Rosetta Code and programming chrestomathy. 2) Wikis and more effective cross-referencing.

--

I once had a conversation with a guy I think might have been high, where we thoroughly enjoyed a conversation that didn't keep a single context and topic for longer than three sentences.

--
:wq

To learn, to Create, to Think

2010-05-09T13:23:00.001-04:00

To learn: Surround yourself with people who know more than you. Can't find any? Try another topic. Whatever you do, don't stop.

To create: Think about different things at the same time.

To think: You'll have to figure this one out on your own.

--
:wq

Miscellaneous things on my mind today.

2010-05-07T16:37:00.001-04:00

There's a striking similarity between hidden markov models and neural networks. For a 1st-order HMM, you could map it to a 4-layer (1 input, 1 output, 2 hidden) feed-forward ANN by having a single input (what symbol you're facing), making every known symbol a 2nd-layer node, with a simple input value of 0/1 (the first layer's node signals the input of the corresponding second layer depending on which symbol corresponds to that node). Use the set of known symbols for the 3rd layer as well, and the mapped weights from the 2nd to the 3rd layer corresponds to the state change probability in the HMM.

I don't think this is necessarily an efficient way to do it, but I think it might be an exact mapping. Cool. :)

If you want a second-order Markov model, use two nodes in your input layer, and use S1,S2 pairs for your 2nd layer, with a single-symbol set for your third. Definitely not a space-efficient way to do it, but also an exact mapping. If, after training (you have to get your symbol set from somewhere, which means you're going to grow your ANN as you build your symbol set), you're comfortable with the idea, you can cull your absolute weakest relationships from your 2nd-order set. Only cull as space limitations force you to, or you'll lose weak-but-significant relationships.

A 3rd-order Markov model would have three input nodes, and use S1,S2,S3 tuples for the 2nd layer, and a single-symbol set for the third. Again, cull as required.

For particularly interesting results, you can use a three-dimensional matrix I1 going to a 1st-order ANN mapping, I1 and I2 going to the 2nd-order ANN mapping, I1, I2 and I3 going to the 3rd-order ANN mapping, and map to the same output node for all three orders. (Or higher orders, if you have unbelievable processing and storage capability...)

--

I'm happy.

--

If every piece of software got as much detail-oriented attention to it as Toyota's code is getting right now, but *before* disaster struck, there'd be millions of happier developers out there, software wouldn't suck as much and development and debugging tools would be absolutely incredible (far beyond anything I've even heard of, including things like valgrind, lint and VS2010's rather impressive debugging aid feature set) as a result of finding ways to reduce developer time spent in analysis and bugfixing. If software were nearly as difficult to consider "ready for release" as hardware is, I think you'd see something akin to Moore's Law occur in software utility. (As a mixed metric of performance and feature set.)

People got used to mediocre software. They hate it, but they got used to it. As a result, software pricing models more or less standardized around writing mediocre software, and people get sticker shock when they hit a requirement for non-mediocre software. Put another way: "Fast, well, cheap" -- except that popular priority heavily weighs toward fast and cheap.

--

Flashbacks to Ender's Game when I accidentally try to call CString::GetBugger(). See also, Start->Run->%APPDADA% instead of %APPDATA%. Dargo, resolve his %APPDADA% for him!

--
:wq

Things on my mind today...

2010-05-06T16:42:00.001-04:00

Prime words. If one were to present thousands of people with the words in the dictionary, and ask them to replace the word with a small set of simpler words with near-identical resulting meaning, would a set of fundamental, or "prime" words result? What would they be? Some circular definitions might result, but their presence might help identify fundamental concepts.

What happens if you could influence the Shadow ships' pilot with LSD or PCP? (If you haven't seen through Season 4, don't bother guessing. If you don't know what I'm talking about, then nothing's spoiled.)

What happens if a vampire or lycanthrope bites a bio-tech vehicle?

I need to make a habit of going to work earlier so I can get enough hours in before early afternoon. OTOH, I've got a ton of vacation hours saved up, and that would be a worthwhile way to spend them. :)

I broke someone's nerd-o-meter today. I feel good.

I seem to be discovering something akin to alchemy and herbal remedies, but using fast and processed foods for ingredients. That can't be a good sign.

SanDisk seems to have abandoned the SD+USB product area. [trope]BigNo[/trope].

--
:wq

Dave, my mind is going. I can feel it.

2010-05-03T11:04:00.001-04:00

Gotta do some thought-out codin'
but my heart is all a'flutter.
Wrote a line of code
but I can't write out another.
Gotta get my brain in gear
but it's slippin like it's butter.
If I don't seee Firefly soon,
I'll be drinkin' like a Mudder.

There's one kind of argument I particularly hate to have

2010-04-27T04:33:00.001-04:00

...and that's rationalizing. Typically, it occurs when the person I'm arguing with has a gut feeling about what's Right, but can't maintain a logical defense of the position. I can chip away each path of apparent logic they use in defense of a position, one at a time, and then they'll come up with some new path, or switch back to an old path with a different tact.

What's frustrating about it is that it tells me there isn't a logical basis behind the defense of their position, but an emotional one, and logic is simply the mechanism being used to defend it. That means that they're exceedingly unlikely to ever change positions, unless and until that emotional basis is identified and resolved. That's pretty difficult to do when the person holding the position can't acknowledge the basis behind their core position.

The particular argument inspiring this rant was my expressed frustrations that AdBlock Plus is overly aggressive, and blocks the queries that build Amazon referral links--I was going to use those links in-line in on-wiki citations and user-recommended resource links. With ABP running, one gets a big, fat, red "Unable to retrieve data" blob of text. I'll have to mod the MW extension to deal with the error case.
I can understand wanting to block distracting and attention-grabbing advertising content. Blocking referral-building and search AJAX queries strikes me as going rather far, and smells of anti-commercialization intent rather than simply consumer protection.

HSV remixing some of my photos

2010-04-24T20:56:00.001-04:00

Tried some simple stuff on one photo, liked the results, tried it on several more.

--
:wq

2.5" -> 3.5" internal bay adapter

2010-04-21T15:26:00.001-04:00

I know 2.5" -> 3.5" bay adapters exist, but do any exist that allow you to put multiple 2.5" devices in the same 3.5" bay? I'm specifically pondering replacing a single 3.5" spinning disk drive (my root disk) with two 2.5" drives (one for root, and a small SSD to put mysqld on; I'm disk-bound, and the lifetime and safety of the data is less interesting than the speed it can be manipulated at.)

Yup. That StumbleUpon surge changed things a bit.

2010-04-20T13:02:00.001-04:00

Not in a way they haven't been changed before, mind you. Every now and again, there's enough of a traffic surge that the traffic never fully dies back to near-original levels, even if the referrals aren't coming from StumbleUpon any more.

Usually, it's a combination of two things: 1) a steady trickle of referrals from dozens of forums like Stack Overflow, Reddit and language-specific phpBB installs. 2) A lot more people add "rosetta code" or "rosetta" to their search queries; I get the impression that this method of learning how to do stuff is getting more popular.

So, yeah, the ambient traffic appears to have permanently bumped up a notch. This has happened three times before, that I can think of.

[TSoC] Automated virtual plays - machinema for the social web

2010-04-19T20:24:00.001-04:00

Ok, so here's the thought: Take a social medium like Twitter or blogging, and take such things as stage plays.

Tie each Twitter or blog account to a character in the play. Now, how do you automate it?

I see three pieces that need to be addressed to cover each member cast:
(character)
(service)
-(account)
--(contacts/friends)

I see four bits of information that need to be conveyed for each active line:

(trigger time)
(service)
(who is the message directed at)
(the message)

Now, think about this. How would you organize this data?

--
:wq

[TSoc] Abusing TinyURL, bit.ly, and your browser--making the work someone else's problem.

2010-04-12T09:32:00.001-04:00

So, lately, I've had a few ideas for simple, interesting websites, and there's one thing they have in common--they have no server-side state. Server-side state means you need a database, and databases don't scale easily or cheaply.

So here's basis behind one of the ideas: http://www.translationparty.com

TranslationParty works by taking your English phrase, using JavaScript to call into Google Language Tools to convert it to Japanese, repeats the process back to English, and back to Japanese--on and on until it finds "equilibrium", or is confident the phrase will never reach it. Of course, it only works with English-Japanese translation. You'll notice, though, that it uses an identifier to allow it to recall previous "parties". I can show you one I just created a moment ago, by having you click on this link: http://translationparty.com/#7180967

They're storing the seed phrases in a database, so that visitors can refer back to them. That's fine, I suppose, but that database is eventually going to get large and expensive, and most of the past seed phrases will have been forgotten about in the mean time.

I showed TranslationParty to a couple folks last night, and they noticed it only did English-Japanese translation. I'd just hit on the idea of doing something like TranslationParty, but using virtualhosts to allow language selection.

en.ja.example.com would behave largely like the existing TranslationParty site.
ja.en.example.com would start with Japanese as the seed language, rather than English.
en.de.example.com would bounce between English and German
de.fr.example.com would bounce between German and French.

So, $1.$2.example.com would start with $1 as the seed language, and #2 as the intermediate language.

Problem is, I wouldn't want to store that database. It's largely a waste of server resources, considering most of it would be forgotten ten minutes after it was created. The obvious solution is to store the seed phrase as an HTTP GET parameter. http://en.ja.example.com/?seed=What%20lies%20in%2the%2deepest%20regions%20ones%20our%2memory would have "What lies in the deepest regions of ones memory" as the seed phrase, and be equivalent to http://translationparty.com/#7181046 ...

Now, that's a long, nasty URL, not suitable for copying and pasting in polite company, but it means that I don't need a database for these things. Besides, that's what bit.ly and tinyURL are for, right?

--
:wq

I'm a DLL?

2010-04-06T03:29:00.001-04:00

src="http://www.bbspot.com/Images/News_Features/2004/10/file_extensions/dll.jpg" width="300" height="90"
border="0" alt="You are .dll You are dynamic. You are constantly in danger of bringing down the house, because you don't play well with others.">
Which File Extension are You?

What we owe Apple and Mac users

2010-04-04T18:11:00.001-04:00

I tend to hate Apple software; it's overly simplistic and makes too many assumptions about what I need to do. I tend to hate Apple hardware; it's good stuff, but you pay too much for it.

However, they've got one heck of a marketing budget, and they manage to make their products mainstream, or, at least, 'hip.' That's something I can appreciate. Why? Because if it weren't for Apple making their hardware and software hip, there wouldn't be the demand for lookalikes and workalikes that leads to products I *do* want to use to become available.

(Yes, the Newton was a failure, but Palm followed up on the concept and made a device I enjoyed using for years.)

Consider the iPod. MP3 players had already been around for a while, but they weren't big sellers. Along somes the iPod, which takes the MP3 player market by storm, and drives the expensive little lossy-audio playback device into popularity. People discover the convenience of having a few thousand songs in their pocket, and show off to their friends. Their friends like the feature set, but balk at the price tag. Along come the products by other manufacturers and vendors, with similar functionality[1] and a lower price tag, and those friends buy. Apple made the MP3 player popular.

The iPod went through a few iterations. It added video, then it added video.

Now we consider the iPhone. Smartphones have been around for far longer than the iPhone, but I normally only heard about them being used by big corporate and institutional types. You didn't get them unless you had a financially-driven need for them, so they were only marketed to the organizations with pockets. Along comes Apple, with their iPhone and their marketing budget, and it *sells*, despite being an open-developer-unfriendly platform. Same as with the iPod, they show their friends this cool stuff, friends want something like it, and so you get things like Android and Windows Mobile[2] getting bought at cell phone retailers all over the place. Meanwhile, people got used to touch screens and video interfaces thanks to things like the iPod Touch, so adding a phone to the feature set doesn't seem too odd.

Now look at the iPad. Everyone teases it. I don't want one, because I don't like the development environment. Everyone teased iPod, too. Tablet PCs are nothing really new, either; they've been around for over a decade. The first one I remember seeing had an 80486 processor, and it was used for mobile logistical record keeping. It ran Windows 3.1, and had handwriting recognition. More recently, HP and other PC vendors have been selling laptop/tablet convertibles. My mother had one. It's kinda fun to use. Yes, they were never really popular. That's why people keep expecting the iPad to fail. They forget that MP3 players weren't popular before the iPod, and that smartphones weren't core culture until the iPhone. I don't know if it's a marketing budget, superb planning or superb timing, but Apple tends to take a known concept and get it to sell.

I expect the iPad will sell nicely. It's like a supersized cross between an iPhone and iPod Touch, with the phone features pulled. What with the requisite technologies for AR getting better and better, I suspect I know where Apple would like to go next.[3]

[1] Greater in some areas, lesser in others. I don't want to debate it.
[2] Yes, Windows Mobile has been around for a while. I didn't say it was *created* as a response to the iPhone.
[3] And a brief plug for an anime I've been enjoying lately. Dennō Coil

--
:wq

[question] Testing Virtualized XP installs with more than two CPUs.

2010-04-02T17:13:00.001-04:00

So Windows XP Pro allows the user to run at most two physical processors, but each such processor can have as many cores as you like. That's great for quad-core and above processors, if the operating system is running directly on top of the hardware.

So what if you have a quad-core or dual-quad-core system, you're running some virtualization package (KVM, VMWare Server, whatever), and you want your guest XP Pro instance to have more than two cores? On QEMU+KVM, there's no way to lump virtual CPUs as cores on a virtual physical processor, and there doesn't appear to be one on VMWare ESX Server, either. So how would you do it?

The particular scenario, in this case, is automated testing of software that's supposed to run on XP Pro with systems between four and eight cores. While it stands to reveal interesting bugs, running it on only two cores is non-optimal.

--
:wq

Quote from my boss from a couple minutes ago

2010-04-02T00:34:00.001-04:00

Working late tonight, wrapping things up. Boss just gave me this little gem:

I don't know if I said it
So you'll have to have to ignore the repeat
But I need to make sure I said it
...I forgot what I was going to say.

[TSoC] [UPS, USB, X10] X10 UPS and USB power supplies.

2010-03-26T12:46:00.001-04:00

Observations:
* X10 was once used to allow remote control of power outlets. It operated by adding a small set of pulsed high-frequency modulations on top of your home's internal AC. Take a peek at it with an oscilloscope some time; it's pretty cool.
* UPS devices report themselves well over USB, but that takes up one of a computer's external USB ports.
* In most systems I've seen and built, there are spare USB headers left on the board after building a system.

Proposal:
Taking these three in context, consider having a UPS self-report its status by doing a controlled modulation of its AC output. Have a computer's power supply be capable of receiving and interpreting these signals. Have that power supply plug into a spare USB header on the motherboard, to provide those communications onward to the system.

Benefits:
* Avoid using an external USB port
* Allow multiple computers powered off of the same UPS to be simultaneously aware of their power source's state.

Drawbacks:
* AC output of UPS can't be a perfect sine wave, much as that would be ideal. (Though most output a stepped approximation, anyway.)
* Requires that the PC's power supply be able to interpret the signals.

--
:wq

For the last time, no, I Won't be a fan of "Google Fiber for Grand Rapids"

2010-03-25T18:10:00.001-04:00

(This is going out to a whole bunch of places, not just Facebook.)

I Won't become a fan of Google Fiber for Grand Rapids, for a couple reasons. Let's start with the big one: I don't want my Internet traffic primarily channeled by a company whose primary revenue source stems from figuring out how to show me the ads I'm most likely to click on. That's a very, very direct incentive for them to do traffic analysis and directly watch the sites I visit so that they can figure out what I'm interested in, and thus what I might buy, and thus what ads to show me.

No, AdBlock doesn't fix things; avoiding being shown ads wouldn't stop the incentive for analysis any more than a boycott on Wendy's for serving meat helps PETA.

I'm sorry, but I'm just not interested. Even if they promise not to do privacy-invasive things, they ultimately can't help it; it makes too much business sense for them to set up a transparent HTTP proxy and build up their Analytics and Adsense statistical bases. *I* would, if I were them. If they don't, then they're stupid; they would be ignoring the opportunity to blend together geographical and demographic awareness into their ad business, and sell localized Adsense ads with better accuracy than the ones you already see. (From Jenison, I get offers to "meet singles in Kalamazoo" ... Kalamazoo is a long ways away from me, but happens to be where the other end of this DSL connection sits.)

So, no, I'm not a fan of Google Fiber.

The second reason I won't become a fan of Google Fiber is the same reason I don't plaster my car with bumper stickers; I don't care to broadcast my taste, appreciation, like or dislike of every single cause, organization or business that flies across my radar.

--
:wq

[TSoC] A different kind of cell phone service

2010-03-20T21:39:00.001-04:00

Anyone notice that cell providers these days tend to not charge roaming fees? Anyone notice that their phone itself might report itself as roaming as often as not, without it making a practical difference? What about pre-paid phones? How many of those aren't tied to a particular network at all?

If a cell service provider can provide phone service that costs the same whether you're on their towers or not, what's to stop a towerless cell provider from providing service using a phone that doesn't care whether it's connected to the service provider via CDMA, GSM or an encrypted SIP connection tunneled across Wifi or WiMAX? Heck, any of the existing SIP/IAX2 trunking providers could potentially expand into that arena, and people wouldn't necessarily require a POTS phone number to call or be called. (Though, rather than calling a Skype ID, someone could simply call me at phone.michael.mol.name.)

Admittedly, toggling between cell and 802.11abgn radios will reduce battery life, and the phone would have to be a tad smarter to manage more of the service provider hand-offs itself, but that price is already being paid by smartphone owners.

Ordinarily, I'd be one of those arguing back "you can take my landline out of my cold, dead hands", but I've somehow managed to never have a landline to my name, nor one that people call when they want to reach me since before I even turned 18. With the drop in landline usage, and the pervasive increase in mobile phone usage, it strikes me as imminently doable.

--
:wq

Have I fallen so far?

2010-03-19T16:59:00.001-04:00

I once commanded legions of GI Joe action figures; they would march to their death at my guidance and whim. Have I fallen so far?

Water once obeyed my guidance, flowing through pipe, hose and tube before sprinkling me amidst glorious sunshine. Have I fallen so far?

The sands themselves once moved aside under my machinations; the formation rivers, gorges, dams and lakes occurred at my influence. Have I fallen so far?

The frozen wastelands would strip themselves bare, the ice and snow forming men as tall as taller than I was. Have I fallen so far?

The trees would shed their leaves, simply for my joy and benefit. Have I fallen so far?

Yes, I have fallen, but the world has turned up-side down. It is now my role to serve those who will come after me. I have risen far.

--
:wq

[Photography] 16:9

2010-03-18T14:59:00.001-04:00

I want to get into photographing at a 16:9 aspect ratio.

When composing for 4:3, the rule of thumb is to divide your image into a 3x3 grid, and keep a distinct element of the picture in each grid component. I suspect that, in a 16:9 aspect ratio, the composition rule might best change to 5x3. I already know how I'd want to use the six extra grid squares in various centered and offset scenarios.

Anyone know of anamorphic lenses for photography? I haven't upgraded from my FujiFilm Finepix yet, so I'm pretty flexible as far as compatibility...

[Linux, libvirt, kvm, snapshots] Piecing together an open-source equivalent of VMWare Workstation

2010-03-18T14:47:00.001-04:00

Well, first, I should mention I've been spending a lot of time in KVM lately; paycheck depends on being able to code on Windows, and I don't have any hardware I want to put Windows on the bare metal of. libvirt+KVM works nicely as a virtual machine, and some parts of its feature set rival that of VMWare Workstation right out of the box. For example, the latest version of Workstation I have access to doesn't support direct access to PCI devices. KVM does (though I haven't played with it). Of course, there are things that Workstation does better than libvirt+kvm, as well. Let's see what's missing:

* Snapshots, full clones, linked clones
* Resize guest display to match viewer viewport. (I had to resort to some XML editing in order to get a video device I could make larger than 1024x768, and that's still not as nice as having the guest framebuffer resize automagically)
* Drag-and-drop between guest and host.
* Application-in-host (Workstation calls this "Fusion", and it's awesome if you need it. I still need to send that fruit basket to the couple developers who wrote that feature.)
* Suspend guest VM.
* Has D3D9 acceleration (for Windows guests on Windows hosts, which wouldn't work for me anyway), which is better than the no-graphics-hardware-acceleration I face with libvirt+kvm. (There is an SDL client, but libvirt's dynamic privilege toying with PolicyKit doesn't manage to get it to work on my system.)

Things that libvirt+KVM does better than Workstation (at least as of the latest version I have access to)
* Will give as many as 16 virtual processors to a guest.
* The VM definition file is XML, and well-documented on their website; manual tweaks are pretty easy.
* Has the ability to not affix a MAC address to a guest. While this causes Windows guests to destroy old NICs and create new ones every time you reboot, it will make cloning much easier.

So let's see what we can do about the things that libvirt+kvm doesn't quite do on it own yet.

D3D9 Acceleration
Not going to happen; host doesn't have Windows. On the other hand, I can put in another video card and give a guest VM direct access to it. I haven't tried it, but it could work, so long as I don't allow the host's X server to use that card.

Susped guest VM
Not going to happen until there's a to specify a file to hold a RAM image, at least. (I could easily see them mmaping that file, too; it seems like a perfectly kosher way to leverage the x86 MMU on 64-bit systems.) No such option exists, currently.

Application-in-host
On Windows and Linux guests both, this requires some awareness of and integration with the guest window manager. There are probably VNC servers that can handle that individually, and there are already VNC servers for both guest operating systems that will serve up individual windows. If you put a VNC client on the host in Listen mode, you could find a way to have every launched application connect to that client. It'd require some real script-fu in the guest, though, and there's also the problem that VNC doesn't support drag-and-drop between clients. (And typically isn't aware of it at all...)

Drag-and-drop between guest and host
That'd require some extension to the VNC protocol, so I can't do that right now. (OTOH, it would be a very, very good general enhancement to the protocol!)

Resize guest display to match viewer viewport

VMWare uses the same core protocol between its viewing interface and its virtualized video card, and its viewing interface supports the viewport and guest triggering each other to resize. (Admittedly, I don't know for certain that the viewport-triggers-guest event is communicated using the same channel.) It seems likely that it could be done, but will require an extension to the VNC protocol.

Snapshots, full clones, linked clones

Some of this is something I might actually be able to do soonish (and I'll certainly be working on it!) Linux has LVM, which, at the very least, will let me take a snapshot of a base volume, and do copy-on-write for maintaining things. I don't know that it will let me take a snapshot of a snapshot, though, or give me simultaneous r/w access to multiple snapshots of the same base view. If it does, that would put it pretty close to VMWare Workstation's capability. If not, it puts me closer to VMWare Server's. VMWare Workstation allows you to take a snapshot after running on a snapshot, which gives you base->snapshot->snapshot, and you can choose to run any of those you like without affecting the others. It also allows you to create a linked clone of any of them, which amounts to creating a snapshot that you can run at the same time as you run a snapshot from that tree. VMWare Server, on the other hand, only has two states for a VM: current snapshot, and current state. In VMWare Server, you can always replace your current snapshot with your current state ("Take snapshot") or your current state with your current snapshot ("Revert to snapshot"), but that's it.

Notice I'm not talking about VMWare Server ESX*; that beast's snapshot model is a little closer to Workstation's.

Out of all of these things, the snapshots and clones are the part I need to work on the most. Right now, snapshots and clones are pretty much the same thing; copy the disk image, copy the XML file, and modify the XML file to point to the copied disk image's location. That gets pretty expensive, diskwise, pretty fast. LVM's copy-on-write should help with that a lot. So would a filesystem with built-in deduplication, but ext4 (what I'm running on right now) doesn't have that. (I can think of a bunch of other reasons a filesystem with built-in deduplication would be awesome, too. Maybe I'll brainstorm on some different performance tradeoff option ideas in the near future.)

--
:wq

[Techtalk Tuesday] Bittorrent and swarm stability

2010-03-16T02:03:00.001-04:00

I don't know if I've blogged on this before, but it's been on my mind a while. I've noticed that Bittorrent swarms tend to be unstable.

Bittorrent chops up the data into multiple pieces, and then the various clients in the swarm play mix and match until they have all the pieces they need for a full copy. Then (if they're well behaved) they stick around for a bit longer and give more copies of the pieces to other members of the swarm.

When a client joins the swarm, it gets a mapping of all the pieces in the swarm, and a listing of who has copies of which pieces. It then asks whoever it can for the pieces it still needs to complete its copy.

The problem I've observed is that if one piece has copies in more clients than another piece, then that first piece will tend to be copied to more clients still, simply by being more available, while that rarer piece becomes more relatively rare. This causes data distribution in swarms to grow "clumpy", with the availability gap between the most common and the least common piece growing wider and wider. If the only copies of a piece exist on swarm members which already have full copies, and those members drop out, then the swarm can't form more complete copies until someone with a full copy rejoins the swarm. Meanwhile, the rest of the swarm members copy from each other until they're all not-quite-complete, but don't get any farther than that.

What I'd like to suggest are three changes to the Bittorrent swarm behavior, both on the "server" and "client" side of the connection.

The change to the client is pretty simple; look at the map of available pieces and nodes, and try to grab the rarest ones first. Even if the client is misbehaved and doesn't plan on contributing to the health of the swarm, it still makes sense to grab the rarest pieces before they may disappear.

The first change to the server is a bit trickier, and would require support from the client. If the client asks for piece A, the server should give it piece A--after it gives it piece B, which is currently rare in the swarm, and it knows the client doesn't already have it.

Aside from requiring the client to recognize that the data it was handed wasn't initially the data it asked for, I can see one other problem with the latter approach. If the client is already asking for piece B from another node, then getting a copy of piece B from the node from which it asked for piece A is a waste of time and bandwidth. It could inform the server which pieces it's grabbing, but that would also be loss to efficiency. A compromise might be met where the swarm would only support slipping in pieces from the rarest N% set of pieces, and so the client would only be inclined to report any from that set that it was already grabbing.

Varying the client's pull pattern to either fall within, overlap with, fall near or be distant from that N% segment each have their advantages and disadvantages, but I'm not sure which would outweigh the others. On one hand, avoiding the N% bracket is a destabilizing influence on the swarm, but saves incident overhead. On the other hand, favoring the N% bracket means the client grabs the rarest packets quickly, but increases the incident overhead. Favoring just outside the N% bracket means avoiding the incident overhead, but destabilizes the swarm, and makes the assumption that there are a number of "slipping" seeders.

The second change to the seeder side would be to *proactively* push data to other clients, based on observed connectivity--to some extent, each client is aware of the bandwidth and uptime of other clients. If a client is observed to have a lot of bandwidth available to it, or is observed to be a stable member at a lower level of bandwidth, it makes sense for the swarm to push rarer pieces to places where they can be copied from more quickly. (For the stable, low-bandwidth clients, this had the added benefit of reducing their particular risk of missing out on rare pieces even more.)

Just some thoughts...

--
:wq

Memetic Mutation

2010-03-15T16:12:00.001-04:00

We're no strangers to memes
You know the rules, and so do I
A full mutation's what I'm thinking of
You wouldn't get this from any other guy
I just gotta tell you what I'm thinking
Gotta make you lose the game.

Unqualified domains

2010-03-15T13:40:00.001-04:00

So I've got a home network and my own bind install. Let's say my local domain is x, so dodo might be dodo.x, alice might be alice.x, and whiterabbit would be whiterabbit.x. If I ping alice.x or alice.x, they both resolve to the same IP address.

Now let's throw another domain in the mix, accessible over VPN. This domain is also unqualified, and let's call it y, and say it has hosts alpha, beta and gamma, for alpha.y, beta.y and gamma.y.

Now, in Windows, I can tell the system to try resolving unqualified domains with a domain of x first, followed by y. That way, I can ping alpha, and it resolves to alpha.y, and I can ping whiterabbit, and it resolves to whiterabbit.x.

Under Linux, when I try to ping gamma, it doesn't resolve, but when I try to ping gamma.y, it does. Under a Windows VM routed through the same machine, pinging gamma resolves to the same thing as gamma.y. How do I get the Linux host to exhibit the same behavior? (No, I'm not going to route the host's DNS through the Windows guest.)

--
:wq

[StepMania] I've still got it. [blogging] LJ and ping.fm

2010-03-14T04:07:00.001-04:00

So I fired up StepMania again on Saturday. It'd been over a year since I'd played. Life's been busy, opportunity slim, the pad needed tuning, and opportunity has been slim.

I dropped into stepmania a couple weeks ago and got the compile issue resolved; they gave me a revision to update to that worked great. Saturday, I tuned the pad.

Fired up StepMania, and all was good.

I gave a run through a few DDR 2nd mix and DDR 3rd mix songs, Franka Potente's "Believe", and a couple variants of Sonic the Hedgehog songs. (Most of those songs make pretty good hard techno. I'd venture a guess it's because of the hardware they were originally written to play from.)

I've discovered that the only thing that really stops me from getting a good score on a five-foot song is how many songs I played leading up to it; my read speed and agility are good, but I need to work on my endurance so I can get more songs out.

I also need to work on breathing while I dance; three breaths for a 120s song is probably not a healthy rate. This is supposed to be aerobic exercise, not anaerobic.

In other news, I've finally got posting to LJ working decently. I write to my post-by-email address, copy the message body into a text editor, replace n (or rn, if I'm on a Windows box) with
, and paste that into ping.fm. Then I send the original message.

Conveniently, this gives me a place to put draft posts, even though I'm using ping.fm. Very nice.

(And while LJ strips my email signature, I leave it in for the ping.fm posts. I like it. That's why it's my signature.)

--
:wq

Applying StumbleUpon to code.

2010-03-13T20:27:00.001-05:00

While I don't know any of the specifics behind StumbleUpon's recommendation and rating systems, I've observed a large number of up-trends and down-trends of StumbleUpon referral traffic while watching Rosetta Code's analytics. Based on my observations of how traffic flow works, I suspect the same system could be adapted to random-walking code bases.

Why would you want to random-walk a code base? I can think of a couple reasons. First, you're bored, stuck or stalled, and you need a distraction from the function you're working on. Second, perhaps you need to familiarize yourself with the code base a bit better better by randomly jumping around to different files, classes and functions.

Here are a few ways to identify a place in code, thinking strictly from a C++ perspective. (Other languages have their own ideal ways of thinking about code at a component level):
* File::Line
* Class
* Function

At a slightly broader perspective (from a Visual Studio perspective. Other development environments have their own terminology):
* Project
* Solution

Getting meta:
* Bug report
* Bug comment

Of course, a line in a file probably exists in association with a class, and with a function. So you would likely have an implicit relationship there. A class or function is likely to have a relationship with a project and solution (or several, if the file it exists in is shared among multiple projects). So there are more implicit relationships. A bug report may have some association with one of the other location identifiers. More implicit relationships.

So that's the content side of things. What about the user side of things?

StumbleUpon is Yet Another Social Network. You have contacts whose likes and dislikes affect where their recommendation algorithm sends you, and your likes and dislikes affect their browsing experience.

While the "social networking is going to solve all your business problems" fad went away with the dot-com bust, there's still some value in positional association. Let's say you've got a project development team of four people, and they all tend to work on the same product or product component. Or maybe you've got a company with thirty developers, and they're all working on the same gigantic codebase. That's akin to StumbleUpon's contacts.

While they're random-walking through their codebase, the developers' individual interests and disinterests would draw each others' attention to various parts of the code base, not as something to direct their focus on, but as something to let their brains passively absorb and toy with while they're taking a step back from the draining problem they're already focused on.

Anyway, just a thought.

--
:wq

Hardware RNGs

2010-03-12T00:47:00.001-05:00

Generation of good random data is hard. Even operating systems have a tricky time doing it. Tools like puttygen implore you to move your mouse around a lot while it's generating a key, in order to add entropy to the system. Generating a gpg key has similar problems, except that the tool discards data given to it that's not random enough, and this drains the kernel of entropy used to feed the internal PRNG. (Read the man page; it warns you about it.) If you're on Linux, and you want to see this in action, do something like this:

"cat /dev/urandom"

You'll get a flood of garbage data, then a trickle, then it will stop. You've drained the kernel's entropy pool. Move your mouse around (you're not doing this on some server, right?), and you'll see the trickle resume.

So you can get good random numbers by reverse-biasing a diode and listening for thermal noise, but you only get those numbers at a low rate.

That has to be the most trivially parallelizeable hardware problem I've read of in ages. How many millions of transistors can we fit in a 1mm

So the new campaign world...

2010-03-11T02:14:00.001-05:00

I'm building a new campaign world. I don't know that I'll be able to fully describe this without pics, but here goes:

(Note, this isn't a final description, even of the parts I'm describing. Things are subject to change.)

Premise: The material plane is formed by the intersection of several other planes. The intersection is caused by a loop of (something) that acts somewhat like a gravitational field in a multidimensional space, drawing all the planes together, with the strongest point of intersection following the ring. (Not the interior of the ring, but the ring edge itself.) The ring itself is deep underground. (How deep? I dunno. Deep enough for dwarves to get at it, anyway.)

Whenever I talk about the ring, I'm talking about the actual torus itself, not the entire region it encompasses. Also, N, for now, is 20. I may scale it up or down. I don't know. "I" needs to be calculated based on the concept of the inverse cube law mentioned below, but I don't feel like doing the calcs right now.

There are six primary zones, relating to the ring:
* A2 -- Region 0-I miles from the ring.
* A1 -- Region I-2I miles from the ring.
* B -- Region 2I-(N-2I) miles from the ring.
* C1 -- Region (N-2I)-(N-I) miles from the ring.
* C2 -- Region N-(N-I) miles from the ring.
* D -- Region greater than N miles from the ring. There are two of these, one on the outer side of the ring, and one circular region on the inside of the ring, a sort of interior dead zone.

Notable artifacts of this:

* Magic is strengthened in proximity to the ring, though the strengthening effect fades according to an inverse-cubed law. (See "inverse square law", and bump it up a dimension)

Region D: magic has faded to uselessness, and the material plane has faded to void. Not space, not vacuum, just void. No rules of physics apply there. Some who've touched it are said to have ascended, while most are never heard from again.

Region C2: the material plane takes on a strong form of the chaos property. (It's been a while since I read the old 3.0 D&D splatbooks, but I remember there being a chaos plane or some such.) These are known as the "outer strangelands."

Region C1: the material plane takes on a weak form of the chaos property. These are known as the "inner strangelands."

Region A2: the material plane takes on the strong form of the "wild magic" property.

Region A1: the material plane takes on the weak form of the "wild magic" property.

Region B: the material plane behaves normally.

* Climate varies greatly near the ring

Region A1: Small, adjacent regions have very, very different climates. Storms are common as these incompatible climates mix at their boundaries.
Region A2: As with A1, but the climate regions are much larger.

Other notable things about the ring:

There are two tall mountains on the "south east" corner. One of them I've nicknamed "nattie" for the time being. Deep in the core of Nattie is an area where the hypermagic ring is twisted into a Gordian knot, leading to stronger localized effects. Closer to the ring's center is another mountain, which I've nicknamed "Everest." At an altitude fairly close to Nattie's peak, the "sun" (some bright light source; I haven't figured out exactly how hot it is) follows the hypermagic ring as something like a track. Day and night in the world are simply the sweeping of Everest's shadow as one, large sundial. The "dead zone" region at the center of the ring also casts a shadow; the lack of remotely consistent physics prevents light from traversing it. As a result, most of the world has two periods of darkness for every cycle of the sun.

There is one small anomaly at the "north west" corner. Dwarves dug deep under the surface of the world, and managed to intentionally create a loop twist of their own in one spot on the hypermagic ring. The hypermagic ring would very much prefer to remain circular, and so it will untwist itself eventually, given the chance. When that happens, all sorts of things will happen. That's not a key specified plot point, by the way; it's just a fact of the world. You make your own plots; I'm just defining the world at a static point.

There is one ocean covering the "south west" quadrant of the world, lined with a short region of hills followed by a mountain range. (The same mountain range that contains Nattie, Everest and the Dwarven magic project. The ocean extends from the outer C2 through the outer C1, B, A2, A1, inner A1, inner A2 and into the inner B regions.

The oceanic section of the A2 and A1 regions are considered perilous, and almost nobody crosses it except in times of emergency. Think of it like the dead region of wind flow near the equator as feared by wind-powered sailors hundreds of years ago, except that instead of a lack of motive energy, magic just goes wonky there.

Instead, trade between the south and west edges of the ocean follows two primary routes. One that sticks to the outer B region, and one that sticks to the inner B region. The Inner Sea is relatively crowded, with privateers and pirates, but also a large number of coastal trade points. The Outer Sea is fairly empty, but there's no room for error; if you have problems, or if you run low on supplies, you're either going to have to cross the A1/A2 regions to get to the Inner sea where you might find help, or you're going to be lucky enough to be near shore or "Atlantis" (nickname, natch) the island city that spans from the outer B region to the outer A2, used as a nigh-lawless research, trade and pirate stronghold.

At the border of outer B and outer A1, on the ocean shores, are the two largest port cities. The Inner Sea has ports that are far more numerous and dispersed.

...and that's all I have figured out for now.

Putting together a new campaign world

2010-03-11T00:01:00.001-05:00

I put together the premise behind a new "sandbox"-style campaign, along with the geologic, botanic and economic portions of the map. Need to do the demographic portion, still, as well as follow the intersections into the other planes. Then comes the political, followed by finishing the biome and building the NPC cast. I'm going to need HTML, CSS, JS and transparent PNGs to tie this all together, legibly.

I have a feeling the table code is going to break horribly.

2010-03-10T19:07:00.001-05:00

(click here to view)

Ed Stein
Self
Mar 10, 2010

EditorialCartoonists.com

So I've put in nearly 3000 miles on my car in the last two weeks.

2010-03-09T00:40:00.001-05:00

Two solid runs of 1000 miles, a solid(ish) run of about 450 miles, and, just Monday morning, a solid run of about 300 miles.

I think she (the car) likes it; she didn't give me a single complaint. I like driving distance. I'm looking forward to putting more miles on her at the right opportunities.

I have a family member who's pushing her car (A '91) to hit the 500k mark, and it's almost there. My 1996 Buick Regal has about 130k on it, but I'm not pushing for a mile marker, I'm pushing for 2016, when it will be 20 years old. Theoretically, it'd be eligible for a Michigan historical vehicle license. I bet I can still have her running as well as she is today. I get anywhere from 26 to 38 mpg, depending on road conditions and how I drive, but I haven't tracked my mileage too closely.

I like my car. We work well together.

Magnetized gorillapod

2010-03-08T22:23:00.001-05:00

So my brother asked me if I could cross a camera magmount with my gorillapod, and I've been considering how to do that.

The tricky part is replacing a gorillapod foot with a magnet. My current though is to take the existing foot off the link chain, put the head of a bolt in the recess left behind, and secure the bolt by filling the recess with some sort of ceramic or plaster. Then I'd screw on one of these.

The only thing I really don't like about it is that, compared to my existing magmount, which uses one of these rubber feet, the magnetized gorillapod is going to be hell on surfaces I attach it to; K&J doesn't make the rubber feet for the right size of magnet, and I definitely don't need the 75lb of surface force provided by their 32mm mounting magnets, which is the smallest size they offer the rubber feet for. (I use one of those 32mm magnets as the basis to my existing magmount, and that's more than enough to keep my camera affixed to the top of my car while driving down the freeway. (By the way; the reactions you get from people when you start drivingdown the road with what looks like a DSLR on top of your camera can be hilarious; the magmount is low profile enough to not be obvious. I should paint the camera orange/black or something so people get the idea that it's supposed to be there.) So I've got two things to resolve before I build it. First, how to make plaster (or some better material equally as cheap to get/make/use.) Second, How to put a protective coating on the magnets so they don't scratch painted surfaces.

The problem I see with IPETEE

2010-03-08T00:34:00.001-05:00

So I just stumbled across this article. It's funny, cause I gave it some thought a few months ago, and there are a few problems with end-to-end encryption as a means of concealment.

First, activity indicates activity. So you connect to a tracker that manages prohibited data. Prohibited how? Doesn't matter. Whether encrypted or not, you have a connection between you and a server managing prohibited data.

Ok, fine. You jump that hurdle by using something like Tor.

Second, there's more than just data, there's metadata. So you don't know what exactly is in a TCP or UDP stream, but you can learn things about it, just the same. What do you know about the packet size? Time between packets? Is a constant data rate maintained? How long does the connection last? Certain types of traffic aren't bandwidth-sensitive, but rather latency-sensitive. VOIP is going to have a distinct pattern. Torrenting is going to have a specific pattern. It's techniques like these that allow forensic specialists to detect hidden TrueCrypt volumes.

Third, encryption doesn't eliminate the data, it just obscures it, and not perfectly. Even aside from the possibility of cracking the encryption key, it's possible to guesstimate what kind of data an encrypted bitstream represents. I've read of that being used on encrypted hard drives, for example.

Ground drove my computer loopy.

2010-03-04T23:20:00.001-05:00

So I've spent the last seven hours trying to get my desktop to boot something, anything, that's a full operating system. For much of the time, the system was hanging randomly between POST and loading the boot sector, and for much of the rest of the time, it was consistently hanging when I tried to load a Linux kernel. I even wound up going so far as to flash my BIOS, because I couldn't think of anything else to try that made any sort of sense.

The flash upgrade had gotten me to the point where I could at least load boot sectors again, and I was able to run memtest off of live CDs, but I couldn't seem to boot into 32-bit or 64-bit Linux, either my installed version or from a couple Xubuntu live CDs.

I was beginning to suspect some sort of weird flash corruption that was preventing me from using graphics card features, or possibly from switching to protected mode or x64 mode. (I don't know how memtest86+ works, as far as accessing all 8GB of my RAM. I'm pretty sure the BIOS is still in Real mode when it runs its initial sweep, but maybe it's bouncing back and forth between Real and Protected during POST.)

The thought of another outlay to continue having a nice computer at home was not appealing.

Finally, a few minutes ago, I realized something. I had two relatively new pieces of hardware attached to the computer: An APC UPS and a powered USB hub. I disconnected the USB connection between the UPS and the computer, and disconnected the hub, and rebooted the computer. The 32-bit Xubuntu live CD came right up. Huh. Reboot, throw in the 64-bit Xubuntu live CD, and *that* came right up. Huh.

I haven't tried booting off my hard disk yet, and I think that'll require some grub command-line magic to deal with device reordering stemming from the BIOS upgrade and CMOS changes. However, I think it's ultimately workable.

What I think happened is that the connection of the UPS and the powered hub to the computer via USB led to a ground loop that was messing with the internals of the USB controller on the motherboard. See, the powered hub isn't plugged into the UPS; it's a fair bit away from the computer, and I'll have to run an extension cord to get the UPS's power to it. As long as the operating system didn't try to do too much with that USB controller, things worked fine. That meant I could get into BIOS and tweak things, and it meant I could get into grub and memtest without too much trouble. Well, sortof. Remember how, before the flash upgrade, the system would hang at a random point between POST and loading the bootloader. I think the flash upgrade may have changed part of how it dealt with the USB controller, with the newer version inadvertently working around some of the ground-loop-induced weirdness.

So, yeah. A little more experience for those weird situations.

Depth perception and optics

2010-03-04T01:48:00.001-05:00

It's often an assumption that two or more vision sources are required for depth perception. That's not strictly true; on one hand, there's simple experience that can teach a monocular person to recognize distances.

On the other hand, there's this idea I had...

*sidetrack*
It stemmed from thinking about image stacking in HDR photos. Image stacking is simply taking several photos of the same object, where the only change between them is a known, measured characteristic of the photograph recording. Typically, in HDR, it's your exposure adjustment, or how much light you demand your camera collect before saving off the frame. Since your camera's sensor can only precisely measure a finite range of light levels, taking several frames where you move that range around allows you to increase the amount of detail you've captured in bright and dark areas.

However, image stacking needn't *only* be done with exposure adjustments. The concept of stacking applies for any variable you can measurably control while observing a scene, and it just happens that exposure adjustment is the most immediately useful setting to vary.
*end sidetrack*

*second sidetrack*
And now a brief bit about aperture width and depth-of-field. A camera's aperture is the hole that allows light to pass through and land on its sensor. It's exactly analogous to your pupil. The larger your pupil, the more light passes through and lands on your retina. The smaller your pupil, the less light lands on your retina.

One weird side effect of optics and aperture size, though, has to do with focusing. With a narrow aperture (such as when you're outside on a bright, snow-covered day), your depth-of-field is very large, meaning that you can see near objects in focus just as well as you can see distant objects, with no additional effort on the part of your lens. (either that of your eye or of your camera) On the other hand, when the aperture is very wide (Indoors, lights off, etc.), the depth of field is very narrow. That means that you (or your camera) need to adjust the configuration of your lens in order to focus on near or distant objects, regardless of whether you have one or two eyes.

All this boils down to one interesting fact: You can know where your field of focus is if you know your aperture size and your lens configuration.
*end second sidetrack*

What all this means is that you could take several images, each with known aperture and lens characteristics, and learn how far away objects in the scene are simply by observing how in-focus each area of your scene for each known field of focus.

So, yes, you could have calculable measured distances to objects in your scene simply by stacking images of that scene where you know the focus range of each of those snaps.

Honestly, though, I think this applies more to machine vision than human vision. It's also probably most useful right now for telescopes.

A quick note and idea on fuel efficiency...

2010-03-04T00:57:00.001-05:00

...because it's been churning in my head since last Tuesday, and I haven't written it out yet.

When I was driving to South Carolina and back, I discovered I was going 80mph without pushing the car's engine much at all. (The amount of work I was asking the engine to do might have kept me going 55 or 60mph, normally. Maybe 65mph on the average downgrade on I-77.) It took me a while, but I finally came up with a plausible reason this happened.

I had merged onto the freeway, found myself embedded in a large pack of big-rigs semis, hadn't settled into traffic enough to set cruise, and had to move to get out of the way of traffic shifts to let another batch of traffic merge in. I signaled, changed lanes to the left and accelerated--and felt my jaw drop when I saw I was effortlessly going 80mph.

What I think happened was that the sparse pack of semis (and filler of smaller vehicles) spanning three lanes had caused the volume of air covering I-77 to be moving roughly uniform in line with traffic, not too far from the speed of traffic. In effect, the traffic had created its own wind tunnel, and you didn't have to be dangerously close to the back-end of a semi to get a drafting effect.

So what would happen if you took a long stretch of road (such as an under-river or through-mountain tunnel) and set up blowers pushing air in in the direction of traffic flow? Given the reduction in air drag, how much would you save on fuel economy? Would the energy involved in maintaining a 15mph tail wind in dense-traffic areas be greater than the aggregate saved energy in vehicle fuel? How much of an impact on local pollution would it have? By pulling energy from the electrical grid rather than car engines, you can move the atmospheric cost of energy generation away from areas where it could cause health problems. (Places with high pollution due to vehicle emissions might find that useful...)

Anyway, it was an interesting experience, and leads to a lot of interesting questions.

I finally realized why shutter glasses make sense.

2010-03-03T14:25:00.001-05:00

A shutter glasses system allows the video source to provide two video channels, one for each eye. That lets you have stereovision, so you get depth perception for whatever you're watching.

So do polarized light systems like IMAX's linearly-polarized glasses, and Real3D's circularly-polarized glasses.

So why do shutter glasses make sense? There's nothing about the concept that limits you to two video channels. You could have four, or eight, or even sixteen, once the technology's chronometric precision increases enough. It's time-division multiplexing for the visual field.

That's going to have major, major implications for video games. No more splitscreen forcing you to have only half or a quarter of your normal visual field; you and your three buddies can each have the full frame to themselves.

It might even give rise to a new form of cinema, one where both the protagonist's and antagonist's stories are told at the same time, but the viewer has to choose which one they're watching.

Software vs hardware RAID

2010-03-01T12:53:00.001-05:00

There are three common types of RAID implementations: "hardware"(high-end RAID cards), BIOS-level software(low-end RAID cards) and system daemon-level software(such as the md daemon on Linux).

The RAID support that comes built-in with your motherboard is *probably* software RAID implemented in your system BIOS. Most small RAID cards have their functionality implemented in on-card BIOS that gets loaded when the system boots. As with system-software RAID setups like md, these RAID setups consume your system CPU and RAM as they churn to perform the calculations associated with your RAID configuration, including things like ECC and parity calculations.

All of this is best-known as "software" RAID.

The funky thing? "Hardware" RAID is also done in software, albeit software that executes on a dedicated processor on your RAID card. (As opposed to your system CPU.) You upgrade this software whenever you upgrade the firmware on the card.

So here's what I'd like to see...An Open Source hardware RAID card. All it would amount to is a multi-port SATA controller connected to an onboard CPU, passing data back to the host operating system via the PCIe bus.

The firmware would be built by, well, I dunno. The folks who like md, but don't like running it on their core system. The folks who want to play around with experimental RAID configurations and ideas. The folks who want to try putting LVM into "hardware." There are plenty of possibilities.

Put a fallback firmware set on the card, in case of flash upgrade failure, to avoid bricking the thing.

I'd hit that.

Loving "Baccano!"

2010-02-27T16:59:00.001-05:00

Things I love about it.

The characters are colorful -- "Yeah, that was impossible. At least we did the next best thing, stealing the entrance!" "Guns? Guns? I hear guns! This is getting exciting! This is getting exciting!"
The dialog is a blast -- "YesYesYesYesYesNoNoNo, Still no, Hell No!"
The animation is excellent -- See Ladd Russo's initial fight in the dining car.
The story presentation is a blast -- Three stories presented concurrently, containing some common characters. Many scenes get revisited from different characters' perspectives. Most of the story is presented out of chronological order; if it weren't different parts of the stories would spoil parts of other stories.

Other notable components:

Set in the late 1920s/early 1930s.
There's a lot of violence. You may or may not enjoy it.

A realization...

2010-02-26T15:13:00.001-05:00

Rather than get a new domain for each idea I may or may not complete, I should get one short domain, and set up all those ideas on subdomains.

HDR of a Charleston alleyway

2010-02-24T23:47:00.001-05:00

My second HDR result from the trip this past weekend. I don't have any more bracketed sets, so that's the last HDR from that trip.

Other interesting pictures from this past weekend.

2010-02-24T17:35:00.001-05:00

Click through for descriptions on Flickr.

My first HDR that only sucks a little bit.

2010-02-24T12:43:00.001-05:00

Mix tapes for driving.

2010-02-18T23:46:00.001-05:00

So I'm making a pair of 90-minute-per-side mix tapes for driving, for listening to during a 15 hour drive.

Yes, tapes. My laptop's being repaired, and I don't have any other decent audio player device at the moment.

To make these tapes, I created a playlist of songs in Rhythmbox that I would be willing to listen to. I don't know how many are there, or how many minutes' worth of music is there. Suffice it to say it's a lot. I also threw in all of the ads and callsign announcements from Streets of SimCity, because they give me a chuckle. I have the playlist playing in Shuffle mode, and I have a dual-tape deck sitting between my computer and stereo syste, configured to continuously record, looping through all four sides of tape.

When I wake up, I'll pop the tapes out. They'll have six hours worth of music on them, and I won't really know what's on there and when.

This will be interesting...

Picked up a Wiimote today.

2010-02-17T19:18:00.001-05:00

Picked up a Wiimote today. I don't have a wii. I want to see if I can get it working as a mouse.

The mapping I have in mind:

As long as the trigger is being held, its motion will affect the mouse cursor's position. When the trigger isn't being held, the mouse cursor's position isn't affected.
The D-pad will be up/down and left/right scroll events.
The A button will be the Left mouse button. That leaves buttons 1 and 2 for Right and Middle.

Hopefully, that will cover most use cases. I don't really care for having the right and middle buttons there, but I prefer the D-pad for scroll events more.

A notice to the DIY folks.

2010-02-15T14:42:00.001-05:00

I don't normally go for the chain letters, but this one is worth bringing attention to.

Don’t let this happen to you.
Women often receive warnings about protecting themselves at the mall and in dark parking lots, etc. This is the first warning I have seen for men. I wanted to pass it on in case you haven’t heard about it. This will only become more commonplace.

A ‘heads up’ for those men who may be regular Lowe’s, Home Depot, or Costco customers. This one caught me by surprise. Over the last month I became a victim of a clever scam while out shopping. Simply going out to get supplies has turned out to be quite traumatic. Please, don’t be naive enough to think it couldn’t happen to you or your friends.

How the scam works:

Two seriously good-looking 20-something girls come over to your car as you are packing your shopping bags into the trunk. They both start wiping your windshield with a rag and Windex, with their breasts almost falling out of their skimpy T-shirts. It is impossible not to look. When you thank them and offer them a tip, they say ‘No’ and instead ask you for a ride to McDonald’s. You agree and they get into the back seat. On the way, they start undressing. Then one of them climbs over into the front seat and starts crawling all over you, while the other one steals your wallet.

I had my wallet stolen November 4th, 9th, 10th, twice on the 15th, 17th, 20th, 24th, & 29th. Also December 1st & 4th, twice on the 8th, 16th, 23rd, 26th & 28th, three times last Monday and very likely
again this upcoming weekend. So tell your friends to be careful. What a horrible way to take advantage of men.

Warn your friends to be vigilant.

P.S. Wal-Mart has wallets on sale for $2.99 each. I found cheaper ones for $1.99 at K Mart. Also, you’ll never get to eat at McDonald’s. I’ve already lost 11 pounds just running back and forth to Lowe’s, Home Depot, and Costco.

Adaptive keyboard layout

2010-02-15T00:26:00.001-05:00

This idea came up as a result of a lot of ruminations on data compression, and more specifically on huffman encoding.

Now, I'm a QWERTY touch typist. I can get by just fine on a blank Das Keyboard, if you give me a QWERTY layout. I realize, of course, that a lot of folks prefer DVORAK; having your more-frequently-used keys on home-row has to be great.

Even as a QWERTY touch-typist, I think there may be room for improvement. Consider the advantage Dvorak offers, with the most-frequently-used keys on home row. That only works under an assumed use case, which may or may not be accurate depending on anything from your locale to how you normally use a computer.

Let's say your keyboard kept a running record of your last twenty-thousand keystrokes. Every time you pressed a non-modifier key, it would push that key on to the queue, pop the twenty-thousand-and-first key off, and then run the entire buffer through huffman encoding.

The keyboard now has an internal representation of which characters you use most frequently, in the form of a huffman tree. Because it knows its own physical form factor, it can estimate the cost of each key on the board based on how much travel there is from home row to that key.

Simply sort the key cost in ascending order, and then walk through the huffman tree, assigning each character encountered to the least-cost key remaining.

You now have a possible keyboard layout optimized specifically for you and your use case.

For me, what I'd like the keyboard to do (at that point) would be to sit tight unless it discerns that a cost reduction threshold would be crossed; say, 10% cost reduction compared to my current activity and use case. At that point, it should beep at me, or flash a little light. Once I acknowledge, the keyboard layout would change.

Of course, something like this would depend on a visual feedback mechanism; I would have to be able to quickly see what the keys were remapped to. Under normal circumstances, the rearrangements shouldn't be much, and most of the layout would remain the same. I should only need to note that two or three keys (not key pairs, mind you) changed places.

I'd want to switch away from typing my password in, though; someone could get a lot closer to guessing it by noticing one or two strange characters that are closer to my home row than for most.

There's also the round-robin key buffer, which I think may be necessary for preventing old keystrokes from polluting the huffman tree. Perhaps simply randomizing which key in the buffer is replaced would be sufficiently optimal for the purpose, rather than storing keystrokes in order.

I think I could get used to a keyboard like that.

LOLCODE and GAFFERS

2010-02-14T15:45:00.001-05:00

Someone once asked me if Rosetta Code would accept LOLcode as a language. I told them that if they could provide a working implementation of the language, and examples that executed correctly with that implementation, that it was fine by me.

We never did see any submissions of LOLcode, but that's beside the point; it's technically possible, because a working implementations do exist.

The reason I'm remembering this today is the discussion of a language named GAFFERS that's recently shown up on the site. There was question of whether or not the language was a hoax, as one of the editors didn't turn up any reliable references while looking it up. I dug into the information provided and didn't find anything to back it up. It appears to have simply been a prank by a vandal. (Not a particularly good one, either; if they'd at least have gone to the trouble of creating convincing-looking references, I might have been amused and impressed.)

If the LOLCODE want to try their hand at getting a decent task coverage on Rosetta Code, they're welcome to. Just follow the steps for adding a language.

If whoever is behind GAFFERS wants to, well, they've got to show an implementation first.

Things I hate about posting to Facebook.

2010-02-14T11:42:00.001-05:00

I use ping.fm to get most of what I write to places where people prefer to read it. Pretty much all of the services ping.fm will post to support a decent subset of HTML, embedding of Flash applets, etc.

Except Facebook. And that's not entirely ping.fm's fault.

With Facebook, I can't post a "note" (Called a blog entry anywhere else) that has text content, followed by an embedded video, followed by text content. At best, ping.fm can convert that to a video post with attached text. At worst, it'll show up as the raw HTML data for embedding the video. I've found I prefer the latter approach, because there's *no* other way to embed multiple videos.

Also, Facebook's HTML subset is ridiculously limited, almost as bad as Slashdot JE's. (At least FB allows inline images!) That means that simple formatting instructions that might otherwise have worked fine, like the "div" blocks that were (and maybe still are? I stopped using them) used by places like blogthings get defanged and look as ugly as embedded flash videos. No, I don't want to add an app to show that particular survey. No, I don't want to add an app for the "which character/what personality type/what are your politics" widgets. Facebook app privacy is so ridiculously poor, I can't even refuse to add the stupid things in defense of my own privacy; Farmville and Mafia Wars already have access to my profile information, just because my contacts added them.

You think Google is a privacy threat? At least a savvy person can use them while not giving them data, thanks to their preference towards open protocols and open APIs. With Facebook, use it or lose it. I got a Facebook account to deal with all the invites I was getting. It's also the de-facto way of staying in passive touch with people you've known from the past; the modern replacement of the Rolodex.

I love being a geek...

2010-02-14T11:09:00.001-05:00

(10:26:14 AM) Mike: I found part of the source of our communication problem!
(10:26:30 AM) Mike: http://tvtropes.org/pmwiki/pmwiki.php/Main/BuffySpeak
(10:36:41 AM) J: :)
(10:37:12 AM) Mike: "BuffySpeak" should be our clarity codeword from now on. It'll drive T nuts. :D
(10:37:29 AM) J: lol
(10:37:35 AM) J: thas almost enough reason to do it :)
(10:37:42 AM) J: probably drive me more nuts though
(10:38:30 AM) Mike: Hey, it's faster than "You're using too many unresolved pronouns!" or "Linker error: symbol 'thing' is ambiguous."
(10:38:53 AM) J: Hey, I like the 2nd
(10:39:03 AM) J: I imagine that P would fall in love with it
(10:39:17 AM) Mike: *chortle*
(10:39:24 AM) Mike: I love being a geek.
(10:39:44 AM) J: chortle?
(10:39:47 AM) J: seriously?
(10:39:52 AM) J: I havent heard that in years
(10:39:59 AM) ***Mike double-checks the definition.
(10:40:07 AM) Mike: er.
(10:40:11 AM) ***Mike inverted that one.
(10:40:20 AM) Mike: What's 1/chortle?
(10:40:33 AM) Mike: You know, reversing the amplitude, but not the sign?
(10:40:34 AM) J: Umm, you used it correctly. its just obscure afaik
(10:40:43 AM) J: um, no idea
(10:41:15 AM) Mike: ROFLMAO, I suppose, except it wasn't quite that strong.

Loving "Baccanno!"

2010-02-13T11:55:00.001-05:00

I'm loving Baccanno!

"That was our first kiss!"
"Wait, you two have been an item for ten years, and he never kissed you? I mean there's gentlemanly, and then there's stupid! Hey! I'm talking about you, too, 'princess!'"

Why having been a tutor does not mean you communicate well.

2010-02-12T16:40:00.001-05:00

I tutored computer subjects in community college for about five years. I figured that being able to teach people how to do things in one-on-one sessions made me a good communicator.

It didn't, and doesn't.

As a tutor, my chief strategy was to provide the student with all the bits and pieces of information required to build an understanding of what they were trying to do, but to systematically avoid actually connecting the dots for them. I found that that system worked very well for the students who wanted an understanding, not just a grade. (Which of those two types of students were normally the ones to come in for tutoring is a topic best avoided.)

It's only taken me three years of real-world experience to realize that while that coworkers aren't students, and I need to explain my thought process, rather than teach it.

In fact, that's probably a good habit to get into outside of work, too.

So I finally sent my laptop in for repairs.

2010-02-12T14:26:00.001-05:00

I'd worn out my alt key, my wifi chipset seemed to be overheating, and the display was exhibiting these annoyances:

Before sending it in, I put on a blank install of xubuntu, a weak password, auto-login, and had Firefox automatically start, maximized, with a local homepage including a complete description of the problems, and localized playback (which I tested without network access) of the above photos and video. I also explained some things they could skip, being merely cosmetic fixes, and told them not to bother putting Windows back on if it was going to be a hassle for them. (I didn't have a restore CD to provide.)

Since all the problems I've been seeing are based in hardware, they should be able to either fix it or declare it a lost cause. If they declare it a lost cause, my warranty says I'm supposed to get a replacement or check to go towards a replacement, so it's no skin off my teeth.

It'll be interesting to see what happens.

Awsome. Just awesome!

2010-02-11T09:38:00.001-05:00

I predicted this five years ago, while chatting with some Computer Club members.

Well, not exactly this. They're using Pentium-class cores, where our discussion compared i686 and i386 cores.

I'm torn on their using a 32-bit core as a base, though. I spend enough time on 4GB+ PCs that I loathe the idea of dropping back to a 32-bit memory model, though; There really is an advantage to having more than 4GB of directly-addressable address space.

Benefits of tech support

2010-02-10T22:13:00.001-05:00

Years and years ago, I was The phone support tech for a dial-up ISP. For the past few years, I've done phone support for the software I help write.

While doing phone tech support can be a grating, humiliating, degrading experience, if you do it right, it can also be incredibly rewarding; I just came across a couple Thank You notes that were sent to that ISP years ago, addressed to me. A couple years ago, one of the people I helped as part of my current job sent the company a fruit basket.

Whatever you do in life, if you're helping someone as a matter of course, chances are it's a worthwhile habit. If you're lucky, they'll remind you of that fact.

Ok, this song is stuck in my head.

2010-02-08T17:08:00.001-05:00

Now it can be stuck in your head, too.

Watched Avatar again yesterday...

2010-02-08T12:09:00.001-05:00

...And was mildly disappointed at one point.

At the point, a researcher is trying to teach Jake Sully the intricacies of the Na'vi language. It goes something like this:

Researcher: (Na'vi)
Jake Sully: (Na'vi) (incorrectly pronounced)
Researcher: (repeats)
Jake Sully: (repeats), looks up with an exasperated grin.
Researcher: "Literally, it means 'I see you,' but it's not just 'I see you before me', but 'I see into you.'"

While a well-spent ten-seconds of film, I think that would have been a heck of lot funnier/more interesting/whatever as:

Researcher: (Na'vi)
Jake Sully: (Na'vi) (incorrectly pronounced)
Researcher: (repeats)
Jake Sully: (repeats), looks up with an exasperated grin.
Researcher: "Literally, it means 'I see you,' but it's not just 'I see you before me', but 'I see into you.' Like 'grok.'"
Jake Sully: "'Grok?'"

(And would have had a nice side-effect of catapulting 'grok' back into popular awareness...)

"real men just upload their important stuff on ftp, and let the rest of the world mirror it ;) "

2010-02-06T15:16:00.001-05:00

Thus spake Linus Torvalds.

I've been thinking about a recurring issue...Namely that none of RCo's remote backup targets are very good. My home system has a problem remaining alive for any extended duration, and I don't have any other good prospective places to I can trust to send the data. (No offense to anyone who's offered, but it's hard for me to totally trust someone I've never met in person.*)

I may have hit on a novel solution, but I want to run it past a bunch of people (namely, you), before I do something this crazy.

Step 1: Take backup on server
Step 2: Compress backup to tarball.
Step 3: Encrypt tarball using a GPG and a long, long public key.
Step 4: Build a torrent.
Step 5: Add torrent to RSS feed.
Step 6: Anyone who wants to help can point their torrent client at the RSS feed. Data's encrypted, so I don't have to worry. With enough seeder boxes out there, there can be several full copies out there.

Bonus: Server data migration happens much, much faster. :)

* Yes, this is me we're talking about, and I realize the irony.

Cool trick of the day: "Uploading" URLs.

2010-02-05T13:53:00.001-05:00

I know this works on Windows 7. I don't know about Vista or XP. I also don't know if the file dialog Firefox uses on Linux duplicates the behavior.

So you want to upload a file to some web form. Click Browse, that traditional file upload dialog pops up.

Into the "Filename" field, type (or paste) the full URL to the file you want to upload. For example, if the file already exists on Flickr or some other site, and you have the URL to the file at that location, paste it here.

Click Open; the dialog will appear to hang for a bit, but that's because it's downloading the file to your local system. When the dialog disappears, the path to its local location will appear in the upload field.

Proceed with however you would normally use the web form; the file was already copied to your local system, Firefox knows where it is, and it will be uploaded when you submit the form.

So the EFF is doing a fundraiser, hosted by Adam Savage.

2010-02-03T13:12:00.001-05:00

They're hosting it at the DNA Lounge. What they don't mention is that the Jamie Zawinski works there. Who's he? The guy who maintains the XScreenSavers package.

Man, I wish I could be in SF next week.

EFF, Adam Savage, but best of all? JWZ. His screensavers package was the reason I dropped gaming on my Win95 box over a decade ago and switched to Linux.

Linux answers the greatest of questions.

2010-02-03T01:37:00.001-05:00

shortcircuit@dodo~
01:35:48 $ whoami
shortcircuit
shortcircuit@dodo~
01:35:50 $ who
shortcircuit tty7         2010-01-30 18:29 (:0)
shortcircuit pts/0        2010-02-02 20:32 (:0.1)
shortcircuit pts/1        2010-02-03 01:28 (:0.1)
shortcircuit pts/3        2010-02-03 01:30 (:0.1)
shortcircuit@dodo~
01:35:52 $ whois shortcircuit
No whois server is known for this kind of object.
shortcircuit@dodo~
01:35:55 $ whatis shortcircuit
shortcircuit: nothing appropriate.
shortcircuit@dodo~
01:35:57 $

That's going to smart.

2010-02-02T16:03:00.001-05:00

I wonder how many businesses are using Google Apps, as well as legacy web systems that only work with IE 6. Quoth an email I got from them today:

Dear Google Apps admin,

In order to continue to improve our products and deliver more sophisticated features and performance, we are harnessing some of the latest improvements in web browser technology. This includes faster JavaScript processing and new standards like HTML5. As a result, over the course of 2010, we will be phasing out support for Microsoft Internet Explorer 6.0 as well as other older browsers that are not supported by their own manufacturers.

We plan to begin phasing out support of these older browsers on the Google Docs suite and the Google Sites editor on March 1, 2010. After that point, certain functionality within these applications may have higher latency and may not work correctly in these older browsers. Later in 2010, we will start to phase out support for these browsers for Google Mail and Google Calendar.

Google Apps will continue to support Internet Explorer 7.0 and above, Firefox 3.0 and above, Google Chrome 4.0 and above, and Safari 3.0 and above.

Starting this week, users on these older browsers will see a message in Google Docs and the Google Sites editor explaining this change and asking them to upgrade their browser. We will also alert you again closer to March 1 to remind you of this change.

In 2009, the Google Apps team delivered more than 100 improvements to enhance your product experience. We are aiming to beat that in 2010 and continue to deliver the best and most innovative collaboration products for businesses.

Thank you for your continued support!

Sincerely,

The Google Apps team

Email preferences: You have received this mandatory email service announcement to update you about important changes to your Google Apps product or account.

Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043

WaitForSingleObject

2010-02-01T20:06:00.001-05:00

Coworker: "I've got a wait open, so when you're done there."
Me: "WaitForSingleObject?"
Coworker: "Yeah."
Me, to boss: "He's got a handle on you."
Boss: *facepalm*
Coworker: *grabs a large piece of equipment*
Me: *ducks*

Ok, what was the name of that thing?

2010-01-31T17:48:00.001-05:00

While it's a crossword, it's more a specific type of crossword, without clues to what the words are except knowing their length.

Consider an NxN square, where no cells are blocked, and every row and column contains a whole word.

There's a specific name for those; they're notable because there are few known solutions for each N. I can't remember the name!

Wait, what?

2010-01-28T10:54:00.001-05:00

I'd just received a .hack//SIGN OST 1 audio CD. I love the music, but...

This is a zoomed in view of a particular part of the spectrum, as shown in Audacity. It's a solid, loud 16kHz present through nearly all of the song. At the volume I was listening at, it was painful!

I doubt it's an encoding artifact; The CD was ripped to FLAC. Here, I imported the FLAC into Audacity.

I don't know if the master was like this, or if I wound up buying a counterfeit disc.

Note to self: Check these out later.

2010-01-27T14:28:00.001-05:00

http://www.projectrho.com/rocket/rocket3i.html

http://www.irrationaldesigns.com/TSOD/products.html

http://www.brokenthorn.com/Resources/OSDev7.html

http://www.project-euh.com/word_up/

Colonel Panic!

2010-01-27T12:11:00.001-05:00

Most of us who've used Linux for any significant length of time have had a kernel panic. Lately, those events have probably been while in X, making accessing the actual details of the event difficult if you haven't memorized the magic SysRq keys (assuming your kernel was compiled with the relevant support).

A couple workarounds are available to deal with this; You can set up a serial console so that system messages get dumped via good old RS-232 to another machine, or even a console on your parallel port so that the messages get printed.

Don't try that last unless you don't mind going through a ton of paper, and have ways of dealing with the printer noise.

It occurred to me, a few days ago, that x86 systems have a (very) small amount of non-volatile RAM, battery backed by the same battery that keeps your computer from losing time when you power cycle it.

While that amount of memory is very, very small, it may be large enough to hold a Panic message; That'd be a nice place for me to be able to look to investigate why my computer randomly hard-locked one evening while I was away.

Some mnemonics for "ls" parameters

2010-01-26T12:56:00.001-05:00

It can be a pain remembering all the different options to "ls", but I've found that most of my needs (outside of simple "ls" or "ls -l") fall into one of these three, easy-to-remember mnemonics:

ls -ltr # Sort by time, reversed, show more details.
ls -Ssh # Show and sort by human-readable size.
ls -Ssl # show and sort by human readable size, but show more details.

Graphic background doubles nicely as a desktop background

2010-01-26T01:12:00.001-05:00

This is one of the images I came up with as a component for images on fundraising items. However, I happen to think it makes an awesome background image. It's made for tiling, not stretching. Enjoy.

(Sorry, I'm keeping the 300dpi version for myself...)

Fast track to a massive parallel computer

2010-01-25T21:41:00.001-05:00

The vast majority of people who will read this have Adobe Flash installed. (Most will probably have Adblock or Flashblock installed, which has relevance, but not that much)

I've been thinking about parallel computation for for a while, and looking at different ways to do it. The best explicitly-known parallel computation setup out there are the @home projects like SETI@Home and folding@Home. These projects divide their workload into chunks, pass those chunks around, and programs running on volunteers' computers take these chunks, chew on them and pass them back.

A few weeks ago, I had the idea that someone could distribute an @home-style client as a flash applet using the one distribution mechanism many of us already hate--website banner ads. Have a flash applet distributed via advertising networks that:

Connects to a central server

Grabs a data chunk

Chews on it

Uploads the results to the central server

Flash already allows applets to pull remote files and perform GET requests. (I don't know about POST, though.) All that remains is grabbing and running the workload, and have a server that can verify that the workload results are uncorrupted. (i.e. not tampered with.)

Yes, this results in spinning an end-user's processors more than they would be otherwise, which may cost them a few more cents per month on their electric bill, or a notable reduction in their laptop's battery life. It would be helpful if Flash had a mechanism to change applet resource consumption based on system power state and/or user preferences. Still, yes, it's coopting a user's computer to do something they didn't explicitly agree to. Someone else can deal with the ethics; I'm just spouting an idea.

Also, yes, a large number of pages are only loaded for a very short period of time; The amount of time required to process a chunk of work may well require more time than the page will be loaded (and, thus, more time than the applet will be running). There's a workaround for that, too. First, Flash allows applets from any given domain to store a configurable amount of non-volatile data on the user's machine; That could be used as a data store for intermediate state. Second, the applet could wait a couple minutes before beginning the number crunch, looking for times when a page is left loaded for an extended duration. (Quite likely on sites like Youtube, Pandora and others with streaming media.)

Turning down Maven

2010-01-25T13:05:00.001-05:00

I've gotten a couple invites to Maven. I ignored the first, then replied to the second with this:

I appreciate the offer, but reading through just this email, it strikes me as an Amway equivalent where the product is professional services, not hard goods. Roughly intriguing, as that's something I can do, but I've known a bit too much about such pyramid product-selling systems. Eventually, there's nobody left that the tail-end guy would want to bring into the system, meaning all he gets is selling his own time (minus a cut to the Maven system). With the focus on knowledge-based sales, that pool gets even narrower.

I won't be joining; You don't need to send me any more invitations. Thank you for the offer.

It's all fun and games...

2010-01-22T02:44:00.001-05:00

A friend of mine (related to another friend of mine) who has a glass eye has this to say:

"It's all fun and games until someone loses an eye--and then it's hilarious!"

He's nice to have on the gun range, too.

Pics on the wing

2010-01-19T21:03:00.001-05:00

No, pics on the wing, not pigs.

Anyway, I took this while flying from ATL to MDW. Came out really neat; I don't know why or how. I like it, though.

Complete set from that flight is here.

Looking for a keyboard/mouse combo device

2010-01-19T17:15:00.001-05:00

I'm looking for a decent, wired keyboard/mouse combo device. I'd love one of these, but they're prohibited at the office. (The noise is apparently too distracting.) I have one of these, and I like it, but I don't care to be replacing or charging batteries on a keyboard I'm continually using.

Ideas or recommendations?

I didn't realize how much of a loser I was until Google spelled it out for me.

2010-01-18T15:56:00.001-05:00

moar funny pictures

"Ghost in the Shell: Standalone Complex 2nd Gig" and "Toccata & Fugue in D Minor"

2010-01-17T02:03:00.001-05:00

I think one could create an interesting AMV that took GitS:SAC 2nd Gig, the corresponding Tachikomatic Days episodes, and mapped them to Bach's Toccata & Fugue in D Minor.

If I were to do it, I think the way I would map it would be based on how you can see the piece played by an organist:

Parts of the song that had more significant use of the right hand would map more greatly to Section 9, their actions and their influences, parts that had more significant use of the left hand would map more greatly to Hideo Kuze and his actions and influences, while parts that had more significant use of the foot pedals would map more greatly to the actions of Kazunda Gouda and his actions and influences.

There are a couple parts that only have high note components, and those, of course, would need to map to the comic relief of Tachikomatic Days.

New Year's Resolution(s)

2010-01-02T03:37:00.001-05:00

Get that check to Woodstock, and in the hands of its intended recipient.

Accomplish at least some of the goals I noted for Rosetta Code for 2010.

Build a NAS box for off-box backups of dodo.

Find my Slayers DVDs.

Get a NexusBot dev team running. Avoid contention over choice of programming language.

Solidify some more of my thoughts and ideas for image and video editing and compositing.

Make an AMV or two.

Give up trying to get JT and RL to come back to coffee. (Dunno what we did to scare them off...)

Get some more anime-tolerant friends to attend the weekly get-togethers.

Best Christmas Eve Ever

2009-12-25T04:53:00.001-05:00

My brother is in town. We watched a bunch of movies together. I introduced him to The Usual Suspects, he then introduced me to Cypher, to which I responded with Paprika.

Coincidentally, that turns out to be an awesome sequence if you like mysteries and plots that twist your brain into pretzels. It's like the hard-liquor cocktail of mystery/thrillers. Watched in that order, those movies' themes flow into each other; The first two have someone unseen pulling the strings, while the latter two make it difficult for the character to discern what's real and what's not. (Paprika takes this to extremes...)

I need to show him Dark City next if I still have the BD from Netflix. I expect doing all four of those in a marathon session would be like a Pan Galactic Gargleblaster. I'd probably throw Dark City somewhere between Cypher and Paprika in the sequence; It shares some of the 'unseen strings' characteristics with Cypher, albeit not quite so dense, and some of the uncertainty about reality, albeit not as dense as that of Paprika.

Techtalk Tuesday: Nexus bot

2009-12-22T00:37:00.001-05:00

This was an idea I've been chewing on for a while, now, and something I've been thinking about doing. Simply put, it's a chat room bridge, but it's not that simple.

Normally, I sit in rosettacode, haskell, proggit, perl, tcl and any number of other channels, ready to offer insight or assistance, or even just observe, if someone mentions Rosetta Code. What Would Be Nice would be if I could just sit in rosettacode, and let a bot handle it for me.

The general sequence might go as follows:

proggit - * soandso thinks Rosetta Code needs better algorithms
rosettacode - * soandso thingks Rosetta Code needs better algorithms
proggit - soandso: What are you looking for?
rosettacode - <#jubilee> soandso: What are you looking for?
rosettacode nexusbot: soandso: Did you see this category? (some url)
proggit - soandso: Did you see this category? (some url)

Nexusbot has to perform several complicated behaviors there, so let's look at them.

First:
* soandso thinks Rosetta Code needs better algorithms

"Rosetta Code" matches one of nexusbot's highlight rules for forwarding to rosettacode, so nexusbot relays the message to rosettacode, thinks of it as a "connection", and associates soandso as a primary for that connection, with a most recent related activity timestamp attached to his association with the connection.

Next:
soandso: What are you looking for?

soandso is associated with a current connection (and that association hasn't timed out), and jubilee just said something to him. nexusbot associates jubilee with soandso, and, through soandso, to the relay to rosettacode. jubilee is attached to the relay with his own related activity timestamp, copied from soandso's.

rosettacode - nexusbot: soandso: Did you see this category? (some url)

shortcircuit addresses nexusbot, and indicates he's addressing soandso through nexusbot. Nexusbot sees that soandso is associated with a connection between rosettacode and proggit, associates shortcircuit with that connection (along with a recent activity timestamp), and passes shortcircuit's message along to proggit

Each time someone triggers a highlight, they're considered a primary for the connection that highlight creates (or would create, if it exists already), and their "recent related activity" timestamp is updated. Each time someone talks to a primary for a connection, they're also associated with the connection, and their "recent related activity" timestamp is set to that of the primary's.

Whenever a primary or secondary talks, their communications are relayed across the connection, but their RRAs are not updated.

When a primary's RRA grows old past a certain point, they're disassociated from the connection. When all of a connection's primaries are gone, the connection is ended.

There are a couple scenarios this logic doesn't quite resolv. What if jubilee is a channel champion, someone who talks to everyone, and who everyone talks to? It's probable that his side of a conversation with someone else would leak across the channel. What if someone talks to a secondary on a related subject, but he doesn't trigger a hilight keyword? Well, that line would be lost.

No solution is perfect.

Now to deal with the Big Brother concerns. Ideally, nexusbot would only be in a channel if he were legitimately asked to be there. That means only a /invite, and preferably checking to see if the user who sent the invite is, in fact, in the destination channel. Ideally, nexusbot would only be in a channel until he were asked to leave. That means no autojoin after a /kick.

There's also the consideration that it should let someone who's in authority in the channel know they're there and what they are, and offer a command set to control the bot's behavior in the channel.

Random braindump of possible commands:

HILIGHT LIST/ADD/REMOVE [#channel] -- lists, adds or removes a hilight rule, optionally associated with a channel. Lists include who requested the hilight rule, and when.

RATELIMIT GET/SET -- get or set the maximum number of lines per minute.

LINEBUFFER GET/SET -- get or set the size of the buffer for queuing lines if the ratelimit is hit.

REPLYMODE USER/HIGHLIGHT/CHANNEL/AUTO +/- b/m/v -- treat connections derived from highlights or associated with particular remote channels as channels themselves, and allow some channel modes like +/- m to be applied to them. Likewise, allow user modes like +/- b and v to be associated with remote users. AUTO means having the bot automatically sync its remote user modes (as they apply in that channel) with the channel's mute, voice and bans.

Ideally, only channel members with +O or +o would have access to the setter commands.

Frameworking realtime communications

2009-12-21T10:29:00.001-05:00

While looking into flexible ways of writing an IRC bot(the nature of which will probably be in a "Tech Tuesday" post for tomorrow; It's a comm routing bot, so don't complain about IRC != IM quite yet.), I tried using IM libraries. I started with libpurple, because I use Pidgin all the time. That was utter failure, as I couldn't find enough organized documentation to even write a stub quickly.

Then I tried playing with telepathy. I wasn't able to get it to work in the few minutes I had left that evening, but I did learn a few things.

First, it's awesomely flexible, and there are tools that exist that allow a GUI user to essentially interact with it the same way your program might, which let me immediately dive in and try testing some capabilities.

Second, it's missing a decent IRC backend. The closest was "haze", which I couldn't get to connect to more than one IRC network at a time. Turns out Haze is just a glue module between Telepathy and libpurple, and the one-IRC-connection-at-a-time thing is a libpurple limitation. (I'm glad I didn't waste my time thoroughly studying the libpurple header files; It wouldn't have done what I needed, anyway.) I might be able to use multiple libpurple instances, but I don't know how safe that would be; libpurple uses a common filesystem location, I don't have control over it through Telepathy, and I can't trust it to be multi-instance safe in its access patterns.

Telepathy is interesting because it allows multiple connections, any of which routed through any number of connection modules. One could conceivably create a program that talks to fifteen different networks using one or as many protocols.

Darn cool. I wish it had better support for IRC as a client (and maybe as a server...who knows what kinds of options that opens up?). Support for things like identi.ca and Twitter would be pretty cool as well.

Computer cable rerouting and bundling (and pics)

2009-12-20T02:29:00.001-05:00

I rerouted some cabling, replacing a hodgepodge of miscellaneous SATA cables with a bunch 90-degree-ended cables I'd bought from Digi-Key.

This is the best I can do for now, until I get another SATA controller (so I can use SATA instead of PATA fo that bottom drive) and replace the power supply with one with modular cables.

In a pinch, I also wound up using some blue electrical tape to bind a coil of some other cable. I wrapped it sticky-side out, then doubled over and wrapped it sticky-side in. Net effect is that it's just rubbery plastic, no sticky adhesive anywhere.

Bass by phase modulation.

2009-12-18T16:17:00.001-05:00

I want to take two 50KHz synced tone sources, invert them, then phase modulate so that the difference between the two represents my actual signal.

I'm really seeking to produce a high power signal in the 0-50Hz range, but the phase modulation approach has three advantages. First, the high carrier frequency can be deadened and insulated much more easily than a pure low frequency signal. Second, the emitting devices don't necessarily need to be as large as a corresponding cone. Third, they don't need to be directly attached, as with bass shakers, making seating and such easier to manage.

The first problem with the phase modulation approach is the carrier signal; You don't want it to be within the audible range of any human or other hearing animal.

The biggest up-front problem, though, is managing the positioning of the interference nodes. Having a high carrier frequency solves part of the problem by reducing the distance between the nodes. The smaller the distance, the lower the likelyhood of a positive node being in one ear, and a negative node in the other. (Wouldn't want to scramble your brains, now, would we?)

QOTD

2009-12-14T15:01:00.001-05:00

"I went back in after I wrote them and added all sorts of weasel words. It sort of saps the punch from the statements, but it's one of those things I've learned that I have to do to avoid the more egregious forms of willful misunderstanding. " -- Raymond Chen

"Willful misunderstanding" ... That's an issue I've been contemplating for a few months, now.

Reboots and Automatic Updates

2009-12-14T10:19:00.002-05:00

I couldn't check last week, as I was out of the office sick, but someone had asked if anyone's system had done an uncommanded reboot in response to automatic updates.

I thought I'd go through my workstation's event log, and I've found a two interesting log entries, so far:

Installation Ready: The following updates are downloaded and ready for installation. This computer is currently scheduled to install these updates on ‎Wednesday, ‎December ‎09, ‎2009 at 3:00 AM: - Windows Malicious Software Removal Tool x64 - December 2009 (KB890830)
and

Installation Ready: The following updates are downloaded and ready for installation. This computer is currently scheduled to install these updates on ‎Wednesday, ‎December ‎09, ‎2009 at 3:00 AM: - Cumulative Security Update for Internet Explorer 8 for Windows 7 for x64-based Systems (KB976325) - Windows Malicious Software Removal Tool x64 - December 2009 (KB890830)

Now, I have Windows set up to download only. It's supposed to wait for my confirmation before I install. Notice that those two events include the phrase, "This computer is currently scheduled to install these updates on Wednesday, December 09, 2009 at 3:00AM" ... I didn't tell Windows it could do that. Or, at least, that's not what the normal interface indicated it was set up to do. (I don't make a habit of digging through Administration Tools and tweaking things, as I don't know the full impact of most of what's in there.)

Without finding the setting that automatically schedules updates for next-day installs, even when you tell it to download and wait for confirmation, the only obvious solution is

The only solution to uncommanded reboots, as far as I can see, is to tell it to not even download the updates unless instructed. Not as convenient as having it download in the background, but it saves you the hassle of having a machine reboot on its own, leaving you scratching your head as to why.

Update

Ok, something reset my Windows Update preferences. I just checked again, and it was set to "Install Automatically."

Coax and Engrish

2009-12-12T00:44:00.001-05:00

A few quick photos I took this evening.

Remote control

2009-12-11T23:55:00.001-05:00

@coderjoe: You don't keep the remote for the receiver over here any more?

Me: I haven't seen it in a week and a half...

A very simple idea for a browser extension

2009-12-09T23:33:00.001-05:00

DOM-walking shortcut keys: hjkl. If you've used vi or Google Reader, you probably know where I'm going with this.

H: Switch focus and highlight to parent DOM node.
J: Switch focus and highlight to previous sibling DOM node.
K: Switch focus and highlight to next sibling DOM node.
L: Switch focus and highlight to first child DOM node.

Something like that would make browsing most blogs' and forums' comment sections much more convenient.

It's simple enough I'd probably do it if I knew how to make a Firefox extension. :-|

Hypoallergenic detergent

2009-12-09T21:54:00.001-05:00

So we now have one of those front-loading washers that uses a lot less water. Problem is, even using the "extra wash" and "second rinse" cycles without soap, I still have to put it through a second run of the above to get all of the soap out, and we're already using less than the normal amount of soap specified in all the directions.

More details on the washer: It supports a normal wash, rinse, spin-dry sequence, same as any other. It also supports an "extra wash" and a "2nd rinse". I usually run loads with "extra wash" and "2nd rinse" enabled, to get more water through to dilute out the soap.

Why is it important to get all the soap out? It gives my grandmother hives. We're using "Hypoallergenic Purex UltraConcentrate h.e", and even with five soap-free rinses (one normal run with extra wash and 2nd rinse enabled, one soap-free run with the same), there's still enough soap remaining to cause allergic reactions.

Does anyone know of something more hypoallergenic than the stuff we're already using?

Vinegar, Venom and salt

2009-12-09T00:20:00.001-05:00

Vinegar, venom and salt
I'm counting all your faults
When the time is right
The pen could write
And joy would come to a halt

Vinegar, venom and salt
They're quite the painful thoughts
I see the fool
I see the tool
Words can be quite the assault

Vinegar, venom and salt
They must be kept in a vault
Padlocked, lost key
No-exit, you see
Void the words and their poisonous waltz.

Looking at a longer Winter in Michigan

2009-12-08T22:33:00.001-05:00

We're looking at having a longer winter here in Michigan.
You see, there are only two seasons here: "Winter" and "Under Construction"
They're talking about losing federal dollars for road maintenance, as Michigan can't put up the matching dollars.
So we're looking at the road maintenance budget being cut to less than half.
The "Under Construction" season would thus be shortened.
Ergo, we're looking at having a longer winter here.

GPG and signed email

2009-12-08T15:24:00.001-05:00

I've started using GPG to sign my email. It was easy; Install FireGPG and generate a key.

In order to send signed emails, FireGPG contact's GMail's SMTP server directly. Fair enough, but that got me thinking...What about having an SMTP server that only delivered signed emails where the signature checked out against some public keyring, and the signer wasn't marked as unauthorized due to abusive behavior? You could have an anonymous relay that operated in that fashion.

Add in a "X-Server-GPG-Signature" header in the email, and an email provider using such a technique could garner a decent reputation, and thus get more or less a pass by any anti-spam filters in the next stage of the email relay.

I'm sure the idea isn't new. I suspect, though, that all that's needed are a few seed SMTP servers that operate in this fashion.

Two puffs every four hours and a vaccine

2009-12-08T13:20:00.001-05:00

So all the crap that's in my lungs is dead, but there's still a lot down there. Plain old albuterol inhaler is going to increase coughing and accelerate getting all that crap out. Codeine cough syrup apparently keeps me awake, so some numbing agent from the Novocaine family will help me sleep at night instead.

Meanwhile, I also got the H1N1 vaccine. Whether or not I'm just getting over H1N1 (they just don't test for it any more; Tests come back positive in Kent County more often than not), the vaccine is a good move for me avoiding carrying something that could infect my grandmother. I specifically asked my doctor about whether or not I could take the vaccine, and, as it turns out, high fever is the disqualifier, and I'm already over that.

Techtalk Tuesday -- Video editing and image compositing

2009-12-08T04:32:00.001-05:00

So I've been thinking about video editing, Linux, and how much I hate Cinelerra.

Now, I don't know a lot about the internals and features of existing video editing tools, but I at least know some of the basics. First, you produce a series of images at a rate intended give the illusion of movement. Let's look at a single point in time, and ignore the animation side of the equation. Let's also focus on visual (as opposed to auditory) factors.

You have source images, you have filters and other transformations you want to apply to them in a particular order, and you want to output them into a combined buffer representing the full visualization of the frame.

Let's break it down a bit further, along the lines of the "I know enough to be dangerous" areas.

Raster image data has several possible variations, aside from what is being depicted. It may have a specific color space, be represented in binary using different color models (RGB vs YUV vs HSL vs HSV), may have additional per-pixel data (like an alpha channel) thrown in, and the subpixel components can have different orderings(RGB vs RBG), sizes(8bpp to 32bpp), and even formats (integer, floats of various sizes and multiplier/mantissa arrangements). ICC color profiles fit in there somewhere, too, but I'm not sure where. There's even dpi, though not a lot of folks pay attention to that in still imagery, much less video. Oh, don't forget stride (empty space often left as padding at the end of an image data row, to take advantage of performance improvements related to byte alignment.).

Now let's look at how you might arrange image transformations. The simplest way to do it might me to organize the entire operation set as an unbalanced tree, merging from the outermost leafs inward. (Well, that's the simplest way I can visualize it, at least). Each node would have a number of children equal to the number of its inputs. A simple filter would have one input, so it would have one child. Any more inputs, and you have a compositing node. An alpha merge, binary (XOR/OR/AND) or arithmetic(subtract, add, multiply, etc) merge would be two-arity, while a mask merge might be three-arity.

Fortunately, all of this is pretty simple to describe in code. You only need one prototype for all of your image operations:

void imageFunc(in ConfigParams, in InputCount, in BUFFER[InputCount], out BUFFER,)
{
}

An image source would have an InputCount of 0; It gets its data from some other location, specified by ConfigParams.

So assuming you were willing to cast aside performance in the interests of insane levels of flexibility (hey, i love over-engineering stuff like this; Be glad I left out the thoughts on scalar filter inputs, vector-to-scalar filters, multiple outputs (useful for deinterlacing), and that's not even fully considering mapping in vector graphics.), you probably want to be able to consider all that frame metadata. Make it part of your BUFFER data type.

One needs to make sure to match input and output formats as much as possible, and minimize glue-type color model and color space conversions. For varying tradeoffs of performance to accuracty, you could up-convert lower-precision image formats to larger range and higher-precision ones, assuming downstream filters supported those higher-precision ones. Given modern CPU and GPU SIMD capabilities, that might even be a recommended baseline for stock filters.

Additionally, it *might* be possible to use an optimizing compiler for the operation graph. From rearranging mathematically-equal filters and eliminating discovered redundancy to building filter machine code based on types and op templates. But that's delving into domain-specific language design, and not something I want to think too hard about at 4AM. In any case, it would likely be unwise to expose all but the most advanced users to the full graph, instead allowing the user interface to map more common behaviors to underlying code.

There's also clear opportunity for parallelism, in that the tree graph, being processed leaf-to-root, could have a thread pool, with each thread starting from a different leaf.

That's an image compositor. Just about any image editing thing you could want to do can be done in there. One exception I can think of are stereovision video, though the workaround for that is to lock-mirror the tree and have the final composite a map-and-join. (If you want to apply different fiters to what each eye sees, you're an evil, devious ba**ard. I salute you, and hope to see it.) Another is gain-type filtering, where a result from nearer the tree root could be used deeper in the tree (such as if you wanted to dynamically avoid clipping due to something you were doing, or if you simply wanted to reuse information you lost due to subsequent filtering or compositing steps). Still another is cross-branch feeding; I can think of a few interesting effects you could pull off with somethig like that. There's also layering and de-layering of per-pixel components.

As a bonus, it's flexible enough that you could get rid of that crap compisitor that's been sitting at the core of the GIMP for the past twenty years.

Flu

2009-12-07T17:00:00.001-05:00

My grandmother clipped this out of some county newsletter and left it for me:

How do I know if I have the H1N1 flu?

The symptoms of this influenza virus are similar to just about every 'flu' bug out there. Common effects are a cough, sore throat, headache, fever and chills, severe fatigue with body aches. Some people are over the fever and chills phase in about 2 to 3 days while others suffer for more than a week. Complications can occur days into the illness when lower chest congestion progresses to pneumonia. This is a secondary bacterial infection that comes about because the immune system is so taxed fighting the flu it cannot fend off the bacterial pneumonia. This disease is so prevalent that hospitals and the health department have stopped testing specifically for H1N1 2009. It is always coming back positive, so if you have the above symptoms you are presumed to have H1N1 influenza.

(Yeah, whoever wrote that article needs to work on their language skills.)

Cough, sore throat, fever and chills, severe fatigue with body aches. All there, at one point or another in the last two weeks. Pneumonia last week.

Doctor's appointment tomorrow, because I hate the idea of missing a third week of work.

Thinking about backups.

2009-12-06T15:48:00.001-05:00

Thinking about backups.

I've got a 3TB RAID5 volume (three 1.5TB disks) that reads between 150-200MB/s, but only writes at 25-50MB/s.

I would like to have full backup capacity of all 3TB of data, but the question becomes "how"?

If we assume that the reason for the slow write speed to the software RAID 5 array stems from parity calculation, then it stands to reason that a RAID 0 array wouldn't suffer the same speed limitation. Additionally, a RAID 0 array of two 1.5TB disks would hit a 3TB volume size, as opposed to requiring a third disk as in RAID 5.

I'm considering having a second, weaker box run software RAID0, and do a nightly rsync from primary box to the backup box. A dedicated 1Gb/s link would facilitate the copy.

If a drive in the RAID0 array fails, I replace it, rebuild and re-run the backup. If a drive in the RAID5 array fails, I replace it and rebuild. If the rebuild kills a drive and the RAID5 fails, I've got a backup. Meanwhile, I've got an isolated power supply, reducing the number of single points of failure. I'm using fewer drives in the backup machine, reducing cost. I'm reusing older hardware for the backup machine, reducing cost.

Tricky part is figuring out offsite backups from there, but my data isn't that valuable yet.

IR->Bluetooth->IR

2009-12-06T15:05:00.001-05:00

You know what would be nice to have? A near-proximity IR->Bluetooth->IR adapter.

By "near-proximity", I mean having it attached to the original IR transmitting device, have an IR sensor, convert that to a 2KHz 1-bit bitstream, send that via BT to a receiver that converts it back to an IR transmission near whatever device needs to receive it.

Two significant problems remain: Bulk and power. For bulk, one could take advantage of paper-ICs or other film integration. (When I was a kid, I saw thick-film ICs dating back to the 70s. They may have been around longer than that.)

For power, I don't know. Probably the best way to go about it is to leech off the existing remote control's battery pack. I can think of a couple ways one might do that without interfering with the remote's internal expectations of its power source.

Of course, you could just build the thing into a lithium battery pack, rechargeable via USB, and tout it as both a range and life extension of the remote. Lithium power density is such that you might be able to pack the lithium, charging circuitry and bt transmitter in the space of a couple AA batteries. Some mechanical finagling and shoe-horning might be necessary to fit different battery compartment configurations.

Tis the Season to Barter

2009-12-06T00:18:00.001-05:00

So my router was dropping 70-80% of packets, making it nightmarish trying to do anything via SSH. I called up a friend and asked him to pick up a cheap router from Best Buy, along with some new CAT6 ends (These are a *lot* nicer than the crap ones that took me 45 minutes apiece to do badly...).

Of course, an order like that comes to about $70, and I don't have that laying around in cash. PayPal was inconvenient for technical reasons, so we came up with a convenient solution...I just bought an equal amount of stuff from his Amazon wishlist, and am having it shipped directly to his registered address.

Migrating from a solid disk to software RAID.

2009-12-04T15:41:00.001-05:00

So I've now got three 1.5TB disks in RAID 5, the array is assembled and running on bootup, and is mounted as my /home. Previously, I was using a 1TB drive for /home.

(I'm going to abbreviate this, including only the steps that worked, and initially omitting some of the excruciatingly long retries)

After installing the three disks into the machine, my first step was using Ubuntu's Palimpset Disk Utility to build the software RAID device. That took about seven hours, unattended.

The next step was copying my old /home filesystem to the new RAID array, using dd. That took about nine hours, unattended.

The next step was expanding the filesystem and tuning some parameters. ext3 isn't aware of the size of the block device it sits on, but it does remember a similar value related to that of the device it was created on. I had to use resize2fs to expand it from having sat on a 1TB volume to occupying a 3TB volume.

I looked at tune2fs and enabled a few options, including dir_index (I have a few folders with thousands of files in them), sparse_super (That saved a *lot* of disk space) and uninit_bg (Supposed to speed up fsck). I didn't read the man page clearly, and didn't discover until afterwards that by enabling uninit_bg, I'd given tune2fs the go-ahead to convert my filesystem from ext3 to ext4. Oh well...Seems to be working, and there are a few features of ext4 (such as extents) that I expect will come in handy.

The next step was to reboot and ensure that I could mount the array after rebooting; I didn't want some screw-up on my part to lead to all that time being wasted* by failing a RAID volume. After establishing I could mount it, it came time to modify mdadm.conf. and seeing that the array would come up on bootup. After that, all that was left was modifying /etc/fstab to mount the RAID volume at /home, rebooting, and restoring compressed tarballs and such from my overflow drive.

Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              2.8T  1.2T  1.4T  47% /home

I've gone from having 8GB free on /home to having 1.4TB free. Can't complain.

root@dodo:/home/shortcircuit# dd if=/dev/md0 of=/dev/null
10985044+0 records in
10985043+0 records out
5624342016 bytes (5.6 GB) copied, 27.5768 s, 204 MB/s
18288001+0 records in
18288000+0 records out
9363456000 bytes (9.4 GB) copied, 46.1469 s, 203 MB/s
22992060+0 records in
22992059+0 records out
11771934208 bytes (12 GB) copied, 57.7066 s, 204 MB/s

Getting over 200MB/s in raw streaming read. Can't complain about that, either; I only read at about 70MB/s when pulling from a single (mostly) idle disk that's not part of the array.

Of course, it's not as good as I'd get with a hardware RAID card, but it's a heck of a lot better than I'd get otherwise. My comparative write speed sits down at about 25MB/s when dd'ing from a USB drive to the md0 device. I probably should have tried testing the write speed while reading from /dev/zero before putting the filesystem in place, but the bonnie disk benchmark at least gives some non-destructive results:

Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
dodo            16G 38025  73 38386   8 25797   5 47096  85 161903  16 353.8   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
dodo,16G,38025,73,38386,8,25797,5,47096,85,161903,16,353.8,0,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

For ext4 on top of the software RAID5 volume (consisting of three Seagate ST31500541AS), I get 38MB/s sequential output, 161MB/s sequential input, and 353 random seeks per second. Little to no difference between per-character writing and block writing.

Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
dodo            16G 50964  96 61461  15 29468   6 49902  87 84502   6 202.1   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++

For ext3 on top of a single disk (a Seagate ST3500630AS), I get 51M/s sequential per-character write, 61M/s sequential block write, 50M/s sequential character read, 84M/s sequential block read, and 202 random seeks per second.

Long and short of it, a single block disk kicks my software RAID5 volume's butt for sequential write, but the software RAID5 blows away the single disk for sequential reads, and gets a 50% improvement over the single disk's random seek rate.

One thing I find particularly interesting about this is that the three disks in the RAID volume are 5900 RPM spindle speed, while that single disk is 7200 RPM spindle speed. I suppose having three heads is better than one. :)