2006-05-03
A frequently-requested ImageIngester feature is for it to number images consecutively, keeping the numbering going from ingestion to ingestion forever, or maybe until reset by the user.
For example, today's images might be MJR_000000.DNG through MJR_000025.DNG, and tomorrow's would start with MJR_000026.DNG, and the numbering would continue over months and years, for a million images, until MJR_999999.DNG.
This is one of those things that's easy to implement if you just do it, and very hard to do, maybe even impossible, if you really think about it.
The easy way is to keep the last number used in some persistent storage, such as in a file or wherever the preferences are stored. Each run of the program picks up the number, increments it for each image, and then updates the file or the preferences. Maybe for reliability, in case the system or ImageIngester crashes, you update the stored number for every image, rather than waiting for the ingestion to end.
That's easy enough. Since I'm storing a bunch of preferences anyway, the sequence number would be just one more, and so most of the code to do it is already there. The rest is pretty simple, too. I think I could have the whole thing working in less than an hour.
So, why haven't I done it already? Because I don't think it would work.
What if the preference file gets restored from a backup? Or, what if the user tosses the preferences out completely, as smart users know to do when a program seems to be acting up (a few ImageIngester users have done this already)? What if there are two ingestions at once? (Locking! Ouch!) What if the user ingests to a laptop when on the road, and to a desktop when in the studio?
One solution is to define the feature in such a way that there's no guarantee of consecutive numbers, or even monotonic ones. My experience tells me that solutions like that result in dissatisfied users, because when something goes wrong the caveat in fine print is not only no consolation, but actually adds to their annoyance if you point it out. ("Yeah, I see that, but what moron would do it that way???!!!!!")
The other solution is to try to make the counting reliable. That's an interesting software design challenge, surprisingly difficult when you consider how simple it was to state the problem: Count reliably over long stretches of time.
I don't have a solution worked out yet, but here are some ideas I've had, along with the reasons why I haven't used any of them:
- Keep the number in a file, so it survives the user deleting the preferences, but make that file very reliable by duplicating it, possibly with the backup on another drive. (Probably very complicated to implement, and what if there's no other drive?)
- Keep the number in a database that provides transaction management and backup. (ImageIngester users generally don't have databases like that.)
- Store the number in a reliable internet storage server, like Amazon's new S3. (Doesn't do much for the laptop on the road.)
- Same as the previous idea, but rename images ingested to the laptop when it's reconnected to the desktop. (Getting really complicated now.)
- Somehow use the computer's clock to at least keep the numbers monotonic. (Clocks get reset, too, and doesn't address the laptop-on-the-road situation.)
- Look at the last image file actually written, rather than keeping the number in its own file. (But the user might have offloaded the image, and doesn't address the laptop issue.)
- If the numbers gets messed up, ImageIngester will fix the numbering by renaming the files. (Won't work--other programs such as iView rely on fixed file names.)
As you can see, some of these ideas are outrageously complicated. Wildly inapproriate for a little utility like ImageIngester, especially since its primary reason for being is increased reliability.
Am I over thinking this? Maybe users won't care that much if the numbers overlap on rare occasions? I really don't know.
Suggestions anyone? (Email link at top of page.)