Daniel Pitts’ Tech Blog

Well, as you can see, this blog has had no activity for quite a while. As a matter of fact, my web server (a desktop machine running from my living room) had been out of commission for a while. It was booting most of the way, and then the kernel panicked and restarted.

Recently, my ex-wife had asked me if I had an old password to an old account of hers. I couldn’t remember it, but I thought it might be in firefox on my old Windows laptop. Well, the old laptop wouldn’t start properly. No love from safe-mode. It would BSOD and reboot, no matter what I tried… Except it would boot into “startup recovery”.

So I had two problems to fix.

For the laptop I bought a 2TB external hard drive for ~$120 at Fry’s. In the startup recovery mode, I was able to open a command prompt, and the drive was recognized immediately. A few xcopy statements later, all my important information was backed up in the drive. I then tried to reinstall windows, which caused the laptop drive to fail completely. At least I got my data off before that happened.

The Web Server was a little trickier. For my Web Server, I bought a new 500GB internal harddrive, for ~$70 at Fry’s. I needed to create a new os Boot Disc. I failed several times to burn a working disc, but eventually I got one running. Apples Disc Utility isn’t intuitive as it aught to be for this.

I installed the latest FreeBSD on the new internal drive, and everything was running fine. Then I tried to mount my old hard disks partitions. Wouldn’t you know it, the kernel panicked and the computer rebooted! I was afraid I lost everything on that server. This blog, all my projects, my other sites.

Well, then I remembered a little unix utility called dd. I was able to copy most of the raw data of the partition by telling dd to skip over the errors. This program created a file which matched the data exactly, except the “bad” locations. Then I was able to use mdconfig to create a virtual disk based on that file. That virtual disk mounted just fine, and I was able to save all my old files.

As you can see, I’ve got that server back up and running. I’m going to look into more robust back-up solutions to prevent this kind of white-knuckle experience ever again. I’m also considering setting up RAID (either 1 or 5) to minimize the down-time for a drive failure.

Oh, and in case you can read *and* count, the title said Three Data Recovery Wins, and I’ve only told you about two… The third win was my iPhone. Apple released it’s latest iOS (6.1.2), and I foolishly started the update without backing-up my phone (the old back-up was lost on my now-broken laptop).

When my iPhone turned back on, all it showed was a USB cable and the iTunes logo. Plugging into iTunes, I was told that the device was in Recover Mode, and must be reset to factory defaults to repair it, and all my data would be lost. Uhh, not good…

A quick google and I found TinyUmbrella. It allowed me to take the iPhone out of recovery mode with no data loss. I was then able to back-up the iPhone to my iTunes.

So, lessons learned:

1. Storage devices fail sometimes.
2. Back-up storage devices fail sometimes too.
3. Never trust an update. They may be safe 99% of the time, but one failure can be catastrophic.
4. Never panic. Sometimes with a little work, you can recover at least some of your valuable data.
5. Keep multiple back-ups of important data.

I’ve been spending more time taking care of my little girl lately, and I was thinking “this isn’t anything like programming.” I realized that it kind of is, just no one has written the JavaDocs so that programmers like me can understand it clearly.

So, I’ve taken it upon myself. Here you go.

// Baby.java
// Copyright(C) 2010, Daniel Pitts, All Rights Reserved.
import java.util.Collection;

/**
 * A Baby. A small child which acts similar to an adult, but has some
 * differences which may be surprising. Any method can throw TantrumException
 * at any time for no decernable reason. All methods are abstract, because
 * every baby is different.
 * <p>
 * Baby's aren't collected by the standard JVM Garbage collector, but
 * instead may be collected by Child Protection Services (CPS for short) if
 * not properly handled.
 */
public abstract class Baby {
    /**
     * Attempt to put baby down for a nap.  You may need to call any of the
     * following methods before this method will succeed.
     * <ul>
     *   <li>{@link Baby#recieve(Bottle)}
     *   <li>{@link Baby#change(Diaper)}
     *   <li>{@link Baby#feed(Meal)}
     *   <li>{@link Baby#change(Diaper)}
     *   <li>{@link Baby#giveAffection(java.util.Collection)}
     *   <li>{@link Baby#playWithToys(java.util.Collection)}
     * </ul>
     * @throws InterruptedException randomly
     * @throws TantrumException when the mood strikes
     * @throws IllegalStateException if the child doesn't want to sleep for
     * some reason or another.
     */
    public abstract void sleep() throws InterruptedException, TantrumException;

    /**
     * Attempt to give the baby a bottle.  Once the baby is done with the
     * bottle, the bottle may be returned or thrown.  This may happen
     * immediately, or after the bottle has been emptied.
     *
     * @param bottle a bottle of formula or milk.
     * @throws TantrumException when the mood strikes.
     * @throws Bottle the bottle, if the baby *really* doesn't want the bottle.
     * @return the bottle.
     */
    public abstract Bottle recieve(Bottle bottle)
            throws TantrumException, Bottle;

    /**
     * Attempt to change the baby's diaper.
     * @param diaper the new, clean diaper.  An attempt to use a dirty
     * diaper will result in a JVM error, and CPS will come by shortly to
     * remove the Baby from your care.
     * @return a dirty diaper. This diaper should be disposed of as soon
     * as possible.
     * @throws TantrumException when the mood strikes.
     * @throws PoopyDiaper if the parent doesn't remove the diaper quickly
     * enough.
     */
    public abstract Diaper change(Diaper diaper) throws TantrumException,
            PoopyDiaper;

    /**
     * Gives a snack to the baby.  May help prevent TantrumExceptions from
     * being thrown for some time.
     * @param snack something delicious and healthful.
     * @throws TantrumException if snack is null, or the wrong kind for the
     * moment.
     */
    public abstract void feed(Snack snack) throws TantrumException;

    /**
     * Gives a prepared meal to the baby.  May help prevent TantrumExceptions
     * from being thrown for some time, also prevents the baby object from
     * being reclaimed by CPS.  Must be called at least three times a day.
     * @param meal something nutricious and yummy.
     * @throws TantrumException at whim.
     * @return a mess which needs to be cleaned up. Older babies may return
     * null more frequently, but don't count on it.
     */
    public abstract Mess feed(Meal meal) throws TantrumException;

    /**
     * This method is very important to call, many times per day.  Remember,
     * the more love you pass in, the more you get back. The love can be Hugs,
     * Kisses, and Talking.
     * @throws TantrumException usually happens only when any of the
     * playWith methods are executing concurrently.
     * @return some love, often more than you give.
     */
    public abstract Collection giveAffection(Collection love)
            throws TantrumException;

    /**
     * Reads a story the baby. This is often a prerequisite for
     * {@link Baby#sleep()}.
     * @param story the story.
     * @throws TantrumException if the story is the babies favorite at the
     * moment, or if the baby realizes that bed-time comes after the story.
     */
    public abstract void read(Story story) throws TantrumException;

    /**
     * This method will be executed on its own from time-to-time.  Not all
     * objects passed in are actually toys, and may cause all kinds of
     * exceptions.  A strong attempt should be made to intercept all calls to
     * this method and filter out non-toys, or otherwise dangerous objects.
     * @param something something the baby thinks is a toy.
     * @throws TantrumException randomly, or sometimes if some object is
     * filtered.
     */
    public abstract void playWith(Object something) throws TantrumException;

    /**
     * Give the child a toy to play with.
     * @param toys something fun to play with.
     * @throws TantrumException once the child is tired of the toy, or if it
     * isn't the toy they wanted.
     */
    public abstract void playWithToys(Collection toys)
            throws TantrumException;

    /**
     * Have this baby play with another group of babies.
     * @param babies a group of babies to play with.
     * @throws TantrumException for many reasons.  Sometimes babies will take
     * toys from other babies, which often causes a TantrumException.
     */
    public abstract void playWithBabies(Collection<Baby> babies)
            throws TantrumException;

    /**
     * Bathe the baby.  Keeps the baby clean and happy. If you go too long
     * without calling this method, CPS will collect the baby.
     * @param bath a warm bath which should have some toys in it.
     * @throws TantrumException whenever.
     */
    public abstract void bathe(Bath bath) throws TantrumException;
}

So, I finally got off my butt and created a SourceForge project for my AT-Robots clone.

In order to play around with this, you should download the original AT-Robots. It has the documentation about creating robots, as well as a selection of sample robots.

Over the next few months, I’ll work on creating documentation in the wiki. If I can get permission from Bones’, I’ll put the original documentation, but no matter what, I want to re-write it to be more specific to the clone.

I’ve been working on my clone of AT-Robots 2.  I’m nearly ready to release a beta version.  I’ve got a couple of screen shots to show.

Does anyone play AT-Robots any more? Someone mentioned it last year on my guestbook. I apparently don’t check the guestbook often enough

If there is still interest in AT-Robots, I might consider starting work on my clone again. Heck, I might do that anyway, but its much more likely if other people are interested.

Please leave a comment here if you are interested.  Also, if you’d like me to reopen my forums, let me know.

If you have been using computers for any amount of time, you’ve come across an error message.  From nearly useless “access violation” dialog boxes, to the more helpful “User names must not contain spaces” form messages, error messages are frequent on computers.  Whether you’re a Mac, PC, Penguin, or Other, error messages that mean only “something broke” are frustrating.

Programmers often treat all kinds of errors the same, especially when a user interaction is involved.  The problem is, not all errors have the same “meaning”, and not all errors should be handled the same way.  If, as a User, I attempt to use the program for something, and it reports a message target at a programmer (displaying a stack-trace, for instance), it can be confusing.  As a High-Level programmer, if a library call fails but my code gets back few details (getting only ‘There was an SQL error’, without the SQL statement that caused the error), it can be just as frustrating.

What do you mean by low-level or high level?

There usually different “levels” to an application, and each level has a specific type of concern.  Errors on lower levels should be filtered up in as appropriate a manor as possible, so that the higher levels can handle them as gracefully as possible.  Applications can have many different layers, but conceptually they can usually be represented in exactly four levels. What are those four layers? I’m glad you asked, because I’ve listed them below, from highest to lowest.

The User

This, of course, is the person using your application. Most often they are a human being, but someday users may include Artificial Intelligence or Extra Terrestrial entities.  Users are often task-oriented, although some of them are naturally explorers.  Task-Oriented users are more damaged by unexpected errors. Exploring users are more interested in seeing what is and isn’t possible, so errors often define a boundary for them, but these users are also most likely to find bugs in your application.

High-Level Code

This is sometimes called the “Domain” level.  The public methods in these classes are often task-oriented, and the class often define nouns that the User would be able to identify with.  Code in this level translates between user requests and low-level requests. This is of course an idealized definition, but the closer to this definition you can make your reality, the easier to maintain your application will be.

Low-Level Code

This is the library or framework code.  Code in this level usually is used to handle resources, and to actually get things done.  Most Low-Level code is “possibility” oriented. The API defined allows you to do simple, atomic, operations that might not be meaningful on their own, but can be strung together in a useful manor.

Resources

These are “external” things used by the application. Some examples are memory, files, databases, and sockets . Consumable (memory, disk-space, etc…) resources can run out unexpectedly, even if you “check” before hand.  Many resources fail for various reasons, for example sockets might fail because of a disconnected wire, files might fail because of a physical disk error, and database operations might fail due to overloaded CPU or low available memory.

Note, that there can be some “blurring” of levels, depending on the project you are working on.  For example, the developer of an HTTP Client library is probably writing “high-level” code, in that it represents the domain of HTTP, and provides “task-oriented” interfaces.  However, when that HTTP Client library is used in an application, it becomes “low-level”, because the User probably doesn’t care about the HTTP Protocol, and instead only cares about “downloading a file” or “viewing a web-page”.

From my experience, each of the levels correspond to a different kind of error, and those errors should all be handled differently. The likelihood of most of the types of errors can be managed, but not eliminated completely.

Error Handling

Handling User errors

This is the highest-level error.  It is caused by a user doing something they shouldn’t.  Most commonly this means the user entered an invalid input somewhere.  Whenever possible, the User Interface should be designed to prevent user errors.  For example, if you need a date then you provide a “Date Widget” and if you need a number you disallow non-numeric input.

Sometimes, it isn’t feasible to simply “prevent” user error.  In this case, the user input must be validated.  The best place to validate it is in the High-Level code.  Since this code is closest to the user, it has more knowledge about the user.  The High-Level code knows whether to pop-up a dialog box, print a line to the console, or update a form with error messages.

It may sometimes useful abstract the “error reporting” interface, so that it can be reused by different  UI interfaces. For example, domain level code might be used in a Web App, as well as a native GUI app, or even a command-line app.  Having an reusable interface for reporting multiple user errors helps in porting to new user interfaces.

Usually, it should be considered a bug in the high-level code if invalid user input causes a lower level exception. Low-Level code must verify the input it receives from higher-level code.

There are times, however, when it is infeasible to verify before hand whether an input is valid, and the validation must be handled a lower-level code.  In this case, the lower-level code should report the error to the higher level code, which should handle it and pass that information on to the user in an appropriate manor.

Handling Bugs

Bugs can happen in either low-level code, or high-level code.

In High-Level code, it means that some method (often a user task-oriented method) has violated some constraint, or failed to handle a lower-level error. In Low-Level code
it means that a method (part of a framework or library) has violated some constraint, or failed to properly handle a Resource Error.

First and foremost, it is rarely appropriate to show all the detail of “bug” caused errors to the user.  Most users would prefer not to experience bugs at all, but because this is the real world, we have to handle the bugs some how.  Most times, its enough to tell the user “There was an internal error.  Click ‘Report’ to send a bug report.”  The bug report should include any useful state information, but absolutely must include a full stack trace.  If possible, the high-level state should be recoverable, so that the user doesn’t lose any of their work, and can try either a slightly different approach, or wait for a bug fix.

The number of errors of these kinds can be reduced by Unit Testing, improving API design, Integration Testing, and User Testing.

Resource errors

These are Exceptional Circumstances.  Resource errors can be caused by bad data, connectivity issues, database constraint violations, file corruption, file-not-found, out-of-memory, and many more uncontrollable problems.

Resource errors should be reported to the user with as much detail as is likely to be useful to them.  If the user specifies a file, and that file is in the wrong format, this could be treated as a user error, and the user should be informed of the file they chose, and what was wrong with it.  Low-level code should report to higher-level code something along the lines of “I expected a FOO file,” and the higher-level code should report something like “The file c:\my.bar is not a FOO file. “  If a file can not be found, an appropriate message should be displayed, etc…

For this reason, low-level code which handles resources should report errors with as much structured information as feasible to higher-level code.  The higher-level code can then query that structure and produce an error message which is appropriate for the user.  For example, an SQL error at the lower level might mean different things; Typically duplicate a record might a User Error, but a syntax error is likely Low-Level Programmer Error.

There is far less that can be done to reduce resource errors, as they are often caused by external circumstances.  Often times using redundancy can reduce the frequency of fatal errors, but the errors themselves will still occur, and should still be handled appropriately.

It seems to me that the technology is here, or at least almost here, and all it would take is one company to put all the pieces together.  I have idea of where to get many of these, but due to time and budget constraints, I haven’t been able to start on this.

• Miniature computer. Maybe gumstix Overo Fire based computer running Linux.
• Wifi and/or Cellular internet connectivity.  (The Overo Fire supports WiFi and Bluetooth)
• Glove based input devices (virtual keyboard and pointer/mouse).
• Video glasses/HUD. Myvu, eMagin, or preferably Lumus glasses.
• Solar cells to augment battery life.

The bare minimum is the core computer with wifi.  Overo supports an LCD touch-sensitive screen, so you can get input/output through that.

The glove based keyboard is the next important feature. I have yet to see anything on the radar for this type of device.  A pointer device is less important, but would be useful.  It would have to be part of the glove system to be useful.  Oh, and the gloves would have to be rain-proof (water resistant).

The next that I would need is the video glasses. The Lumus glasses look promising, as they allow basically an overlay, rather than off-to-the-side or block-your-vision display.  Unfortunately, they won’t be available to consumers directly from lumus, so we’ll have to rely on vendors.

Finally, the Solar power would enable longer (perhaps unlimited) up-time.  A back-pack made with flexible solar fabric may be the best solution, giving you both the power boost, and an accessory to store additional physical material (real-paper books, lunch, etc…)

This feels real close to me.  If someone had been focusing on this vision, I believe it would have been done by now .  This is the sort of thing shown in science fiction, but it is actually possible in the next few years.

I’m a firm believer in equality and non-discrimination.  California Proposition 8 embodies the exact opposite of that principal.  It is word very specifically to change the California constitution to include discrimination.

It flabbergasted me when I came to my own blog, and saw that Google was serving a Yes on 8 ad on my own page.  Not only that, but it was implying that Obama endorses Proposition 8, when in fact he does not.

I have removed google ads from my site, and replaced it with the words “Please vote no on 8″, until the election is over.

If anyone has seen this, or any other yes-on-8, ad on my page, please know that I am firmly against Proposition 8.

Recently I’ve been faced with the same lesson over and over again.  Polishing is a necessary and oft overlooked step in the creative process.  It takes significantly more effort polishing your final product than just making it work.

By “polishing” I mean putting the final pieces in place, cleaning up any mess, and making sure that your creation shines!  Often times we think about polishing only in regards to one aspect of what we create. For application programmers, it may be improving the user interface; for writers it could be reorganizing the essay.  But there is so much more that needs to sparkle.

I’ve come to the realization that I need to spend more time polishing in almost everything I do. From writing these blogs, to my personal relationship, and many levels of my programs.  Often, since I’m so enthusiastic about a topic,  I am so anxious to share the enthusiasm that I skip vital information, don’t proof read, or try to expand too much on all the implications.  Even now, I feel like expanding on other “things” that software developers need to worry about polishing (I’ll make that another post).

I’m now determined to write more polished articles, as well as and code. You can expect my future articles to be better written. I might even cleaned up and re-post so old articles.  I’ll even proof and edit before I publish. Hey, I even edited this article before I published!

I know, Applets are almost dead, but we actually had use for them recently in a project.  We needed to upload multiple images at once, and since no one on our team knows Flash, we decided to use an Applet. Anyway, the trouble we came across was the Java to JavaScript bridge is flat-out broken.  What you are supposed to be able to do is well documented at Sun’s java_js page and Mozilla’s LiveConnect page. The trouble is, that they are wrong when it comes to calling a Java method from JavaScript. They also fail to mention the security implications of that altogether.

Signed applets with unsigned JavaScript

The first challenge we came to (which is actually the easiest to solve) was that we were getting AccessControlExceptions, even though we went through the trouble of signing the Applet jar. As it turns out, the permission context used is that of the JavaScript, so you need to elevate the permission to your Applets context, using AccessController, and PrivilegedAction.
Java:

public void javascriptCallsMe() {
    AccessController.doPrivileged(new PrivilegedAction() {
      public Void run() {
         // We can now
         readOrWriteFilesOrWhatever();
         return null;
      }
    });
 }

That solved that problem.

Passing JavaScript objects to Java

The next problem, which plagued me for a week, was that getting a JSObject in Java seems to be broken. There are two ways to get an instance of the netscape.javascript.JSObject class.  The first way, and this always seems to work, is the JSObject.getWindow(Applet applet).  This will get the JSObject wrapping the “window” browser object.  This is useful if you know the “path” to the JavaScript object your code cares about.  It is akin to using a static reference, and isn’t good design. The other way, is to have a Java method that takes an Object or JSObject reference: In Java:

public class MyApplet extends JApplet {
  public void doStuff(JSObject params) {
    System.out.println(params.getMember("foo"));
  }
}

Then in JavaScript you should be able to do this: documents.applets[0].doStuff({foo: "bar"}); Unfortunately, what really happens is you get a “broken” instance of JSObject.  Debugging the Applet, I found that the JSObject instance has a field called nac, which has a value for the JSObject.getWindow(…), but is null for values passed in from JavaScript. So, what solutions and work arounds have people come up with? None that I could find.  I searched high and low. Plenty of people have discovered this bug, but none of come up with a solution. Until now!  I thought about it and realized, JSObjects I get from the “window” JSObject all work, so maybe I can put my broken object into a working object, and pull it back out to get a working object. A little experiment proved that it worked (at least on FireFox, I’ll guess it works on IE too, anyone want to verify?). So, I decided to go ahead and create a JSObject resolver, that will fix any possibly broken object:

public class MyApplet extends JApplet {
  private JSObject appletTmp;

  public JSObject resolveObject(Object o) {
    final int hashCode = System.identityHashCode(o);

    appletTmp.setMember("toResolve" + hashCode, o);
    return (JSObject) appletTmp.getMember("toResolve" + hashCode);
  }

  public void init() {
    if (appletTmp == null) {
      final JSObject window = JSObject.getWindow(this);
      final String tmpName = "_AppletTmp" + System.identityHashCode(this);
      window.eval("var "+ tmpName +" = {}");
      appletTmp = (JSObject)window.getMember(tmpName);
    }
  }
}

Granted, this code doesn’t clean up after itself, so if you use it for a long time or on a lot of JS objects, you will need to add some clean up code to it. With that, I was finally able to fully use the Java Applet the way I wanted to: As a “service provider” to the JavaScript on our existing page.