Grant's Space

Do you think Siri is cool? Then you probably didn’t use the original app, and you probably also never heard of the 60′s program “Eliza”.

I used to be able to ask Siri for actual information, and get real answers. I could ask it to reserve a table for two at Beausejour at 8 o’clock, and it would place the reservation. I could ask it what good movies were playing nearby, and it would show a list of movies ranked high (on rotten tomatoes) that were playing near me with a list of theaters. I could even tap to buy tickets.

But no more. Siri’s become stupid. Like tragic car crash stupid. The kids in school make fun of it (literally). And that’s all it’s good for. If I ask it to make a reservation at a restaurant at 8pm, it’ll respond with “there’s a restaurant named Beausejour near you”. Great, I already knew that. Ask it for movies and it’ll just display a list of theaters. Ask it almost anything else and it’ll say “I don’t understand”. It’s not good at scheduling, texting, or calling; not better than just tapping and typing at least.

Why does it take hackers to try to revive what used to be a great app?

I miss Siri.

Developing a great web site or web application is really very easy, but there are many ways to do it badly, and that makes it difficult.

Here’s how to do it “right”.

1. A URI identifies each and every resource
2. Gather your content, and code it in straight, standard, valid XHTML
3. Lay out your content using CSS
4. Ignore the flashy new technology, stick with well-established basics

These 4 simple rules are all you need to design great sites. Companies are constantly trying to push their fancy new technologies, and browser makers put proprietary features into their browsers hoping you’ll use them, and force people to use their browser. Unfortunately, this just makes your site unusable to people not using that browser.

A URI identifies each and every resource

URI means “Unique Resource Identifier”. Originally, and sometimes still, this meant that each page on a server was identified by a Unique string. It’s a great, simple concept: any given set of content should have a unique URI (yes, that’s redundant). This blog page, for example, has a unique string that identifies it. You can send that string to your friends (hint hint), and if they put it in a web browser, they’ll see this article. Seems straight forward, but many many web sites break this rule. For example, if I’m searching for flights on a travel site and have a certain set of flights displayed, I should be able to copy the URI and send it to my friend. Unfortunately, this rarely works. If I load up my shopping cart with stuff at Amazon.com, I should be able to send that cart to my friend. I can’t.

Make your sites RESTful. That’s a fancy word meaning “use the web the way it was designed”. The O’Reilly “RESTful Web Services” book is excellent extended reading on this subject:

Restful Web Services

URIs are fancy things that can include a LOT of information. Let’s take the examples I gave above and do them right:

Filtering flights

I see filters on web sites done wrong all the time, and it drives me nuts. It’s so easy to do right, and would be so convenient. I apologize, because now you’re going to be driven nuts too.

Each element of a filter should be included in the URI. Lets say you’re setting up a travel site. People fill in departing and arriving airport, dates, and preferred time range. They click a “search” button. The button should encode that search into a GET request (NOT A POST!), which then displays the result. So it might look like

http://www.mytravelsite.com/flightsearch?dep_loc=lax&arrive_apt=nyc&dep_time=12&dep_date=2011-10-20&ret_date=2011-10-30. (I’ve left a lot of potential detail out, like return time, for brevity).

Say your result page then has additional filters. It’s really fancy, and clicking a column header sorts by that column. When a user clicks a filter, say selecting a max price of $500 per flight, the URI should change to, say, include “max_price=500″.

http://www.mytravelsite.com/flightsearch?dep_loc=lax&arrive_apt=nyc&dep_time=12&dep_date=2011-10-20&ret_date=2011-10-30&max_price=500

Now we sort by price:

http://www.mytravelsite.com/flightsearch?dep_loc=lax&arrive_apt=nyc&dep_time=12&dep_date=2011-10-20&ret_date=2011-10-30&max_price=500&sort_by=price

Now that’s a nice URI I can email to my friend and say “look at the cool flights to New York, let’s go!”. It’ll have the exact filters I selected and even be sorted the same. This URI represents “Flights from LAX to NYC departing Oct 10, 2011, returning Oct 20, 2011 that cost less than $500 ordered by price”.

Sadly, you’ll see similar results on sites that might include the flight info, but once you start clicking on options like price, time windows, and sorts, those don’t factor into the URI.

Shopping Cart

I’ve never seen this done right, but it’s so easy to do right.

To be RESTful, (and have unique resources to identify), you shouldn’t maintain “state” on the server. That means no session cookies (although I’ll concede to an authentication token, but I’ll get to that later). “But how do I store my user’s shopping cart” you ask? Let’s follow the rule: A shopping cart with certain items in it is a resource. That resource should have a unique identifier. If I go to that identifier (URI) later, I should see the same cart. If I send the URI to my friend, they should see the same cart.

http://www.myshoppingsite.com/cart/myusername

So as I shop and click “add to cart”, my items get POSTed to /cart/myusername. A GET to /cart/myusername returns the items in the cart. Ideally, this would be a basic XML document that gets wrapped with a page layout via an appropriate CSS style sheet (sticking to the content/layout separation rules). That way, you could use it in an API to an iPhone app, or as a mobile web page, or as a normal web page by just using the right style sheet. But we’re not talking about that yet.

From here you might want to let the user set up multiple shopping carts. You could have:

http://www.myshoppingsite.com/cart/myusername/12345

A POST to http://www.myshoppingsite.com/cart/myusername/ now would create a new cart and return its ID (e.g. in the resulting web page). Subsequent pages would then POST to /cart/myusername/12345. The user could then create carts specifically to email to friends, e.g. “here’s a cart of all the stuff you need to build the game PC we were talking about”.

For details on how to formulate RESTful URIs, read that O’Reilly book above. And notice that handy URI that takes you to a page on Amazon.com and tells them I sent you.

I’ve mentioned “GET” and “POST” above. These are HTTP methods. There are four total methods: GET, PUT, DELETE, and POST. These tell a server to GET a resource, PUT (change or replace) it, DELETE it, or add to it (POST). So if we were maintaining a database of products, product number 12345 might be identified by the URI http://www.mysite.com/product/12345. A PUT to this URI would change the product. A DELETE would delete the product. A GET would return the product’s info. You wouldn’t POST to this product. However, to create a new product, you would POST to http://www.mysite.com/product/, which could create a new product and return a new product number for it. You can read all about that in the RESTful Web Services book.

I mention HTTP methods because there’s a sub-rule that’s frequently broken in web pages: GET should be used to retrieve information, and POST should be used to change it. When you’re writing a web form in HTML, you can specify the method in the “form” tag. If you’re doing a search form, this method should be “GET”. If you’re changing a user’s password, it should be “POST”. HTML doesn’t support PUT or DELETE. If you use the wrong method, you’re hindering your site, and you could end up doing real damage in some cases. For example, if you POST a search form, the user can’t bookmark it and send the results to his friends. That could be (lots of) lost business. If you use GET in a form that adds a new product, you could accidentally hit “back”, and that product would be created again. If you reload the page, it’d be created again and again. What if you did that with a credit card order form? That could be hard to explain. Stick with good URIs, and use GET to get, POST to change, and you’ll make great sites.

Gather your content, and code it in straight, standard, valid XHTML

People frequently do this step backwards, and it gets confusing. They think they want a site to look cool, so the focus on graphics and/or use a template. That’s like painting a picture before you know what you want to paint.

One thing is true of every single web site: its sole purpose is to get some information to someone. Someone wants something, and a great site gets whatever that is to the user as easily as possible. If your site is a store, the perfect site would read the customer’s mind, charge their card, and ship the item they wanted to them. Your job is to get as close to that as possible.

So when you start designing your site, get your content together before ANYTHING else. If you’re starting a store, gather your inventory and marketing text. If you’re advertising your company’s services, get the info about your company together.

Break the information into chunks that are easy to search and navigate. e.g. on most basic company sites, you’ll have a page of contact/location information, a basic introduction page, a page with more detailed info about the company, and a page about your products or services.

Each chunk becomes a resource, and gets its own URI (for a basic web site, that means each page gets a URI, which is what happens if you put a page on a web server anyway. So you’ll get http://www.mysite.com/about.html, for example.)

The content goes into straight, boring XML or XHTML. No layout information. If I look at it, it should look like a linear outline. This means any device or browser will be able to display it. Or, an iPhone app could read it.

If you find yourself thinking about layout here, stop it. If you output a navigation menu, it should look like a bullet-point list. Use attributes to give classes and IDs to your elements. Then the CSS can use that information to create any layout you want.

Your page output should look something like this:

<xhtml>
  <head>
    <title>My Company - About Us</title>
    <LINK REL=StyleSheet HREF="style.css" TYPE="text/css" MEDIA=screen>
  </head>

  <body>
    <ol>
      <li><a href="/index.html">Home</a></li>
      <li>...
    </ol>

    <h1>About Us</h1>

    <p>We are a company that does stuff and things ...</p>
  </body>
</xhtml>

Notice there’s nothing about layout in there. The navigation information is just in “li” tags, and the content of the page, if displayed without a style sheet, will just look like a boring outline.

The navigation menu, in REST parlance, provides information to the client about available resources based on the request for the current resource. In short, it tells you places you could go from here. The client (e.g. the browser) could choose to display that information, turn it into buttons in an app, or hide it completely.

Once you’ve got your content in XHTML, run it through the validator at w3c.org. If you ever find yourself testing in different browsers, or writing Javascript that checks what browser the user’s using, slap yourself and stop it. The world wide web works on standards – write your code to the standards and it should work on all browsers. The browser’s job is to render a page according to the standards. If it doesn’t, that’s its problem. If developers follow the standards, browsers that don’t follow them will disappear, because users will switch to a browser that works. If you need content in a different format (e.g. XML for your iPhone app), that should appear at a different URL (for example http://api.mysite.com/cart/myusername/). But if you stick to delivering your content straight up in standard XHTML, you won’t have to go through the trouble of providing different “views” for it.

Lay out your content using CSS

CSS is magic. It can do incredible things, including making the boring XHTML above look like a full-blown web page. You can add background colors and images. You can place things anywhere on the page you want. You can even have the content resize based on the browser window width. You can specify layouts for different media types, including “screen”, “handheld”, “print”, and “speech”. You can change the color of a button when you click on it. You can turn the navigation list above into a tab-like navigation menu that changes colors when you click on each tab. You can anchor the nab bar to the side of the screen while the rest of the page scrolls. Read all about it at w3.org.

The point here is, use CSS to lay out the content. If you want to support a different layout on a small screen than a big one, do it in CSS.

I’d like to say a word about mobile content: some sites provide a subset of their main site’s content for mobile devices. This is stupid. Decide if your information is useful enough to include on your site or not. When you find yourself thinking “well, this page has too much information for a smaller screen”, maybe your page just has too much information. Consider breaking it up into smaller resources, or consider just dropping the “too much” part of the information. Don’t clutter the user’s browser and bandwidth just because it’s there. Again, this goes back to the “gather your content first” rule. If the user just wants to know the time, don’t give them a map of the world along with it.

Ignore the flashy new technology, stick with well-established basics

This is where most developers run into trouble. Which browser should I code for? What technology should I use? Ooh, Cold Fusion looks cool, Javascript is cool, this new programming language is cool, .NET looks like it makes things easy, I can write nifty menus in Flash, etc etc etc…

There’s always something new and cool. The problem is that most of it doesn’t work well, or doesn’t run on a user’s browser, or breaks the rules above.

Stick with what works and you’ll write great sites that won’t give you headaches: UNIX, Apache, PHP, Python, Perl, MySQL, Postgres, Javascript, Java. For frameworks, Django, Ruby on Rails, and maybe Cake. Stick to those and you’ll be writing great sites that’ll actually work. These technologies also let you stick to the rules above easily (well, Cake might not). Try to use MVC design (look that up on Wikipedia). Avoid flash (use Javascript instead), Cold Fusion (breaks rules), and anything by Microsoft (breaks rules, isn’t reliable).

A note about REST and sessions

RESTful services don’t store “state” on the server. That means it doesn’t matter what browser I use, where I come from, or how long I wait between requests. On a RESTful site, I could start a flight search on one computer, bookmark my URI, go to another computer, click the bookmark, and keep going. Or, I could start a shopping cart order on one system, email my friend the URI, and have him place the order for me. Of course, this isn’t how sites work. One big issue is “authentication”. In a RESTful web service, there are a few extra components to that URI request: Authentication (who are you?) and Permission (are you allowed to perform this method on this resource?).

So for our shopping cart, my friend would first have to prove to the server who he is, and then he’d have to have Permission to see my cart. It’s the Authentication step that gets tricky. Ideally, the user would prove who he is (usually using a username and password) with every request. Browsers do support this, but it’s not very friendly. This is usually worked around by using session cookies, which is difficult, non-RESTful, and causes annoying problems like taking the user to a page they didn’t request after logging in, forgetting the information in a huge form they entered, etc.

Instead of using cookies, use HTTP Auth Digest. If you want a pretty login form, you can create one with Javascript. There’s a good article with code samples by Paul James that shows you how.

Sadly, it looks like Intuit’s programmers have infected the once good Mint.com

Mission
Utilize cross-media e-services to grow revolutionary communities.

Point-by-point Plan

• incubate open-source interfaces
• target synergistic models
• matrix front-end schemas
• morph bricks-and-clicks web services
• embrace one-to-one niches
• (profit!)

Plan developed with the aid of this tool.

If I get a bunch of Google hits from BS keywords, I’m going to laugh.

I program a lot in Perl. I used to be a big fan of “modules should never die”. Now I’m the opposite. I want a method to die any time it can’t succeed in doing what it’s been asked to do.

Here’s why:

Without exceptions:

$module->do_something( %args ) or die "I couldn't do something";
$module->do_something_else( %args ) or die "I couldn't do something else";
[...]


With exceptions:

$module->do_something( %args );
$module->do_something_else( %args );


Now imagine that with hundreds of lines of code. I love having my script or module simply contain a list of commands – Do this then this then this – without having to babysit every single line to make sure everything went ok.

It also lets you handle things on a more transactional basis. Say you want to connect to a database and update some data. With exceptions you can do this:

use Error.pm qw(:try);

my $dbh = DBI->connect($data_source, $username, $auth, { RaiseError => 1, AutoCommit => 0 } );
# RaiseError turned on exceptions. We’re free to roam.

try {
$dbh->do(“update mytable set myfield=’gobble’ where myotherfield=’turkey’”);
$dbh->do(“update anothertable set anotherfield=’somevalue’ where yetanother=’anothervalue’”);
$dbh->commit;
} otherwise {
$dbh->revert;
};

(Don’t quote me on syntax here – this is very perl-like psuedocode. )

I also like building exceptions into my own modules using Error.pm. This lets me throw specific types of exceptions. I usually end up with an exception type for data errors, IO errors, and Database errors, just in case I want to catch those. For example, I may wait and retry IO and Database errors, but a Data error (bad user data) I’ll let go uncaught up to the UI level so that the UI can complain to the user.

That brings me to the other reason I like exceptions, and the reason I started using them in the first place. Say you’re building a complex module, or set of modules, with many layers of subroutine calls. If your error-trapping mechanism is the ever-popular “return undef”, you now have, say, 5 layers of subroutine/method calls to check and pass undef up. You forget an “or return undef” at any level, and you let an error slip by. Usually something’ll break later on in your program as a result, and you’ll have a hard-to-track bug. I prefer to throw an exception and include a stack trace (see the Error.pm docs for sample code). Then you can catch the exception at any point, and if you don’t, you still have a stack trace that shows exactly where the error occurred.

“But I don’t want my script killed by a module I’m using, I want to decide what to do if there’s a problem”. I used to think this. But the fact is, 99.9% of the time you’re going to do one of two things:

1. $module->do_something() or die “Couldn’t do something: ” . $module->errstr;
2. $module->do_something();

In the first case, if do_something threw an exception, it’d save you the babysitting code. In the second case, if “do_something” didn’t do what it was supposed to, chances are your program can’t really continue anyway, right? And if it can, maybe you don’t need to “do_something()” anyway. And for that .1% of the time you want to ignore errors, you can “eval { $module->do_something(); }”.

So there. I like throwing exceptions. Maybe you will too.

Further reading: “man Error”, http://www.perl.com/pub/2002/11/14/exception.html