Grant's Space

Developing a great web site or web application is really very easy, but there are many ways to do it badly, and that makes it difficult.

Here’s how to do it “right”.

1. A URI identifies each and every resource
2. Gather your content, and code it in straight, standard, valid XHTML
3. Lay out your content using CSS
4. Ignore the flashy new technology, stick with well-established basics

These 4 simple rules are all you need to design great sites. Companies are constantly trying to push their fancy new technologies, and browser makers put proprietary features into their browsers hoping you’ll use them, and force people to use their browser. Unfortunately, this just makes your site unusable to people not using that browser.

A URI identifies each and every resource

URI means “Unique Resource Identifier”. Originally, and sometimes still, this meant that each page on a server was identified by a Unique string. It’s a great, simple concept: any given set of content should have a unique URI (yes, that’s redundant). This blog page, for example, has a unique string that identifies it. You can send that string to your friends (hint hint), and if they put it in a web browser, they’ll see this article. Seems straight forward, but many many web sites break this rule. For example, if I’m searching for flights on a travel site and have a certain set of flights displayed, I should be able to copy the URI and send it to my friend. Unfortunately, this rarely works. If I load up my shopping cart with stuff at Amazon.com, I should be able to send that cart to my friend. I can’t.

Make your sites RESTful. That’s a fancy word meaning “use the web the way it was designed”. The O’Reilly “RESTful Web Services” book is excellent extended reading on this subject:

Restful Web Services

URIs are fancy things that can include a LOT of information. Let’s take the examples I gave above and do them right:

Filtering flights

I see filters on web sites done wrong all the time, and it drives me nuts. It’s so easy to do right, and would be so convenient. I apologize, because now you’re going to be driven nuts too.

Each element of a filter should be included in the URI. Lets say you’re setting up a travel site. People fill in departing and arriving airport, dates, and preferred time range. They click a “search” button. The button should encode that search into a GET request (NOT A POST!), which then displays the result. So it might look like

http://www.mytravelsite.com/flightsearch?dep_loc=lax&arrive_apt=nyc&dep_time=12&dep_date=2011-10-20&ret_date=2011-10-30. (I’ve left a lot of potential detail out, like return time, for brevity).

Say your result page then has additional filters. It’s really fancy, and clicking a column header sorts by that column. When a user clicks a filter, say selecting a max price of $500 per flight, the URI should change to, say, include “max_price=500″.

http://www.mytravelsite.com/flightsearch?dep_loc=lax&arrive_apt=nyc&dep_time=12&dep_date=2011-10-20&ret_date=2011-10-30&max_price=500

Now we sort by price:

http://www.mytravelsite.com/flightsearch?dep_loc=lax&arrive_apt=nyc&dep_time=12&dep_date=2011-10-20&ret_date=2011-10-30&max_price=500&sort_by=price

Now that’s a nice URI I can email to my friend and say “look at the cool flights to New York, let’s go!”. It’ll have the exact filters I selected and even be sorted the same. This URI represents “Flights from LAX to NYC departing Oct 10, 2011, returning Oct 20, 2011 that cost less than $500 ordered by price”.

Sadly, you’ll see similar results on sites that might include the flight info, but once you start clicking on options like price, time windows, and sorts, those don’t factor into the URI.

Shopping Cart

I’ve never seen this done right, but it’s so easy to do right.

To be RESTful, (and have unique resources to identify), you shouldn’t maintain “state” on the server. That means no session cookies (although I’ll concede to an authentication token, but I’ll get to that later). “But how do I store my user’s shopping cart” you ask? Let’s follow the rule: A shopping cart with certain items in it is a resource. That resource should have a unique identifier. If I go to that identifier (URI) later, I should see the same cart. If I send the URI to my friend, they should see the same cart.

http://www.myshoppingsite.com/cart/myusername

So as I shop and click “add to cart”, my items get POSTed to /cart/myusername. A GET to /cart/myusername returns the items in the cart. Ideally, this would be a basic XML document that gets wrapped with a page layout via an appropriate CSS style sheet (sticking to the content/layout separation rules). That way, you could use it in an API to an iPhone app, or as a mobile web page, or as a normal web page by just using the right style sheet. But we’re not talking about that yet.

From here you might want to let the user set up multiple shopping carts. You could have:

http://www.myshoppingsite.com/cart/myusername/12345

A POST to http://www.myshoppingsite.com/cart/myusername/ now would create a new cart and return its ID (e.g. in the resulting web page). Subsequent pages would then POST to /cart/myusername/12345. The user could then create carts specifically to email to friends, e.g. “here’s a cart of all the stuff you need to build the game PC we were talking about”.

For details on how to formulate RESTful URIs, read that O’Reilly book above. And notice that handy URI that takes you to a page on Amazon.com and tells them I sent you.

I’ve mentioned “GET” and “POST” above. These are HTTP methods. There are four total methods: GET, PUT, DELETE, and POST. These tell a server to GET a resource, PUT (change or replace) it, DELETE it, or add to it (POST). So if we were maintaining a database of products, product number 12345 might be identified by the URI http://www.mysite.com/product/12345. A PUT to this URI would change the product. A DELETE would delete the product. A GET would return the product’s info. You wouldn’t POST to this product. However, to create a new product, you would POST to http://www.mysite.com/product/, which could create a new product and return a new product number for it. You can read all about that in the RESTful Web Services book.

I mention HTTP methods because there’s a sub-rule that’s frequently broken in web pages: GET should be used to retrieve information, and POST should be used to change it. When you’re writing a web form in HTML, you can specify the method in the “form” tag. If you’re doing a search form, this method should be “GET”. If you’re changing a user’s password, it should be “POST”. HTML doesn’t support PUT or DELETE. If you use the wrong method, you’re hindering your site, and you could end up doing real damage in some cases. For example, if you POST a search form, the user can’t bookmark it and send the results to his friends. That could be (lots of) lost business. If you use GET in a form that adds a new product, you could accidentally hit “back”, and that product would be created again. If you reload the page, it’d be created again and again. What if you did that with a credit card order form? That could be hard to explain. Stick with good URIs, and use GET to get, POST to change, and you’ll make great sites.

Gather your content, and code it in straight, standard, valid XHTML

People frequently do this step backwards, and it gets confusing. They think they want a site to look cool, so the focus on graphics and/or use a template. That’s like painting a picture before you know what you want to paint.

One thing is true of every single web site: its sole purpose is to get some information to someone. Someone wants something, and a great site gets whatever that is to the user as easily as possible. If your site is a store, the perfect site would read the customer’s mind, charge their card, and ship the item they wanted to them. Your job is to get as close to that as possible.

So when you start designing your site, get your content together before ANYTHING else. If you’re starting a store, gather your inventory and marketing text. If you’re advertising your company’s services, get the info about your company together.

Break the information into chunks that are easy to search and navigate. e.g. on most basic company sites, you’ll have a page of contact/location information, a basic introduction page, a page with more detailed info about the company, and a page about your products or services.

Each chunk becomes a resource, and gets its own URI (for a basic web site, that means each page gets a URI, which is what happens if you put a page on a web server anyway. So you’ll get http://www.mysite.com/about.html, for example.)

The content goes into straight, boring XML or XHTML. No layout information. If I look at it, it should look like a linear outline. This means any device or browser will be able to display it. Or, an iPhone app could read it.

If you find yourself thinking about layout here, stop it. If you output a navigation menu, it should look like a bullet-point list. Use attributes to give classes and IDs to your elements. Then the CSS can use that information to create any layout you want.

Your page output should look something like this:

<xhtml>
  <head>
    <title>My Company - About Us</title>
    <LINK REL=StyleSheet HREF="style.css" TYPE="text/css" MEDIA=screen>
  </head>

  <body>
    <ol>
      <li><a href="/index.html">Home</a></li>
      <li>...
    </ol>

    <h1>About Us</h1>

    <p>We are a company that does stuff and things ...</p>
  </body>
</xhtml>

Notice there’s nothing about layout in there. The navigation information is just in “li” tags, and the content of the page, if displayed without a style sheet, will just look like a boring outline.

The navigation menu, in REST parlance, provides information to the client about available resources based on the request for the current resource. In short, it tells you places you could go from here. The client (e.g. the browser) could choose to display that information, turn it into buttons in an app, or hide it completely.

Once you’ve got your content in XHTML, run it through the validator at w3c.org. If you ever find yourself testing in different browsers, or writing Javascript that checks what browser the user’s using, slap yourself and stop it. The world wide web works on standards – write your code to the standards and it should work on all browsers. The browser’s job is to render a page according to the standards. If it doesn’t, that’s its problem. If developers follow the standards, browsers that don’t follow them will disappear, because users will switch to a browser that works. If you need content in a different format (e.g. XML for your iPhone app), that should appear at a different URL (for example http://api.mysite.com/cart/myusername/). But if you stick to delivering your content straight up in standard XHTML, you won’t have to go through the trouble of providing different “views” for it.

Lay out your content using CSS

CSS is magic. It can do incredible things, including making the boring XHTML above look like a full-blown web page. You can add background colors and images. You can place things anywhere on the page you want. You can even have the content resize based on the browser window width. You can specify layouts for different media types, including “screen”, “handheld”, “print”, and “speech”. You can change the color of a button when you click on it. You can turn the navigation list above into a tab-like navigation menu that changes colors when you click on each tab. You can anchor the nab bar to the side of the screen while the rest of the page scrolls. Read all about it at w3.org.

The point here is, use CSS to lay out the content. If you want to support a different layout on a small screen than a big one, do it in CSS.

I’d like to say a word about mobile content: some sites provide a subset of their main site’s content for mobile devices. This is stupid. Decide if your information is useful enough to include on your site or not. When you find yourself thinking “well, this page has too much information for a smaller screen”, maybe your page just has too much information. Consider breaking it up into smaller resources, or consider just dropping the “too much” part of the information. Don’t clutter the user’s browser and bandwidth just because it’s there. Again, this goes back to the “gather your content first” rule. If the user just wants to know the time, don’t give them a map of the world along with it.

Ignore the flashy new technology, stick with well-established basics

This is where most developers run into trouble. Which browser should I code for? What technology should I use? Ooh, Cold Fusion looks cool, Javascript is cool, this new programming language is cool, .NET looks like it makes things easy, I can write nifty menus in Flash, etc etc etc…

There’s always something new and cool. The problem is that most of it doesn’t work well, or doesn’t run on a user’s browser, or breaks the rules above.

Stick with what works and you’ll write great sites that won’t give you headaches: UNIX, Apache, PHP, Python, Perl, MySQL, Postgres, Javascript, Java. For frameworks, Django, Ruby on Rails, and maybe Cake. Stick to those and you’ll be writing great sites that’ll actually work. These technologies also let you stick to the rules above easily (well, Cake might not). Try to use MVC design (look that up on Wikipedia). Avoid flash (use Javascript instead), Cold Fusion (breaks rules), and anything by Microsoft (breaks rules, isn’t reliable).

A note about REST and sessions

RESTful services don’t store “state” on the server. That means it doesn’t matter what browser I use, where I come from, or how long I wait between requests. On a RESTful site, I could start a flight search on one computer, bookmark my URI, go to another computer, click the bookmark, and keep going. Or, I could start a shopping cart order on one system, email my friend the URI, and have him place the order for me. Of course, this isn’t how sites work. One big issue is “authentication”. In a RESTful web service, there are a few extra components to that URI request: Authentication (who are you?) and Permission (are you allowed to perform this method on this resource?).

So for our shopping cart, my friend would first have to prove to the server who he is, and then he’d have to have Permission to see my cart. It’s the Authentication step that gets tricky. Ideally, the user would prove who he is (usually using a username and password) with every request. Browsers do support this, but it’s not very friendly. This is usually worked around by using session cookies, which is difficult, non-RESTful, and causes annoying problems like taking the user to a page they didn’t request after logging in, forgetting the information in a huge form they entered, etc.

Instead of using cookies, use HTTP Auth Digest. If you want a pretty login form, you can create one with Javascript. There’s a good article with code samples by Paul James that shows you how.

On August 29, 2010, Santa Monica’s Big Blue Bus took a step backward in fare handling. Ironically, to announce the change, they ran ads implying that you wouldn’t need to use money on the bus any more. In reality, they were replacing the Little Blue Card (which allowed you to load cash onto the card and pay for your bus fare, transfers, express bus fees, etc) with various trip-based or multi-day-based passes (which don’t allow you to pay for transfers or express bus fees).

So, Dear Los Angeles Public Transportation providers, here’s why spewing advertising billboards about saving gas all over the city isn’t getting people on your busses and trains:

• Make paying simple
  Seriously, I’ve lost count of the times I’ve driven to work instead of figuring out if I have exactly $1.50 for the trip there and $1.85 for the trip back (and calculating that cost in the first place). Heaven forbid I get on the wrong version of the bus I need to get back (see a lower point) and need to pay an extra amount. Take a hint from Hong Kong and London:
  • Make it easy to load cash on the TAP card (hint: online, in EVERY Metro station, and on EVERY bus)
  • Make the fares simple! Seriously, it’s one rate if you’re staying local, another if you’re going into a different “zone”, another if you’re a student, another if you’re a senior, another if you’re taking more than one bus. GEEZ! It’s easier to buy a car than take the damn bus once! Remember the old KISS rule – Keep It Simple, Stupid
• Make the busses and trains run more frequently
  Some are fine, some aren’t. If a line runs less frequently than every 20 minutes, it’s basically useless. 5-10 minutes, and you’re talking about replacing my car. 10-20 minutes, and I’ll consider it if I don’t want to drive. 20 minutes, and I’m just gonna be spending my life at bus stops – no longer worth it.
• Make the busses run reliably
  This one’s huge. I was all set to bus to work and cancel my parking until the day I got left on the street for over 40 minutes one cold night because a bus (scheduled every 20 minutes…) just didn’t show. The busses sole purpose is to get me somewhere – if it fails that purpose, I have to find something else to get me there. LA Metro repeatedly fails that purpose.
• Make all transit lines use the TAP card
  “Can’t we all just get along?” Really, everyone’s got to use the same payment system to make the system usable. LA’s too big to have to guess at the three systems you might need to use to get somewhere more than a couple miles away (or even less than a couple miles away). The EZ transit pass is ok, but is only affordable if you’re going to only use public transportation; it loses the occasional rider.
• Make the bus/train lines simple
  Sure, it’s a big city, and the system’s going to be a bit complicated. But at least try for simplicity: Let’s take the 108 Metro bus for example. One out of every three trips, the route extends into Marina Del Rey. Same bus number, different destination. Really? Who thought that would be a good idea? So if you’re waiting for the 108 and need to go to Marina Del Rey, and don’t notice that the head sign says “Fox Hills Mall” instead of “Marina Del Rey”, you just got stuck at the mall, and have to hope you have the extra buck fifty or whatever it is to get the rest of the way (whenever the right version of the 108 comes). Again, the bus just failed to get me where I need to go – time for another solution.

Yeah, I know “running the bus every 10 minutes is too expensive”, and “there aren’t enough people on the bus to warrant it running more than every 45 minutes through this one section, but there is through this other section, and we don’t really want to have it be a transfer because the people coming in from Marina Del Rey would have to transfer blah blah blah”. And so, we widen the 405, because if you take the 761 into the valley, good luck figuring out how to get back (hint: it’s not on the 761). Make it easier to pay, faster to get places, and easier to comprehend, and maybe LA can join the list of world cities in which people who can afford a car use public transportation to get around anyway.

I’m on the computer a lot (over 40 hours/wk), and there are a few tools I’ve noticed I have on every computer that have become a standard part of my setup.

• Firefox
  On my Mac, I tend to use Safari, but at work and on any PC I’m on, I use Firefox. Safari pissed me off a couple of days ago (Flash crashed it again again again), so I set Firefox as my default browser (I’m using it to write this, in fact). It seems that even the worst AJAX-abusing web sites work in Firefox, it’s pretty standard across all platforms, and it wraps plugins well enough that when certain Adobe products explode, the browser just displays “Sorry, the flash plugin has crashed” or similar, instead of crashing itself. Firefox pre-3.6 was frickin’ slow, but 3.6 is about as fast as Safari. Of course, I do lose syncing of bookmarks to my iPhone, but honestly I almost never use bookmarks on my iPhone.
• Lastpass
  I used to use 1Password, but Lastpass took over because 1Password didn’t work on Linux (it may now, I don’t know). I still run 1Password (Pro) on my iPhone, but that’s mainly to look up passwords that aren’t in Lastpass yet. Lastpass is web based, which makes it really convenient if you end up on a hotel computer and need to log into a web site, and may not have your iPhone handy. Of course, there’s also an iPhone app, but I haven’t gotten that yet. 1Password also requires too much setup: you need to store your password file somewhere and sync it yourself (via Dropbox, iDisk, whatever), and the iPhone sync requires you to manually run the program on your Mac and phone at the same time. See, Winbloz users just went away. Lastpass just lets you store passwords. It handles the syncing (to their servers), and is still secure, because they’re just storing encrypted information.
• WordPress
  I’ve used a lot of blogging products over the years, starting with straight HTML in a text editor. WordPress has great features, and is the standard for setting up a blog. I use the installed version, which is conveniently a 1-click install through Simplescripts on my current web host. Before this host, I installed it myself (and even wrote scripts to do installs/upgrades automatically before that was built in). Current wordpress is stable, feature-rich, and very easy to use. Themes and plugins usually work with little modification (you used to have to tweak the heck out of things to make a new theme work – “Widgets” fixed that). Updates are automatic like most desktop software these days. The Askimet plugin combined with the “Bad Behavior” plugin stop 99% of my comment spam (which is a huge problem and time sucker if you’re setting up a blog these days). WordPress lets you set up a blog very quickly and easily, and add/change features as you go. In short, it’s quite flexible, but easy to get started with. Although you can use it as a CMS to set up web sites (and I have), it’s really designed as a blog platform, so I wouldn’t recommend it if you’re trying to find a web site builder. If your web site is a blog with some extra pages, however, WordPress is fine.
• Dropbox
  This is a must on all my computers now. Dropbox’s syncing is impeccably handled, it works on all platforms, and it allows 1-click sharing of folders. I have an “Assistant Share” folder (shared with my assistant and girlfriend), a “Bookkeeping” folder (shared with my bookkeeper), and an “Accounting” folder (shared with my accountant). If I need to get tax documents to my accountant, I just drop them into my Accounting folder, and they’re on her computer immediately. Need to update some procedures for my assistant – I update the document in my “Assistant Share” folder. The syncing is immediate and I have never had a problem with file conflicts (well, I did once, but my girlfriend was copying files into her Dropbox folder and I moved them on mine while they were syncing. Even then, cleanup was pretty quick, and Dropbox didn’t break anything – just moved the files I said to move, while leaving others in place because she’d just put them there). To share a folder, you control-click (or right-click) and select “Share” (the option varies slightly depending on the OS you’re running). Works on Linux, Mac, and Winblows.
• Ubuntu
  
  
  I had to use a PC for work, and I tolerated Windows for a couple days of instability on it before I borrowed my friend’s Ubuntu CD. I still prefer MacOS X, but if I can’t use it for some reason, Ubuntu’s next in line. In some ways I even prefer it to MacOS. Software updates (for all software) are built in. You can customize the desktop to behave like Windows or MacOS X, or some weird hybrid if you want. It’s almost as stable as MacOS X (although I do run into occasional glitches that require a Google search to remedy). It’s got a good user base, so most issues you run into can be solved by Googling and following the instructions someone’s spelled out. It’s 99% an end-user-friendly OS now, including Firefox, OpenOffice, an iTunes-like music player/store (which even has Last.fm built in), and a host of other applications pre-installed. Getting new software is unique – you go to the “Software Center” menu item and search (or browse by categories such as Office, Entertainment, etc). Most software is free (open-source). Software updates are usually automatic, as when you “download” software, you’re not actually downloading an application, but installing a package description, which then tells the software update tool (“package manager”) to include that software in its updates. Fancy. To the end user, that means that the software just magically stays up to date. Plus, it’s free, and in many cases better than certain commercial equivalents. (I don’t bother with MS Office any more, as OpenOffice runs on PC/Mac/Linux).
• Netflix
  I don’t know anyone who isn’t a Netflix subscriber, but I figured I’d mention it anyway. I use it both for DVDs and to Watch Instantly on my Mac and the PS3. Netflix jumped in as part of the beginning of the switch from broadcast to on-demand content. (Tivo was the other part).

So that’s a few of my favorite non-obvious tools. I also use the iPhone, running Shazam, Omnifocus, Now Playing and Siri on my home screen. (Now Playing is there to add movies to my Netflix queue when I see cool previews). Shazam is awesome for figuring out “what’s that song”, and Omnifocus is the best GTD-based organization software for Mac/iPhone (although the lack of a Linux version is problematic for me – if there was an equivalent web-based tool with an iPhone app I’d use that instead).

What do avoiding car accidents, being on time, and successful inventory control all have in common?

Buffers.

People aren’t precise. Everything we do has a margin of error. It’s cool and movie-like to drive as fast as your mind, and car, can handle, or run in the door just in the nick of time for something important; but in the real world, chances are that you’ll get pulled over, or hit something, or arrive at the important event just after it started (or 30 minutes late).

There are natural limits in the world: the fastest your car can go, the fastest your mind can process information, the actual time the meeting starts, the actual amount of toothpaste in the tube. If you pass those limits, it’s too late: your car breaks down, you hit something you didn’t expect because you didn’t see it, you’re late for your friend’s performance, or you run out of toothpaste.

There’s an easy way to have those things almost never happen: set a different limit, and treat it as your actual
limit. Drive around the speed limit (you’ll notice other people are doing it too). Stay 2-3 seconds behind the car in front of you. Stop at the yellow light (it’s a buffer, btw). Have an unopened tube of toothpaste in a drawer. (When you need to open it, add “toothpaste” to your shopping list.)

The difference between your limit and the “hard” limit is a “buffer”, or a layer that protects you from the hard limit (just like those yellow barrels on the freeway stop you before you hit the hard cement wall). The buffer means you won’t get a red light ticket (or accident), or a speeding ticket, or run out of stuff.

You can also apply it to car maintenance (avoid breakdowns), your checking account balance (avoid overdrafts), and on and on.

You’ll find that once you’ve relieved the pressures of pushing the limits in mundane areas, you’ll suddenly have time, and mental freedom, to spend on more exciting things. This might take some getting used to, as you’ll have a void of time and thought to fill. Then you can push the important limits: your fears, your comfort zone, and why you’re spending time on this earth.

So, for some reason after returning from asia, I became obsessed with public transportation, and was determined to learn how to use Los Angeles’ obscenely difficult public transportation.

The net result of this is the location-aware, bookmarkable Metro Trip Planner mobile site, which, unlike Google Maps, includes ALL the LA area busses and trains, and calculates fares. Works on iPhone and Android phones (2.1 or higher). Should work on any phone with a decent JavaScript implementation and GeoLocation integrated into the browser. (Not sure if your phone has it? Try the link…). On the iPhone, click the “+” button at the bottom of your screen and “Save to Home Page” to make it accessible as an app. I think the Android has a similar capability.

The site is simply Metro.net’s mobile trip planner with a bunch of JavaScript that gets your current location and lets you add bookmarks (which are stored in your phone using a cookie – never sent anywhere).

I wrote it because I got tired of having to choose between the speed and convenience of Google Maps and the accuracy of the Metro Trip Planner. Now you can access the Metro Trip Planner fast, find out where you are, and how to get where you want to go in a few taps.

Hint: The trip planner accepts major landmarks, including “disneyland”, various shopping malls, etc. Try entering one as your destination.

Enjoy!