The Potential of Smartphones

So often in the mobile phone business, people have approached these devices as merely mobile versions of immobile technology. Thus the "mobile web", "mobile mail", "mobile phone", and so on. But what if we approached smartphones from the perspective of what they are, what they can do, and what we could do with that?

An example of a technology that is completely "home grown" to the mobile community is SMS. This was not a planned service in the same way as MMS or WAP -- the operators and handset manufacturers did not carefully design and market SMS. It was simply a capability available on the phones and network that suited people's needs, and so it took off. (Note that SMS was not "mobile instant messaging", it was simply a messaging facility for operator use which turned out to work wonderfully peer-to-peer. It's mode of operation is, in fact, quite different from IM, and it is only recently that efforts have been made to fit it into an IM type of UI, such as on the iPhone.)

So what could we come up with in the future, and how do we go about it? Or do we have to rely on accidents?

I think the first thing we need to understand with this approach is exactly what a phone is, and how it fits into people's lives. So let's tackle that one first.

What is a smartphone?
If you are intimately familiar with smartphones, you can skip this section. Still, it's fascinating to take stock of just how much functionality modern smartphones pack into them, and to think about the uses of that technology independent of the actual features.

A smartphone:

• Is a small computer with substantial CPU and memory resources
• Carried almost everywhere

Smartphones have:

• Relatively small screen (sometimes touch sensitive)
• Small keypad or keyboard
• Built-in phone, for telecommunications with other people
• Microphone and the ability to record from it
• Speaker and the ability to play music and sounds
• Usually a camera (or two) with the ability to capture stills and video
• Internet connection which is usually, but not always, available
• Bluetooth connection which can detect and communicate with neighbouring devices
• Infrared connection which can communicate with neighbouring devices
• Positioning information, available via cell ID or built-in GPS
• Databases for contacts and calendar information

Some smartphones have:

• Light sensor
• Motion detectors (eg. iPhone, Nokia 5500)
• Near field RFID units (eg. mobile Suica)

What can we do with this?

So, the question is then, what can we do with the (fairly impressive) bundle of functionality that millions of users carry around with them every day?

Well, some ideas are pretty straight-forward:

• Use it as a PDA (to keep contacts, calendar, and notes)
• Use it as a mobile web browser (slowly starting to become viable as screens get bigger and, more importantly, CPUs get fast enough to present the web in readable ways on a small screen)
• Use it as a mobile email terminal (RIM has been particularly successful in this area, although something like an E61 or P990i/M600i on an operator-provided IMAP push service is just as good, and much cheaper)
• Use it as a constantly-updated weather chart
• Use it as a navigation unit, with maps, current location (via GPS), dynamic routing, and even dynamically updated traffic status
• Use it as an e-book reader

What's obvious about these ideas are that they have all been transferred from devices that already exist. PDAs, web browsers (on desktops and laptops), email, online weather, GPS navigation, and e-book readers are all technologies that have been around for quite some time. Putting them on a smartphone certainly makes them more accessible, and thus more useful, but doesn't really transform the way they integrate with people's lives.

Are there other ideas that can be built from the smartphone's capabilities itself? Yes, of course there are, and here are some we've seen:

• Lifeblog: using the camera, location and time information, and recording snippets of your life along with some comments on it, creating a multimedia "life blog". http://www.nokia.com/lifeblog
• Sensor: using bluetooth and a personalised profile to discover and meet people in your
  immediate vicinity with matching interests. http://www.nokia.com/sensor
  

The problem with these ideas, and why they haven't taken the market by storm, is that there really just curiosities. They don't meet a real need. How many people complain that they don't have a sufficiently rich record of their lives? Not many.

New ideas

Are there other ideas that take advantage of the capabilities of a smart phone and meet a real need? I think there are many, and I'll talk here about one, which I would love to see implemented.

Imagine a service on your smartphone which took advantage of its computing ability, its knowledge of your location, and its connection to the internet. Imagine that you could specify a destination and a desired arrival time, and this service could go off and discover all the ways you could get to that destination at that time, then present you with the options, and then book your chosen options, and finally remind you about when you needed to get moving to get there, and guide you through the process.

For example, I might want to fly from my home on the Gold Coast, Australia, to a hotel in Hong Kong. The software would start by attempting to find a route from start to end. Once it knew this, it would attempt to find services on this route, starting with the most irregular and expensive (ie. the flight from Australia to Hong Kong), and then work down to the simplest (eg. getting to and from the airports). Then it would present me with a range of "best-case" options (no point confusing me with lots of almost-identical options), and I would choose what I wanted for the various legs. Remembered preferences (for example, I prefer the train over driving) would make the choices easier by prioritising them so my most likely choices are the first ones I see. Finally, it would take my choices, book them for me, and keep the e-ticketing information.

Then, when the time came to make the trip, the software would remind me when to start (maybe by putting appointments in my calendar), would present the e-ticket information when I needed it, and would guide me through the confusion of interchanges, and so on.

All of this information is available on-line today. All of the technology required to do this is available. Much of the infrastructure, such as mapping and routing technology is freely available for this type of "mash-up".

What's missing is a good UI running on the phone, which integrates well with the phone's capabilities (its small keyboard and screen, its calendar database and positioning technology), and provides fluid, friendly interaction.

There are obvious add-ons to this service, such as the ability to find and book accommodation given your parameters and choices. An even more powerful addition would be the ability to carpool with others who are heading in the same direction. Sharing aggregate data with transport providers could even allow them to improve the quality and efficiency of their services.

Mass market?

The question is, are these types of services useful for many people? Do they have mass market appeal?

If the purpose is to plan large-scale trips like Austalia to Hong Kong (or even interstate within large countries), the answer would have to be no. However, if the technology can handle small-scale trips like meeting someone in an unknown pub at a certain time, then this is far more useful to the general user.

The tipping point is based on usability and price. It has to be easier to use the phone to discover, book, and schedule your trip than it is to do it yourself. If you are looking at a regular trip (such as a commute), it's unlikely that you'd use such technology, unless it gave you benefits such as carpooling or a discount ticket (from the transport provider to encourage use of such services so that they could better implement their services). But an irregular, but still planned trip using public transport -- such as a weekend outing, or a meeting with the mates or for work -- presents planning and information gathering demands that a smartphone could easily perform.

Personally, the idea of being able to quickly and easily find my way to a meeting, without wasting time at connections or stressing about figuring out the best way there, sounds like a dream come true.

The key to these ideas

Perhaps the key to this approach is to understand the smartphone as an "invisible" tool. A tool that is simply the conduit for desires and information. The idea I've mapped out can include peer-to-peer functionality (with carpooling), which many believe to be a key to success, as well as information and service delivery.

This is the beauty of smartphones: they can span so many "worlds" that they can do all sorts of exciting things. Let's not just create "mobile" versions of existing, desk-bound services, let's try to create truly unique services with the capabilities available right now!

A Future for Symbian Smartphones

Come dream with me for a while. Dreaming about technology, especially when those dreams are firmly rooted in currently reality, is a truly helpful way to determine what trends and features are important to the big picture.

Imagine it's ten years in the future.

The mobile internet is a given, devices are tightly integrated, with precise positioning available, along with sophisticated mapping information, routing, and so forth. RFIDs (radio frequency IDs) abound, allowing RFID readers in smartphones to detect what is around them at any time. Business have moved much more of their catalogs online, encouraging richer consumer-level e-commerce.

But your smartphone is no longer the monolithic device we're used to. It's functionality has now spread much farther. Your watch alerts you, shows simple information (such as what this alert is about), and allows simple interaction (for example, accept, reject, or delay). Your headset now sits inside your ear, offering noise cancellation or transparency, depending on context, as well as voice recognition. Your glasses offer a large, high resolution heads-up display. A lightly textured piece of cloth on the side of your leg is pressure sensitive, allowing discrete, chorded input (pressing different combinations of fingers together to indicate a single letter). All of these are networked via a low-power personal area network (the descendant of Bluetooth) to the phone which you hardly ever take out of your pocket.

Your phone itself roams from network to network, using whatever resources are available to it at the time, depending on your preconfiguration. It has Terabytes of storage, a powerful CPU, and the various high-speed wireless data connections. But you rarely take it out of your pocket, except to take a high-resolution photo or video.

Imagine grocery shopping

It's grocery shopping time. Can't remember what to buy? A few quick commands brings up your phone's memory of the RFIDs it detected in the fridge that morning (you have an old fridge which isn't on the internet), your Food Management software compares that to your observed weekly requirements (the average of your requirements each week), and makes a tentative list.

As you walk through the supermarket shelves, the heads-up display brings up alerts when you approach items you want (as the phone detects their RFIDs), even showing you what they look like, and emphasizing if they're on sale (as specified in the stores online catalog).

While shopping, your list suddenly gets some urgent updates. Your spouse, Kris, has just added some triggered shopping requests for you, and since you're already in the shop, the trigger has added the items to your list. At the same time your watch buzzes subtly. A quick glance shows that those shopping items have a purpose: friends are coming to dinner in two hours, the recipe will take an hour to prepare (according to the online database it was snarfed from), and you're half an hour from home. Not much time to waste. You accept the alert.

Imagine travelling

As you leave the shop, pushing the trolley through the RFID reader, and typing your PIN in acceptance of the charge to your account (offered by your phone), your schedule suddenly changes, though you're not yet aware of it. Your friend's flight has been delayed and their phones have informed yours.

You become aware of the extra time you have when you walk past the newsagent, and a triggered event reminds you that you want to check out the range of birthday cards. Surprised, you actually look at your schedule, and note that the dinner has been delayed by half an hour. Still, you're not in the mood to look at cards, so you reject the suggestion and continue home.

On the way home you listen to the latest e-book on your phone, read out over the car's sound system. Close to home you need to make a detour to avoid an accident. The e-book is interrupted with a gentle request: "Jenny's going to be ready to go home from school in four minutes, the detour could go by the school, would you like to take her home with you?"

You query the system, "Where is Kris?" And it responds that your spouse is still at home.

You accept the opportunity to share journeys with your daughter, and stop at the school, listening to the e-book for the couple of minutes until Jenny gets into the car. Then on the way home you talk about your days. Waiting at a set of lights, Jenney unfolds her phone's display to show you a diagram she drew at school -- it's clarity of layout impresses you, although Jenny points out that it works even better on a display bigger than the unfolded A4 display of her phone.

At home

When you arrive home, your spouse asks you to prepare the dinner. You find the activity for cooking already tentatively in your schedule, with a link to the recipe. The cooking schedule is linked to the ETA for your friends, and needs to start soon (they travelled a little earlier than expected). So, you put away the food you don't need, and start cooking, following the instructions and pictures on your glass's heads-up display (even for things you don't need glasses for, the heads up display is so useful that you tend to use what used to be called "cosmetic spectacles", ie. glasses that have no corrective function).

While waiting for a particular stage you check out the afternoon's headlines (who wastes time sitting down to watch the news anymore?). As you approach the last stage, you see that your friends are scheduled to arrive in a few minutes, and so they do. You spend the rest of the evening happily offline, catching up. Your phone withholds all but emergency alerts (of which, thankfully, there are none).

How does it work?

This sort of scenario seems so much science-fiction to us, doesn't it? And yet there's not much there that current technology can't manage (the prevalence of RFIDs, the integration of data systems, and the more advanced interfaces to the phone are the major advances).

The point is that this seamless integration, of phone software and services with online services, and of phone hardware and systems with other interface systems, is the way that smartphones are heading.

For example, doing voice recognition in a headset has significant advantages in keeping personal area (or near field) communications to a minimum. Having glasses with heads-up displays capable of running substantial chunks of software has the same benefit, in addition to ensuring smooth animations without high network bandwidths or embarrassing dropouts.

Achieving this type of distributed, highly integrated software requires a system that is both a light user of resources (glasses have very little space or weight for CPUs and batteries, and native software, rather than resource-consuming software layers will be required) and able to be distributed (i.e. a microkernel based system with strong Inter Process Communications). That's Symbian (but not Windows or Linux).

And note what happens in the scenario: there is little need for a PC for most everyday tasks. Rather than desktop OS's scaling down, it seems likely to me that powerful device OS's will scale up.

Is this possible? Yes. Is it likely? Well, that depends on the vision of the handset and accessory manufacturers, operators, ISVs, and even retailers. Open interfaces are critical to this vision, and are the most likely thing to never come true.

Let's all hope that there are enough people with vision to help make this a reality.

Smart Phone or Mobile Browser

[This post has been languishing for a month. I think it's time to post it's beginning.]

It seems that some people are expecting smart phones to evolve into what is effectively just a mobile browser, providing a "thin client" for web-based applications (eg. AJAX apps like Google Mail, etc.).

This sounds familiar to me -- I've been through this before, in the PC industry. So is it different this time? Let's take a closer look.

History

First, some history.

In the mid-nineties, when the Web was taking off, a number of companies saw the opportunity to make a revolutionary type of personal computer to replace the existing model of the PC (running Windows 3.1 back when this started). This type of computer was called the NC, short for Network Computer. It's greatest proponents were the Acorn/Oracle partnership, and Sun. The idea was that the NC would be a mediumly thin client (thicker than a dumb terminal, such as an X-Windows terminal, but thinner than a PC). Basically the NC would run apps natively, but they would be downloaded from the network, and data would reside on the network.

This is a very similar model to the new AJAX style web-based applications.

Of course, history tells us that this approach was a massive flop, and it killed Acorn, amongst any number of other companies. Since the winning technology could hardly be considered to have won through techical superiority (we're talking about Windows 95 and NT 3 here), it must have been the underlying architectures that had significant advantages and disadvantages.

Interestingly, I was in favour of the NC at the time, mostly because I was a dedicated Acorn user, and didn't want to see the most innovative PC company go under. This time I find myself "naturally" on the other side.

So why did the NC fail? Well, the answer's two-fold, but pretty simple:

1. Implementing sophisticated applications in Java (the language of choice for NC's) was probably as easy as doing so for Windows (back then), but the results were vastly inferior. NC's back then were running 30MHz ARM CPUs or the like, and Java was just too heavy for this (or PC) hardware. PC applications were simply more responsive than Java app's.
2. The network simply wasn't up to handling all these apps and associated data. I remember working in Novell KK (Japan) in the latter half of '94 when Mosaic and the Web were gathering steam, and we still only had a 64kbps ISDN line shared between the 200 or so staff there!

The burning question is whether things have changed enough so that these major issues are no longer severe enough to simply kill the approach. (And, for mobile, I think the answer is pretty clearly, "no", see Michael Mace's post here for some insights.)

Japan

So, what about Japan? Clearly their phones are mobile browsers, and not smart phones. Perhaps they provide an example of the future.

Yes, well Japan has always marched to the beat of a different drum. In the nineties, when the NC was being hyped, the PC was still rare amongst home users in Japan. Much more common were turnkey wordprocessors (a type of technology almost completely dead in the West at that time).

To really understand Japan, you would have to understand what they're actually doing with their mobile browsers. I have to confess that I don't know what, exactly, they're doing. But I'm fairly sure it's merely consuming content and creating trivial content, rather than generating sophisticated content, or doing sophisticated processing that on-device app's are so good at.

In addition, you need to be aware of the geographical character of Japan and other target markets. Japan is heavily urbanised, with around a quarter of the population in greater Tokyo. The rest of the population are scattered through densely settled valleys and plains, with almost unpopulated mountains between them. This makes it very easy to build a mobile network that covers almost everyone. (Especially given the population and relative size of the country.)

Compare Japan to Australia, which, despite being heavily urbanised, has the population density such that travelling a few hundred kilometres from the population centres will dump you into a sparsely populated (but still populated) vast land, almost impossible to network with mobile networks.

So Japan can (as usual) be viewed as a special case, and not an indicator of things to come.

Next post I'll look further at issues with browser-based applications on mobile devices, including:

• Responsiveness (a common issue, partly addressed by technologies like AJAX)
• Reliability (a problem for mobile devices that no longer really exists for desktops)
• Privacy (a problem that no-one seems to be taking seriously, on any platform)