The Potential of Smartphones

So often in the mobile phone business, people have approached these devices as merely mobile versions of immobile technology. Thus the "mobile web", "mobile mail", "mobile phone", and so on. But what if we approached smartphones from the perspective of what they are, what they can do, and what we could do with that?

An example of a technology that is completely "home grown" to the mobile community is SMS. This was not a planned service in the same way as MMS or WAP -- the operators and handset manufacturers did not carefully design and market SMS. It was simply a capability available on the phones and network that suited people's needs, and so it took off. (Note that SMS was not "mobile instant messaging", it was simply a messaging facility for operator use which turned out to work wonderfully peer-to-peer. It's mode of operation is, in fact, quite different from IM, and it is only recently that efforts have been made to fit it into an IM type of UI, such as on the iPhone.)

So what could we come up with in the future, and how do we go about it? Or do we have to rely on accidents?

I think the first thing we need to understand with this approach is exactly what a phone is, and how it fits into people's lives. So let's tackle that one first.

What is a smartphone?
If you are intimately familiar with smartphones, you can skip this section. Still, it's fascinating to take stock of just how much functionality modern smartphones pack into them, and to think about the uses of that technology independent of the actual features.

A smartphone:

• Is a small computer with substantial CPU and memory resources
• Carried almost everywhere

Smartphones have:

• Relatively small screen (sometimes touch sensitive)
• Small keypad or keyboard
• Built-in phone, for telecommunications with other people
• Microphone and the ability to record from it
• Speaker and the ability to play music and sounds
• Usually a camera (or two) with the ability to capture stills and video
• Internet connection which is usually, but not always, available
• Bluetooth connection which can detect and communicate with neighbouring devices
• Infrared connection which can communicate with neighbouring devices
• Positioning information, available via cell ID or built-in GPS
• Databases for contacts and calendar information

Some smartphones have:

• Light sensor
• Motion detectors (eg. iPhone, Nokia 5500)
• Near field RFID units (eg. mobile Suica)

What can we do with this?

So, the question is then, what can we do with the (fairly impressive) bundle of functionality that millions of users carry around with them every day?

Well, some ideas are pretty straight-forward:

• Use it as a PDA (to keep contacts, calendar, and notes)
• Use it as a mobile web browser (slowly starting to become viable as screens get bigger and, more importantly, CPUs get fast enough to present the web in readable ways on a small screen)
• Use it as a mobile email terminal (RIM has been particularly successful in this area, although something like an E61 or P990i/M600i on an operator-provided IMAP push service is just as good, and much cheaper)
• Use it as a constantly-updated weather chart
• Use it as a navigation unit, with maps, current location (via GPS), dynamic routing, and even dynamically updated traffic status
• Use it as an e-book reader

What's obvious about these ideas are that they have all been transferred from devices that already exist. PDAs, web browsers (on desktops and laptops), email, online weather, GPS navigation, and e-book readers are all technologies that have been around for quite some time. Putting them on a smartphone certainly makes them more accessible, and thus more useful, but doesn't really transform the way they integrate with people's lives.

Are there other ideas that can be built from the smartphone's capabilities itself? Yes, of course there are, and here are some we've seen:

• Lifeblog: using the camera, location and time information, and recording snippets of your life along with some comments on it, creating a multimedia "life blog". http://www.nokia.com/lifeblog
• Sensor: using bluetooth and a personalised profile to discover and meet people in your
  immediate vicinity with matching interests. http://www.nokia.com/sensor
  

The problem with these ideas, and why they haven't taken the market by storm, is that there really just curiosities. They don't meet a real need. How many people complain that they don't have a sufficiently rich record of their lives? Not many.

New ideas

Are there other ideas that take advantage of the capabilities of a smart phone and meet a real need? I think there are many, and I'll talk here about one, which I would love to see implemented.

Imagine a service on your smartphone which took advantage of its computing ability, its knowledge of your location, and its connection to the internet. Imagine that you could specify a destination and a desired arrival time, and this service could go off and discover all the ways you could get to that destination at that time, then present you with the options, and then book your chosen options, and finally remind you about when you needed to get moving to get there, and guide you through the process.

For example, I might want to fly from my home on the Gold Coast, Australia, to a hotel in Hong Kong. The software would start by attempting to find a route from start to end. Once it knew this, it would attempt to find services on this route, starting with the most irregular and expensive (ie. the flight from Australia to Hong Kong), and then work down to the simplest (eg. getting to and from the airports). Then it would present me with a range of "best-case" options (no point confusing me with lots of almost-identical options), and I would choose what I wanted for the various legs. Remembered preferences (for example, I prefer the train over driving) would make the choices easier by prioritising them so my most likely choices are the first ones I see. Finally, it would take my choices, book them for me, and keep the e-ticketing information.

Then, when the time came to make the trip, the software would remind me when to start (maybe by putting appointments in my calendar), would present the e-ticket information when I needed it, and would guide me through the confusion of interchanges, and so on.

All of this information is available on-line today. All of the technology required to do this is available. Much of the infrastructure, such as mapping and routing technology is freely available for this type of "mash-up".

What's missing is a good UI running on the phone, which integrates well with the phone's capabilities (its small keyboard and screen, its calendar database and positioning technology), and provides fluid, friendly interaction.

There are obvious add-ons to this service, such as the ability to find and book accommodation given your parameters and choices. An even more powerful addition would be the ability to carpool with others who are heading in the same direction. Sharing aggregate data with transport providers could even allow them to improve the quality and efficiency of their services.

Mass market?

The question is, are these types of services useful for many people? Do they have mass market appeal?

If the purpose is to plan large-scale trips like Austalia to Hong Kong (or even interstate within large countries), the answer would have to be no. However, if the technology can handle small-scale trips like meeting someone in an unknown pub at a certain time, then this is far more useful to the general user.

The tipping point is based on usability and price. It has to be easier to use the phone to discover, book, and schedule your trip than it is to do it yourself. If you are looking at a regular trip (such as a commute), it's unlikely that you'd use such technology, unless it gave you benefits such as carpooling or a discount ticket (from the transport provider to encourage use of such services so that they could better implement their services). But an irregular, but still planned trip using public transport -- such as a weekend outing, or a meeting with the mates or for work -- presents planning and information gathering demands that a smartphone could easily perform.

Personally, the idea of being able to quickly and easily find my way to a meeting, without wasting time at connections or stressing about figuring out the best way there, sounds like a dream come true.

The key to these ideas

Perhaps the key to this approach is to understand the smartphone as an "invisible" tool. A tool that is simply the conduit for desires and information. The idea I've mapped out can include peer-to-peer functionality (with carpooling), which many believe to be a key to success, as well as information and service delivery.

This is the beauty of smartphones: they can span so many "worlds" that they can do all sorts of exciting things. Let's not just create "mobile" versions of existing, desk-bound services, let's try to create truly unique services with the capabilities available right now!

A Future for Symbian Smartphones

Come dream with me for a while. Dreaming about technology, especially when those dreams are firmly rooted in currently reality, is a truly helpful way to determine what trends and features are important to the big picture.

Imagine it's ten years in the future.

The mobile internet is a given, devices are tightly integrated, with precise positioning available, along with sophisticated mapping information, routing, and so forth. RFIDs (radio frequency IDs) abound, allowing RFID readers in smartphones to detect what is around them at any time. Business have moved much more of their catalogs online, encouraging richer consumer-level e-commerce.

But your smartphone is no longer the monolithic device we're used to. It's functionality has now spread much farther. Your watch alerts you, shows simple information (such as what this alert is about), and allows simple interaction (for example, accept, reject, or delay). Your headset now sits inside your ear, offering noise cancellation or transparency, depending on context, as well as voice recognition. Your glasses offer a large, high resolution heads-up display. A lightly textured piece of cloth on the side of your leg is pressure sensitive, allowing discrete, chorded input (pressing different combinations of fingers together to indicate a single letter). All of these are networked via a low-power personal area network (the descendant of Bluetooth) to the phone which you hardly ever take out of your pocket.

Your phone itself roams from network to network, using whatever resources are available to it at the time, depending on your preconfiguration. It has Terabytes of storage, a powerful CPU, and the various high-speed wireless data connections. But you rarely take it out of your pocket, except to take a high-resolution photo or video.

Imagine grocery shopping

It's grocery shopping time. Can't remember what to buy? A few quick commands brings up your phone's memory of the RFIDs it detected in the fridge that morning (you have an old fridge which isn't on the internet), your Food Management software compares that to your observed weekly requirements (the average of your requirements each week), and makes a tentative list.

As you walk through the supermarket shelves, the heads-up display brings up alerts when you approach items you want (as the phone detects their RFIDs), even showing you what they look like, and emphasizing if they're on sale (as specified in the stores online catalog).

While shopping, your list suddenly gets some urgent updates. Your spouse, Kris, has just added some triggered shopping requests for you, and since you're already in the shop, the trigger has added the items to your list. At the same time your watch buzzes subtly. A quick glance shows that those shopping items have a purpose: friends are coming to dinner in two hours, the recipe will take an hour to prepare (according to the online database it was snarfed from), and you're half an hour from home. Not much time to waste. You accept the alert.

Imagine travelling

As you leave the shop, pushing the trolley through the RFID reader, and typing your PIN in acceptance of the charge to your account (offered by your phone), your schedule suddenly changes, though you're not yet aware of it. Your friend's flight has been delayed and their phones have informed yours.

You become aware of the extra time you have when you walk past the newsagent, and a triggered event reminds you that you want to check out the range of birthday cards. Surprised, you actually look at your schedule, and note that the dinner has been delayed by half an hour. Still, you're not in the mood to look at cards, so you reject the suggestion and continue home.

On the way home you listen to the latest e-book on your phone, read out over the car's sound system. Close to home you need to make a detour to avoid an accident. The e-book is interrupted with a gentle request: "Jenny's going to be ready to go home from school in four minutes, the detour could go by the school, would you like to take her home with you?"

You query the system, "Where is Kris?" And it responds that your spouse is still at home.

You accept the opportunity to share journeys with your daughter, and stop at the school, listening to the e-book for the couple of minutes until Jenny gets into the car. Then on the way home you talk about your days. Waiting at a set of lights, Jenney unfolds her phone's display to show you a diagram she drew at school -- it's clarity of layout impresses you, although Jenny points out that it works even better on a display bigger than the unfolded A4 display of her phone.

At home

When you arrive home, your spouse asks you to prepare the dinner. You find the activity for cooking already tentatively in your schedule, with a link to the recipe. The cooking schedule is linked to the ETA for your friends, and needs to start soon (they travelled a little earlier than expected). So, you put away the food you don't need, and start cooking, following the instructions and pictures on your glass's heads-up display (even for things you don't need glasses for, the heads up display is so useful that you tend to use what used to be called "cosmetic spectacles", ie. glasses that have no corrective function).

While waiting for a particular stage you check out the afternoon's headlines (who wastes time sitting down to watch the news anymore?). As you approach the last stage, you see that your friends are scheduled to arrive in a few minutes, and so they do. You spend the rest of the evening happily offline, catching up. Your phone withholds all but emergency alerts (of which, thankfully, there are none).

How does it work?

This sort of scenario seems so much science-fiction to us, doesn't it? And yet there's not much there that current technology can't manage (the prevalence of RFIDs, the integration of data systems, and the more advanced interfaces to the phone are the major advances).

The point is that this seamless integration, of phone software and services with online services, and of phone hardware and systems with other interface systems, is the way that smartphones are heading.

For example, doing voice recognition in a headset has significant advantages in keeping personal area (or near field) communications to a minimum. Having glasses with heads-up displays capable of running substantial chunks of software has the same benefit, in addition to ensuring smooth animations without high network bandwidths or embarrassing dropouts.

Achieving this type of distributed, highly integrated software requires a system that is both a light user of resources (glasses have very little space or weight for CPUs and batteries, and native software, rather than resource-consuming software layers will be required) and able to be distributed (i.e. a microkernel based system with strong Inter Process Communications). That's Symbian (but not Windows or Linux).

And note what happens in the scenario: there is little need for a PC for most everyday tasks. Rather than desktop OS's scaling down, it seems likely to me that powerful device OS's will scale up.

Is this possible? Yes. Is it likely? Well, that depends on the vision of the handset and accessory manufacturers, operators, ISVs, and even retailers. Open interfaces are critical to this vision, and are the most likely thing to never come true.

Let's all hope that there are enough people with vision to help make this a reality.

State of Play for Symbian ISVs

This article is a quick summary of the state of play for Independent Software Vendors creating software and services for Symbian OS devices. Each section contains a short history, current situation, and likely future directions (including suggestions).

Issues tackled include why ISVs should be supported at all, the range of devices available to Symbian ISVs, the possibilities for the ecosystem in ISV-provided services, the channels to market for third party software, the range of technical documentation available to support development, the process of Symbian Signing, the state of APIs available to ISVs, and the development tools available for software creation.

While not all of these may be of concern to you, certainly one or two will be of interest to anyone involved in the Symbian ecosystem.

1. Development tools

Development tools are critical to ISVs -- they can make the difference between being able to make an application profitably and being unable to do anything at all.

In the Symbian world, we've had three "preferred" development tools over the last seven or so years:

Visual C++ -> Metrowerks -> Carbide

Carbide has finally become usable in v1.2, and is a quite reasonable IDE. However, it still has some serious limitations that are very obstructive in terms of developing Symbian apps.

1. On device debugging doesn't work properly (very clunky with static DLLs, which Symbian encourage developers to use, and doesn't work at all for memory shortage debugging)


2. RAD development for UIQ 3, there are simply no tools, and given the complexity of UIQ 3's very powerful resource files, this would be a great help.

2. Application Program Interfaces (APIs)

Until recently Symbian support three separate APIs, S60, S80, and UIQ. In the last year or so that has narrowed down to two: S60 3rd Edition and UIQ 3. This is obviously an improvement.

However, getting here has been very painful. Constant changes to S60 (including breaking compatibility in many areas) in addition to radical change between UIQ 2 and UIQ 3, has necessitated serious porting effort for many vendors. This effort would have been better spent improving the products.

At least with UIQ 3, the effort spent on porting now works on two classes of phone (touch screen and soft key).

In the future, this should be able to be better managed, and it seems as if this will be the case. Nokia has a "platform evolution" page which shows that future editions of S60 have a "Compatibility promise". So long as Nokia stick to this, ISVs will be much more productive. As far as UIQ is concerned, they seem to be inherently more careful about compatibility than Nokia, and took the opportunity of the forced platform incompatibility caused by PlatSec to totally revamp their APIs, ready for the future.

3. Signing

The whole Platform Security (PlatSec) issue, which requires application signing for certain purposes, started out as a bit of a nightmare. Signing was expensive, clumsy, and non-trivial. The simple fact of an application requiring resigning on even a change to text resources prevented many developers from signing applications due to the way it "locked in" the release version, preventing incremental fixes and improvements. Furthermore, the expense of signing versus the potential return was all out of whack.

Symbian have been slowly working on this, introducing initiatives such as freeware signing (very slow and quite heavily restricted though it is), cheaper Publisher certificates, cheaper test houses, and so on.

They have just announced a new set of proposals, which are very encouraging, especially new signing methods such as the "Express Signed" process (which allows instant signing with the submitted apps being audited later). It seems that Symbian recognise the limitations of the signing process, as well as the strengths, and are making sincere efforts to minimise the pain while maintaining the advantages. I think Symbian are doing a good job here.

However, automated test tools must be delivered (UIQ 3's automated test tool hasn't worked for well over a year, preventing developers from pre-testing their applications before submission), and more precise test plans must be available (or how can you prepare an application for testing?).

4. Documentation

Back in EPOC days, Symbian started with very poor documentation but this was partially mitigated by the availability of much of the UI source (which also helped in debugging).

With the release of S60 and UIQ SDKs, we were moved on to slightly better (but still poor) doco with no UI source -- a significant (and very frustrating) backwards step.

The current situation is certainly much better, with more complete documentation and a growing set of examples. However the SDKs are still incomplete. (For example, where is the UIQ 3 documentation describing how stand-in controls work? This is a fundamental UI concept in UIQ 3, and yet there is nothing to assist an ISV to create their own stand-ins.)

The SDKs need serious work to complete them. The S60 and UIQ developers need to remember that this is an OO framework, so deep knowledge of the framework is needed for subclassing, etc. It seems that none of the OS/UI providers fully understand this.

Microsoft is still the role model in this regard.

5. Channels to market

Symbian development started with no official channels to market. Developers simply sold on their own websites. Over the years, independent resellers (such as Handango) have popped up. Some of these have been adopted by operators or handset manufacturers as official channels (and some have lost that official status).

The situation now is that premium channels (vendor and operator) have higher requirements (eg. signing), making it more difficult to get quick, light product to market, or to provide incremental updates. Generic channels (such as Handango or Mobile2day) offer significantly less barriers, but still demand a substantial cut of the sale (often almost half) for relatively little effort on their part (they do no front-line or Level 1 support for example, merely sales support). Furthermore, agreements between these distributors are often incompatible (for example, Handango requires no references to anything but Handango in software they sell, which, combined with signing required for textual changes such as that, places a premium on dealing with Handango). Finally generic channels don't always have great results, anyway, since they are pretty much invisible to the average smartphone user.

Premium channels now starting to be promoted on phones (such as Sony Ericsson's move to put their application shop on the home screen of the P1i), but this still needs more work.

I think that, in the future, we need time-of-handset-sale channels. Many users still don't realise smartphones are extensible, and so don't try to improve their user experience even though they may want to if presented with that option at time of sale. Operator's shops could do this, but have been very poor at implementing this.

SE (with their choice program in Asia) and Nokia (with their Downloads facility) seem to be attempt to improve this situation, but these initiatives need plenty of promotion and some retail-level training (ie. by the operators) would probably help.

6. Services

In the past there has been no provision of a services platform by the operators or handset manufacturers, so any services provision (such as Mobimate's weather downloads) have been purely ad-hoc and provided by the ISV themselves.

Nokia's Ovi may be the first approach to making service provision easier. Hopefully Ovi will be open to ISV's, providing a framework for services, including access to infrastructure such as mapping, routing, etc. This would allow much richer provision of services, which benefits everyone: users, operators (service provision has to travel over their networks), and handset manufacturers (their handsets have more, richer features).

SE needs to mirror this with a UIQ services facility.

Important service infrastructure that would be very beneficial to provide in such offerings would include: better sync platforms, including open support for sync plugins (at both ends); generic databases for data storage; web-scraping engines for collating information; location, mapping, and routing tools; and so on.

7. Devices

In the area of device availability and capability, the Symbian licensees (at least Nokia and SE) haven't made too many mistakes. Certainly Nokia's proliferation of devices with minor compatibility issues has (and still is, judging from the new N-Gage issues) caused some problems, and SE's struggle to produce new devices has caused it's own problems. But compared to the other platforms, Symbian has been well served.

Some real improvements that I would like to see in the near future would include features like video out that scaled to a larger screen and other features to help the phones double as PCs. Symbian OS is a powerful, real OS (even theoretically capable of being distributed across multiple devices thanks to it's microkernel and IPC model), so there's no reason to limit it.

Of course, better development tools would be helpful here, too, as well as remote testing capability such as already provided by Nokia.

8. Why should ISV's be supported at all?

Now, given how demanding ISV's seem, given the article so far, handset manufacturers may just be asking themselves, "What is the point of all this? Why bother with these demanding ISV's?"

It's not hard to provide an answer. Just look at Microsoft.

How successful would Windows be without it's vast ecosystem of ISV's? Even when MS has been predatory towards particularly successful types of third party software (such as office applications and web browsers), MS have still benefited enormously from all that innovation that they simply cannot do themselves.

In fact, considering how unexciting PC's and Windows are by themselves, could you ever imagine their extraordinary level of success without ISVs? The phone manufacturers have devices that are, inherently, much more exciting than a PC, but ISV's are capable of multiplying that wow factor, transforming these cool little camera/music-player/phones into devices that co-ordinate, communicate, track, instruct, protect, inform, educate, capture, assist, remind, and guide your life.

As for the operators, well, they stand to benefit even more -- what use is a data network without useful applications to use that data?