CIO Dashboard » Handwriting Recognition is iPad’s Next Hurdle

Handwriting Recognition is iPad’s Next Hurdle

post by Chris Curran on June 3, 2010

A recent survey of 500 companies by Citrix says that 80% of companies plan to buy iPads. The big question is why? The iPad is a no-brainer for a personal laptop replacement, super media device, advanced remote control, and all-around great home computing device. But for work, the rationale is less clear.

At the latest All Things Digital, when asked is the iPad would replace laptops or PCs, Steve Jobs said:

PCs are gonna be like trucks; they’re still gonna be around and still gonna add value, but they are going to be used by 1 out of X people.

Many uses we haven’t thought of yet will make the iPad and possibly other tablets valuable to the enterprise, especially in places where we interact with customers. The size, accelerometer, graphics and multi-touch interface make the iPad a perfect device for situations that would benefit from simultaneous interaction with 2 or more people.

Aside from to-be-determined custom apps, I just don’t see the iPad as a general purpose business computer - yet. While much has been written about the bad adaptation of office document apps such as Keynote for iPad, these software and usability issues will be fixed soon. What remains, is the issue of getting text for emails, documents, blog posts, spreadsheet data, etc. into the iPad.

iPad Data Entry Options

You have 4 choices:

Use the virtual keyboard
Use an external keyboard
Use a stylus and note taking app geared for it
Voice entry (thanks Nalneesh)

The virtual keyboard is cramped and requires the iPad to be on a flat or slightly inclined surface, which defeats the purpose of the tablet form factor. Some say “stop complaining about data entry, just get a keyboard” (about $900 w 3G and external keyboard). I say just get an Apple Air ($1499) for a full fledged computer and only a little more than a pound of weight. The stylus method uses the iPad as a tablet, but the output is locked in images. The final option is voice recognition using something like Dragon Dictate, which requires a mic or quiet room, an internet connection and varies wildly by person. So, to fully realize the potential of this great device in the enterprise, school or doctors office, it must support handwriting recognition.

The single biggest barrier in making the iPad a real general purpose business computer is the lack of handwriting recognition.

Apple has over 20 handwriting recognition related patents dating back to 1997 and the Newton experiences, so why did they leave it out? Maybe because they felt it sent a “this is a complicated computer” message to the marketplace. Or, maybe its just part of a calculated roll-out strategy, during which they would also add multitasking, a forward facing camera for videoconferencing, a better way to manage files and -gasp- a stylus.

Taking Notes with the iPad

So, to prepare myself for the day handwriting recognition comes (fingers crossed), I’ve spent the last month using my iPad as my primary note taking device (option #3 above). I chose to do this with a Pogo stylus and Note Taker HD (written by Diamond advisor and spreadsheet inventor Dan Bricklin). While NoteTaker does not add handwriting recognition, it does everything else you need to write, tag, and share (in PDF or image) your handwritten notes and drawings. The major innovation NoteTaker adds is in shrinking down your handwriting to get around the thick penstrokes from your finger and even a stylus. This leaves you with a great looking page with small but legible handwriting. In his next release, Dan is adding support for multiple pages, colored pens and VGA output.

Bottom line is that until handwriting recognition is available, the iPad is an excellent option for innovative application development, but not yet a great general business computing device. But get one for the house - it’s the next transistor radio!

cc licensed flickr photo shared by cote

Nalneesh Gaur

If dictation is an option then Dragon (http://itunes.apple.com/us/app/dragon-dictation/id341446764?mt=8) should be another one worth considering. Of course its not ideal for all situations but something that should minimize keyboard usage.

For notes taking, porting decent OCR software to iPad should make it possible to digitize notes taking. Hopefully, the software will be really smart or maybe Doctors will improve their handwriting 🙂
Nalneesh Gaur

If dictation is an option then Dragon (http://itunes.apple.com/us/app/dragon-dictation/id341446764?mt=8) should be another one worth considering. Of course its not ideal for all situations but something that should minimize keyboard usage.

For notes taking, porting decent OCR software to iPad should make it possible to digitize notes taking. Hopefully, the software will be really smart or maybe Doctors will improve their handwriting 🙂
nick brigman

Thanks to Ed Anderson, I am a frequent reader of your blog. Two comments. a) corporate impact of iPad - a community within CompuCom is testing the potential value and impact in a variety of roles and business uses. We (the community) are surprised by the many applications with which we are having success. Many of us feel that we are on the forefront of an interesting change. From Skype to dictation, from citrix to collaboration meetings, from SharePoint to mind maps (iThought HD), there is little we cannot accomplish. b) text recognition - last year a few of us played with digital pens (devices that record the writing and record audio in stereo). It might make an interesting combination.
Please keep us all thinking!

Sent from iPad
nick brigman

Thanks to Ed Anderson, I am a frequent reader of your blog. Two comments. a) corporate impact of iPad - a community within CompuCom is testing the potential value and impact in a variety of roles and business uses. We (the community) are surprised by the many applications with which we are having success. Many of us feel that we are on the forefront of an interesting change. From Skype to dictation, from citrix to collaboration meetings, from SharePoint to mind maps (iThought HD), there is little we cannot accomplish. b) text recognition - last year a few of us played with digital pens (devices that record the writing and record audio in stereo). It might make an interesting combination.
Please keep us all thinking!

Sent from iPad
Josh

I’m not convinced that handwriting recognition for the iPad and the swag of tablets on their way, is a barrier to entry for use as a business tool.

At the start of the year i used a pad and pen to take notes, then I bought an electronic pad and pen to save notes to pc and OCR. Then I stopped using this and went back to pad and pen (OCR inaccurate and wasn’t used to saving notes as separate word documents). Today i use an iPad with a Pendel (pendel.com.au) stylus to take notes, which I can sync and save to my dropbox. I can organize and print my notes and refer to them if need be. I don’t OCR as I still don’t understand how to organize and read them, like a notebook. Besides I can still share my notes.

In essence, OCR is a couple of steps too far ahead. Let’s first get our heads around saving and storing handwritten notes digitally, far more advanced than pad and pen. Then when we are comfortable with this notion, take the next step to OCR.
Josh

I’m not convinced that handwriting recognition for the iPad and the swag of tablets on their way, is a barrier to entry for use as a business tool.

At the start of the year i used a pad and pen to take notes, then I bought an electronic pad and pen to save notes to pc and OCR. Then I stopped using this and went back to pad and pen (OCR inaccurate and wasn’t used to saving notes as separate word documents). Today i use an iPad with a Pendel (pendel.com.au) stylus to take notes, which I can sync and save to my dropbox. I can organize and print my notes and refer to them if need be. I don’t OCR as I still don’t understand how to organize and read them, like a notebook. Besides I can still share my notes.

In essence, OCR is a couple of steps too far ahead. Let’s first get our heads around saving and storing handwritten notes digitally, far more advanced than pad and pen. Then when we are comfortable with this notion, take the next step to OCR.
Florent Buisson

Actually, “really smart softwares” does exists!

But unless Apple decide to open their API and allow new input methods in their products (iPad & iPhone) as for AndroÃ¯d tablets, you will have to wait for Apple to recode such a software.

Here is some example of what is possible today:
http://www.visionobjects.com/handwriting_recognition/downloadstylus3.htm
Florent Buisson

Actually, “really smart softwares” does exists!

But unless Apple decide to open their API and allow new input methods in their products (iPad & iPhone) as for AndroÃ¯d tablets, you will have to wait for Apple to recode such a software.

Here is some example of what is possible today:
http://www.visionobjects.com/handwriting_recognition/downloadstylus3.htm
Peter Kretzman

Just a short comment here, speaking solely for myself, and recognizing that others may have a different practice: I grew away from taking handwritten notes years ago. In fact, writing more than a sentence or two by hand now strikes me as woefully inefficient, even painful, given the speed and accuracy I can muster on a keyboard. The obvious exception is diagrams, but capturing these from a white board with a camera phone and incorporating them into electronic notes seems better anyway.

I do recognize that the idea of having meetings with everyone typing away on their laptops raises a whole host of other issues that have to be dealt with (IMs, email, even (!) solitaire).
Peter Kretzman

Just a short comment here, speaking solely for myself, and recognizing that others may have a different practice: I grew away from taking handwritten notes years ago. In fact, writing more than a sentence or two by hand now strikes me as woefully inefficient, even painful, given the speed and accuracy I can muster on a keyboard. The obvious exception is diagrams, but capturing these from a white board with a camera phone and incorporating them into electronic notes seems better anyway.

I do recognize that the idea of having meetings with everyone typing away on their laptops raises a whole host of other issues that have to be dealt with (IMs, email, even (!) solitaire).
Julie

There is a hand writing recognition app for the ipad already. Its called WritePad and it’s available in the app store for $9.99. It works well with the popo stylus and your finger as well. I love it. What I wish it had was the ability for the on screen keyboard to pop up, so you could switch back and forth between the inputs. And if you could write and not have it change to text, as some notes (such as math) are just made easier when hand written.
Julie

There is a hand writing recognition app for the ipad already. Its called WritePad and it’s available in the app store for $9.99. It works well with the popo stylus and your finger as well. I love it. What I wish it had was the ability for the on screen keyboard to pop up, so you could switch back and forth between the inputs. And if you could write and not have it change to text, as some notes (such as math) are just made easier when hand written.
Chris Curran

From my friend Dan Bricklin:

My feeling is that recognition is best used for indexing and searching. You use handwriting when you want to express yourself that way, and there is more to that expression than just the letters themselves. Keyboards can be much faster than writing for most people, even on touch screens I’ve seen. There are times when you just can’t use a keyboard (such as with a touch screen and not looking at the screen) and you’d mainly want to what you write to be turned directly into text, but for note taking I think the layout and other marks many people make are an important part of what they write. This is based on what I remember from the days when I worked along with the people at DayTimer Corporation.

-Dan
Chris Curran

From my friend Dan Bricklin:

My feeling is that recognition is best used for indexing and searching. You use handwriting when you want to express yourself that way, and there is more to that expression than just the letters themselves. Keyboards can be much faster than writing for most people, even on touch screens I’ve seen. There are times when you just can’t use a keyboard (such as with a touch screen and not looking at the screen) and you’d mainly want to what you write to be turned directly into text, but for note taking I think the layout and other marks many people make are an important part of what they write. This is based on what I remember from the days when I worked along with the people at DayTimer Corporation.

-Dan
Tom

I am an old Newton and PC user so I’m not just an Apple fanatic writing here. Even though Apple holds the patents to Rosetta (the software behind the Newtons handwriting recognition engine) I believe that part of the hold up with Steve Jobs and handwriting recognition is that he doesn’t want to create another Newton because he didn’t like it in the first place. One of the first things he did when he retook the helm was to kill the Newton. I am, however, hoping that with Apple purchasing FingerWorks and adding their developers to their staff that will breathe some new technology and possibility for inking into many of their platforms.

On the usability front, I used my Newton for all my note taking, when I was in school, when I was in meetings. Did it reach 100% recognition of my handwriting, no, usually 85%-90% but one of the advantages of the Newton over many inking apps that I have seen so far is it allowed me to save my ink and convert it all, or in parts, later so I could verify it’s correction. This was both fast and allowed for “human” correction. Unlike what other people said previously, I didn’t have to look at my screen to know where I was writing any more than I have to look at my keyboard to know where I am typing. Keyboards have their places, don’t get me wrong, but so does writing and I believe by offering handwriting recognition to the iPad (more than the iPod/iPhone) it could open up many avenues in both business and personal lives, and especially education. In fact, other than the larger form factor, handwriting recognition is the main reason I haven’t purchased an iPad.
Tom

I am an old Newton and PC user so I’m not just an Apple fanatic writing here. Even though Apple holds the patents to Rosetta (the software behind the Newtons handwriting recognition engine) I believe that part of the hold up with Steve Jobs and handwriting recognition is that he doesn’t want to create another Newton because he didn’t like it in the first place. One of the first things he did when he retook the helm was to kill the Newton. I am, however, hoping that with Apple purchasing FingerWorks and adding their developers to their staff that will breathe some new technology and possibility for inking into many of their platforms.

On the usability front, I used my Newton for all my note taking, when I was in school, when I was in meetings. Did it reach 100% recognition of my handwriting, no, usually 85%-90% but one of the advantages of the Newton over many inking apps that I have seen so far is it allowed me to save my ink and convert it all, or in parts, later so I could verify it’s correction. This was both fast and allowed for “human” correction. Unlike what other people said previously, I didn’t have to look at my screen to know where I was writing any more than I have to look at my keyboard to know where I am typing. Keyboards have their places, don’t get me wrong, but so does writing and I believe by offering handwriting recognition to the iPad (more than the iPod/iPhone) it could open up many avenues in both business and personal lives, and especially education. In fact, other than the larger form factor, handwriting recognition is the main reason I haven’t purchased an iPad.
PB

Hi Chris,

Iâ€™ve been looking for a stylus for the iPad and was wondering if the solution you mentioned and/or the Pogo stylus would allow me to replace paper and pen? Iâ€™m looking to be able to take quick notes and drawings like one would on paper.

Do the stylusâ€™s allow one to rest their hands on the screen and write finely i.e. I donâ€™t want the page filling up fast because the tip/text is very thick.

Iâ€™d love to hear your feedback.

Thanks.
PB
PB

Hi Chris,

Iâ€™ve been looking for a stylus for the iPad and was wondering if the solution you mentioned and/or the Pogo stylus would allow me to replace paper and pen? Iâ€™m looking to be able to take quick notes and drawings like one would on paper.

Do the stylusâ€™s allow one to rest their hands on the screen and write finely i.e. I donâ€™t want the page filling up fast because the tip/text is very thick.

Iâ€™d love to hear your feedback.

Thanks.
PB
Store

There is a new stylus that is much better than the pogo.

Go to http://www.apppen.com/storedetails.php?SId=39&page=2, This is the cheapest price you will find on it
Pingback: iPad Uses for Airlines In-Flight â€” CIO Dashboard()
Pingback: 7 Great iPad Apps for CIO â€” CIO Dashboard()
Stepheng

I agree with the article, the iPad is a killer way to consume information but lousy for creating it. Apple needs two things:

1 standrd input via handwriting recognition and a stylus. Does anyone remember the Palm’s letter system? Took a little time to learnbut the accuracy was always way better than any other writing system and that is what we need to create content.

2 have a touch typing mode for the onscreen keyboard. Allow me to rest all my fngers on the keyboard and then figure out that I picked one up and hit another letter. A bit of fuzzy logic and knowing Mavis Beacon Teaches Typing and it would work.

Give me these two things and I can now take my iPad into a requirements meeting, and take actual notes in realtime and then share them (spell checked and formatted) with others. Using blutooth several of us could collaborate on the same document and also have access to our personal notes and other information
Chris Curran

Thanks for your thoughts Magnus. I too have continued to experiment with typewritten, handwritten and recognized inputs with the iPad over the last year, using no fewer than 20 apps in this space. I have found that I’m most productive with handwritten notes and drawings w/o recognition. What I would like, however, is the note taker to do some recognition behind the scenes to build a keyword list. I know that Evernote is supposed to be able to do this but I have not been able to get it to work for my handwriting yet :).

-Chris