Microsoft Windows for Pens

Contents

Windows for Pens

• Why Pens?
• Technical Highlights
• The Internal Architecture of Pen for Windows
• RC Manager
• Pen for Windows Support Resources

Why Pens?

With pen computing, you can use a pen-like stylus on a computer with a special display to print, draw, and annotate information much as you would with pen and paper. Beneath the computer’s display is a digitizer, an electronic sensing device that captures data and displays it as “ink” or interprets it as text or as a command. Pen computing supports:

• Creation of highly portable devices. With the pen, hardware developers are no longer con­strained by the keyboard, and can deliver smaller, lighter machines that can be used while stand­ing or walking.
• Development of new markets. With smaller, more portable pen PCs, new categories of applications such as notetakers and calendars become possible.
• More intuitive features for users, combining the familiar feel of pen and paper with the power of the computer. Pen computers may re­semble clipboards, notebooks, and pocket calendars. 
• Improved efficiency. The pen is a superb pointing device for accurate data selection and object manipulation. Gestures (described later in this paper) make data manipulation econo­mical because they can simultaneously com­municate position (or object selection) and initiate an action.
• Flexibility. The pen is a great drawing tool for creating detailed graphics and, with the Windows graphical user interface and Object Linking and Embedding features, can support creating com­pound documents. The pen is ideal for taking notes, sketching, and annotating documents.

Who Will Use Pens?

Consider the following scenarios:

• Nurses and doctors can track and record patient histories as they make their rounds, using clip­board-like computers and pens to look up and enter patient information over a wireless net­work from servers throughout the hospital.
• Insurance claims adjusters can assess auto­mobile damages on site, looking up relevant cost infor­mation with the tap of the pen, then printing the estimate and writing a check to the repair shop at the end of the visit.
• Sales representatives can track inventory and the effect of promotional campaigns in retail stores, using a pen computer. At the end of the day, the information is transmitted through a phone line back to headquarters.
• Financial analysts can perform “what if” analy­ses in meetings with customers or managers, using a pen to change a number or drag a line on a chart.
• Business people can maintain their calendars on computers that feel more like notepads and calendars than current computers. They jot down notes during meetings, conduct interactive presentations, annotate documents, or edit using simple handwriting.
   

Pen users will include:

• Service providers such as field sales and serv­ice representatives, field engineers, route man­agers, and insurance adjusters who need to look up infor­mation on site, or capture it in a format that is machine-readable to be used in the field or to be communicated to other users through cellu­lar com­munications or other network solutions.
• Users of  laptop and notebook PCs, who will find that the next generation of notebook PCs will most likely be “convertible,” allowing users to detach the display for pen-exclusive uses. These pen compu­ters will be used as data entry devices, and may support smaller, less expensive compact disk storage and appli­cations for data retrieval.
• Professionals who  will add pen devices to existing hardware, controlling the pace and cost of the migration to pen computing. 

What Is Windows for Pens?

The modular design of the Windows operating sys­tem allows the integration of pen extensions into the basic Windows 3.1 product. Windows for Pens is an OEM product to be pack­aged and sold with a variety of new pen computers.

Windows for Pens offers a base of pen-specific APIs to allow Windows developers to build pen-centric applications. Windows for Pens supports existing Windows applications without special modifi­cation through a special compatibility layer so users can:

• Point and tap as with the mouse.
• Issue commands using a standard set of 13 sys­tem-wide gestures.
• Issue commands using 26 user-assigned gestures.
• Print characters in any text field. Hand-printed entries in Windows applications are translated and appear as typed text.

Windows for Pens provides an open, modular approach to handwriting recognition, so you can replace or supplement the Windows for Pens recognizer with third-party recognizers.

Windows for Pens uses the Windows-specific func­tions such as scalable fonts, Advanced Power Man­age­ment, support for Flash EPROM, IC card storage, and the ability to run in ROM. Windows for Pens also includes pen-specific DLLs and de­vice drivers so that applications can run on new pen hardware platforms. Applications can run in por­trait or landscape mode and can support scaling for different sized displays.

Why Choose Windows for Pens?

Windows for Pens, a complete operating environ­ment for pen computers, offers compelling benefits for all groups involved in developing and using pen applications:

• For new computer users, Windows for Pens provides the building blocks for creating easy-to-use pen-centric applications. Unlike other environments, Windows for Pens does not im­pose a specific metaphor on end-users, leaving the choice of user interface design and func­tionality to the application developer.
• For existing computer users, Windows for Pens draws on the Microsoft Windows environ­ment, so they can use the same applications in the office or on the road. Users can use the pen with Windows applications to point and tap as they would with a mouse, issue commands us­ing gestures or hand-printed characters to be translated as text.
  Windows for Pens, because it is based on the Windows environment, supports a multitude of communication options, printing devices, database engines, and file formats.
• For Independent Software Vendors (ISVs), Windows for Pens provides: comprehensive API support for controlling ink, recognition, and gestures; extensive driver support for vari­ous pen platforms and OEM hardware; support for multiple recognition engines to facilitate access to domestic and international markets, and all the capabilities pro­vided by the Windows 3.1 environment.
  Windows for Pens offers developers maximum flexibility to design and implement applications using the Windows interface or innovative meta­phors. With Windows for Pens, software developers can use their pro­gramming experi­ence and the code base created for Microsoft Windows, and can choose from over 120 dif­ferent development tools— third generation languages, object oriented devel­opment envi­ronments, quick UI and application generators, and database front ends.
  Because of the large and growing installed base of Windows users, ISVs have access to established markets and to new market segments that require connec­tivity solutions.
• For corporate MIS, Windows for Pens provides an environment where developers can draw on a market of skilled developers and consultants, or use their in-house Windows experience with develop­ment tools and application generators that shorten the development cycle. New pen-centric applications can be integrated with exist­ing desktop solutions to create enter­prise-wide solutions, avoiding the inefficiencies asso­ciated with supporting specialized niche environ­ments, and eliminating the need to maintain dif­fer­ent operating systems, code bases, and skill sets.
• For Original Equipment Manufacturers (OEMs), Windows for Pens offers a systems solution that can be scaled across platforms and form factors. Its modularity ensures that it can run on small silicon-memory–based pen com­puters as well as larger clipboard-type ma­chines and convertible notebooks. Its modular design means that devel­opers can scale down Windows for Pens to under 2 MB — most of which can be run in ROM. Be­cause Windows for Pens can run on a variety of hardware platforms, OEMs can capitalize on their investment in hardware design and im­plementation.

Technical Highlights

Windows 3.1 includes several special pen fea­tures and capabilities. These features, de­scribed in detail later in this paper, include:

• Gestures, implemented as 13 standard system-wide gestures available across all applications plus 26 application-mappable gestures. 
• The Pen Palette is a floating toolbar that fea­tures buttons for invoking the Writing window, the Trainer, the Gesture Manager, and the On-Screen Synthetic Keyboard.
• Control Panel settings for pen and ink, so that users can specify their own custom data­base of handwriting samples.

The core components are de­scribed in “The Internal Architecture of Pen for Windows”  later in this paper. These components include:

• The RC Manager. This includes the pen, display drivers, and the recognition engine. These components handle the data that flows between the pen and the application.
• The Windows for Pens Compatibility Layer. Also referred to as the Pen Message Interpreter, this function interacts with standard Windows applications, so they can use gestures and hand­writing recognition.

Gestures

A gesture is a special handwriting symbol that is­sues a command, such as Copy, or produces a non-printing character, such as a carriage return or back­space. 

Figure D.1   Gestures

Gestures can save time, especially for editing or correcting text. With gestures, the user can simul­taneously specify an object or function and initiate an action. Windows for Pens gestures are similar in form and purpose to common hand­written editing symbols. Three types of gestures are supported by Windows for Pens:

• Thirteen system-wide gestures that are uniform across applications.
• Application-mappable gestures that allow ap­plication-specific commands to be mapped to an unused set of symbols.
• User-defined gestures, so that users can define their own symbols.

The Pen Palette

The Pen Palette is displayed whenever Windows for Pens is started, providing a pen interface (menus and dialog boxes to execute the most common pen com­mands), an expandable writing area, and an on-screen keyboard. The Pen Palette is not required for applica­tions that call the pen API directly. To sup­port the feel of pen and paper, Windows for Pens was designed so that users can write anywhere in the text area of an application rather than in a special writing window.

Figure D.2  The Pen Palette

The other elements that support the Pen Palette are:

• The Writing window and Edit box, invoked with an edit gesture that resembles a check ­mark.
  The Writing window and Edit box provide a large area for entering, editing, and reviewing hand-printed text. The Writing window appears below the Pen Palette.
• The Trainer supports sophisticated on-the-fly handwriting training. Users are not required to train the system before using it and can call the Trainer as needed, without closing work in progress.
• The Gesture Manager controls the definition of 26 circled character ges­tures based on the 26 characters in the alphabet.

Figure D.3   The Wiring window

• The On-Screen Synthetic Keyboard provides a tool for entering characters, CTRL and ALT key combinations, and keyboard shortcuts that can be used in conjunction with the Gesture Manager to replace keyboard-based menu ac­celerators with circled character-based gestures.

The Control Panel

By defining values in the Control Panel, users can control the width of the ink generated by the pen, the orientation of the menus, and the type of user (left- or right-handed). The Control Panel supports multiple users and different display orientations and also de­fines when recognition occurs—for example, when the pen is lifted or after a time-out.

Figure D.4   Control Panel items for pens

This discussion describes how Windows for Pens is for applications programming, and is intended for readers who understand some basics of Windows programming.

Adding the pen to Windows involves enhancing both input and output. For input, the pen device driver provides pen data to the system. For output, pen data is echoed on the display as “ink” while drawing takes place. This ink layer is handled through a new device driver entry point and involves no changes to the GDI interface. The presence of ink has no impact on Windows appli­cations. At this lowest level in the system, connec­tion between pen input and ink on the dis­play is immediate. This is identical to the mouse cursor location being updated at interrupt time.

Above the device-driver level, pen input is inter­preted and given meaning. The device driver-to-device driver communication in the hardware guar­antees that users see the pen as a writing tool that can freely leave ink on the display. The levels above interpret the ink as text, shapes, and gestures.

Windows applications interact with the pen through the RC Manager, which handles the pen device driver, recognizers, and dictionaries on behalf of all programs. The RC Manager sends pen input to the appropriate window and coordinates pen input processing for that window.

The diagram in Figure D.5 shows the various compo­nents of the pen enhancements to Windows. The RC Manager is in the Systems components layer, which also includes MS-DOS functionality, input handling (labeled “Windows” to designate the functionality in the Windows Kernel and USER.EXE libraries), and graphics output (handled by GDI.EXE). The “RC” in RC Manager stands for Recognition Context.

Figure D.5   The pen enhancements to Windows

RC Manager

The RC Manager is the heart of the extensions to Windows provided in Windows for Pens, providing the architectural component that orchestrates pen input handling. Code for the RC Manager is con­tained in PENWIN.DLL, so the terms “RC Manager” and “Pen DLL” are synonymous.

Whenever a windows program wants to work with the pen extensions, that program obtains an RC, fills in the fields of the RC appropriately, and hands the RC to the RC Manager. Within the RC data structure are fields that specify exactly how the pen data is to be handled. In this sense, the RC is pat­terned after the Device Context used with GDI.

In general, one RC is used with each inking “transaction” between the user and the program. Often, programs will find it convenient to allocate an RC permanently and fine-tune it as needed (just like an “own” DC).

Fields within the RC identify which recognizer should be applied to the ink and which dictionaries used to select the “best guess” string. Other fields specify what conditions will end the inking transac­tion (such as the pen going up, or the pen going down in a specific location, or a time-out for pauses). The RC also provides parameters used by the recognizer and the dictionaries themselves.

At the end of each inking transaction, the RC Manager returns the results of ink processing to the window that initiated the transaction. If the window does not fill in an RC and initiate an inking trans­action with the RC Manager, the pen will function exactly like a mouse.

The Pen and Pen Driver

Pens come both tethered and untethered, to serve different markets, but both styles of pen act the same under Windows for Pens. The pen works together with electronics in the PC to encode (x, y) locations and other information. The pen must sense the differ­ence between touching and not touching the drawing surface, typically accom­plished with a tip-switch that acts just like the left mouse button. Many pens actual­ly report pressure levels at the surface, but this addi­tional information is not required. Some pen systems also provide lo­cation even when the pen is not quite touching the surface (called proximity sensing).

Many pens will also provide a button on the barrel. An OEM might also implement a pen that records the slant of the pen or rotation of the pen barrel. This information is not required by Windows for Pens, but a standard method is pro­vided for applications to determine whether the in­formation is present and, if so, to access the infor­mation.

All pen devices must be capable of recording at least 120 events each second, and must have a reso­lution of at least 200 spots per inch. Many pens will be able to record data at much higher resolu­tion or rates. These requirements guarantee suffi­cient data for character recognition by the vector-based character recognizers used with Windows for Pens.

All pen data is reported by the Pen Device Driver to the system at 0.001" resolution. This standard reso­lution allows improvements in pen hardware. It also is the same as the HIENGLISH Mapping mode in the GDI, and is therefore convenient for programs.

The Pen SDK includes sample device drivers for WACOM and Scriptel tablets. These drivers can be easily modified to support any pen whose hard­ware uses an interrupt to return data through I/O addresses.

Display Driver

The display driver under Windows for Pens is an enhancement of the standard display driver for Windows 3.1. The extensions support echoing ink on the display as it is drawn by the user, and mov­ing the origin of the screen to support landscape and portrait orientations. Ink support should be provided in such a way that removing the ink does not generate redraw messages to Windows pro­grams. The display driver must also provide a new cursor—the Pen cursor.

Inking is supported by two function calls in the display driver, InkReady() and GetLPDevice(), and is perhaps the easiest of the functions to im­plement. Supporting portrait and landscape modes is somewhat more difficult. The Pen SDK includes a sample driver for VGA devices.

Recognizer

A recognizer is a dynamic-link library (DLL) that collects ink from the device driver and interprets that ink as characters in one or more alphabets. When inking begins, the RC Manager calls the rec­ognizer specified in the RC for the transaction. Windows programs do not directly interact with recognizers, though programs may use application-specific recognizers.

Recognizers are required to handle an inking trans­action according to the fields set in the RC for that transaction. The recognizer will end the transaction when the conditions specified in RC fields are met. The recognizer will examine the ink for characters in the alphabets and character sets named in the RC. The recognizer will only return characters from the char­acter sets named, and, within those sets, only char­ac­ters recognized at the level of confi­dence specified.

During recognition, the recognizer collects and analyzes the ink. The original ink is returned to the calling program along with the results of analysis, which is returned in a symbol graph that is a se­quence of 32-bit values. Each 32-bit symbol is either a recognized character or a meta-character used to group alternate possibilities where more than one character meets the confidence level set for the transaction in the RC. For example, the string of symbols “clea{r|n}” would be a symbol graph representing two possible strings (“clear” and “clean”). In this example, “c,” “l,” “e,” “a,” “r,” and “n” represent recognized characters, each of which is associated with a confidence level. The “{,” “|,” and “}” symbols are meta-characters used to define the graph of alternatives.

All recognized characters in a symbol graph are as­sociated with both the confidence the recognizer assigns to that character and the specific strokes in the ink that make up the character. If the recognizer cannot assign a character to some part of the ink with the required confidence, it specifies “don’t know” at that position in the symbol graph.

A recognizer returns the results of its analysis to the RC Manager, which ar­ranges to return the results to the calling program. The recognizer returns ink, a symbol graph, symbol elements (containing the symbol value and confi­dence), and symbol correlations (which list the strokes for a recognized character). Recognizers may return partial results during an inking transac­tion.

Beyond the requirements for collecting ink and re­turning results, there are no constraints placed on how recognizers work. Windows for Pens was built from the ground up to accommodate recognizers for languages using non-Roman alphabets. The recog­nizer interface is open and published, and third-party recognizers will be widely available. At least some Windows programs will want to provide cus­tom recognizers for special symbols.

Compatibility with Windows Applications

The Pen Message Interpreter allows the pen to be used with existing Windows (and non-Windows) applications by acting as a “middleman” between these applications and Windows for Pens. Most pen input is treated as mouse input (for example, a downward motion that depresses the pen point is treated as a mouse click) and is passed to the appli­cation’s mouse buffer for processing. Existing Windows applications already buffer mouse input separately from keyboard input, simplifying the pen message interpreter’s tasks.

However, when the user moves the pen into a text entry box, Windows for Pens recognizes that the user is now over a writing area. This causes several changes: the cursor shape changes to that of a pen; the display device driver temporarily throws a transparent overlay over the existing screen to show the “ink” left by the pen; and pen input is sent to the handwriting recognition engine for processing, then sent to the application’s keyboard buffer. Gestures are handled similarly and mapped to command events, such as “copy.”

Support Resources

CompuServe

CompuServe features a special forum for Windows for Pens beta developers. The forum is used daily by engineers and support specialists who an­swer tech­nical questions and place technical articles that are of wide interest to Windows for Pens devel­opers. To access this forum, developers need to be on the Windows for Pens beta program. Section 12 under “winbtdev” is dedicated to Windows for Pens issues.

Contact: 800/ 848–8199

We recommend using TAPCIS, The Access Program for the CompuServe Information Service. Put together by Support Group, Inc.

Contact:
Phone:    800/ 872–4768 or 301/ 387–4500
Fax:         301/ 387–7322

Pen for Windows Consultants

Andersen Consulting

Andersen Consulting is a world wide information systems integrator, with offices in 146 cities. In the U.S., there is an Andersen Consulting office in all major metropolitan areas. Since installing the first business computer in 1954, Andersen Consulting has helped leaders in virtually every industry to apply information technology for competitive ad­vantage and to manage the profound change associ­ated with it. We offer clients our expertise in indus­try, business, and advanced technology, plus the services necessary to integrate complex business and technology systems. This unmatched blend of capabilities has made Andersen Consulting one of the world’s largest commercial systems integrators. A single source for the total solution, from strategic planning and systems design and development to change management, facilities management, hard­ware, and software.

As one of the largest systems integration firms in the world, our clients list is extensive. We provide professional services to a significant proportion of the Fortune 500 and to the leading companies in most local markets. In industries like Insurance, Aerospace, and Telecommunications, we are pres­ently working with nearly all the major entrants.

Product Specific Experience: Andersen Consulting is a leader in the development of custom Windows-based applications for major corporations. As part of our ongoing application platform research, we have followed the development of pen-based tech­nology with interest since its inception. We have been a participant in the Windows for Pens beta pro­gram since its inception, evaluating it as a basis for client solutions and demonstrating its capabilities to existing clients.

Pen Consulting Services: For the organization considering the application of pen technology, Andersen Consulting can provide a number of services, including:

• Information technology strategy
• Information planning
• Analysis of competitive and management is­sues
• Systems development planning
• Systems design and installation
• Organizational change management
• Systems support and management

Contact for Andersen Consulting Information:
Timothy M. Boudreau        
69 West Washington Street
Chicago, IL 60602
Phone:    312/ 507–6526      
Fax:         312/ 507–2548      

K.C. Watkins
801 Second Avenue
Seattle, WA 98104
Phone:    206/ 233–8241
Fax:         206/ 386–8615

Joseph McCall & Company

Joseph McCall & Company provides professional consulting services in North America in the follow­ing functional areas:

• Strategic services: information planning and technology audits
• System design and implementation
• Software evaluation and selection
• Project management and planning
• System testing and production systems support
• Training design and execution

Major accounts include: Department of Business Regulation, State of Florida; Allis–Gleaner (AG Co.); Continental Telecom (Contel); Alabama Gas Company; Blue Cross-Blue Shield of Florida.

Pen-Based Experience: In early 1990, Joseph McCall & Company made a strategic decision to pursue pen-based technology. Since that time we have established ourselves as strategic marketing partners with Microsoft and NCR Corporation and have successfully implemented one of the largest applications of pen-based technology. Our goal is to provide innovative pen-based solutions that meet the information processing needs of our clients. With the experience and knowledge gained from our involvement in pen-based projects, we have successfully applied our expertise across various in­dustries including: Insurance and Healthcare; Consumer Products; Federal, State and Local Government; and Manufacturing.

Contact for Joseph McCall & Co. Information:
Scott J. Brady, Manager, Pen-Based Development
1819 Peachtree Street
Suite 520               
Atlanta, GA 30309
Phone:    404/ 352–4241
Fax:         404/ 351–3728

Knowledge-Based Solutions (KBS)

The areas of expertise for Knowledge-Based Solutions include our QUICK START rapid proto­typing program for early adopters of pen-based sys­tems; domain experience includes field mainte­nance, sales force automation, financial, health care, and insurance, with projects throughout U.S., though primarily New York and New England. Representative clients include:

• Startup software company (health care)
• Windows for Pens hardware OEMs
• Windows for Pens ISVs
• Government agency

Product Experience: Experience with all major Windows–based development tools, including but not limited to ObjectVision, Visual Basic, Windows for Pens SDK, as well as connectivity with data­bases, wireless networks, and other Windows applications.

Pen Consulting Services: We are experts in the de­ployment of field automation systems, specifically involving pen-based systems. Our early work on pen computing with Microsoft and other pioneers, combined with extensive contacts, provides KBS clients with access to the most complete base of ex­perience with this technology.

Our low-cost, low-risk QUICK START program provides clients with a chance to quickly produce a “proof of concept” system to get started with pen-based computing. Other KBS services include full system development, executive overviews, technol­ogy transfer, and strategic analysis. Please contact us for a copy of our QUICK START brochure.

Contact for KBS Information:
Randy Parker
65 Westbourne Terrace, Suite 300
Brookline, MA 02146
Phone:    617/ 566–2451
Fax:         617/ 734–8442

Microsoft Consulting Services

Microsoft Consulting Services are available worldwide. MCS Consultants have assisted organi­zations throughout the world. MCS has already served hundreds of clients with tasks ranging from developing feasibility studies to providing onsite, long-term consultants. We’ve worked with dozens of industries, including aerospace, public utilities, law, finance, government, hotels and resorts, tele­communications, health care, automobile, manufac­turing, and others.

Microsoft Consulting Services is staffed by consult­ants with a wide range of expertise in the field of information technology management and world-class technical consultants in areas such as the Microsoft Windows graphical environment, OS/2 operating systems, Microsoft Visual Basic modern programming system for Windows, Microsoft Mail for PC Networks, LANs, multimedia, enterprise networking, cooperative processing, development tools, and office automation. And because MCS is an integral part of Microsoft, clients have excellent resources. MCS is directly linked to Microsoft product development and support groups and is able to obtain their tools, services, and expertise.

Major clients include: Baxter Healthcare, British Petroleum, Coca-Cola Company, Dun and Brad­street, Eli Lilly and Company, JP Morgan Com­pany, Nike International, and Texaco are a partial list of organizations served by MCS world­wide.

• Planning
• Design
• Development
• Implementation and support

We are committed to helping you leverage your in­vestment in information technology to your best ad­vantage. We can provide assistance with all phases of a project in developing systems that take advan­tage of the portability and the familiarity of pen computing. Our mission is to help you apply the leading informa­tion technologies such as pen com­puting to the infor­mation needs of your organiza­tion.

Contact for MCS Information:

Western Regional Office:
Microsoft Consulting Services
Microsoft Corporation       
One Market Plaza, Suite 1550
Spear Street Tower             
San Francisco, CA 94105
Phone:    415/  905–0235

Eastern Regional Office:
Microsoft Consulting Services
Microsoft Corporation
200 State Street    
Boston, MA 02109
Phone:    617/  951–2780

Sales Technologies (ST) develops, installs, and supports tailored, networked systems that improve sales force productivity for corporations with mobile field sales forces. ST’s software product consists of five core application modules: Territory Management, Electronic Mail, Electronic Forms, Electronic Reports Distribution, and Spreadsheets. Pen-based systems are already being used with the Territory Management module, and further devel­opment of pen-based software is underway. ST also offers a broad range of services to keep customers’ systems running smoothly, including design con­sulting, database development, 24-hour help line, data center, hardware repair and replacement serv­ices, and comprehensive user training.

ST has developed sales force productivity solutions for companies in many different industries, includ­ing consumer package goods, industrial, oil and gas, and health care.

Major clients include: 3M, General Mills, Gillette, Mobile Oil, SmithKline Beecham, The Upjohn Company.

Pen Consulting Services: Sales Technologies offers custom solutions based on its core products. Our primary services include:

• Planning
• Design
• Development
• Implementation and support.

Contact for Sales Technologies Information:
Gitesh Ramamurthy
3399 Peachtree Rd. NE
The Lenox Building, Suite 700
Atlanta, GA 30326
Phone:    404/ 841–4000
Fax:         404/ 841–4115