Today the US Patent & Trademark Office published a patent application from Apple that relates to a future version of Apple Pencil with a huge twist. On one hand the invention relates to delivering tactile feedback to a future version of Apple Pencil. On the other hand, Apple's engineers describe how an Apple Pencil could be used with a mixed reality headset for both augmented and virtual reality. Users will be able to write on virtual paper or have this Apple Pencil used as another tool interacting with virtual objects.

Haptics are used on a touch display found on an iPhone or iPad to convey a variety of different information to a user, such as information regarding one or more touch inputs that a user has provided such as alerts or status of the electronic device or one or more applications.

However, when using Apple Pencil, as it is today, the user misses out on feeling the particular haptic related to alerts and so forth.

In accordance with Apple's invention, improvements to Apple Pencil will be able to produce shear forces that act on a user to provide unique tactile sensations. For example, a shear device can be included at a grip region of a stylus to provide shear sensations at the user's hand (e.g., fingers). The sensation can be targeted directly to the user's hand, rather than generally across the entire stylus.

Augmented and Virtual Reality Application

Apple's patent takes an interesting twist at one point and notes that "Shear forces can be used with a virtual reality, augmented reality, or mixed reality system to simulate tactile sensations of interacting with physical objects even when no such objects are present.

More specifically, the invention at this point veers off and doesn't limit 'Shear Forces' to a future version of Apple Pencil.

Apple specifically states that "A touch-based input device in accordance with embodiments disclosed herein can include any device that is held, worn, or contacted by a user for providing input and/or receiving feedback. The touch-based input device can be used alone or in conjunction with another device."

For the sake of simplicity, "another device" is shown to be an iPad in patent FIG. 1 below as Apple's patent returns to non-virtual applications until we get to patent FIG. 6 further below.

In patent FIG. 2, Apple points us to the grip area of this future Apple Pencil and focuses on where the 'Shear' tactile forces will be delivered. In this one design, the shear feedback is delivered by vertical types of strips.

Beyond tactile haptic feedback, Apple notes that shear forces include unaligned forces that urge one part of the user's hand (e.g., finger) in one direction and another part of the user's hand in an opposite direction.

The shear feedback components include sliding elements #220 shown further below in patent FIG. 11. Apple clarifies that the grip region could actually take up 10% to 90% of the surface of the grip area, even though the patent figures limit the size of the shear feedback components/elements.

In patent FIG. 5 we see that this future version of the Apple Pencil will include an accelerometer #170, a gyroscope #172, and/or a compass #174. During use, the accelerometer can track and record acceleration of Apple Pencil. Acceleration can be measured in a three-dimensional (x, y, and z) coordinate system.

The haptic device #178 can be implemented as any suitable device configured to provide force feedback, vibratory feedback, tactile sensations, and the like.

Free Space Application

In Apple's patent FIG. 6 below Apple Pencil can be used to provide feedback during use in free space. For example, this future version of Apple Pencil can simulate tactile sensations that correspond to a user experience in a mixed reality system.

As shown in FIG. 6, Apple Pencil will be operated by a user in a manner that moves the Pencil through space. Each point in space corresponds to a point within a virtual environment of a mixed reality system.

A region #198 of space can correspond to a region of the virtual environment. For example, a virtual object can be rendered in the virtual environment and displayed or otherwise output for reference by the user. The display can include a headset, a head-up display, and/or an optical head-mounted display in communication with Apple Pencil.

A surface, volume, interior, or other portion of the virtual object can correspond to the location of the region #198 in space. Accordingly, when the future Apple Pencil is moved to the region in space, it is understood by the user to be positioned at the virtual object in the virtual environment.

In Apple's patent FIG. 9 above we're able to see the bridge segments #212 in the sliding elements #220 can be evenly distributed along a length of the user grip region. For example, the user grip region can present radial and/or bilateral symmetry.

In Apple's patent FIGS. 10 and 11 above, the future Apple Pencil can be used to provide feedback during use in free space. As shown in FIG. 10, the future Apple Pencil could be operated by a user in a manner that rotates through space.

Apple's patent application that was published today by the U.S. Patent Office was filed back in Q4 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Apple's Inventors

Benjamin Jackson: Failure Analysis Engineer - iPad Tech Ops

David Bloom: Sr. Quality Assurance Manager in Portland, Oregon

Steven Taylor: Mechanical Design Engineer

Go here to see the original:

Apple Reveal a Future Apple Pencil that delivers tactile feedback to users in a Real or Virtual World using a Headset - Patently Apple