Biography

Vincent Levesque is a Research Scientist with Immersion Canada (Montreal, Canada). He was a Postdoctoral Fellow in the Computer Science Department at the University of British Columbia (Vancouver, Canada) from 2009 to 2011. He received a B.Eng. in Computer Engineering (2000), and a M.Eng. (2003) and PhD (2009) in Electrical Engineering from McGill University (Montreal, Canada). His research interests include interaction design with novel haptic interfaces, tactile displays and rendering, and applications of haptics for persons with visual impairments. He is the recipient of several awards including Best Paper Awards at the 2012 Haptics Symposium and the 2011 ACM CHI Conference for his work on touch interaction with programmable friction, the Best Reviewer Award at the 2010 Haptics Symposium, the Best Demonstration Award at the 2008 Haptics Symposium for his work on dynamic tactile graphics, and a Best Paper Award at the 2007 IEEE World Haptics Conference for his work on refreshable Braille.

Research

Enhancing Touch Interactions with Programmable Friction

This project investigates the benefits and outcomes of augmenting touchscreen interactions with programmable surface friction. Using a technology developed by Ed Colgate and his team at Northwestern University, we vary the coefficient of friction at the surface of a touchscreen to create haptic effects that enhance the sense of realism, engagement, and enjoyment experienced by users.

Selected Publications:
      CHI'11 (Best Paper Award), HAPTICS'12 (Best Paper Award)

Haptics for Persons with Visual Impairment

My work on laterotactile stimulation brought me to consider the broader needs of visually impaired persons and the ways in which technology - particularly haptic technologies - could improve their quality of life. In an effort to share my findings, I am writing a survey that I hope will serve as a primer for haptics researchers interested in applying their research to visually impaired persons. An early version of the survey was published as a technical report.

Selected Publications:
      PhD Thesis, Technical Report

Tactile Graphics

The main topic of my Ph.D. thesis was the display of refreshable tactile graphics by laterotactile stimulation. I have developed rendering methods that produce tactile patterns with vibration, an undulating texture and small dots.

Selected Publications:
     PhD Thesis, EuroHaptics'10, ASSETS'08, HAPTICS'08, CHI'06

Learn more on laterotactile.com:
     Tactile Graphics, Haptic Memory Game

Braille

This project investigated the feasibility of using laterotactile stimulation for the refreshable display of Braille. As a first step, we implemented a 1D laterotactile display and showed that it could be used to produce a line of Braille dots. We more recently extended this work to the display of complete Braille cells with the STReSS2, our lab's latest 2D laterotactile display.

Selected Publications:
      PhD Thesis, WHC'07 (Best Paper Award), TAP 2005

Learn more on laterotactile.com:
     Virtual Braille, Virtual Braille Dots

Mobile Interaction

I played a supporting role in a project that looked at ways in which laterotactile stimulation can be used in a mobile context. My colleague Jerome Pasquero, in collaboration with Joseph Luk and Karon Maclean of the SPIN lab at the University of British Colombia, integrated a miniature laterotactile display into a PDA-like device and is now looking at how the novel feedback it allows can be used to improve interaction with such devices.

Selected Publications:
      Trans. Multimedia 2007, CHI'06 (Best Paper Award)

Learn more on laterotactile.com:
     Haptics for Mobile Interactions

Measurement of Skin Deformation

The research topic for my Master's thesis was the measurement of skin strain by tracking of anatomical landmarks of the fingertip. Techniques borrowed from the field of biometry were used to extract fingerprint features from movies of a finger sliding over a transparent surface (with or without simple geometrical features.) Variations in skin strain were measured by observing changes in inter-feature distance over time. 

Selected Publications:
      J. R. Soc. Interface 2011, EuroHaptics'03

click here for more details

Publications

2012

Exploring the Design Space of Programmable Friction for Scrolling Interactions
Vincent Lévesque, Louise Oram and Karon MacLean
Proc. Haptics Symposium 2012, Vancouver, Canada, March 2012.
Best Paper

Abstract

Scrolling interactions are an important aspect of the design of usable touchscreen interfaces, particularly for handheld devices that can only display a limited amount of information at once. Using a touchscreen capable of dynamically altering its surface friction, we explore the design space of haptically-augmented scrolling interactions and investigate programmable friction's ability to provide appropriate feedback in envisioned usage scenarios. We performed five user experiments to evaluate respectively the identifiability of a set of iconic detents, the countability of detents, the perception of detent density, the synchronization of tactile feedback to on-screen events, and the optimal friction pattern for a spring-like resistance. The results of these experiments provide valuable information that will inform the design of scrolling interactions that leverage programmable friction for an improved user experience.

Programmable Friction in Scrolling Interactions
Vincent Lévesque, Louise Oram and Karon MacLean
Demonstration, Haptics Symposium 2012, Vancouver, Canada, March 2012.
Nominated for Best Demonstration

Abstract

Scrolling is a pervasive interaction for accessing information that is currently off screen. Devices with small screens further leverage scrolling, and are often used in scenarios where the visual attention of the user is divided between the screen and the environment. Using a touchscreen that can dynamically alter its surface friction, scrolling interactions can be augmented, providing a non-visual source of feedback about the state of the system. Detents, and other distinct patterns created from changing friction, can be used to signal when a new item has entered the screen, and the density of this feedback can be seen as the rate of scrolling. Circular scrolling, variable rate scrolling (or scrubbing), as well as other exemplar applications will be used to demonstrate how these ideas, and others, can be used to effectively augment scrolling interactions.

Adaptive Level of Detail in Dynamic, Refreshable Tactile Graphics
Vincent Lévesque, Gregory Petit, Aude Dufresne and Vincent Hayward
Proc. Haptics Symposium 2012, Vancouver, Canada, March 2012.

Abstract

We investigate gains in user appreciation and performance when the level of detail of tactile graphics is dynamically altered either at the press of a button or automatically, as a function of exploration speed. This concept was evaluated by asking 9 visually impaired participants to perform hierarchical spatial search tasks in a concert hall illustration. The tasks could be simplified by first searching for a section in a sparse illustration, and then a seat in a detailed illustration. The results show no improvement in task performance but indicate a user preference for explicitly controlling the level of details with the manual toggle.

Designing Interactions for Novel Haptic Technologies
Vincent Lévesque
Invited Talk, Haptic Interaction Design Workshop, Haptics Symposium 2012, Vancouver, Canada, March 2012.

Abstract

The haptics community is a great source of innovation, with novel means of stimulating the human sense of touch being constantly devised. Developing compelling applications for these innovative technologies, however, can be a significant challenge due to the initial lack of understanding of relevant human factors and the dedication required to design high-performance hardware. A common approach is to first optimize a device’s hardware design, and then methodically build up knowledge of its psychophysics prior to investigating applications. In this talk, I will propose alternative approaches that skip ahead to the exploration of a haptic technology’s interaction design space and to the evaluation of its benefits for the user experience in concrete applications. Drawing from my experience with laterotactile displays and programmable friction touchscreens, I will illustrate how rapid prototyping and user centered design can demonstrate the value of novel haptic feedback early on, justifying and guiding further investments in hardware refinements and human factor studies.

2011

Enhancing Touch Interactions with Programmable Friction
Vincent Lévesque
Invited Talk, Workshop on Vibrotactile Haptics for Touch Screens, IEEE World Haptics Conference 2011, Istanbul, Turkey, June 2011.

Abstract

Touch interactions take place through flat surfaces that lack the tactile richness of physical interfaces and hence do not fully exploit our sense of touch. This collaborative work explores the reintroduction of rich tactile feedback in touch interfaces through the use of programmable surface friction. Using Northwestern University's Large Area Tactile Pattern display (LATPaD), we demonstrate that programmable friction can not only improve performance in targeting tasks but also have a positive impact on the enjoyment, engagement and sense of realism experienced by users of touch interfaces.

Enhancing Physicality in Touch Interaction with Programmable Friction
Vincent Lévesque, Louise Oram, Karon MacLean, Andy Cockburn, Nicolas D. Marchuk, Dan Johnson, J. Edward Colgate and Michael A. Peshkin
Proc. ACM Conference on Human Factors in Computing Systems (CHI '11), Vancouver, Canada, May 2011, pp. 2481-2490.
Best of CHI

Abstract

Touch interactions have refreshed some of the 'glowing enthusiasm' of thirty years ago for direct manipulation interfaces. However, today's touch technologies, whose interactions are supported by graphics, sounds or crude clicks, have a tactile sameness and gaps in usability. We use a Large Area Tactile Pattern Display (LATPaD) to examine design possibilities and outcomes when touch interactions are enhanced with variable surface friction. In a series of four studies, we first confirm that variable friction gives significant performance advantages in low-level targeting activities. We then explore the design space of variable friction interface controls and assess user reactions. Most importantly, we demonstrate that variable friction can have a positive impact on the enjoyment, engagement and sense of realism experienced by users of touch interfaces.

Video

Frictional Widgets: Enhancing Touch Interfaces with Programmable Friction
Vincent Lévesque, Louise Oram, Karon MacLean, Andy Cockburn, Nicolas D. Marchuk, Dan Johnson, J. Edward Colgate and Michael A. Peshkin
CHI'11 Extended Abstracts on Human Factors in Computing Systems, Vancouver, Canada, May 2011, pp. 1153-1158.

Abstract

Touch interactions occur through flat surfaces that lack the tactile richness of physical interfaces. We explore the design possibilities offered by augmenting touchscreens with programmable surface friction. Four exemplar applications - an alarm clock, a file manager, a game, and a text editor - demonstrate tactile effects that improve touch interactions by enhancing physicality, performance, and subjective satisfaction.

Video

Effect of skin hydration on the dynamics of fingertip gripping contact
Thibaut André, Vincent Lévesque, Vincent Hayward, Philippe Lefèvre and Jean-Louis Thonnard
Journal of the Royal Society Interface, published online 13 April 2011.

Abstract

The dynamics of fingertip contact manifest themselves in the complex skin movements observed during the transition from a stuck state to a fully developed slip. While investigating this transition, we found that it depended on skin hydration. To quantify this dependency, we asked subjects to slide their index fingertip on a glass surface while keeping the normal component of the interaction force constant with the help of visual feedback. Skin deformation inside the contact region was imaged with an optical apparatus that allowed us to quantify the relative sizes of the slipping and sticking regions. The ratio of the stuck skin area to the total contact area decreased linearly from 1 to 0 when the tangential force component increased from 0 to a maximum. The slope of this relationship was inversely correlated to the normal force component. The skin hydration level dramatically affected the dynamics of the contact encapsulated in the course of evolution from sticking to slipping. The specific effect was to reduce the tendency of a contact to slip, regardless of the variations of the coefficient of friction. Since grips were more unstable under dry skin conditions, our results suggest that the nervous system responds to dry skin by exaggerated grip forces that cannot be simply explained by a change in the coefficient of friction.

Restoring Physicality to Touch Interaction with Programmable Friction
Vincent Lévesque, Louise Oram, Karon MacLean, J. Edward Colgate and Michael A. Peshkin
Proc. International Conference on Consumer Electronics (ICCE) 2011, Las Vegas, IEEE, 2 pages, January 2011.

Abstract

Rich tactile interaction with control surfaces has been compromised in the transition to touch interfaces. This paper discusses new haptic effects that restore physicality to touch interaction by dynamically altering a touchscreen's frictional properties.

Do-It-Yourself Haptics: A Practical Introduction to Haptics for Consumer Electronics
Vincent Lévesque and Karon Maclean
Invited Tutorial, IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, ICCE, January 2011.

Abstract

What was once accomplished by directly interacting with physically embodied objects, tools and controls is now increasingly performed through abstract, intangible interfaces. The distinct edges, textures and clicks of physical buttons are, for example, gradually being replaced by the blank feel of a touchscreen, with only audio-visual feedback remaining to guide actions. While flexible, these interfaces impose greater attentional demands and do not fully exploit the richness of the sense of touch. Haptic feedback promises to restore natural, direct and responsive interaction to these digital interfaces and to thereby improve their physical esthetics and reduce their reliance on the overloaded senses of vision and audition.

This tutorial assumes no knowledge of haptics and aims not only to provide a broad overview of the topic to consumer electronics practitioners, but also to engage you in a dialogue about the challenges and benefits of integrating this technology in their products. The tutorial will introduce you to current uses of haptics in consumer electronics, the basics of haptic perception in humans, the design and effective use of simple haptic interfaces, and cutting edge haptic interfaces being explored in research labs around the world. It will provide a solid basis for consumer electronics professionals to further investigate this important emerging technology.

2010

Laterotactile Rendering of Vector Graphics with the Stroke Pattern
Vincent Lévesque and Vincent Hayward
Proc. EuropHaptics 2010, Part II, Kappers, A.M.L. et al. (Eds.), LNSC 6192, Springer-Verlag, pp. 25-30.

Abstract

Raised line patterns are used extensively in the design of tactile graphics for persons with visual impairments. A tactile stroke pattern was therefore developed to enable the rendering of vector graphics by lateral skin deformation. The stroke pattern defines a transversal profile and a longitudinal texture which provide tactile feedback while respectively crossing over the stroke and tracing its length. The stroke pattern is demonstrated with the rendering of lines, circles and polygons, and is extensible to other vector graphics primitives such as curves. The parametric nature of the stroke allows the representation of distinctive line types and the online adjustment of line thickness and other param- eters according to user preferences and capabilities. The stroke pattern was informally evaluated with four visually impaired volunteers.

2009

Virtual Display of Tactile Graphics and Braille by Lateral Skin Deformation
Vincent Lévesque
Ph.D. Thesis, McGill University, August 2009

Abstract

Graphical content is increasingly pervasive in digital interfaces and documents yet it remains accessible to visually impaired persons almost exclusively on media with limited flexibility such as embossed paper. Textual content is more accessible but nevertheless limited by the cost and functionality of refreshable Braille displays and voice synthesis. This thesis explores the use of a novel tactile stimulation approach that relies on lateral skin deformation for the computerized display of virtual Braille and tactile graphics.

Tactile synthesis by lateral skin deformation is initially explored in the context of Braille. The feasibility of producing virtual Braille by laterotactile stimulation is first demonstrated by creating the illusion of brushing against a line of Braille dots through the synchronization of a travelling wave of skin deformation with the displacement of a tactile array of eight actuators. This principle is then extended to complete 6-dot Braille cells by distributing lines of virtual dots onto the rows of actuators of a general-purpose STReSS2 tactile array. Reading the resulting virtual Braille is shown to generally be feasible but demanding, suggesting that a specialized laterotactile Braille display should be devised or that dots be rendered for contrast rather than realism.

Tactile rendering by lateral skin deformation is then further explored with the gradual development of a virtual tactile graphics framework that emulates conventional features such as raised lines and areal textures through a coherent set of patterns that includes grating textures, stroked and dotted shapes, bitmap-based masks, and composite patterns. Dynamic rendering is also exploited to produce novel effects such as tactile flow, reactive textures dependent on the exploration behaviour, and interactive content with alternate views. The usability of the framework is informally evaluated with visually impaired volunteers and early tactile patterns studied through formal experiments. The tactile patterns are presented on the Tactograph, a haptic interface redesigned specifically for the display of tactile graphics that combines a STReSS2 display with an instrumented planar carrier.

This thesis demonstrates the potential of lateral skin deformation for the display of Braille and tactile graphics, and explores in the process the ways in which this novel approach to tactile stimulation can be applied to produce meaningful tactile sensations.

The effects of moisture on fingertip skin deformation during loading and slipping
Thibaut André, Vincent Lévesque, Vincent Hayward, Philippe Lefèvre and Jean-Louis Thonnard
39th Annual Meeting of the Society for Neuroscience, Chicago, USA, 2009.

Abstract

During tactile interaction, skin deformation stimulates the mechanoreceptors, enabling the nervous system to become aware of the properties of the touched surface. Even on perfectly smooth, glassy surfaces, the fingertip skin deforms in complicated ways when slip occurs. The aim of the present study is to assess the effects of moisture content and interaction forces on the fingertip skin deformation occurring in this condition.

Twelve subjects having a large range of skin moisture (from dry to wet) participated in this study. They were asked to exert a constant normal force against a glass surface with the index fingertip and to increase the tangential force until slip occurs. Each subject performed five slippages for six different levels of normal force (0.2, 0.5, 1, 2, 5, 10N). Forces and moisture were measured with a force-torque sensor (ATI F/T Mini40) and a moisture evaluator (Corneometer®), respectively. Skin deformation at the contact surface was acquired with a specifically designed optical apparatus that allowed the evaluation of the relative sizes of the slipping and sticking regions.

The area of the sticking region decreased when the tangential force increased. It varied linearly with the ratio of net tangential/normal force components. The slope of this relationship was strongly influenced by both the normal force exerted and the moisture content of the skin. Finally, the static coefficient of friction could be extracted when the sticking regions vanished and the slipping region spread to the whole surface of contact.

This study described the deformation of the skin occurring during interaction with smooth surfaces and evaluated the effect of interaction force components and moisture on this deformation. A skin deformation model that takes into account these two parameters is presently being developed. This research was supported by a grant from Prodex (contract numbers 90063, 90064, 90231, 90232), ESA (European Space Agency), PAI, FNRS, FRSM, ARC, and NANOBIOTACT-project (EU-FPG-NMP-033287).

2008

Exploration multimodale d'images pour des utilisateurs ayant une déficience visuelle
Gregory Petit, Aude Dufresne, Vincent Lévesque, Vincent Hayward
Sciences et Technologies pour le Handicap, 2(2), pp. 175-186, 2008.

Résumé

Cet article preésente une recherche visant à rendre accessibles des images aux usagers ayant une déficience visuelle. Le logiciel MaskGen a été développé afin de transformer interactivement ces images « visuelles » en images multimodales (tactiles et audio). Une méthodologie a été développé pour transposer ces images et les afficher sur le Tactograph, un appareil multimodal dynamique.

Abstract

This article presents research on making images accessible for people with visual impairment. The MaskGen system was developed to interactively transpose these "visual" images into multimodal images (tactile and audio). A methodology was designed to transpose the images and prepare them to be displayed on the Tactograph, a refreshable multimodal device.

Refreshable tactile graphics using a lateral skin deformation device
Vincent Lévesque, Vincent Hayward, Gregory Petit, Aude Dufresne
Abstract selected for oral presentation, 9th International Conference on Low Vision, July 7 - 11, 2008, Montreal, Quebec, Canada.

Abstract

Tactile graphics are useful to convey spatial information and concepts to visually impaired persons with maps, mathematical diagrams and other types of illustrations. Unfortunately, tactile graphics are currently produced on physical media, a slow and cumbersome process that results in static material. Tactile graphics also present unique challenges due to the relatively limited resolution of the tactile sense, often requiring a reduction in information density. Refreshable tactile graphics could alleviate these problems by allowing instant access to online illustrations as well as interactive control over features such as the visibility of layers of information.

Research on refreshable tactile graphics has focused on programmable arrays of moving pins that approximate 3D surfaces. For the past decade, we have been investigating an alternative approach based on lateral deformation of the skin. This approach relies on miniature bending motors that pull laterally on the skin, locally stretching or compressing it to stimulate tactile receptors. The latest prototype covers the surface of a typical fingerpad with a matrix of 8 by 8 actuators. Once mounted on a mouse-like planar carrier, the device can alter the tactile sensation felt in response to displacements by controlling the activation of the actuators, thereby creating a large virtual surface out of a small active area. The system can currently be used to draw refreshable tactile graphics with three types of features. The first produces a strong, localized vibration sensation. The second produces a sensation similar to that of brushing against Braille dots. The third produces the sensation of brushing against a grating texture with a programmable roughness and orientation. Recent experiments have shown that these features can be combined to display distinguishable textured shapes.

Recent joint work between McGill University and University of Montréal focuses on the adaptation of schoolbook illustrations for visually impaired children. Experiments were recently conducted to evaluate the effectiveness of the system at conveying the pedagogical content of graphics adapted from a high school history textbook. Participants answered questions based on their interpretation of a world map of the first civilizations, a bar chart of the population of ancient Athens, and an illustration of a cathedral. Results suggest that the system could become a valuable tool for the education of visually impaired children.

Refreshable Tactile Graphics Applied to Schoolbook Illustrations for Students with Visual Impairment
Gregory Petit, Aude Dufresne, Vincent Lévesque, Vincent Hayward, Nicole Trudeau
Proc. ASSETS 2008, Oct. 13-15, 2008, Halifax, Canada.

Abstract

This article presents research on making schoolbook illustrations accessible for students with visual impairment. The MaskGen system was developed to interactively transpose illustrations of schoolbooks into tactile graphics. A methodology was designed to transpose the graphics and prepare them to be displayed on the STReSS2, a refreshable tactile device. We experimented different associations of tactile rendering and audio feedbacks to find a model that children with visual impairment could use. We experimented with three scientific graphics (diagram, bar-chart and map) with forty participants: twenty sighted adults, ten adults with visual impairment, and ten children with visual impairment. Results show that the participants with visual impairment liked the tactile graphics and could use them to explore illustrations and answer questions about their content.

Graphisme tactile appliqué aux illustrations de manuels scolaires à l'usage d'enfants ayant une déficience visuelle
Gregory Petit, Aude Dufresne, Vincent Lévesque, Vincent Hayward, Nicole Trudeau
Proc. IHM 2008, Sept. 3-5, 2008, Metz, France.

Résumé

Cet article présente une recherche visant à rendre accessible, aux étudiants ayant une déficience visuelle les illustrations de manuels scolaires. Le logiciel MaskGen a été développé afin de transformer interactivement ces illustrations en graphiques tactiles. Une méthodologie a été développée pour transposer ces graphiques et les afficher sur le STReSS2, un appareil tactile dynamique. Nous avons expérimenté différentes associations de rendus tactiles et de retours sonores pour obtenir une version compréhensible pour les étudiants ayant une déficience visuelle. Nous avons testé trois graphiques tactiles (un plan, un histogramme et une carte) avec quarante participants : vingt voyants, dix adultes non-voyants et dix enfants non-voyants. Les résultats de l'expérimentation montrent que les participants ayant une déficience visuelle ont aimé les images tactiles, ont été capables de les explorer correctement et de répondre avec succès aux questions posées sur ces images.

Abstract

This article presents research on making schoolbook illustrations accessible for students with visual impairment. The MaskGen system was developed to interactively transpose illustrations of schoolbooks into tactile graphics. A methodology was designed to transpose the graphics and prepare them to be displayed on the STReSS2, a refreshable tactile device. We experimented different associations of tactile rendering and audio feedbacks to find a model that children with visual impairment could use. We experimented with three scientific graphics (diagram, bar chart and map) with forty participants: twenty sighted adults, ten adults with visual impairment, and ten children with visual impairment. Results show that the participants with visual impairment liked the tactile graphics and could use them to explore illustrations and to answer questions about their content.

Refreshable Tactile Graphics with the STReSS2 Laterotactile Display
Vincent Lévesque and Vincent Hayward
Technical Demonstration, EuroHaptics 2008, June 11-13, 2008, Madrid, Spain.

Abstract

This demonstration will allow visitors to feel virtual tactile graphics rendered with the STReSS2, a tactile display that relies on lateral skin deformation to produce tactile sensations. The display consists of a fingerpad-sized array of 8 by 8 laterally moving actuators that are programmed to produce deformation patterns in response to movements of the device over a letter-sized workspace (approx. 28 by 22 cm). Tactile graphics are drawn using three tactile rendering methods that respectively produce virtual dots, virtual gratings, and localized vibrations. Despite resulting from lateral deformation rather than indentation, virtual dots and gratings give a convincing illusion of brushing against raised dot patterns and embossed undulating textures. Localized vibrations produce less natural but stronger, sharper sensations. Visitors will be invited to explore a variety of tactile graphics produced by combining these three rendering methods, ranging from simple shapes to more complex illustrations. A similar demonstration of this technology was recently awarded the best demonstration prize at the 2008 Haptics Symposium in Reno, Nevada. More details on this work can be found on the project webpage at laterotactile.com.

Tactile Graphics Rendering Using Three Laterotactile Drawing Primitives
Vincent Lévesque and Vincent Hayward
Proc. 16th Symposium on Haptic Interfaces For Virtual Environment And Teleoperator Systems, March 13-14, 2008, Reno, Nevada, pp. 429-436.

Abstract

This paper presents preliminary work towards the development and evaluation of a practical refreshable tactile graphics system for the display of tactile maps, diagrams and graphs for people with visual impairments. Refreshable tactile graphics were dynamically produced by laterally deforming the skin of a finger using the STReSS2 tactile display. Tactile features were displayed over an 11 x 6 cm virtual surface by controlling the tactile sensations produced by the fingerpad-sized tactile display as it was moved on a planar carrier. Three tactile rendering methods were used to respectively produce virtual gratings, dots and vibrating patterns. These tactile features were used alone or in combination to display shapes and textures. The ability of the system to produce tactile graphics elements was evaluated in five experiments, each conducted with 10 sighted subjects. The first four evaluated the perception of simple shapes, grating orientations, and grating spatial frequencies. The fifth experiment combined these elements and showed that tactile icons composed of both vibrating contours and grated textures can be identified. The fifth experiment was repeated with 6 visually impaired subjects with results suggesting that similar performance should be expected from that user group.

Refreshable Tactile Graphics with the STReSS2 Laterotactile Display
Vincent Lévesque, Andrew H. C. Gosline and Vincent Hayward
Technical Demonstration, 16th Symposium on Haptic Interfaces For Virtual Environment And Teleoperator Systems, March 13-14, 2008, Reno, Nevada.
Best Demonstration

Abstract

This demonstration will allow visitors to feel virtual tactile graphics rendered with the STReSS2, a tactile display that relies on lateral skin deformation to produce tactile sensations. The display consists of a fingerpad-sized array of 8 by 8 laterally moving actuators that are programmed to produce deformation patterns in response to movements of the device over a letter-sized workspace. Tactile graphics are drawn using three tactile rendering methods that respectively produce virtual dots, virtual gratings, and localized vibrations. Despite resulting from lateral deformation rather than indentation, virtual dots and gratings give a convincing illusion of brushing against raised dot patterns and embossed undulating textures. Localized vibrations produce less natural but stronger, sharper sensations. Visitors will be invited to explore a variety of tactile graphics produced by combining these three rendering methods, ranging from simple shapes (as investigated in the accompanying paper) to more complex illustrations.

Laterotactile Displays and Rendering: from Illusion to Application
Vincent Lévesque and Jerome Pasquero
Invited Talk, IEEE VR 2008 Tutorial, Integration of Haptics in Virtual Environments: from Perception to Rendering, March 9, 2008, Reno, Nevada.

Abstract

This tutorial will discuss the process through which seemingly minor observations about haptic phenomenon have led to the design of simple yet powerful interfaces in the McGill Haptics Laboratory. The focus will be on a tactile display technology that relies on lateral skin deformation to produce a variety of tactile sensations, including some that are typically associated with normal indentation of the skin. This work, which was initially inspired by a simple tactile illusion produced by running a pen along the teeth of a comb, has resulted in the design of a family of practical laterotactile displays and the development of tactile rendering algorithms. Applications of this technology for mobile interaction and for the display of refreshable Braille and tactile graphics for visually impaired persons will also be covered.

2007

Haptically Enabled Handheld Information Display with Distributed Tactile Transducer
Jerome Pasquero, Joseph Luk, Vincent Lévesque, Qi Wang, Vincent Hayward, Karon E. MacLean
IEEE Transactions on Multimedia, 9(4), pp 746-753, 2007.

Abstract

This paper describes the design, construction, and initial evaluation of a handheld information device that supports combined tactile and graphical interaction. The design comprises a liquid crystal graphic display co-located with a miniature, low-power, distributed tactile transducer. This transducer can create electronically-controlled lateral skin deformation patterns which give the sensation of sliding over small shapes. It is integrated within a slider mechanism to control scrolling. It also functions as a detent when pushing on it. Tactile feedback and the combination of visual and tactile feedback in a mobile context enable the development of new functions, such as multimodal navigation within large graphic spaces.

Braille Display by Lateral Skin Deformation with the STReSS2 Tactile Transducer
Vincent Lévesque, Jerome Pasquero, and Vincent Hayward
Proc. World Haptics 2007, March 22-24, Tsukuba, Japan.
Best Paper Award for Haptic Application

Abstract

Earlier work with a 1-D tactile transducer demonstrated that lateral skin deformation is sufficient to produce sensations similar to those felt when brushing a finger against a line of Braille dots. Here, we extend this work to the display of complete 6-dot Braille characters using a general purpose 2-D tactile transducer called STReSS2. The legibility of the produced Braille was evaluated by asking seven expert Braille readers to identify meaningless 5-letter strings as well as familiar words. Results indicate that reading was difficult but possible for most individuals. The superposition of texture to the sensation of a dot improved performance. The results contain much information to guide the design of a specialized Braille display operating by lateral skin deformation. They also suggest that rendering for contrast rather than realism may facilitate Braille reading when using a weak tactile transducer.

2006

A Role for Haptics in Mobile Interaction: Initial Design Using a Handheld Tactile Display Prototype
Joseph Luk, Jerome Pasquero, Shannon Little, Karon MacLean, Vincent Lévesque, and Vincent Hayward
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'06), Montreal, Canada, April 2006, pp. 171-180.
Best of CHI

Abstract

Mobile interaction can potentially be enhanced with well- designed haptic control and display. However, advances have been limited by a vicious cycle whereby inadequate haptic technology obstructs inception of vitalizing applications. We present the first stages of a systematic design effort to break that cycle, beginning with specific usage scenarios and a new handheld display platform based on lateral skin stretch. Results of a perceptual device characterization inform mappings between device capabilities and specific roles in mobile interaction, and the next step of hardware re-engineering.

Laterotactile Devices and Rendering
Vincent Hayward and Vincent Lévesque
Invited Talk, EuroHaptics 2006, Workshop II - Tactile Display Design, Paris, France, July 2006
A Haptic Memory Game using the STRESS2 Tactile Display
Qi Wang, Vincent Lévesque, Jerome Pasquero, and Vincent Hayward
CHI'06 Extended Abstracts on Human Factors in Computing Systems, CHI'06, Montreal, Canada, April 2006, pp. 271-274.

Abstract

A computer implementation of a classic memory card game was adapted to rely on touch rather than vision. Instead of memorizing pictures on cards, players explore tactile graphics on a computer-generated virtual surface. Tactile sensations are created by controlling dynamic, distributed lateral strain patterns on a fingerpad in contact with an electronic tactile display called STReSS2. The tactile graphics are explored by moving the device within the workspace of a 2D planar carrier. Three tactile rendering methods were developed and used to create distinct tactile memory cards. The haptic memory game showcases the capabilities of this novel tactile display technology.

Video

2005

Display of Virtual Braille Dots by Lateral Skin Deformation: Feasibility Study
Vincent Lévesque, Jerome Pasquero, Vincent Hayward, and Maryse Legault
ACM Transactions on Applied Perception, 2(2), pp 132-149, 2005.

Abstract

When a progressive wave of localized deformations occurs tangentially on the fingerpad skin, one typically experiences the illusion of a small object sliding on it. This effect was investigated because of its potential application to the display of Braille. A device was constructed that could produce such deformation patterns along a line. Blind subjects' ability to read truncated Braille characters ('lowered lowered', 'lowered raised', 'raised lowered', and 'raised raised') using the device was experimentally tested and compared to their performance with a conventional Braille medium. While subjects could identify two-character strings with a high rate of success, several factors need to be addressed before a display based on this principle can become practical.

Blindness, Technology and Haptics
Vincent Lévesque
CIM Technical Report (CIM-TR-05.08), McGill University, Montreal, Canada, 28 pages, October 2005.

Abstract

The blind and the visually impaired are in a unique position to appreciate and make functional use of haptic devices. Designing devices for the blind is, however, more arduous than many researchers and inventors expect. It is thus important to fully understand the needs and requirements of that community before attempting to create devices for them. It is also important to learn from past research and development in the application of technology for the blind. This survey provides an overview of current knowledge on blindness and rehabilitation technology relevant for the design of aids for the blind, and more particularly for the use of haptics with the blind.

The survey begins with a demystification of blindness and a discussion of the differences between blind and sighted. Follows a broad overview of the many attempts at applying technological solutions to problems encountered by the blind. The survey ends with a discussion of lessons learned from previous failures and successes in rehabilitation technology as well as speculation on the future of haptics and other technologies for people living with blindness.

2004

Display of Virtual Braille Dots by Lateral Skin Deformation: A Pilot Study
Jerome Pasquero, Vincent Lévesque, Vincent Hayward, and Maryse Legault
Proc. Eurohaptics 2004, Munich, Germany, June 2004

Abstract

When a progressive wave of localized deformations occurs tangentially on the fingerpad skin, one typically experiences the illusion of a small object sliding on it. This effect was investigated because of its potential application to the display of Braille. A device was constructed that could produce such deformation patterns along a line. This enabled us to test blind subjects' ability to read the truncated Braille characters 'lowered lowered', 'lowered raised', 'raised lowered', and 'raised raised'. While subjects could identify two-character strings with a high rate of success, several factors need to be addressed before a display based on this principle can become practical.

Video

Display of Virtual Braille Dots by Lateral Skin Deformation
Vincent Lévesque, Jerome Pasquero, Vincent Hayward, and Maryse Legault
Poster, 14th Annual Canadian Conference on Intelligent Systems (IRIS), Ottawa, Canada, June 2004
3rd Prize, Best Poster
Display of Virtual Braille Dots by Lateral Skin Deformation
Vincent Lévesque, Jerome Pasquero, Vincent Hayward, and Maryse Legault
Technical Demonstration, 14th Annual Canadian Conference on Intelligent Systems (IRIS), Ottawa, Canada, June 2004
Experimental Evidence of Lateral Skin Strain During Tactile Exploration
Vincent Lévesque
Online presentation, International Society for Haptics, May 2004

2003

Experimental Evidence of Lateral Skin Strain During Tactile Exploration
Vincent Lévesque and Vincent Hayward
Proc. Eurohaptics 2003, Dublin, Ireland, July 2003

Abstract

This paper describes an experimental platform for the study of stretch and compression of the human fingerpad skin during tactile exploration. A digital camera records the sequence of patterns created by a fingertip as it slides over a transparent surface with simple geometrical features. Skin deformation is measured with high temporal and spatial resolution by tracking anatomical landmarks on the fingertip. Techniques adapted from the field of online fingerprinting are used to acquire high-contrast fingerprint images and extract salient features (pores, valley endings, and valley bifurcations). The results of experiments performed with surfaces with a bump or hole and flat surfaces are presented. This work is motivated by the need to provide meaningful 'tactile movies' for a tactile display that uses distributed lateral skin stretch.

Apparatus to reproduce tactile sensations
Vincent Hayward, Jerome Pasquero, and Vincent Lévesque
United States Patent 7,077,015

Abstract

A tactile sensing transducer provides as its output a skin deformation signal for storage or transmission to a tactile stimulation display transducer. A skin surface imaging means operating optically, acoustically or otherwise is positioned for viewing and providing an output image signal corresponding to said deformed skin surface. An electronic processing means connected to said imaging means provides an output skin deformation signal corresponding to the deformation of said skin. The display transducer may incorporate a plurality of individually actuatable, bendable cantilevered arms mounted in line on a base support, each of the arms having tip ends which are displaceable, upon actuation, to provide a tactile display.

2002

Measurement of Skin Deformation Using Fingerprint Feature Tracking
Vincent Lévesque
M. Eng. Thesis, McGill University, November 2002
Dean's Honour List

Abstract

This thesis describes an experimental platform for the study of skin stretch of the human fingerpad during tactile exploration tasks. A digital camera records the sequence of patterns created by a fingertip as it slides over a transparent surface with simple geometrical features. Skin deformation is measured with high temporal and spatial resolution by tracking anatomical landmarks on the fingertip. Techniques adapted from the field of online fingerprinting are used to acquire high-contrast fingerprint images and extract salient features (pores, valley endings, and valley bifurcations). The performance of the method is evaluated with surfaces embossed with a ridge or valley and flat surfaces. This work is motivated by the need to provide meaningful 'tactile movies' for a tactile display using distributed lateral skin stretch.

Minitature Tactile Display
Jerome Pasquero, Vincent Lévesque, and Vincent Hayward
Technical Demonstration, 12th Annual Canadian Conference on Intelligent Systems (IRIS), Calgary, Canada, May 2002
Best Demonstration
Lateral Skin Stretch Measurement by Fingerprint Minutia Tracking
Vincent Lévesque and Vincent Hayward
Poster, 12th Annual Canadian Conference on Intelligent Systems (IRIS), Calgary, Canada, May 2002

Abstract

This poster proposes a technique for the measurement of lateral skin stretch on the fingertip. A sequence of images is acquired as the fingertip moves across a transparent surface. Fingerprint minutia are detected using techniques borrowed from the fingerprint identification field (Stosz and Alyea, 1994) and tracked between pairs of successive images. Variations in inter-minutia distance are used to evaluate lateral skin stretch.