Vigdís Vala Valgeirsdóttir - Skemman · 2016). One of the best examples of SS is the use of...

Influencing factors in vibrotactile acuity

Vigdís Vala Valgeirsdóttir

Lokaverkefni til M.Sc.-gráðu í sálfræði

Leiðbeinendur:

Dr. Árni Kristjánsson & Dr. Ómar Jóhannesson

Sálfræðideild

Heilbrigðisvísindasvið Háskóla Íslands

Júní 2017

Ritgerð þessi er lokaverkefni til M.Sc.-gráðu í sálfræði og hana má afrita í einu eintaki

til einkanota.

© Vigdís Vala Valgeirsdóttir 2017

3

Preface

First of all I want to thank my excellent instructors, Árni Kristjánsson and Ómar Jóhannesson,

for whom it has been a true privilege to learn from and be inspired by professionally as well as

personally. I would also like to thank Rúnar Unnþórsson, the leader of the Sound of Vision

project and all my colleges on the team. Special thanks go to the wonderful Ingiríður Þórisdóttir,

and all the faculty members of the Psychology department for their assistance and positive

feedback throughout my studies. I am also grateful to Sigurður J. Grétarsson, Jörgen L. Pind for

giving me the opportunity to teach in an academic setting. Furthermore, I would like to thank

the Ministry of Welfare for supporting me financially in my research as well as Karl Ægir

Karlsson for examining my thesis. I also thank my parents, Ásta Kr. Ragnarsdóttir and Valgeir

Guðjónsson for their undying support for the past 24 years. Finally, good friendship is

invaluable during demanding periods and therefore I want to thank my closest friends Maríanna

Eva Dúfa Sævarsdóttir and Guðrún Fjóla Guðmundsóttir, for having been there for me when

needed and encouraging me to work hard and becoming the better version of myself in life and

work.

4

Útdráttur

Skynskipti felast í því að bæta upp fyrir skyntap með því að miðla upplýsingum í gegnum heilbrigt

skilningarvit. Á undanförnum árum hefur háþróaður skynskiptibúnaður verið hannaður fyrir

sjónskerta þar sem hljóð- eða snertiskynið er notað til að miðla upplýsingum sem annarst bærust

með sjónskynjun. Gnægð atferlislegra- og taugafræðilegra gagna benda til hentugleika slíks

búnaðar en samt sem áður hefur hann ekki öðlast víðtækar vinsældir utan rannsóknastofunnar,

líklegast af hagfræðilegum-, vinnuvistfræðilegum- og útlitslegum ástæðum. Ennfremur hafa skyn-

og athyglishömlur taugakerfisins oft á tíðum yfirsést í þróun búnaðarins sem bendir til þess að

grunnrannsóknir séu lykilatriði í þróun farsæls skynskiptibúnaðar. Vegna skorts á kerfisbundnum

athugunum á áhrifaþáttum í upplausn snertiskynsins við notkun titringsvaka könnuðum við áhrif

billengdar milli titrara, mismunandi titraragerða auk örvunaráttar á frammistöðu í fjórum

tilraunum. Niðurstöður okkar benda til þess að skynþröskulda skyldi ekki alhæfa milli rannsókna

þar sem mismunandi titraragerðum er beitt og að örvunarátt geti haft áhrif snertiskynjun neðra

mjóbaks við notkun titringsvaka.

5

Abstract

Sensory substitution involves compensation for sensory loss by conveying information through a

functioning sensory modality. In recent years technically sophisticated sensory substitution

devices have been developed for the visually impaired where visual input is substituted for

auditory or tactile input. Even though behavioral and neurological evidence supports the feasibility

of such devices, they have not yet reached widespread popularity outside the laboratory,

presumably due to ergonomic, economic and cosmetic factors. Moreover, the sensory- and

attentional constrains of the nervous system has often been ignored during their development,

highlighting the need for basic research for the development of successful sensory substitution

devices. Due to the lack of systematic research on influencing factors on vibrotactile acuity we

explored the effect of inter-tactor spacing, tactor types and stimulation direction on performance

during point localization tasks in 4 experiments. Our findings indicate that vibrotactile thresholds

should not be generalized between tactor types and that there are considerable spatial asymmetries

in the vibrotactile acuity of the lumbar region.

6

Table of Contents

Preface ..................................................................................................................................................... 3

Útdráttur .................................................................................................................................................. 4

Abstract ................................................................................................................................................... 5

Introduction ............................................................................................................................................. 8

Sensory substitution .................................................................................................................. 9

Sensory substitution through auditory stimulation ................................................................ 9

Sensory substitution through tactile stimulation ................................................................. 10

Neurological evidence on the feasibility of sensory substitution ............................................ 12

Training and cross modal plasticity ..................................................................................... 16

Externalization..................................................................................................................... 17

Outside the laboratory ............................................................................................................. 18

Ergonomics and functionality.............................................................................................. 18

Sensory overload ................................................................................................................. 18

The importance of basic research ............................................................................................ 19

Psychophysics and spatial acuity......................................................................................... 20

Method (experiment 1-3)....................................................................................................................... 24

Participants .............................................................................................................................. 24

Stimuli ..................................................................................................................................... 24

Materials .................................................................................................................................. 25

‘Vibro-sponge’ .................................................................................................................... 25

Tactile vest .......................................................................................................................... 26

Procedure ................................................................................................................................. 27

Statistical analysis ................................................................................................................... 28

Results (experiment 1-3) ....................................................................................................................... 29

Accuracy .................................................................................................................................. 29

Response time ......................................................................................................................... 35

Discussion (experiment 1-3) ................................................................................................................. 37

Method (experiment 4) .......................................................................................................................... 40

Participants .............................................................................................................................. 40

Stimuli ..................................................................................................................................... 40

Materials .................................................................................................................................. 40

ERM (pp) sponge ................................................................................................................ 41

7

ERM (ip) sponge ................................................................................................................. 41

LRA Sponge ........................................................................................................................ 42

Procedure ................................................................................................................................. 43

Statistical analysis ................................................................................................................... 44

Results (experiment 4)........................................................................................................................... 45

Accuracy .................................................................................................................................. 45

Response time ......................................................................................................................... 47

Comparison between tactor types and directions .................................................................... 49

Training effects ....................................................................................................................... 51

Response Time .................................................................................................................... 52

Tactor location and performance ............................................................................................. 53

Effect of tactor rows on performance .................................................................................. 53

Effect of tactor colums on performance .............................................................................. 53

Individual differences .............................................................................................................. 53

Discussion (experiment 4) ..................................................................................................................... 58

General discussion ................................................................................................................................. 60

References ............................................................................................................................................. 62

8

Introduction

Traditionally, it is believed our understanding of the world is derived from information provided

by at least five senses: Sight, smell, hearing, taste and touch. In the eighteenth century the

French philosopher Étienne Bonnot de Condillac held the idea that our mind is comprised of the

aggregation of sensations and the mental impressions they lead to (Condillac & Carr, 1930).

Another eighteenth-century philosopher, George Berkley, went as far as to proclaim that

nothing can exist without being perceived, stating: „the only things we perceive are our

perceptions“ (Berkley, 1874).

Condillac believed that in order to understand our mind and ideas we need to study the

senses separately to infer which ideas or information each sense leads to, how the senses can be

trained and, notably, how they aid one another (Condillac & Carr, 1930). Although

understanding the function of sensory modalities separately and in isolation is informative,

understanding how they work together is equally relevant as nothing exists in isolation. The

world around us is complex and dynamic, and we rely on a multitude of information to make

sense of it. Furthermore, our holistic representation of the world depends as much – if not more

– on inner mechanisms that combine and interpret the multimodal information provided by our

senses, making our brain the most important sensory organ for perception (Kristjánsson, 2012).

Losing the function of a sense due to damage in the peripheral nervous system does not

directly alter the pre-existing structures in the central nervous system designated for

interpretation of the environmental input from the lost sense. In fact, the cortical areas

responsible for the lost sense might be recruited for processing by the remaining intact senses

(Veraart, Wanet-Defalque, Bol, Michel, & Goffinet, 1990). This phenomenon has widely been

known as cross-modal plasticity, a form of neuroplasticity concerning the brain‘s ability to

reorganize itself in response to the environment (May, 2011) on a neuronal- (Mitterauer &

Kopp, 2003), synaptic- (Kercel, 2004), receptor- (Kercel, Caulfield, & Bach-y-Rita, 2003) and

neurochemical level (Weaver, Richards, Saenz, Petropoulos, & Fine, 2013). This neural

flexibility lays the foundation for the vast field of sensory substitution (SS), which applies to

compensating for sensory loss by conveying information through a functioning modality

(Kristjánsson, et al., 2016; Jóhannesson, Balan, Unnthorsson, Moldoveanu, & Kristjánsson,

2016). One of the best examples of SS is the use of Braille by blind and visually impaired

people (VIP), substituting a visual reading system with a tactile one. The white cane is another

example of SS, aiding the VIPs in navigation and obstacle detection via tactile cues. The white

cane can also provide auditory cues when tapped to the ground, producing sounds that can be

9

used for echolocation. In addition to Braille and the white cane, a plethora of devices have been

designed to aid those with sensory impairments, so-called sensory substitution devices (SSD).

As with the use of Braille, evidence suggests that training is an important factor in the

successful use of SSD‘s (Burton, et al., 2002; Stronks H. , Nau, Ibbotson, & Barnes, 2015).

Thus, returning to the topic of Condillac‘s desire to study how the senses can be trained and

how they aid each other, one might ask how the senses can be trained to aid each other. To

answer that question it is crucial to understand the capabilities of the modalities in question as

well as their limitations, making basic psychophysical methods a feasible starting point in the

development of sensory substitution devices.

Sensory substitution

Sensory impairment does not only affect people on an individual level but also society as a

whole, highlighting the importance of efficient SSDs. For instance, visually impaired people

face a number of difficulties in their everyday lives that sighted individuals rarely have to think

about. Everything from making a pot of coffee in the morning to walking to the nearest bus stop

requires a certain amount of preparation and planning so the person can go about their day

unassisted. Consequently, social involvement is often lacking as visually impaired individuals

tend to lead a sedentary, isolated lifestyle (Longmuir & Bar-Or, 2000; Varo, et al., 2003) and

unemployment rates within the visually impaired population are high worldwide, over 60% in

the Nordic countries and over 70% in Central and Eastern European countries (World Health

Organization, 2014). These socio-economical factors are a major loss for society resulting in the

development of both physical and mental health problems and large healthcare costs (Di Cagno,

et al., 2013; Varo, et al., 2003). A lack of adequate assistive solutions is thought to be the main

reason preventing the VIPs from leading a healthy lifestyle and taking an active part in society.

Researchers worldwide have made attempts to develop assistive devices to improve the

mobility and situational awareness of VIPs with some success, however, very few have been

able to develop a product that fulfils the VIPs requirements or has gained widespread popularity

outside the laboratory (Elli, Benetti, & Collignon, 2014).

Sensory substitution through auditory stimulation

Various assistive devices conveying visual information through audio have been designed for

VIPs enabling them to recognize objects, read text and navigate and complete important

everyday tasks.

One such device, the Drishti navigation system, allows the user to ask questions through

a speech recognition component and provides guidance through text-to speech to specific

10

targets. The system relies on GPS localization and pre-existing maps and is therefore only

useful outdoors in locations already available in a database (Helal, Moore, & Ramachandran,

2001). However, the Drishti system does not provide information on obstacles that are not

registered in the database, so the user does not receive warnings about objects that have recently

been placed in the area. Similarly, the SmartGuide system provides information about the

user’s location on a university campus, working both out- and indoors. However, the systems

use is limited to campus areas with predefined sensors (Tee, et al., 2010).

A common drawback to systems such as the Drishti and SmartGuide is the requirement

of a database for information about the environment, or the need for tags or sensors placed in

the location the VIP intends to navigate. Consequently, devices suitable for unknown

environments have emerged as technology has advanced. A popular solution, the vOICe (the

three middle letters stand for "Oh, I See"), converts video images to audio patterns, providing an

auditory representation of the user’s environment. With training, users are able to experience

smooth movement, depth and even colour perception if they have prior visual experience with

the stimuli (Ward & Meijer, 2010) as well as object recognition, even for novel objects never

before “seen” (Auvray, Hanneton, & O'Regan, 2007; Kim & Zatorre, 2008; Meijer, 1992;

Amedi, et al., 2007). Another video-audio SSD, the PSVA (Prosthesis for Substitution of Vision

by Audition) associates a specific frequency with each pixel of the image processed roughly

modelling the human retina (Capelle, Trullemans, Arno, & Veraart, 1998). Not only has visual

pattern recognition been possible with the device (Arno, Capelle, Wanet-Defalque, Catalan-

Ahumada, & Veraart, 1999), but well-known visual illusions have been reproduced with the

device (Renier, et al., 2005).

Sensory substitution through tactile stimulation

In addition to auditory SSD’s, tactual compensation for sensory loss has been studied since the

1960’s and is still a growing field. A great amount of tactile interfaces have been developed to

assist VIP’s in object recognition, obstacle detection, wayfinding and more.

Paul Bach-y-Rita (1969) implemented one of the first and best known studies on tactile-

visual SS in which a dental chair containing 20x20 arrays of tactors conveyed information

obtained by a camera by applying vibration to the user’s back. After receiving 20-40 hours of

training, participants were able to discriminate between horizontal, vertical, curved and diagonal

lines as well as geometric forms. Participants were also able to recognize objects and experience

externalized perception as if the image appeared in front of them. Highly trained observers were

able to report remarkable detail, including the identity of people. They could even determine if

these people were wearing glasses or not.

11

Since Bach-y-Rita’s pioneering study, a plethora of tactual devices have been developed

to compensate for various types of sensory loss. Many of these devices have been designed as

wearables, such as belts or vests lined with tactors. For instance, Van Erp et al. (2005) designed

a vibrotactile belt with 8 tactors placed around the user’s torso to indicate either the direction

that the user had to follow to reach a predetermined destination or the turn necessary to take in

order to move towards this direction. The belt was used in conjunction with a GPS system

guiding the user to certain waypoints enabling them to navigate through a number of different

routes on a university campus site. The tactile display was intuitive and could be used without

extensive practice. A more recent study by Cosgun, Sisbot and Christensen (2014) on the

efficacy of directional and turning motion cues provided by a tactile belt further indicates the

usefulness of tactile devices in navigation. Participants were able to detect patterns with near

perfect accuracy and found it easy to move while wearing the belt.

In addition to wayfinding and navigation, tactile interfaces have also been shown to be

useful in simple obstacle detection. In a study by Johnson and Higgins (2006) participants were

able to detect objects using a vibrotactile belt receiving input from two cameras on a separate

belt, providing stereo information. Similarly, McDaniel et al. (2008) designed a vibrotactile

waist belt with 7 equidistantly spaced tactors on the front of the torso which conveyed

information about the direction and distance to stimuli. Vibration location conveyed information

on direction while vibration duration conveyed information on distance. After training,

participants were able to use the belt to perceive the direction and distance to stimuli, relative to

the user, with a mean accuracy of 95%.

In addition to displays on the torso, application to other areas of the body has been

explored. For example, information can be conveyed with tactile stimulation of the fingertips

(Koo, Jung, Koo, Nam, & Lee, 2008), hands (Ito, et al., 2005) and mouth (Bach-y-Rita,

Kaczmarek, Tyler, & Garcia-Lara, 1998; Tang & Beebe, 2006; Matteau, Kupers, Ricciardi,

Pietrini, & Ptito, 2010).While the fingertips and mouth are some of the most sensitive areas on

the body (Weinstein, 1968), the arms and hands can also be rather sensitive depending on the

location of stimulation. For instance, people tend to localize tactile stimuli with more precision

when they are presented close to anchor points such as the elbow or shoulder joints than

elsewhere on the body indicating that the arms are a highly suitable location for tactile SSD’s

(Cholewiak & Collins, 2003). Sensitivity to tactile stimuli also tends to increase with increased

mobility of the body part in question (Vierordt, 1870, as cited in Cholewiak et. al, 2004).

However, as Cholewiak, Brill and Schwab (2004) argue, the use of the limbs and mouth in

development of SSD’s should be avoided since most everyday activities require free use of the

hands, which a tactile display might inhibit. Furthermore, people tend to use the torso rather

12

than the arms as a reference point to localize themselves in three-dimensional space (Karnath,

Schenkel, & Fischer, 1991), which is crucial for navigation and spatial awareness.

Neurological evidence on the feasibility of sensory substitution

The idea that the intact senses become heightened following visual impairment has prevailed for

years and there are many examples of anecdotal evidence in popular culture. This phenomenon

is often considered to be a compensatory strategy used by the blind to make up for their lack of

vision. There might be a scientific backing to these claims as studies have demonstrated that the

visually impaired perform equally well, or even better than sighted subjects on certain

behavioural tasks such as auditory-pitch discrimination (Gougoux, et al., 2004), spatial sound

localization (Gougoux, Zatorre, Lassonde, Voss, & Lepore, 2005; Lessard, Pare, Lepore, &

Lassonde, 1998; Röder, et al., 1999; Voss, Lassonde, Gougoux, Madeleine, & Guellemot, 2004)

and navigation in a route learning task (Fortin, et al., 2008), speech discrimination (Neimeyer &

Starlinger, 1981), verbal recall (Amedi, Raz, Pianka, Malach, & Zohary, 2003; Roder, Rosler, &

Neville, 2001) and, interestingly for the current purpose, fine tactile-discrimination (Alary, et

al., 2008; Glodreich & Kancis, 2003; Van Boven, Hamilton, Kauffman, Keenan, & Pascual-

Leone, 2000; Alary, et al., 2009). As previously stated, cortical areas responsible for a non-

functional sense might possibly be recruited for processing of input from the intact senses

through cross-modal plasticity (Veraart, Wanet-Defalque, Bol, Michel, & Goffinet, 1990). In

the case of blind individuals, brain imaging studies have shown that the visual cortex is far from

being inactive in the absence of visual stimulation for a variety of tasks. Evidence of cortical

activation in the occipital lobe has been recorded during Braille reading, both in congenitally

blind (Röder, Stock, Bien, Neville, & Rösler, 2002) and early blind (Sadato, et al., 1996)

individuals. Activations have also been found in blind subjects during tactile discrimination

tasks as well as auditory- localization (Weeks, et al., 2000), object discrimination (Arno, et al.,

2001) and during spoken language (Bedny, Richardson, & Saxe, 2015). Perhaps the most

compelling evidence on cross modal plasticity can be found in transcranial magnetic stimulation

(TMS) studies in which behavioural performance has been disrupted through stimulation of the

occipital lobe during Braille reading (Kupers, et al., 2007), verbal generation (Amedi, Floel,

Knecht, Zohary, & Cohen, 2004) and the use of auditory SDD’s such as the PSVA (Collignon,

Lassonde, Lepore, Bastien, & Veraart, 2007) and the vOICe (Merabet, et al., 2009).

Indeed, brain areas customarily regarded as unimodal are likely so only in the sense of

their afferent projections. There are countless examples of multisensory illusions such as the

ventriloquist effect where speech is perceived to originate from a different direction than its true

direction due to the visual influence from the apparent speaker (Bertelson, 1999). The McGurk

effect is another example, in which the perception of phonemes is altered by lip-movements

13

(Massaro, 1999). However, even more interesting is the fact that receiving information about an

event from multiple modalities can result in greater understanding even when only one of the

modalities in question logically applies to the stimulus. For instance, touch itself does not

convey colour information intuitively but can nonetheless improve visual judgments of colour

(Spence, McDonald, & Driver, 2004). Further, in Vroomen and de Gelder (2000) participants

had to localize a specific shape in a visual display. Target detection and localization improved

when coincided with a unique sound, even though the sound provided no information about

visual events. Similarly, in a fMRI study by Macaluso, Firth & Driver (2000) tactile cues

spatially congruent to visual targets enhanced neural responses in the visual cortex.

In light of the growing body of research on perception and visual impairment, the

traditional view that the brain is divided into separate divisions with specific functions has been

substituted by a more multimodal outlook that regards cortical regions as functional-based

rather that sensory-specific (Pascual-Leone & Hamilton, 2001).

Whether the brain is inherently multimodal, with regards to plasticity, or becomes so

after sensory deprivation has been debated (Qin & Yu, 2013) although Pascual-Leone and

Hamilton (2001) maintain that multimodality is a natural characteristic of the cerebral cortex as

it can not only be found in the early- and late blind but also for sighted, but blindfolded

participants. They argue that everyone might possess a visual cortex able to take on different

roles; evidence for this ability however might be masked by visual input in the sighted. In an

fMRI study by Merabet et al. (2008) the occipital cortex in normally sighted people was

activated during Braille reading following five days of blindfolding. The same did not apply to

controls which were not blindfolded; they showed no such occipital activation. On the fifth day

TMS was applied to the occipital cortex during a Braille reading task. Reading was only

disrupted for blindfolded participants, not the controls. Remarkably, it only took around 24

hours after the blindfold was removed for the activation pattern to disappear completely from

fMRI scans and as for the TMS, it no longer had a disruptive effect on Braille reading.

Additional evidence can be found in a study by Striem-Amit et al (2012) where the well known

division of labour between the dorsal and ventral processing streams was utilized. The dorsal

stream is often referred to as the “where” stream due to its role in processing visuospatial

information on object location, visuomotor movement and planning. The ventral pathway on the

other hand is often dubbed the “what” stream since it plays a part in visual processing of object

identity, form and colour (Goodale, 1992). Sighted as well as congenitally blind participants

were trained in using the vOICe system to identify novel auditory stimuli with regard to either

location or form. If the congenitally blind in absence of any visual experience show no

segregation of activation in the dorsal and ventral streams it is likely that visual experience is

14

needed to develop this functional segregation after sensory deprivation. However, if they show

signs of segregation there should at least be biases in connectivity that aren’t visual in nature,

but functionality related. The latter turned out to be true; the congenitally blind did indeed show

signs of segregation in the dorsal versus ventral streams, supporting the view of the inherently

multimodal nature of the brain. The sighted controls in this study also displayed a similar

segregation of activation in the dorsal and ventral streams, although weaker than in the

congenitally blind, suggesting that these multimodal characteristics might not be dependent on

sensory deprivation but strengthened by it (Striem-Amit, Dakwar, Reich, & Amedi, 2012).

Given these points it seems logical to assume that the cortical areas in the occipital lobe that are

generally utilized during visual processing might be recruited for the aforementioned tasks.

However, enhanced abilities and superior performance of the blind when compared to sighted

subjects may depend on the task at hand (Alary, et al., 2009) as well as the age when the

individual becomes visually impaired. It seems that the earlier the person becomes blind, the

better the performance on some tasks (Gougoux, et al., 2004). For instance, although

congenitally blind outperform sighted individuals on some tasks they can be less efficient at

information processing that requires integration across modalities (Collignon & De Volder,

2009). Collignon et al. (2009) compared early- and late blind participants with sighted controls

on lateralizing tactile, auditory and audio-tactile stimuli. The task was carried out in two

conditions: with hands crossed or uncrossed over the body’s midline. By crossing hands the

relationship between peripersonal and personal spatial representations is altered. The early

blind outperformed the sighted and late blind participants in every condition except when

lateralizing auditory stimuli with the hands crossed. By contrast, the performance of the late

blind in the crossed hands condition was similar to the sighted controls. Perhaps, without visual

experience the individual is not able to calibrate kinaesthetic and auditory frames of reference

and construct a common spatial coordinate system. Late blind individuals, however, are more

likely to show a similar pattern as sighed people, presumably because of their visual experience

affecting their frames of reference (Ocelli, Spence, & Zampini, 2013; Cappagli, Cocchi, & Gori,

2015). Furthermore, there are reports on the impairment of auditory spatial localization among

the congenitally blind (Gori, Sandini, Martinoli, & Burr, 2014) and enhanced auditory

perception (Wan, Wood, Reutens, & Wilson, 2010) as well as impaired tactile perception

(Röder, Rösler, & Spence, 2004) only for the early- but not late blind, highlighting the

importance of the task in question and the age when people become visually impaired.

These mixed results may be experience-based since certain abilities might become

enhanced only after a certain amount of training (Alary, et al., 2009). An alternative explanation

is that brain plasticity is optimal at an early age and might not be as effective later on in life

15

(Gougoux, et al., 2004). The latter explanation fits well with research on critical- and sensitive

periods and cerebral development that has demonstrated that certain cortical areas in the

occipital lobe are highly sensitive to visual input early on in life. For example, if visual

deprivation occurs around 6 months of age normal visual acuity or peripheral light sensitivity

will not develop. However, during this period global direction of motion is not affected since it

is the most sensitive around birth and thus only affected if deprivation occurs at that point in

time (Lewis & Maurer, 2005). As previously stated, the visual cortex in the congenitally blind

has been thought to overtake high-level functions such as language (Bedny, Richardson, &

Saxe, 2015) and memory processing (Amedi, Raz, Pianka, Malach, & Zohary, 2003) in the

absence of visual input. Consequently, cortical structure in the occipital lobe has been thought to

differ fundamentally in the blind compared to the sighted (Merabet & Pascual-Leone, 2010).

For instance, there is a common belief that congenitally blind people don’t have the capacity to

process, organize and retain visual input due to a lack of input during sensitive periods resulting

in structural variations when compared to the sighted. However, Striem-Amit et al. (2015) found

that the topographical organization in the visual cortex develops even without visual experience,

and furthermore, can be retained regardless of visual experience based changes. Although

topographical changes have been found in the visually impaired cortex, the changes mainly

pertain to the functional connectivity between the visual cortex and non-visual cortical areas

rather than their structure.

All in all, the full effect of deprivation during critical and sensitive periods cannot be

completely accounted for. Further evidence should be considered, especially since the level and

onset of visual impairment varies throughout different studies as well as the methods employed.

In a PET study by Arno et al. (2001), early blind and sighted subjects showed different

activation patterns while using the PSVA system (Capelle, Trullemans, Arno, & Veraart, 1998)

on a number of tasks, both before and after training. Occipital structures seemed to be recruited

only in the early blind group but not the sighted control group indicating cerebral reorganization

as a result of visual deprivation. Furthermore, stronger primary visual cortex responses were

reported for early rather than late blind during a Braille-reading task (Sadato, Okada, Honda, &

Yonekura, 2002). However, it’s possible that the early and late blind subjects had different

hemodynamic delays, which the method applied did not account for, possibly explaining why

activity was not detected in the late blind (Ollinger, Corbetta, & Shulman, 2001). Higher

activity in the primary visual cortex of late blind subjects has been reported when hemodynamic

responses have been cross-correlated with estimated time-courses (Burton, et al., 2002) making

it difficult to conclude that late blind individuals have less plasticity.

16

Training and cross modal plasticity

Little is known about the exact processes behind cross-modal learning in sensory substitution

and the effect of training has been a rather peripheral topic in the literature concerning SDDs.

Stronks et al. (2015) however, reviewed 20 studies on SSDs and found only one

experiment that did not report a significant training effect. Furthermore they point out the

importance of suitable rehabilitation strategies when training with SSDs and draw the

conclusion that the prognosis of a blind individual relies on the willingness to practice and go

through rigorous rehabilitation.

Moreover, a rigorous training schedule is not necessarily needed to see significant

improvement in performance using SSD’s, although further training should lead to better

performance, at least up to a certain point. Kim and Zatorre (2008) demonstrated that explicit

knowledge on how the vOICe system converts objects into soundscapes results in above chance

performance, similar to that of people who receive short-term training. Receiving additional

training for the duration of three weeks resulted in significantly higher scores where participants

showed signs of generalized learning, applying what they learnt during training to novel

scenarios.

Practice on visual SDDs has been linked to changes in cortical activity, notably in visual

areas. In addition to activation of the dorsal and ventral extrastriate areas during location and

form identification with the vOICe as previously mentioned (Striem-Amit, Dakwar, Reich, &

Amedi, 2012) activation in the same areas of the occipital cortex has been recorded during

pattern recognition before and after training with the PSVA system. Notably, activation

increases after training (Poirier, De Volder, Tranduy, & Scheiber, 2007).

In some cases increased activity in the occipital areas has only been found after

extensive training. In Amedi et al. (2007) the lateral-occipital tactile visual area (LOtv) was

activated after 40 hours of training with the vOICe system. The same pattern was not seen in the

control group which trained for around 6-8 hours.

These findings and the general belief that sensory substitution requires a lot of top-down

attention and effort imply that only by following an extensive training regime will one become

able to use a SDD effortlessly. Stiles and Shimojo (2015), however, argue that the extent of

training might be dependent on certain characteristics of the SSD in question, specifically how

it’s encoding corresponds with pre-existing cross-modal connections. If the encoding of stimuli

is based on evidence on cross-modal mapping between the relevant cortical areas involved (e.g.

visual-auditory) and is intuitive (e.g. scan from left to right similar to reading) it might be

possible to reduce the cognitive load placed on the individual resulting in a more bottom-up

17

understanding. By doing so it might be possible to shorten training time and reduce strain on the

user.

Externalization

The ultimate goal of sensory substitution is externalization: an effortless comprehensive

understanding of the environment, much like vision, that doesn’t require much thought or

interpretation. Ever since Bach-y-Rita’s (1969) pioneering experiment there have been reports

of SDD users feeling as though the sensation from the device comes from the environment

itself. Generally externalization has been reported when users have control over the SDD and

actively use it (White, Saunders, Scadden, Bach-y-Rita, & Collins, 1970; Prolux, Stoerig,

Ludowig, & Knoll, 2008; Ward & Meijer, 2010). In Bach-y-Rita (1972) participants only

reported externalization after training if they had absolute control of the camera delivering input

to the vibrotactile SDD. In a more recent study by Prolux et al. (2008) on the vOICe system the

performance of subjects with different levels and types of training was compared during object

detection. The performance of those who could freely use the system was higher than of those

who could not, indicating that training in a naturalistic setting improves performance more than

in a testing environment.

As can be seen it’s not only the amount of training but also the level of

activation during training that affects people’s fluency using an SDD. A probable

explanation is that learning how to use a visual SSD largely concerns matching the

devices view of the environment with one’s own egocentric body schema. Active or

self-produced movement is important for sensory-motor learning (Held & Hein, 1963)

as reafferent motor signals might contribute to optimized sensory-motor learning

through memory-based prediction (Mountcastle, 1978). That is, we might be able

correlate our physical movement with objects in the environment to anticipate changes

in sensory input. These signals could possibly enable us to perceive stimuli in our

environment as constant despite dynamic changes. For example, the perceived size of an

object does not change even when the size of said object changes on the retina, within a

certain range. Perceptual constancies are important to our understanding of the world as

it would be ever-changing without them. Although there have been uncertainties about

obtaining constancy within the field of sensory substitution, Stiles, Zheng and Shimojo

(2015) demonstrated that people are able to learn constancies using a SDD where

performance improved significantly with training. Interestingly the number of

spontaneous head movements was highly correlated with performance improvement,

18

further indicating that self-produced movement is the key to externalisation. Moreover,

unrestricted head movements should be coupled with proprioceptive feedback during

training for maximal efficiency. Actively matching environmental stimulation with

one’s own movement and body schema has been shown to be the most effective training

method for sound localization without visual feedback (Hodna, et al., 2013).

Outside the laboratory

Despite behavioural- and neurological evidence on the feasibility of sensory substitution for the

blind and successful demonstrations of available systems, their use has more or less been

limited to the laboratory. Nearly five decades have passed since Bach-y-Rita’s (1969) famous

experiment yet for some reason VIPs tend to rely on conventional aids such as the white cane or

guide dog but do not use the many sophisticated assistive devices that have been developed. .

Ergonomics and functionality

Certain ergonomic standards have to be fulfilled so that devices will appeal to the VIP’s. They

should be comfortable, unobtrusive and compact, making it possible to carry the device for

longer durations without restricting the user’s mobility or arm movement (Elli, Benetti, &

Collignon, 2014; Dakopoulos & Bourbakis, 2010). Additionally, cosmetic appeal is important to

VIP’s, and is sometimes deemed as more important than the potential benefit provided by the

SDD (Golledge, Klatzky, Loomis, & Marston, 2004).

A survey by Dakopoulos and Bourbakis (2010) indicates that no single system meets all

the aforementioned criteria and the visually impaired are confident about. However they point

out that long-term experiments have not been carried out yet and most systems are still in the

prototype stage.

In addition to functional and ergonomic issues, commercial access to technically

advanced SDDs is usually limited and although the cost varies it can be fairly high. Pricing is

especially an issue considering that the majority of the blind population lives in low income

settings (WHO, 2014). Furthermore, advantages of SDDs over familiar aids have not been

clearly demonstrated, while aids such as the white cane are both familiar and effective and

appear less complex than SDDs currently being developed (Elli, Benetti, & Collignon, 2014).

Sensory overload

Another key point to realize is that the amount of information the human nervous system can

process at any given moment can never exceed its attentional capacity. Filtering processes

reduce mental load by directing our attention to certain stimuli in the environment, preventing

19

sensory overload (Broadbent, 1958). Although this characteristic is extremely useful it can also

pose several perceptual limitations which have to be kept in mind when developing SSDs.

For one, the visually impaired rely heavily on environmental sounds which should

therefore not be obstructed during sensory substitution (Härmä, et al., 2004). Second,

considering that VIPs are likely to employ certain adaptive strategies (Collignon & De Volder,

2009) and compensate for their lack of vision with their intact senses extreme care should be

taken not to obstruct them. Thirdly, sensory substitution can demand a lot of attentional and

cognitive recourses which could limit the attention available for the processing of environmental

stimuli (Kärcher, Fenzlaff, Hartmann, Nagel, & König, 2012), although with time and practice

the strain could be reduced as use becomes more automatic. Furthermore, the bandwidth and

processing capacity differs between senses. The visual system exceeds the other senses in

bandwidth (Jacobson, 1951), with the optic nerve consisting of around one million neural fibres

(Wurtz & Kandel, 2000); the skin has the lowest capacity (Kokjer, 1987) and the ear is

somewhere in between the two (Jacobson, 1950) with around 30.000 fibres in the auditory nerve

(Wurtz & Kandel, 2000). Therefore, the capabilities of the modalities have to be taken into

account when information usually delivered to one is conveyed through another.

Altogether, ignoring sensory- and attentional constraints of the nervous system might be

the fundamental reason for the limited use of SSDs outside of the laboratory. Basic research on

these issues is crucial to the successful development of assistive devices if they are to gain

popularity and be widely used among the blind community (Loomis, Klatzky, & Guidice,

2012).

The importance of basic research

As evident in Condillac‘s (Condillac & Carr, 1930) ideology, it is neccessary to understand the

different sensory modalities separately before we can understand how they work together and

aid one another. The need for basic research within the field of sensory substitution is great and

has often been underestimated or neglected (Kristjánsson, et al., 2016). The issue is highlighted

by Loomis et al. (2012) where a theoretical framework for task-specific SSD development is

provided. First and foremost, the SSD has to match the capabilities of the human nervous

system. Moreover, different tasks require different types of information and some sensory

modalities are better suited for certain processes. These factors are often overlooked during

development resulting in unsuccessful sensory substitution.

20

Psychophysics and spatial acuity

Psychophysical experimentation has been a useful tool in understanding the link between

sensation and perception since the 19th century and its methods are still employed to this day.

Originating with Ernst H. Weber’s experiments on touch, psychophysical methods were first

properly defined in Gustav Fechner’s Elemente der Psychophysic in 1860 in which he laid the

theoretical basis of psychophysics demonstrating that the mind can indeed be linked with

physical events (Fechner, 1966).

Weber’s tactile experiments were published in De tactu in 1834 (Ross, & Murray, 1996)

and Der Tastsinn und das Gemeingefül in 1846 (Ross & Murray, 1996) in which he – amongst

other things – explored two-point discrimination thresholds (2PT) on numerous body parts. By

applying a compass to the skin and varying the distance between its two ends, he recorded the

subject’s perceived distance between the two points. By doing so he was able to estimate two-

point thresholds where two stimuli are, incorrectly, perceived as one. Additionally he studied

points of localization (PL) by presenting stimuli one after another, either at the same location or

with some distance in between. By increasing the distance between the stimuli he determined

the thresholds where two stimuli are, correctly, perceived as two. From these experiments he

drew the conclusion that different areas of the body have different sensitivity with regard to the

spatial separation of two points of contact presented simultaneously.

Weinstein (1968) picked up where Weber left off, speculating whether sensitivity

differs between the left and right sides on various areas of the body, and between the genders.

Like Weber, he found differences in sensitivity for different areas of the body where the

fingertips were the most sensitive at a 3-4 mm discrimination threshold and the thighs/calves

were the least sensitive at over 40 mm. Furthermore, he found that measuring acuity with PL

yields lower tactile thresholds than when the 2PT method was used, although the results were

highly correlated.

One key point to realize is the different results produced by testing relative localization

such as the 2PT and PL versus absolute localization where points of stimulation are matched

with an absolute frame of reference. For instance, participants in Cholewiak et al. (2004) wore a

vibrotactile waistbelt with tactors located equidistant around the torso and had to match

vibrations on their body to corresponding locations of a clock. Accuracy was higher when the

array contained fewer rather than a larger number of tactors even inter-tactor spacing was

controlled for. A possible explanation is that there are stronger cognitive factors involved in

absolute compared to relative localization and thus cognitive load increases with the increasing

number of tactors. Furthermore, they found that localization accuracy also varies depending on

21

which are of the body is stimulated. Localization on the forearm tends to become more accurate

the closer the stimulation point is to the elbow. These findings are consistent with Vierordt’s

law of mobility which states that sensitivity to location or separation of stimuli increases with

the mobility of the body part (Vierordt, 1870, as cited in Cholewiak et. al, 2004). .

Although most agree on these principles regarding location and sensitivity, different

methods of stimulation (Loomis & Collins, 1978), as well as different psychophysical methods

(Loomis, 1981) have resulted in different estimates of thresholds of the skin. Cholewiak et al.

(2004) point out that Weinstein’s (1968) measurements were based on pressure pulses or

touches. Had he used vibration instead of touch, the results might have been different.

Furthermore, the reason for differences in sensitivity on different locations has been linked with

the distribution of the cutaneous receptors underlying the stimulated area as well as the size of

the corresponding cortical regions devoted to the processing of these signals (Kaas, Nelson, Sur,

Lin, & Merzenich, 1979). Applying vibration to the skin can stimulate tissue further away from

the point of contact than pressure pulses, and thereby recruits a larger number of receptors as

well as their corresponding cortical areas. This could possibly result in different neural

interpretation of the stimuli and therefore different acuity than when this is measured with

pressure pulses. Moreover, different tactor types might cause different wave propagation on the

skin depending on how the vibration is generated by their axis of movement, possibly

stimulating different mechanoreceptors, although there has been no systematic exploration of

the topic. Another important factor to consider is stimulus frequency which can affect

vibrotactile threshold measurements, depending on the location of the stimulated area, tactor

type and size (Morioka, Whitehouse, & Griffin, 2008). For instance, vibrotactile discrimination

on hairy skin likely depends on the activation of superficial receptors for lower frequencies

while it depends on deeper receptors for higher frequencies (Mahns, Perkins, Sahai, Robinson,

& Rowe, 2006). Thus, threshold measurements should not be generalized from one method of

stimulation to another (e.g. vibrotactile vs. touch pulses), even when the stimuli are of the same

nature (e.g. different tactor types), highlighting the importance of the systematic study of tactile

acuity.

Additionally, there are several reports on orientation sensitivity of tactile acuity.

Discrimination of vertical and horizontal stimulation is more accurate than stimulation of a

diagonal orientation; a phenomenon referred to as the oblique effect first reviewed for the visual

modality by Apelle (1972). Since then similar findings have been reported for tactile stimulation

(Lechelt, 1976; Lechelt & Veranka, 1980). Moreover, horizontal sensitivity is greater than

vertical sensitivity which suggests that orientation (or direction) is an important factor

influencing tactile acuity (Lechelt, 1988). The effect of stimulation direction in relative

22

localization has not been addressed systematically and results on the matter have been mixed.

Van Erp (2005) found no differences in vibrotactile acuity between horizontal and vertical

stimulation with an inter-tactor spacing of 20 measured centre-to-centre, using a curve fitting

procedure to estimate acuity instead of direct measurements. Luomajoki and Moseley (2011)

however applied pressure pulses to the lumbar region of participants and found that vertical and

horizontal 2PT thresholds differed significantly. They also found a higher variability amongst

subjects for vertical than horizontal discrimination.

Other factors that can influence tactile acuity include temporal and spatial effects as

well as interactions between them. For instance, percepts of stimuli can be altered by

manipulating their temporal proximity. When vibrotactile stimuli of the same frequency are

separated by a 100-500 ms interval the subjective magnitude of the second stimulus can be

enhanced (Verrillo & Gescheider, 1975). The second stimulus can also be masked if the interval

between the stimuli is from 100-1200 ms when they are in the same location and of the same

frequency (Craig & Evans, 1987). Furthermore, adaptation can occur after prolonged

stimulation and leads to higher discrimination thresholds and lower subjective magnitude

(Capraro, Verrillo, & Zwislocki, 1979). It is possible to prevent such temporal effects by

altering between frequencies below 80 and above 100 Hz (Capraro, Verrillo, & Zwislocki,

1979), although it should be kept in mind that different mechanoreceptors are sensitive to

different frequencies which has to be kept in mind when preventing temporal effects. Spatial

effects include spatial masking where the location of a stimulus is masked by another stimulus

that is presented at the same time at a different location (Verrillo & Gescheider, 1975). Two

stimuli presented at the same time can also lead to a percept of a single stimulus occurring in

between the two (Sherrick, Cholewiak, & Collins, 1990). Finally, by manipulating vibration

duration and vibration intervals it is possible to evoke apparent motion as long as the bursts of

vibrations are longer than 20 ms (Hahn, 1966). The cutaneous rabbit is an illusion induced by

such spatio-temporal manipulations where a train of taps at two locations can lead to phantom

impressions between the two locations resembling a rabbit hopping (Gerald & Sherrick, 1972).

Given these point it’s not surprising that various thresholds and inter-tactor spacing

have been reported for vibrotactile devices through the years. Van Erp (2005) estimated spatial

acuity at 20-30 mm while participants in Novich and Eagleman (2015) were only able to reach

>80% accuracy if the tactors were at least 60 mm apart on the vest used in the study. Others

have utilized elastic waist belts (Edwards, et al., 2009; Cosgun, Sisbot, & Christensen, 2014),

making it difficult to estimate inter-tactor spacing since it’s highly unlikely that all participants

in said studies had the exact same waistline measurements.

23

Due to the lack of systematic research on vibrotactile acuity and the comparison of different

tactor types it is difficult to formulate general guidelines for the development for vibrotactile

devices. Our aim was to shed light on the matter and explore the effects of inter-tactor spacing

as well as stimulation direction on accuracy and response time with three different tactor types

in four experiments.

All experiments were carried out as a part of the Sound of Vision project (SoV) which aims to

develop and test an assistive device for the visually impaired and blind. The SoV device is

intended to consist of cameras which reconstruct a 3D image of the environment in real time

and custom-made software which analyzes the image and identifies environmental stimuli such

as objects and people. This information is then conveyed to the user through auditory and tactile

stimulation.

The first developmental prototype of the tactile component of the device was a tactile vest with

an array of 8x8 tactors with 40 and 52 mm inter-tactor horizontal and vertical spacing,

respectively, on the inner layer of the back. To estimate the number of tactors that could be

placed on the vest it was assessed whether people could distinguish between two side-by-side

tactors on the vest. If the inter-tactor spacing is to narrow, people are not able to distinguish

between two tactors, resulting in fewer potential ways of conveying information with the vest.

Due to the time it took to assemble the vest as well as manufacturing costs it was decided to put

3x3 tactors on a sponge which people could place on their back and secure by tying straps

around their waist. This solution was inexpensive and not as time consuming in construction as

the vest. Furthermore it was relatively easy to manipulate the distance between tactors on the

sponge, making it a suitable instrument to estimate how closely tactors can be placed before

people stop feeling a difference between two adjacent tactors.

24

Method (experiment 1-3)

Three experimental sessions were carried out to explore the effect of inter-tactor spacing on

tactile acuity during a point localization task using two different vibrotactile devices; a tactile-

vest (experiments 1b, 2b, 3b and 3c) and a sponge (experiments 1a, 2a and 3a) laced with

tactors, referred to as a vibro-sponge. Inter-tactor spacing on the vibro-sponge was manipulated

throughout the experiments to assess how close the tactors could be placed to each other with

regard to performance, while it was kept constant on the vest in order to ensure homogeneity in

tactile acuity between experiments as the sample varied from experiment to experiment. An

additional task was carried out with the tactile vest in experiment 3c to assess the effect of

structural differences on acuity.

Participants

Each experiment included 10 participants (5 males and 5 females), who were either students or

staff members at the University of Iceland. The mean age of the participants in experiment 1

was 29,2 years (range = 20-38 years), 24,3 years in experiment 2 (range = 20-31) and 26 years

in experiment 3 (range = 22-31 years). Four participants took part in both experiment 1 and 3

and an additional four participated both in experiment 2 and 3. However, they – as well as all

the other participants – were naïve about the purpose of the experiments and 1-2 months passed

between each experiment. All participants gave a written informed consent before participating.

The experiments were approved by the National Bioethical Committee of Iceland (VSN-15-

107).

Stimuli

Each trial in the point localization task included a pair of tactors. Either the same tactor was

activated twice or two adjacent tactors were activated in succession. If two separate tactors were

activated instead of the same one twice, the second tactor activated was the closest tactor on the

right or left side of the first tactor. Participants had to judge whether the second vibration was at

the same location as the first one (by pressing the spacebar), or if it was to the left or right of the

first one by pressing the left or right arrow keys respectively. The tactors on each trial were each

activated successively for 200 ms with a 50 ms interval. A combination of a 800 ms waiting

time and a random interval of 300-900 ms (in 100 ms steps) passed between trials. The order

and placement of stimuli pairs was randomized.

25

Materials

Both the tactile vest and the vibro-sponge were equipped with coin cell tactors with an eccentric

rotating mass (ERM) producing vibrations in plane with the skin, 8 mm in diameter for the

tactile vest and 10 mm for the vibro-sponge respectively. All experiments were run in PsychoPy

(Peirce, 2007; 2009) on a PC to which the tactile vest and vibro sponge were connected. The

tactors were controlled with an Arduino Uno microcontroller with custom made software.

‘Vibro-sponge’

The vibro-sponge was composed of a sponge placed on a wooden frame, an electronic circuit

board and an array of 3x3 tactors. The inter-tactor spacing measured center-to-center was 30

mm in experiment 1a, 20 mm in experiment 2a and 10 mm in experiment 3a. The sponge itself

was placed on the centre of the lumbar region of participant’s backs, who then secured it by

tying two straps around their waists. Participants were asked to have only one layer of clothing

between their skin and the sponge to ensure that the vibrations would not be diluted through

thick fabric. A total of 135 trials were conducted with the sponge, where 9 combinations of

vibration pairs, each presented 15 times.

Figure 1. The vibro-sponge with a 30 mm inter-tactor spacing.

26

Tactile vest

Although the main goal was to explore differences in acuity as the inter-tactor spacing on the

vibro-sponge gradually decreased from experiment to experiment, it was necessary to ensure

that the differences in acuity between experiments could not be accounted for by individual

differences between the different participant groups in the different experiments. To control for

this it we used a ‘tactile vest’ with constant inter-tactor spacing in all experiments. If accuracy is

similar between groups using the vest, changes in accuracy between groups using the vibro-

sponge cannot be explained with individual differences in tactile acuity.

The vest came in two sizes, one large with a straight cut and one with a curved waistline

intended for those with a smaller waistline. The fabric on the shoulders as well as overall

tightness could be adjusted to suit the needs of each participant, ensuring a good fit regardless of

body type. An inlay containing 8x8 tactors was placed on the inside of the vests covering the

scapular areas to and including the lumbar regions on the participant’s back. Inter-tactor spacing

measured centre-to-centre horizontally was 40 mm in all three experiments. Vertical inter-tactor

spacing was 52 mm. A total of 576 trials were conducted with the whole vest, where 144

combinations of vibration pairs were repeated 4 times each.

27

Figure 2. The tactile Vest with a 40 mm inter-tactor spacing

Due to structural differences an additional task was conducted using a 3x3 tactor array

of the vest. The array was located on the lumbar region of each participant, just like the vibro-

sponge. Thus it was possible to compare accuracy between the vest and sponge and explore the

effect of structural differences. If the accuracy rates turned out to be similar using both devices,

differences in accuracy using the sponge could not be explained by structural differences of the

devices used.

Procedure

All experiments followed the same procedure; except for experiment 3 which included an

additional point localization (PL) task (experiment 3c). Each participant received an email

invitation to take part in an experiment and was provided with an online link to book an

appointment. Upon arrival an explanation of the PL task was provided and the informed consent

form was signed. Thereafter, participants sat at a desk in a private room and carried out the PL

task with both devices after receiving instructions on how to respond. Half the sample began

with the sponge and half with the vest, determined randomly. During the experiment, white

noise was played in headphones to cancel out the noise generated by the tactors to prevent

participants from using auditory cues when solving the task. To summarize, the experiments

28

were all conducted in the same manner, the main difference being the inter-tactor spacing on the

vibro-sponge. Furthermore, an additional task using the vest was carried out in experiment 3c

with a smaller array of tactors activated than the former task with the vest.

For convenience, the experiment numbers 1, 2 and 3 stand for the three different

occasions the experiments were carried out and thus also the three different inter-tactor spacings

used on the vibro-sponge in addition to the different samples used. Experiments carried out with

the vibro-sponge are identified with “a” (experiment 1a, 2a and 3a), experiments carried out

with the full (8x8) array of the vest are identified with “b” (experiment 1b, 2b and 3b) and the

experiment using only a smaller section (3x3) of the vest is identified with “c” (experiment 3c).

Correspondingly, participants in experiment 1 completed the point localization task both with

the sponge (1a) and the full vest (8x8) (1b). On a separate occasion experiment 2 was carried

out where different participants also completed the point localization tasks with the sponge (2a)

and the vest (8x8) (2b). The third occasion, experiment 3, likewise included the sponge (3a) and

the full vest (8x8) (3b) as well as a smaller section of the vest (3x3) (3c).

Statistical analysis

Before analyzing the response time data, all trials where the response was incorrect were

removed, as well as those where response time deviated more than three SD’s from the mean

RT of each participant as well as those who were less than 100 ms. A one-sample t-test was

conducted to assess accuracy rates and a repeated measures ANOVA (RCoreTeam, 2015) was

conducted to explore the influence of the location of tactor rows on accuracy and response time

as well as gender differences. Furthermore, performance throughout each experiment was

analysed to estimate whether participants’ performance improved with practice. All analysis

were conducted in R (RCoreTeam, 2015) in RStudio (RStudioTeam, 2015). Tukey’s honest

significant difference test was used for all post-hoc comparisons (RCoreTeam, 2015).

29

Results (experiment 1-3)

Accuracy

The accuracy rates in all three experiments were significantly higher than chance in all cases,

for both devices as displayed in Table 1. No significant gender differences were found during

the experiments for accuracy, p > .19 in all cases.

Table 1. Accuracy rates in experiment 1-3 for all conditions along with t-tests on whether

accuracy was significantly higher than chance level

Accuracy t-test

Experiment Device Spacing Mean SD df t p

Experiment 1a Sponge 30 mm 0.91 0.29

9 37.91 < .001


9 12.73 < .001


9 9.04 < .001

Experiment 1b Vest (8x8) 40 mm 0.70 0.46

9 13.10 < .001


9 11.45 < .001


9 9.52 < .001

Experiment 3c Vest (3x3) 40 mm 0.75 0.43

9 13.36 < .001

A post hoc comparison revealed that the difference between accuracy rates obtained with the 30

and 20 mm spacing of the vibro-sponge was not significant. However, as seen in Table 2, the

mean accuracy rates with 30 and 20 mm spacing in experiment 1 were and 2 were significantly

higher than those with a 10 mm spacing in experiment 3.

Table 2. Post hoc comparison of mean accuracy score obtained with the vibro- sponge in

experiments 1-3

Experiment

Difference

95% CF

p-value Devices Lower Higher

Exp1a – Exp2a Sponge - Sponge 0.089 -0.058 0.236 .526

Exp1a – Exp 3a Sponge - Sponge 0.272 0.125 0.419 < .001

Exp2a – Exp3a Sponge - Sponge 0.183 0.036 0.330 .006

30

Figure 3 displays the accuracy of each participant in experiment 1a where the sponge was used

with a 30 mm inter-tactor spacing. The high accuracy does not leave a lot of room for variability

between participants.

Figure 3. Accuracy by participant in experiment 1a using the vibro-sponge

with a 30mm inter-tactor spacing. The error bars represent between subject

standard errors of the mean (SEM)

In experiment 2a when the sponge was used with 20 mm inter-tactor spacing, variability

between participants was higher than in experiment 1a. Figure 2 shows that accuracy was

numerically lower in experiment 2a than 1a, although the difference in mean accuracy was not

significant, as shown in table 2.

31




In experiment 3a when the vibro-sponge was used with a 10mm inter-tactor spacing mean

accuracy was significantly lower (see table 2) than in experiments 1a and 2a with, 30mm and

20mm inter-tactor spacing respecitvely. Figure 5 displays the accuracy of each participant in

experiment 3a, which shows greater variability than in experiment 1a and 2a.

32




A post hoc analysis comparing average accuracy rates with our control condition, the vest (8x8),

between experiments did not result in significant differences as seen in Table 3. Furthermore the

differences were always small.

Table 3. Post hoc comparison of mean accuracy scores obtained with the Vest (8x8) in

experiments 1-3

Experiment

Difference

95% CF


Exp2b – Exp 1b Vest (8x8) 0.050 -0.093 0.202 .917

Exp 3b – Exp 1b Vest (8x8) 0.062 -0.085 0.209 .853

Exp 2b – Exp 3b Vest (8x8) 0.008 -0.140 0.155 .999

When accuracy rates of the vest (8x8) were compared with those obtained with the vibro-sponge

post hoc, two significant differences in accuracy were found. Table 4 shows that participants in

experiment 1 acquired higher accuracy rates when using the sponge than the vest. Furthermore,

using the vibro-sponge in experiment 2a resulted in higher accuracy rates than when the vest

33

was used in experiment 1b. No other post hoc comparisons of accuracy scores with the tactile-

vest (8x8) and vibro-sponge were significant, they were p > 0.16 for all instances not displayed

in Table 4.

Table 4. Post hoc comparison of mean accuracy score obtained with the vibro- sponge in and

tactil-vest (8x8) in experiments 1-3 when the difference was significant.

Experiment

Difference

95% Confidence interval


Exp1a – Exp1b Sponge – Vest 0.204 0.057 0.351 .001

Exp2b – Exp1a Vest – Sponge – 0.149 – 0.297 – 0.002 .043

To estimate whether the size of the area stimulated and therefore a larger number of tactors had

an effect on performance we compared the full (8x8) array of the vest with a smaller section

(3x3). As shown in Table 5, mean accuracy rates with the 3x3 array of the vest did not differ

significantly from those obtained with the 8x8 array of the vest in a post hoc comparison.

Table 5. Post hoc comparison of accuracy score obtained with the Vest (8x8) and the vest (3x3)

Experiment

Difference



Exp3c – Exp3b Vest (3x3) – Vest (8x8) -0.011 -0.158 0.136 .999

Exp3c – Exp2b Vest (3x3) – Vest (8x8) -0.003 -0.150 0.144 .999

Exp3c – Exp1b Vest (3x3) – Vest (8x8) 0.051 -0.096 0.198 .934

As stated before, experiment 3c was first and foremost conducted to estimate whether structural

components of the vest and sponge might account for the differences in accuracy found with the

two devices. If accuracy obtained with the vest with the same amount of tactors activated (3x3)

in the same location on the body (the lumbar region) differed between the two devices, we could

infer that inter-tactor spacing is not the only variable affecting performance. Table 6 shows that

the average accuracy rate using the 3x3 array of the vest in experiment 3c did not differ

significantly from those obtained with the sponge in the same experiment (3a) or in experiment

2a. However, when compared with the results from experiment 1a using the vibro-sponge, a

significantly lower mean accuracy was found when the 3x3 area of the vest was used in

experiment 3c than the sponge in experiment 1a.

34

Table 6. Post hoc comparison of accuracy score obtained with the full array of the vest (8x8)

and the smaller array of the vest (3x3)

Experiment

Difference



Exp3c – Exp1a Vest (3x3) – Sponge -0.153 -0.300 0.006 .036

Exp3c – Exp2a Vest (3x3) – Sponge -0.064 -0.211 0.083 .834

Exp3c – Exp3a Vest (3x3) – Sponge 0.119 -0.028 0.266 .193

The main effect of repetition on accuracy was never significant in any of the experiments,

neither with the vibro-sponge nor the tactile vest (p > .26 in all cases).

Tactor rows, the vertical location of tactors on the back, never had a significant effect on

accuracy when the vibro-sponge was used. However, the main effect of tactor rows on accuracy

was significant in all cases when the vest was used, both for the full vest (8x8) and when the

3x3 array was used. In detail, F(7,63) = 2.48, p = .021 in experiment 1b with the full vest,

F(7,63) = 2.05, p = .008 in experiment 2b with the full vest and F(2,63) = 2.68, p = .17 in

experiment 3b with the full vest. Finally, F(2,18) = 4.84, p = .21 in experiment 3c where the 3x3

array of the vest was used.

However, when post hoc comparisons between the effect of tactor rows on accuracy were

conducted, no significant differences were found in experiments 1b, 2b and 3b where the full

array (8x8) of the vest was used, with p > .32 in all cases. In contrast, there was a significant

difference in accuracy between the top and bottom rows in experiment 3c where the (3x3) array

of the vest was used, p = .026. The difference in accuracy between the bottom and middle row

was not significant, p = .092, and neither was the difference in accuracy for the top row and

middle row, p = .062, when the sponge (3x3) was used.

35

Response time

Figure 6. Response time (ms) as a function of experimental condition. The

error bars represent 95% confidence intervals.

Response times were similar in all experiments as figure 6 shows. The main effect of inter-

tactor distance on response time was not significant (F(6, 63) = 1.46, p = 0.207). Table 7 shows

that repetition led to shorter response times in all experiments where the full array of the vest

(8x8) was used as well as when the vibro-sponge was used in experiment 3.

Table 7. Response times in experiment 1-3 for all conditions, including F-tests on the effect of

increased repetitions on response time. The number of trials removed before analysis is also included.

Removed

RT (ms)

F-test

Exp Device Incorrect Outliers Mean SD df F p

1a Sponge 125 29

615 196

(14,126) 1.08 .381

2a Sponge 245 28

741 291

(14,126) 1.26 .243

3a Sponge 492 17

654 240

(14,126) 1.91 .031

1b Vest (8x8) 1710 83

689 230

(3,27) 17.25 < .001

2b Vest (8x8) 1396 98

742 241

(3,27) 17.12 < .001

3b Vest (8x8) 1351 84

596 184

(3,27) 6.27 .002

3c Vest (3x3) 345 27

641 269

(15,135) 1.54 .101

36

The main effect of tactor rows on response time was not significant in experiments 1a, 2a and 3a

when the sponge was used, p > .075 in all cases. The same applies to experiments 2b (p = .12)

with the vest (8x8) and 3c with the vest (3x3) (p = .078). However in experiment 1b and 3b

where the full (8x8) array of the vest was used the main effect of rows on response time was

significant, where F(7,63) = 2.5, p = .025 in 1b and F(7,63) = 2.66, p = .01 inin 3b. When post

hoc comparisons between tactor rows were conducted for experiments 1b and 3b with the vest

(8x8) the differences were not significant, p > .99 in both cases.

Finally, no significant gender differences were found during the experiments for response time

with p > .06 in all instances.

37

Discussion (experiment 1-3)

We assessed vibrotactile acuity for three different inter-tactor spacings with a point localization

task in three experiments using a vibro-sponge: a sponge with an array of 3x3 ERM tactors.

Due to the fact that differnt observers took part in experiments 1-3 it was neccecary to include a

control condition, which in this case involved performing the same task as with the sponge with

a tactile-vest in development for the SoV project. The vest had an array of 8x8 tactors and a

constant 40 mm inter-tactor spacing. If accuracy using the vest turned out to be similar between

groups it should be safe to conclude that their tactile acuity does not vary a lot, allowing for

between group comparisons on performance with the vibro-sponge in different conditions even

though the inter-tactor spacings is manipulated from experiment to experiment. Indeed, there

was no such differences between the samples in experiment 1-3, and thus we concluded that

they were comparable and that the differences for the sponge (30, 20 and 10 mm spacings)

reflect tactile sensitivty rather than simple group differences.

Accuracy using the vibro-sponge with inter-tactor spacing of 30 and 20 mm was higher

than when the spacing was 10mm. However, accuracy was in all cases above chance level

indicating that vibrotactile acuity of the lumbar region is at least 10 mm. The difference in

accuracy between 20 and 30 mm was less than the difference between 10 and 20mm suggesting

that difficulty in distinguishing between adjacent tacors increases with increased proximity to

the vibrotactile threshold. The accuracy of 0.64 found with the 10 mm spacing, although above

chance level, might not be considered adequate, but this depends on the context of where this

will be applied. For example, if important information is to be delivered it‘s neccecary to ensure

that as little information is lost as possible which would in turn probably require more than 10

mm spacing between tactors. These findings suggest that although the threshold is at least 10

mm it might be preferable to keep inter-tactor spacing larger than 10 mm with regards to

accuracy, possibly somewhere between 10-20 mm.

Interestingly, accuracy was generally higher with the vibro-sponge than the vest even

though the vest had a larger distance between tactors. Furthermore, accuracy using the 3x3 array

of the vest was not significantly lower than accuracy obtained with the full (8x8) vest array,

indicating that the difference in accuracy found between the vest and sponge is not a result of

the number of locations or the size of the stimulated area, nor a higher number of tactor pairs

applied during the experiments with the vest. Accuracy with the 3x3 array of the vest in

experiment 3c did not differ significantly from accuracy obtained with the vibro-sponge in the

same experiment, although it was lower than the accuracy with the sponge in experiment 1a,

38

suggesting that inter-tactor spacing is not the only factor influencing vibrotactile acutity.

Furthermore, accuracy and response time tended not to be influenced by the vertical location of

the tactor row the stimuli were presented in.

Together, these results suggest that the differences in accuracy found between the

sponge and full (8x8) array of the vest stem from structural differences of the devices and

perhaps tactor diameter. For one, the sponge presses the tactors closer to the users back due to

its thickness and flexibility, ensuring close contact with skin regardless of physique. Second, the

nature of its fabric is less likely to allow vibrations to spread resulting in a more precise

stimulation of the skin. Finally, the tactors on the sponge are not covered with fabric unlike the

vest.

It seems that along with inter-tactor spacing, the structure of the device used influences

how readily people distinguish between two adjacent tactors and therefore the number of

potential ways of conveying information with tactors.

No differences between the genders were found in any of the experiments, neither for

accuracy nor response time, which is interesting due to the contradiction with Weinsteins (1968)

reports on gender differences in two-point thresholds of pressure pulses. Gender differences in

vibrotactile acuity might therefore be more subtle than for pressure sensitivity, or even

nonexistent.

Response time was similar in all experiments and conditions suggesting that the

difficulty of the tasks was similar. However in experiments using the vest, response time

decreased with repetition although accuracy did not, which indicates that participants spent less

effort and became better at responding during the task, not at sensing the actual difference

between the tactors. A higher number of trials and possibly multiple sessions would be needed

to assess training effects on vibrotactile acuity.

There are several obvious limitations to these experiments. First of all, we were not able

to include the same participants in all experiments. Even though we attempt to ensure

comparabiltiy by including a control condition it would have been preferable to be able to make

comparisons within-subjects between experiments. Second, we used ERM tactors in all

experiments and conditions and therefore it‘s not feasable to generalsie our findings to other

tactor types which generate vibrations differently and might thus lead to different vibrotactile

acuity measurements (Kaas, Nelson, Sur, Lin, & Merzenich, 1979). Third, the diameter of the

tactors used in the vest and sponge differed by 2 mm which makes it difficult to pinpoint exactly

which structoral components between the vest and sponge had the greatest effect on

39

performance. Fouth, all participants were 31 or younger and thus our findings might not apply

to older age groups, especially since tactile acuity might decrease with age (Stevens, Foulke, &

Patterson, 1996). Lastly, we only tested horizontal localization during the experiments, not

vertical or diagonal localization. It is therefore not possible to generalize our findings to acuity

of different orientations of stimulation.

In light of our results, as well as their limitations, we conducted an additional

experiment to further explore vibrotactile acuity with three different types of tactors in addition

to different orientations of stimulation.

40

Method (experiment 4)

Participants

A total of 17 participants (5 male, 12 female) aged between 20-26 years with the mean age of

23.12 (SD = 1.32). The mean age of females was 23 years (SD = 1.47) and 23.4 (SD = 0.80) for

males. All participants were students at the University of Iceland who received course credit for

their participation, gave a written informed consent sheet before participating and were naïve

about the purpose of the experiments. The experiments were approved by the National

Bioethical Committee of Iceland (VSN-15-107).

Stimuli

There were two types of tasks in the experiment, a horizontal and a vertical point localization

task. Each trial in the horizontal point localization tasks included a pair of tactors. Either the

same tactor was activated twice or two adjacent tactors in the same row were activated in

succession. If two separate tactors were activated instead of the same one twice, the second

tactor activated was the closest tactor on the right or left side of the first tactor. The participant

had to judge whether the second vibration was at the same location as the first one (by pressing

the spacebar), or if it was to the left or right of the first one by pressing the left or right arrow

keys, respectively. The vertical point localization task only differed with regard to direction: the

stimuli were presented on the columns of the vibro-sponge instead of the rows. The participant

had to judge whether the second vibration was at the same location as the first one (by pressing

the spacebar), or if it was above or below the first one by pressing the up or down arrow keys

respectively. The tactors in each trial in both tasks were each activated successively for 200 ms

with a 50 ms interval. A combination of a 800 ms waiting time and a random interval of 300-

900 ms (in 100 ms steps) passed between trials. The total number of trials in each condition was

168 divided into seven blocks in which each possible stimulus pair was presented once. Thus

there were 24 trials within a block. Order and placement of stimuli pairs was randomized within

each block.

Materials

Three vibro-sponges were used with three different tactor types. The tactors were controlled

with an Arduino Uno microcontroller with custom made software. Furthermore, all three

sponges were equipped with a battery and a charger circuit. All experiments were run in

PsychoPy (Peirce, 2007; 2009) on a PC to which the microcontrollers controlling the vibro-

sponges were connected.

41

ERM (pp) sponge

The ‘ERM (pp)’ sponge was equipped eccentric rotating mass (ERM) tactors 8.7 mm in

diameter with a body length of 25 mm. The tactors produce perpendicular (pp) vibrations. Each

tactor was driven with a simple Darlington driver. The inter-tactor spacing of sponge 1 was 25

mm in the first session and 20 mm in the second session measured centre-to-centre (c/c). It was

not physically possible to place the tactors on the ERM (pp) sponge much closer to each other

than 20 mm and therefore we could not explore a 10 mm spacing with it as we did with the

other sponges.

Figure 7. Sponge 1 with an inter-tactor spacing of 20 mm measured center

to center

ERM (ip) sponge

The ERM (ip) sponge was equipped with ERM tactors, 8 mm in diameter with a body length of

3 mm. The tactors produce vibrations in plane. Each tactor was driven by a simple Darlington

driver. The inter-tactor spacing of sponge 2 was 20 mm in the first session and 10 mm the

second session.

42

Figure 8. Sponge 2 with an inter-tactor spacing of 10 mm measured center

to center

LRA Sponge

The LRA sponge was equipped with LRA tactors, 8mm in diameter with a body length of 3.25

mm. The tactors produce vibrations out of plane. Each tactor was driven with a Texas

Instruments DRV2605L IC. The inter-tactor spacing of sponge 3 was 20 mm in the first session

and 10 mm in the second session.

43

Figure 9. Sponge 3 with a inter-tactor spacing of 10 mm

Procedure

The experiment was carried out in two separate sessions which were conducted in the same

manner only differing with regard to inter-tactor spacing on the sponges. All participants

received an email invitation to take part in an experiment on two separate occasions and were

provided with an online link to book an appointment for the first session. Upon arrival to the

first session an explanation of the PL task was provided and the informed consent was signed.

Thereafter, the participant sat at a desk in a private room and carried out the PL task with all

three vibro-sponges in both the horizontal and vertical condition after receiving an introduction

on how to respond. The order in which the sponges were presented to participants was

randomized with a balanced Latin square as well as the order of conditions. During the

experiment, white noise was played in headphones to cancel out the noise generated by the

tactors. Participants were asked to have only one layer of clothing between their skin and the

sponge to ensure that the vibrations would not be diluted through thick fabric. When the first

session was completed the sponges were modified and their inter-tactor spacing manipulated.

44

All participants received an online link to book an appointment for the second session, which

was carried out in the same manner as the former session. Three weeks passed between the first

and second session.

Statistical analysis

Before analyzing the response time data, all trials were removed in which an incorrect response

was provided as well as those in which response time deviated more than three SD’s from the

mean RT of each participant as well as those who were less than 100 ms. A one-sample t-test

was conducted to assess accuracy rates within each condition and a repeated measures ANOVA

(R Core Team, 2015) was conducted to explore the influence of direction, inter-tactor spacing,

tactor rows and columns on accuracy and response time, as well as gender differences.

Furthermore, performance throughout each condition was analysed to estimate whether

participant’s performance improved with practice. All analyses were conducted in R (R Crew

Team, 2015) in RStudio (RStudio Team, 2015). Tukey’s honest significant difference test was

used for all post-hoc comparisons (R Core Team, 2015).

45

Results (experiment 4)

Accuracy

Figure 10 displays mean accuracy levels obtained with the three different tactor types, inter-

tactor spacing and direction. No significant gender differences in accuracy were found during

the experiment (F(1,15) = 0.29, p = .60).

Figure 10. Accuracy plotted by tactor types, inter-tactor spacing and

direction. The error bars represent the within subjects standard error of the

mean (SEM).

The accuracy rates obtained with the ERM (pp) tactors was significantly above chance level

(33%) in all conditions, as displayed in table 8. The difference in accuracy between the

horizontal and vertical condition was significant both with the 25 mm (t(16) = -0.255, p = .021)

and 20 mm spacing (t(16) = -3.04, p=.008). The difference in accuracy between 25 mm and 20

mm spacing was neither significant for the horizontal (t(16) = -0.28, p = .786) nor the vertical

condition (t(16) = -0.39, p = .699).

46

Table 8. Accuracy with ERM tactors (pp) by spacing and direction along with t-tests

indicating whether the results differ from chance

Accuracy t-test

Spacing Direction Mean SD df t p

25 mm Horizontal 0.64 0.48 16 5.37 < .001

25 mm Vertical 0.52 0.50 16 3.67 .002

20 mm Horizontal 0.65 0.48 16 6.05 < .001

20 mm Vertical 0.53 0.50 16 6.71 < .001

The accuracy rates obtained with the ERM (ip) tactors was significantly above chance level

(33%) in all conditions, as displayed in table 9. The difference in accuracy between horizontal

and vertical was significant both 20 mm (t(16) = -4.44, p < .001) and 10 mm (t(16) = -4.34, p <

.001). The difference in accuracy between 20 mm and 10 mm was neither significant for the

horizontal (t(16) = 1.14, p = .270) nor the vertical condition (t(16) = 0.48, p = .635).

Table 9. Accuracy with ERM (ip) tactors by spacing and direction along with t-tests indicating

whether the results differ from chance

Accuracy t-test


20 mm Horizontal 0.53 0.50 16 5.49 < .001

20 mm Vertical 0.41 0.49 16 3.48 .003

10 mm Horizontal 0.50 0.50 16 7.05 < .001

10 mm Vertical 0.40 0.49 16 7.71 < .001

Table 10 shows that accuracy rates obtained with the LRA tactors was significantly above

chance level (33%) when stimuli were presented horizontally at both 20 mm and 10 mm. When

stimuli were presented vertcially, however, the accuracy rates were not significantly above

chance. The difference in accuracy between horizontal and vertical was significant both for the

20 mm (t(16) = -3.75, p = .002) and 10 mm (t(16) = -2.28, p = .037) spacing. The difference in

accuracy between 20mm and 10 mm was neither significant for the horizontal (t(16) = 1.02, p =

.321) nor the vertical condition (t(16) = 0.30, p = .771).

47

Table 10. Accuracy with the LRA tactors by spacing and direction along with t-tests indicating

whether the results differ from chance

Accuracy t-test


20 mm Horizontal 0.44 0.50 16 2.99 .008

20 mm Vertical 0.34 0.48 16 0.45 .660

10 mm Horizontal 0.41 0.49 16 3.92 .001

10 mm Vertical 0.33 0.47 16 0.13 .895

Response time

Figure 11 shows response times obtained with all three tactor types in all conditions . No

significant gender differences in RT were found during the experiment (F(1,15) = 0.16, p =

.69).

Figure 11. Response time (ms) plotted by tactor types, inter-tactor spacing

and direction. The error bars represent the within subjects standard error of

the mean (SEM).

The response times obtained with the ERM (pp) tactors are displayed in Table 11. No

significant differences were found in response time between the horzontal and vertical

conditions with a 25 mm (t(16) = 1.63, p = .122) or 20 mm spacing (t(16) = 1.28, p = .229).

The difference in response time between 25 mm and 20 mm was not significant for the

horizontal condition (t(16) = 1.86, p = .081) but was significant in the vertical condition (t(16) =

2.71, p = .016).

48

Table 11. Response times with ERM (pp) tactors by spacing and direction along with

information on removed trials and outliers

Removed Response time (ms)

Spacing Direction Incorrect Outliers Mean SD

25 mm Horizontal 1031 25 668 174

25 mm Vertical 1380 12 749 205

20 mm Horizontal 990 26 616 167

20 mm Vertical 1336 16 647 164

The response times obtained with the ERM (pp) tactors are displayed in table 12. The differene

in response time between horizontal and vertical conditions were both significant for the 20 mm

(t(16) = 3.09, p = .006) and 10 mm (t(16) = 2.10, p =.052) spacing. The difference in response

time between 20 mm and 10 mm was neither significant for the horizontal (t(16) = 2.02, p = .06)

nor the vertical condition (t(16) = 1.82, p = .088).

Table 12. Response times with ERM (ip) tactors by spacing and direction


Spacing Direction Incorrect Outliers Mean SD

20 mm Horizontal 1339 17 702 203

20 mm Vertical 1684 8 759 219

10 mm Horizontal 1430 16 634 173

10 mm Vertical 1717 10 685 223

The response times obtained with the ERM (pp) tactors are displayed in table 13. The differene

in response time between horizontal and vertical conditions was not significant with the 20 mm

spacing (t(16) = 1.09, p = .290). The difference in repsonse time between the horizontal and

vertical direction was however significant with the 10 mm spacing (t(16) = 2.96, p = .009). The

difference in response time between 20 and 10 mm was significant only for the horizontal

condition (t(16) = 3.91, p = .001) but not the vertical condition (t(16) = 1.44, p = .166).

49

Table 13. Response times with the LRA tactors by spacing and direction


Experiment Direction Incorrect Outliers Mean SD

20 mm Horizontal 1590 13 685 231

20 mm Vertical 1871 9 722 225

10 mm Horizontal 1682 14 589 180

10 mm Vertical 1905 9 667 235

Comparison between tactor types and directions

A post hoc comparison revealed that the difference between accuracy rates obtained with the

LRA and the ERM (pp) tactors with a 20 mm spacing was significant in both the horizontal and

vertical conditions, as shown in Table 14. The difference in accuracy was neither significant

when the LRA and ERM (ip) nor the ERM (pp) and ERM (ip) tactors where compared in the

same conditions (ps > .424).

Table 14. Post hoc comparison of the difference in mean accuracy obtained with the LRA and

ERM (pp) tactors with a 20 mm spacing.

95% CF

Comparison Diff. Lower Higher p-value

LRA 20 mm vertical ERM (pp) 20 mm vertical -0.187 -0.357 -0.018 .017

LRA 20 mm horizontal ERM (pp) 20 mm horizontal -0.210 -0.380 -0.040 .004

There was a significant difference between the horizontal and vertical conditions of all three

tactor types with a 20 mm spacing as shown in Table 15.

Table 15. Post hoc comparison of the difference in mean accuracy obtained in the horizontal

and vertical conditions with a 20 mm spacing with the different tactor types

95% CF


ERM (pp) 20 mm horizontal ERM (ip) 20 mm vertical 0.243 0.073 0.413 < .001

ERM (ip) 20 mm horizontal LRA 20 mm vertical 0.186 0.017 0.356 .018

ERM (pp) 20 mm horizontal LRA 20 mm vertical 0.308 0.139 0.478 < .001

50

In Table 16 you can see the post hoc comparisons of mean accuracy with the different tactor

types and stimulation directions. Only the significant differences are displayed, all other ps >

.061.

Table 16. Post hoc comparisons with different tactor types with different spacing and

stimulation direction

95% CF




ERM (ip) 20 mm horizontal LRA 10 mm vertical 0.198 0.028 0.368 .008

Due to the physical size of the ERM (pp) tactors it was not possible to have an inter-tactor

spacing of 10 mm and therefore it was not possible to compare all three tactor types with a 10

mm spacing. Instead, we tested the ERM (pp) tactors at 25 and 20 mm. Table 16 displays

significant differences between the 25 mm spacing of the ERM and the other tactor types. All

other comparisons of the ERM (pp) and the other tactors resulted in ps > .224.

Table 17. Post hoc comparison of the difference in mean accuracy obtained with the ERM (pp)

tactors at 25 mm with the ERM (ip) and LRA tactors with a 10-20 mm spacing

95% CF


LRA 20 mm horizontal ERM (pp) 25 mm horizontal -0.196 -0.365 -0.026 .001

ERM (pp) 25 mm horizontal LRA 10 mm horizontal 0.228 0.058 0.398 < .001



ERM (pp) 25 mm vertical LRA 10 mm vertical 0.184 0.014 0.354 .021



Table 18 displays the significant post hoc comparisons of mean response times in different

experiments and directions. All other ps >.189

51

Table 18. Post hoc comparison of the difference between mean response time obtained in

different experiments as well as direction.

95% CF

Experiments Diff. Lower Higher p-value

LRA 10 mm horizontal ERM (pp) 25 mm vertical -142 -273 -12 .019

ERM (ip) 10 mm horizontal ERM (ip) 20 mm vertical -134 -264 -4 .037

ERM (pp) 20 mm horizontal ERM (ip) 20 mm vertical -146 -276 -16 .013

LRA 10 mm horizontal ERM (ip) 20 mm vertical -179 -309 -48 < .001

LRA 10 mm horizontal LRA 20 mm vertical -138 -268 -8 .028

Training effects

Accuracy

To estimate whether repetition or training had a significant effect on accuracy we compared

mean accuracy within each block of trials throughout each condition.

ERM (pp): The main effect of repetition with 25 mm neither had a significant effect on accuracy

in the horizontal (F(6,96) = 1.38, p = .232) nor the vertical (F(6,96) = 1.16, p = 0.334)

condition. However, repetition had a significant effect on accuracy with a 20 mm spacing, both

in the horizontal (F(6,96) = 2.46, p = .030) and vertical condition (F(6, 96) = 3.95, p = .001).

The mean accuracy of each block with the 20 mm spacing of the ERM (pp) tactors is displayed

in Table 16.

Table 16. Mean accuracy by blocks (B) with the 20 mm spacing of the ERM (pp) & LRA

tactors

Tactor Direction Spacing B1 B2 B3 B4 B5 B6 B7

ERM (pp) Horizontal 20 mm 0.61 0.69 0.68 0.69 0.69 0.66 0.70

ERM (pp) Vertical 20 mm 0.48 0.53 0.54 0.61 0.56 0.53 0.54

LRA Horizontal 20 mm 0.39 0.48 0.45 0.48 0.44 0.46 0.44

ERM (ip): Repetition never had a significant effect on accuracy with the ERM (ip) tactors (all ps

< .325).

LRA: Repetition only had a significant effect on accuracy with 20 mm in the horizontal

condition with the LRA tactors where F(6,96) = 2.46, p = .028, see Table 16 for mean accuracy

of blocks, (all other ps > .349).

52

Response Time

To estimate whether repetition or training had a significant effect on accuracy we compared

mean response time within each of the seven block of trials throughout each condition. Note that

all trials in which RT deviated more than three SD‘s from the mean RT of each participant were

removed before data analysis. Due to this, three participants happened to have the total number

trials within a whole block removed. Therefore, they were not included when training effects in

those conditions were estimated. One participant was excluded in the vertical condition with the

25 mm spacing using ERM (pp) tactors, one for the vertical condition with 20 mm spacing and

one for the 10 mm with the LRA tactors.

ERM (pp): The main effect of repetition on RT was significant with the 25 mm spacing of the

ERM (pp) tactors in both the horizontal (F(6,96) = 2.21, p =.049) and the vertical (F(6,90) =

2.34, p = .038) condition. For mean accuracy by blocks, see Table 17. When the 20 mm spacing

was used the effect of repitition was not significant (ps > .098).

Table 17. Mean RT (ms) by blocks (B) with the ERM (pp) tactors


ERM (pp) Horizontal 25 mm 708 677 682 642 659 658 653

ERM (pp) Vertical 25 mm 783 756 772 732 771 713 727

ERM (ip): The main effect of repetition on RT was significant for the horizontal condition with

a 20 mm spacing (F(6,96) = 2.47, p =.029) but not the vertical (F(6,96) = 1.61, p =.152).

When the 10 mm spacing was used, the main effect of repetition on RT was significant in the

vertical (F(6,96) = 2.85, p =.014) but not the horizontal condition (F(6,96) = 1.81, p =.105).

See Table 18 for mean accuracy by blocks in the conditions where repetition had a significant

effect on response time.

Table 18. Mean RT by blocks (B) with the 20 mm spacing of the ERM (ip) tactors


ERM (ip) Horizontal 20 mm 757 716 706 688 677 692 694

ERM (ip) Vertical 10 mm 732 702 681 665 648 663 662

LRA: The main effect of repetition on RT was significant in the 10 mm condition with the LRA

tactors, both in the horizontal (F(6,96) = 2.95, p =.011) and vertical (F(6,60) = 4.95, p < .001)

condition. The mean response time by blocks is displayed in table 19. The effect of repetition

was not significant when the 20 mm spacing was used (ps > .690).

53

Table 19. Mean RT by blocks (B) with the 20 mm spacing of the LRA tactors


LRA Horizontal 10 mm 639 620 579 593 572 582 555

LRA Vertical 10 mm 747 697 684 601 640 633 637

Tactor location and performance

Effect of tactor rows on performance

Tactor rows, the vertical location of tactors on the back during horizontal stimulation, had a

significant effect on accuracy with the 25 mm spacing of the ERM (pp) tactors (F(3,48) = 3.01,

p = .039) and 20 mm spacing of the LRA tactors (F(3,48) = 4.93, p = .005). Post hoc

comparisons were not significant however, with all ps > .520. Furthermore, tactor rows had a

significant effect on accuracy with the 10 mm spacing of the ERM (ip) tactors (F(3,48) = 12.21,

p < .001). A post hoc comparison resulted in a significant difference between the top and

bottom rows (p = .010) but not between any of the other rows (ps > .091).

Additionally, tactor rows had a significant effect on response time with the 25 mm spacing of

the ERM (pp) tactors (F(3,48) = 3.54, p = .022), the 20 mm spacing of the ERM (ip) tactors

(F(3,48) = 3.31), p = .028) and the 20 mm spacing of the LRA tactors (F(3,48) = 3.90, p =

.014). All post hoc comparisons were however unsignificant with all ps > .723.

Effect of tactor colums on performance

Tactor columns, the horizontal location of tactors on the back during vertical stimulation, had a

significant effect on accuracy with the 20 mm spacing of the LRA tactors (F(3,48) = 3.48, p =

.023) The differences in accuracy between the tactor colums were not significant post hoc, with

all ps > .655. Tactor columns also had a significant effect with the 20 mm spacing of the ERM

(ip) tactors (F(3,48) = 3.97, p = .013).Only the difference between the second leftmost column

and the rightmost column was significant post hoc, p = .050, whereas all other ps > .325.

Furthermore, tactor colums only had a significant effect on response time with the 10 mm

spacing of the LRA tactors (F(3,48) = 3.13, p = .034). A post hoc comparison was not

significant however, with all ps > .285.

Individual differences

There was quite the individual variability in accuracy throughout the experiment. In Figure 12

the mean accuracy of each subject is displayed. Mean horizontal accuracy was numerically

54

higher than mean vertical accuracy for all but one subject although the difference between the

two condition varied quite a bit between subjects.

Figure 12. Mean accuracy of subjects throughout the experiment. The error bars represent the

within subjects standard error of the mean (SEM)

Figure 13 displays the mean accuracy by subjects with the ERM (pp) tactors with a 25 mm

spacing. There is quite the variability between participants in this condition. Furthermore, the

difference in mean accuracy between the horizontal and vertical conditions also varied from one

participant to the next. Curiously, some participants hovered around chance level (0.33) while

others had a much higher mean accuracy well above chance.

Figure 13. Mean subject accuracy with the ERM (pp) tactors and 25 mm

spacing. The error bars represent the within subjects standard error of the

mean (SEM)

55

When the ERM (pp) tactors were applied with a 20 mm spacing a similar pattern appeared: the

variability between participants was still apparent as well as the variability between subjects

regarding the differnce between the two directions of stimulation. Again some pariticpants have

a mean accuracy around or below chance level, as displayed in Figure 14.

Figure 14. Mean subject accuracy with the ERM (pp) tactors and 20 mm


mean (SEM)

When the ERM (ip) tactors were applied the variability in mean accuracy between participants

was still evident although not as great as when the ERM (pp) tactors were used, as shown in

Figure 15.

56

Figure 15. Mean subject accuracy with the ERM (ip) tactors and 20 mm


mean (SEM)

Reducing the inter-tactor spacing of the ERM (ip) tactors from 20 to 10 mm resulted in less

variability between participants as displayed in Figure 16.

Figure 16. Mean subject accuracy with the ERM (ip) tacots and 10 mm


mean (SEM)

The LRA tactors resulted in the lowest mean accuracy throughout the experiment. Figure 7

displayes the mean subject accuracy with a 20 mm spacing of the LRA tactors. There are quite a

few participants with a mean accuracy level around or under chance level (0.33).

57

Figure 17. Figure 7. Mean subject accuracy with the LRA tactors and 20 mm


mean (SEM)

Finally, mean accuracy with the 10 mm spacing of the LRA sponge did not vary as much

between participants as accuracy in the other conditions as displayed in Figure 18.

Figure 18. Mean subject acuracy with the LRA tactors and 10 mm spacing.

The error bars represent the within subjects standard error of the mean

(SEM)

58

Discussion (experiment 4)

We assessed vibrotactile acuity with three different tactor types with two different stimulation

directions and different inter-tactor spacings.

Accuracy was significantly above chance level during the horizontal condition with all

three tactor types and inter-tactor spacings. During the vertical condition, however, accuracy did

not differ significantly from chance with the LRA motors, neither with the larger nor smaller

spacing, although it did with the ERM (pp) and ERM (ip) tactors in both conditions. These

results indicate that vertical vibrotactile thresholds differ depending on the tactor type used.

Furthermore, accuracy obtained the 20 mm spacing was higher using the ERM (pp) than the

LRA tactors, both in the vertical and horizontal condition which suggests that vibrotactile acuity

of the lumbar region is largely dependent on the tactor type applied.

Horizontal stimulation always resulted in significantly higher accuracy rates than

vertical stimulation with all three tactor types, both at the larger and the smaller spacing.

Moreover, horizontal stimulation with a 20 mm spacing was significantly higher with the ERM

(pp) tactors than vertical stimulation with the ERM (ip) and LRA tactors with the same inter-

tactor spacing. In addition, accuracy in the horizontal condition with ERM (ip) tactors at 20 mm

was significantly higher than accuracy obtained during vertical stimulation with the LRA tactors

with the same inter-tactor spacing. Furthermore, horizontal stimulation with greater inter-tactor

spacing tended to result in higher accuracy rates than vertical stimulation with a smaller spacing

when performance was compared between tactor types. Together, these results indicate

considerable spatial asymmetry in vibrotactile acuity of the lumbar region.

There were no differences found between the performance of the genders in any

condition, neither for accuracy nor response time. This finding is consistent with our results in

experiments 1-3 and further strengthens the notion that gender differences in vibrotactile acuity

might be more subtle than in pressure sensitivity (Weinstein, 1968).

Response time was similar in all conditions with the ERM (pp) tactors although vertical

stimulation resulted in shorter response times with the 25 mm spacing than the 20 mm one.

Furthermore, response times were longer with vertical than the horizontal stimulation using the

ERM (ip) tactors at both 20 and 10 mm. Accuracy with the LRA tactors was never significantly

above chance level during the vertical condition and the differences in response time between

the 20 and 10 mm spacings were only significant during the horizontal condition. On the whole,

these results hint that the same task is harder when stimulation is vertical rather than horizontal.

59

Training effects found during the experiments both applied to accuracy and response

time hinting that not only do people become better at replying during the task but in some cases

become better at the task itself. Furthermore, performance tended not to be influenced by the

horizontal and vertical location the tactors were located in.

Finally, individual differences were considerable throughout the experiment where

some individuals seemed to perform consistently better or worse compared to the others in the

same condition. This variability might reflect individual differences in vibrotactile accuracy

although they could also stem from motivational differences in performing the task at hand.

There were two obvious limitations to this experiment. Firstly, all participants were

young adults aged from 20-26 years, making generalizations towards older age groups difficult,

especially since tactile acuity might decrease with age (Stevens, Foulke, & Patterson, 1996).

Secondly, the contact area of the ERM (pp) tactors differed from the ERM (ip) and LRA tactors

which shared identical physical measurements and therefore it’s impossible to determine

whether the differences in performance between the ERM (pp) and other tactors stem from the

differences size of contact area or the difference in their axis of movement.

Nonetheless, the results indicate that there is considerable spatial asymmetry in the

vibrotactile acuity of the lumbar region and that different results can be achieved with different

tactor types.

60

General discussion

The broad field of sensory substitution (SS) involves compensation for sensory loss by

conveying information through a functioning sense modality (Kristjánsson, et al., 2016). For

visually impaired people (VIP) popular solutions for sensory substitution include Braille as well

as the white cane where visual input is substituted for tactile input. Even so, VIPs face a number

of difficulties in everyday life that the sighted rarely have to think about and require a

considerable amount of preperation and planning to go about their day unassisted. As a result,

VIPs tend to lead a sedetary, isolated lifestyle (Longmuir & Bar-Or, 2000; Varo, et al., 2003)

and unemployment rates are high worldwide (World Health Organization, 2014). These socio-

economical factors are a great loss for society and have urged researchers to develop

sophisticated sensory substitution devices (SSD) aimed at improving mobility and thereby the

independence of the visually impaired individuals. This devlopment of advanced auditory SSDs

such as the PSVA and vOICe as well as tactile solutions has been successful in the sense that

people have been able to utilize them to recognice objects (Auvray, Hanneton, & O'Regan,

2007; Kim & Zatorre, 2008; Amedi, et al., 2007; Meijer, 1992) and patterns (Arno, Capelle,

Wanet-Defalque, Catalan-Ahumada, & Veraart, 1999), experience movement and depth (Ward

& Meijer, 2010) and even navigate in the environment (Cosgun et al., 2014; Van Erp et al.,

2005). In addition to behavioral data, a plethora of neural evidence supports the feasability of

such advanced sensory substitution (Kristjánsson, et al., 2016; Jóhannesson, Balan,

Unnthorsson, Moldoveanu, & Kristjánsson, 2016). Despite successful demonstrations of the

available systems, most of their use is limited to laboratory settings presumably due to

ergonomic, economic and cosmetic factors (Elli, Benetti, & Collignon, 2014; Dakopoulos &

Bourbakis, 2010). Moreover, ignoring the sensory- and attentional constraints of the nervous

system during the development of SSDs might be central to the issue, highlighting the

importance of basic research in the development of successful assistive devices (Loomis,

Klatzky, & Guidice, 2012).

We conducted four experiments a part of the development of a vibrotactile vest within

the Sound of Vision project (SoV). The project aims to design an audio-tactile sensory

substitution device for the visually impaired. Due to the lack of systematic research and mixed

reports we explored the effects of inter-tactor spacing, different tactor types and stimulation

direction on vibrotactile acuity.

Our results illustrate the importance of practicing caution when generalizing results

from findings on vibrotactile acuity and thresholds. Although we demonstrated that the

61

vibrotactile threshold using coin cell tactors 8 mm in diameter, vibrating in plane, is at least 10

mm, different tactor types can yeild different results in the same conditions. Moreover,

depending on the importance of the information information transmitted vibrotactilly,

developers of sensory substitution devices should consider keeping a larger spacing than 10 mm

between tactors to avoid loosing information as accuracy tends to decrease with decreasing

inter-tactor spacing. Gender differenes in vibrotactile acuity seem to be more subtle with

vibrotactile stimulation than pressure pulses and might even be nonexistent. Finally, there seems

to be substantial spatial asymmetry in the vibrotactile acuity of the lumbar region and therefore

stimulation direction has to be taken into account along with inter-tactor spacing in the

development of sensory substitution devices.

62

References

Alary, F., Duquette, M., Goldstein, R., Chapman, C. E., Voss, P., La Buissonniére-Ariza, V., &

Lepore, F. (2009). Tactile acuity in the blind: A closer look reveals superiority over the

sighted in some but not all cutaneous tasks. Neuropsychologia, 47(10), 2037-2043.

Alary, F., Goldstein, R., Duquette, M., Chapman, C. E., Voss, P., & Lepore, F. (2008). Tactile

acuity in the blind: a psychophysical study using a two-dimensional angle

discrimination task. Experimental Brain Research, 187, 587-594.

Amedi, A., Floel, A., Knecht, S., Zohary, E., & Cohen, L. G. (2004). . Transcranial magnetic

stimulation of the occipital pole interferes with verbal processing in blind subjects.

Nature Neuroscience, 7(11), 1266-1270.

Amedi, A., Raz, N., Pianka, P., Malach, R., & Zohary, E. (2003). Early ‘visual’ cortex

activation correlates with superior verbal memory performance in the blind. Nature

neuroscience, 6(7), 758-766.

Amedi, A., Stern, W. M., Camprodon, J. A., Bermpohl, F., Merabet, L., Rotman, S., . . .

Pascual-Leone, A. (2007). Shape conveyed by visual-to-auditory sensory substitution

activates the lateral occipital complex. Nature neuroscience, 10(6), 687-689.

Apelle, S. (1972). Perception and discrimination as a function of stimulus orientation: The

'oblique' effect in man and animals. Psychological Bulletin, 78, 266-278.

Arno, P., Capelle, C., Wanet-Defalque, M. C., Catalan-Ahumada, M., & Veraart, C. (1999).

Auditory coding of visual patterns for the blind. Perception, 28(8), 1013-1029.

Arno, P., De Volder, A., Vanlierde, A., Wanet-Defalque, M. C., Streel, E., Robert, A., . . .

Veraart, C. (2001). Occipital activation by pattern recognition in the early blind using

auditory substitution for vision. Neuroimage, 13, 632-645.

Auvray, M., Hanneton, S., & O'Regan, J. (2007). Learning to Perceive with a Visuo-Auditory

Substitution System: Localisation and Object Recognition with 'The Voice'. Perception,

36(3), 416-430.

Bach-y-Rita, P. (1972). Brain Mechanisms in Sensory Substitution. New York: Academic Press.

Bach-y-Rita, P., Collins, C. C., Saunders, S., White, B., & Scadden, L. (1969). Vision

substitution by tactile image projection. Nature, 221, 963-1964.

Bach-y-Rita, P., Kaczmarek, K., Tyler, M., & Garcia-Lara, J. (1998). Form perception with a

49-point electrotactile stimulus array on the tongue: A technical note. Journal of

Rehabilitation Research and Development, 35(4), 427-430.

Bedny, M., Richardson, H., & Saxe, R. (2015). "Visual" cortex responds to spoken language in

blind children. The journal of neuroscience, 35(33), 11674-11681.

63

Berkley, G. (1874). A Treatise Concerning the Principles of Human Knowledge. Philadelphia,

JB Lippincott & Company.

Bertelson, P. (1999). Ventriloquism: a case of crossmodal perceptual grouping. Advances in

psychology, 129, 347-362.

Broadbent, D. E. (1958). Perception and Communication. New York: Pergamon Press.

Burton, H., Sinclair, R., & Agato, A. (2012). Recognition memory for Braille or spoken words:

An fMRI study in early blind. Brain Research, 1438, 22-34.

Burton, H., Snyder, A. Z., Conturo, T. E., Akbudak, E., Ollinger, J. M., & Raichle, M. E.

(2002). Adaptive changes in early and late blind: a fMRI study of Braille reading.

Journal of Neurophysiology, 87(1), 589-607.

Capelle, C., Trullemans, P., Arno, C., & Veraart, C. (1998). A real time prototype for

enhancement of vision rehabilitation using auditory sensory substitution. IEEE

Transactions on Biomedical Engineering, 10(12), 1279-1293.

Cappagli, G., Cocchi, E., & Gori, M. (2015). Auditory and proprioceptive spatial impairments

in blind children and adults. Develompental Science, 1-12.

Capraro, A., Verrillo, R., & Zwislocki, J. (1979). Psychophysical evidence for a triplex system

of cutaneous mechanoreception. Sensory Processes, 3, 340-352.

Cholewiak, R. W., & Collins, A. A. (2003). Vibrotactile localization on the arm: Effects of

places, space, and age. Perception & Psychophysics, 65, 1058-1077.

Cholewiak, R. W., Brill, J. C., & Schwab, A. (2004). Vibrotactile localization on the abdomen:

Effects of place and space. Perception & Psychophysics, 66(6), 970-987.

Cohen, L. G., Celnik, P., Pascual-Leone, A., Corwell, B., Falz, L., Dambrosia, J., . . . Hallet, M.

(1997). Functional relevance of cross-modal plasticity in blind humans. Nature,

389(6647), 180-183.

Collignon, O., & De Volder, A. G. (2009). Further evidence that congenitally blind participants

react faster to auditory and tactile spatial targets. Canadian Journal of Experimental

Psychology/Revue canadienne de psychologie expérimentale, 63(4), 287.

Collignon, O., Charbonneau, G., Lassonde, M., & Lepore, F. (2009). Early visual deprivation

alters multisensory processing in peripersonal space. Neuropsychologia, 47(14), 3236-

3243.

Collignon, O., Lassonde, M., Lepore, F., Bastien, D., & Veraart, C. (2007). Functional cerebral

reorganization for auditory spatial processing and auditory substitution of vision in

early blind subjects. Cerebral Cortex, 17(2), 457-465.

Condillac, E. B., & Carr, M. G. (1930). Traité des sensations. London: Favil Press.

64

Cosgun, A., Sisbot, E. A., & Christensen, H. I. (2014). Evaluation of rotational and directional

vibration patterns on a tactile belt for guiding visually impaired people. IEEE Haptic

Symposium (HAPTICS), (pp. 367-370).

Craig, J., & Evans, P. (1987). Vibrotactile masking and the persistence of tactual features.

Perception & Psychophysics, 42(4), 309-317.

Dakopoulos, D., & Bourbakis, N. G. (2010). Wearable obstacle avoidance electronic travel aids

for blind: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C

(Applications and Reviews), 40(1), 25-35.

Di Cagno, A., Luilano, E., Aquino, G., Fiorilli, G., Battaglia, G., Giombini, A., & Calcagno, G.

(2013). Psychological well-being and social participation assessment in visually

impaired subjects playing Torball: A controlled study. Research in Developmental

Disabilities, 34(4), 1204-1209.

Edwards, N., Rosenthal, J., Moberly, D., Lindsay, J., Blair, K., Krishna, S., . . . Panchanathan,

S. (2009). A Pragmatic Approach to the Design and Implementation of a Vibrotactile

Belt and its Applications. IEEE International Workshop on Haptic Audio visual

Environments and Games, (pp. 13-18).

Elli, G. V., Benetti, S., & Collignon, O. (2014). Is there a future for sensory substitution outside

academic laboratories? Multisensory Research, 27(5-6), 271-291.

Fechner, G. (1966). Elements of Psychophysics. New York: Holt, Rinehart and Winston.

Fortin, M., Voss, P., Lord, C., Lassonde, M., Pruessner, J., Saint-Amour, D., . . . Lepore, F.

(2008). Wayfinding in the blind: larger hippocampal volume and supranormal spatial

navigation. Brain, 131(11), 2995-3005.

Gerald, F. A., & Sherrick, C. E. (1972). The cutaneous “rabbit”: A perceptual illusion. Science,

178(4057), 178-179.

Glodreich, D., & Kancis, I. (2003). Tactile acuity is enhanced in blindness. Journal of

Neuroscience, 187, 3439-3445.

Golledge, R., Klatzky, R., Loomis, J., & Marston, J. (2004). Stated preferences for components

of a personal guidance system for nonvisual navigation. Journal of Visual Impairment

& Blindness (JVIB), 98(3), 135-137.

Goodale, M. A. (1992). Separate visual pathways for perception and action. Trends in

Neuroscience, 15, 20-25.

Gori, M., Sandini, G., Martinoli, C., & Burr, D. C. (2014). Impairment of auditory spatial

localization in congenitally blind human subjects. Brain, 137(1), 288-293.

Gougoux, F., Lepore, F., Lassonde, M., Voss, P., Zatorre, R. J., & Belin, P. (2004).

Neuropsychology: pitch discrimination in the early blind. Nature, 430(6997), 309-309.

65

Gougoux, F., Zatorre, R. J., Lassonde, M., Voss, P., & Lepore, F. (2005). A functional

neuroimaging study of sound localization: visual cortex activity predicts performance in

early-blind individuals. PLoS Biol, 3(2), e27.

Hahn, J. (1966). Vibrotactile adaptation and recovery measured. Journal of Experimental

Psychology, 71(5), 655-658.

Hamilton, R., & Pascual-Leone, A. (1998). Cortical plasticity associated with Braille learning.

Trends in Cognitive Science, 2(5), 168-174.

Härmä, A., Jakka, J., Tikander, M., Karjalainen, M., Lokki, T., Hiipakka, J., & Lorho, G.

(2004). Augmented reality audio for mobile and wearable appliances. Journal of the

Audio Engineering Society, 52(6), 618-639.

Helal, A., Moore, S. E., & Ramachandran, B. (2001). Drishti: An Integrated Navigation System

for Visually Impaired and Disabled. Procedings of the 5th International Symposium on

Warable Computers, 149-156.

Held, R., & Hein, A. (1963). Movement-produced stimulation in the development of visually

guided behavior. Journal of Comparative and Physiological Psychology, 56(5), 872-

876.

Hodna, A., Shibata, H., Hidaka, S., Gyoba, J., Iwaya, Y., & Suzuki, Y. (2013). Effects of head

movement and proprioceptive feedback in training of sound localization. i-Perception,

4(4), 253-264.

Ito, K., Okamoto, M., Akita, J., Ono, T., Gyobu, I., & Takagi, T. (2005). CyARM: An

alternative aid device for blind persons. CHI, (pp. 1483-1486).

Jacobson, H. (1950). The informational capacity of the human ear. Science, 112, 143-144.

Jacobson, H. (1951). The informational capacity of the human eye. Science, 113(2933), 292-

293.

Johnson, L. A., & Higgins, C. M. (2006). A navigation aid for the blind using tactile-visual

sensory substitution. Engineering in Medicine and Biology Society, 28th Annual

International Conference of the IEEE, (pp. 6289-6292).

Jóhannesson, Ó. I., Balan, O., Unnthorsson, R., Moldoveanu, A., & Kristjánsson, Á. (2016).

The Sound of Vision Project: On the Feasibility of an Audio-Haptic Representation of

the Environment, for the Visually Impaired. Brain Sciences, 6(3), 20.

K., I., Okamoto, M., Akita, J., Ono, T., Gyobu, I., & Takagi, T. (n.d.).

Kaas, J. H., Nelson, R. J., Sur, M., Lin, C. S., & Merzenich, M. M. (1979). Multiple

representations of the body within the primary somatosensory cortex of primates.

Science, 204(4392), 521-523.

Kärcher, S. M., Fenzlaff, S., Hartmann, D., Nagel, S. K., & König, P. (2012). Sensory

augmentation for the blind. Frontiers in Human Neuroscience, 6(36), 1-15.

66

Karnath, H. O., Schenkel, P., & Fischer, B. (1991). Trunk orientation as the determining factor

of the "contralateral" deficit in the neglect syndrome and as the physical anchor of the

internal representation of body orientation in space. Brain, 114, 1997-2014.

Kercel, S. W. (2004). The endogenous brain. Journal of integrative neuroscience, 3(1), 61-84.

Kercel, S. W., Caulfield, H. J., & Bach-y-Rita, P. (2003). Bizarre hierarchy of brain function.

Intelligent Computing: Theory and Applications, 5103, 150-161.

Kim, J. K., & Zatorre, R. J. (2008). Generalized learning of visual-to-auditory substitution in

sighted individuals. Brain Research, 1242, 263-275.

Kim, J. K., & Zatorre, R. J. (2008). Generalized Learning of Visual-to-

AuditorySubstitutioninSightedIndividuals. Brain Research, 1242(2008), 263-275.

Kokjer, K. J. (1987). he Information Capacity of the Human Finger-tip. IEEE Transactions on

Systems Man and Cybernetics, 17(1), 100-102.

Koo, I. M., Jung, K., Koo, J., Nam, J., & Lee, Y. K. (2008). Development of soft-actuator-based

wearable tactile display. IEEE Trans Robot, 24, pp. 549-558.

Kristjánsson, Á. (2012). Innra Augað. Reykjavík: Háskólaútgáfan.

Kristjánsson, Á., Moldoveanu, A., Jóhannesson, Ó. I., Balan, O., Spagnol, S., Valgeirsdóttir, V.

V., & Unnþórsson, R. (2016). Designing sensory-substitution devices: Priniples, pitfalls

and potential. Restorative Neurology and Neuroscience, 34(5), 769-787.

Kupers, R., Pappens, M., de Noordhout, A. M., Schoenen, J., Ptito, M., & Fumal, A. (2007).

rTMS of the occipital cortex abolishes Braille reading and repetition priming in blind

subjects. Neurology, 68(9), 691-693.

Lechelt, E. C. (1976). Perceptual orientational asymmetries: A comparison of visual and haptic

space. Perception & Psychophys, 20, 463-469.

Lechelt, E. C. (1988). Spatial asymmetries in tactile discrimintation of line orientation: A

comparison of the sighted, visually impaired, and blind. Perception, 17, 579-585.

Lechelt, E. C., & Veranka, A. (1980). Spatial anisotropy in intramodal and cross-modal

judgments of stimulus orientation: the stability of the oblique effect. Perception, 9, 581-

589.

Lessard, N., Pare, M., Lepore, F., & Lassonde, M. A. (1998). Early-blind human subjects

localize sound sources better than sighted subjects. Nature, 395(6699), 278-280.

Lewis, T. L., & Maurer, D. (2005). Multiple sensitive periods in human visual development:

evidence from visually deprived children. Developmental Psychobiology, 46(3), 163-

183.

Longmuir, P., & Bar-Or, O. (2000). Factors Influencing the Physical Activity Levels of Youths

With Physical and Sensory Disabilities. Adapted Physical Activity Quarterly, 17(40),

40-53.

67

Loomis, J. M. (1981). On the tangibility of letters and braille. Perceptual Psychophysics, 29(1),

37-46.

Loomis, J. M., & Collins, C. C. (1978). Sensitivity to shifts of a point stimulus: An instance of

tactile hyperacuity. Perception and Psychophysics, 24(6), 487-492.

Loomis, J. M., Klatzky, R., & Guidice, N. A. (2012). Sensory substitution of vision:Importance

of perceptual and cognitive processing. In R. Manduchi, & S. Kurniawan (Eds.),

Assistive technology for blindness and low vision (pp. 162-191). Boca Raton, Florida,

USA: CRC Press.

Luomajoki, H., & Moseley, G. L. (2011). Tactile acuity and lumbopelvic motor control in

patients with back pain and healthy controls. British journal of sports medicine, 45(5),

437-440.

Macaluso, E., Frith, C., & Driver, J. (2000). Modulation of human visual cortex by crossmodal

spatial attention. Science, 289, 1206-1208.

Mahns, D. A., Perkins, N. M., Sahai, V., Robinson, L., & Rowe, M. J. (2006). Vibrotactile

frequency discrimination in human hairy skin. Journal of neurophysiology, 95(3), 1442-

1450.

Maravita, A., Husain, M., Clarke, K., & Driver, J. (2001). Reaching with a tool extends visual-

tactile interactions into far space: Evidence from cross-modal extinction.

Neuropsychologia, 39(6), 580-585.

Massaro, D. W. (1999). Speechreading: illusion or window into pattern recognition. Trends in

Cognitive Sciences, 3(8), 310-317.

Matteau, I., Kupers, R., Ricciardi, E., Pietrini, P., & Ptito, M. (2010). Beyond visual, aural and

haptic movement perception: hMT + is activated by electrotactile motion stimulation of

the tounge in sighted and in congentially blind individuals. Brain research bulletin,

82(5), 265-270.

May, A. (2011). Experience-dependent structural plasticity in the adult human brain. Trends in

Cognitive Scienses, 15(10), 475-482.

McDaniel, T., Krishna, S., Balasubramian, V., Colbry, D., & Panchanathan, S. (2008). Using a

haptic belt to convey non-verbal communication cues during social interactions to

individuals who are blind. Haptic Audio Visual Environment and Games, HAVE 2008.

IEEE International Workshop, (pp. 13-18).

Meijer, P. B. (1992). An experimental system for auditory image representations. IEEE

transactions on biomedical engineering, 39(2), 112-121.

Merabet, L. B., Batelli, L., Obretenova, S., Maguire, S., Meijer, P., & Pascual-Leone, A. (2009).

Functional recruitment of visual cortex for sound encoded object identification in the

blind. Neruoreport, 20(2), 132.

Merabet, L. B., Hamilton, R., Schlaug, G., Swisher, J. D., & Kiriakopoulos, E. T. (2008). Rapid

and Reversible Recruitment of Early Visual Cortex for Touch. PLOs ONE, 3(8), e3046.

68

Merabet, L., & Pascual-Leone, A. (2010). Neural reorganization following sensory loss: the

opportunity of change. Nature Reviews Neuroscience, 11, 44-52.

Mitterauer, B., & Kopp, K. (2003). The self-composing brain: towards a glial–neuronal brain

theory. Brain and Cognition, 51(3), 357-367.

Morioka, M., Whitehouse, D. J., & Griffin, M. J. (2008). Vibrotactile thresholds at the fingertip,

volar forearm, large toe, and heel. Somatosensory & motor research, 25(2), 101-112.

Mountcastle, V. B. (1978). Brain mechanisms for directed attention. Journal of the Royal

Society of Medicine, 71(1), 14.

Neimeyer, W., & Starlinger, I. (1981). Do the blind hear better? Investigations on auditory

processing in congenital or early acquired blindness. II. Central functions. Audiology,

20(6), 510-515.

Novich, S. D., & Eagleman, D. M. (2015). Using space and time to encode vibrotactile

information: toward an estimate of the skin’s achievable throughput. Experimental

Brain research, 233(10), 2777-2788.

Ocelli, V., Spence, C., & Zampini, M. (2013). Auditory, Tactile and Audiotactile Information

Processing Following Visual Depriviation. Psychological Bulletin, 139(1), 189-212.

Ollinger, J. M., Corbetta, M., & Shulman, G. L. (2001). Separating processes within a trial in

event-related functional MRI II Analysis. NeuroImage, 13, 218-229.

Pascual-Leone, A., & Hamilton, R. (2001). The metamodal organization of the brain. Progress

in Brain Research, 134, 427-445.

Peirce, J. W. (2007). PsychoPy - Psychophysics software in Python. Journal of neuroscience

methods, 162(1), 8-13.

Peirce, J. W. (2009). Generating stimuli for neuroscience using PsychoPy. Frontiers in

Neuroinformatics, 2.

Poirier, C., De Volder, A., Tranduy, D., & Scheiber, C. (2007). Pattern recognition using a

device substituting audition for vision in blindfolded sighted subjects.

Neruopsychologia, 45(5), 1108-1121.

Prolux, M. J., Stoerig, P., Ludowig, E., & Knoll, I. (2008). Seeing ‘Where’ through the Ears:

Effects of Learning-byDoing and Long-Term Sensory Deprivation on Localization

Based on Image-to-Sound Substitution. PloS one, 3(3), e1840.

Qin, W., & Yu, C. (2013). Neural Pathways Conveying Novisual Information to the Visual

Cortex. doi:10.1155/2013/864920

RCoreTeam. (2015). R: A language and environment for statistical computing. Vienna: R

Foundation for Statistical Computing. doi:http://www.R-project.org

69

Renier, L., Laloyaux, C., Collignon, O., Tranduy, D., Vanlierde, A., Bruyer, R., & De Volder,

A. G. (2005). The Ponzo illusion with auditory substitution of vision in sighted and

early-blind subjects. Perception, 34(7), 857-867.

Roder, B., Rosler, F., & Neville, H. J. (2001). Auditory memory in congenitally blind adults: a

behavioralelectrophysiological investigation. Cognitive Brain Research, 11(2), 289-303.

RStudioTeam. (2015). RStudio: Integrated Development for R. Boston: RStudio. Inc. Retrieved

from http://www.rstudio.com

Röder, B., Rösler, F., & Spence, C. (2004). Early vision impairs tactile perception in the blind.

Current Biology, 14(2), 121-124.

Röder, B., Stock, O., Bien, S., Neville, H., & Rösler, F. (2002). Speech processing activates

visual cortex in congenitally blind humans. European Journal of Neuroscience, 16, 930-

936.

Röder, B., Teder-Sälejärvi, W., Sterr, A., Rösler, F., Hillyard, S. A., & J., N. H. (1999).

Improved auditory spatial tuning in blind humans. Nature, 400(6740), 162-166.

Sadato, N., Okada, T., Honda, M., & Yonekura, Y. (2002). Critical period for cross-modal

plasticity in blind humans: a functional MRI study. NeuroImage, 16, 389-400.

Sadato, N., Pascual-Leone, A., Grafman, J., Ibanez, V., Deiber, M., Dold, G., & Hallett, M.

(1996). Activation of the primary visual cortex by Braille reading in blind subjects.

Nature, 380(6574), 526-528.

Schlaug, G., Halpern, A. R., Press, D. Z., Baker, J., Edleman, R. R., & Pascual-Leone, A.

(1999). Changes in cortical auditory processing after blindfolding. Social Neuroscience

Abstracts, 25, 1628.

Sherrick, C., Cholewiak, R., & Collins, A. (1990). The localization of low- and high-frequency

vibrotactile stimuli. Journal of the Acoustical Society of America, 88(1), 169-178.

Spence, C., McDonald, J., & Driver, J. (2004). Exogenous spatial cuing studies of human

crossmodal attention and multisensory integration. Crossmodal space and crossmodal

attention, 277-320.

Stevens, J. C., Foulke, E., & Patterson, M. Q. (1996). Tactile acuity, aging, and braille reading

in long-term blindness. Journal of Experimental Psychology, 2(2), 91.

Stiles, N. R., & Shimojo, S. (2015). Auditory Sensory Substitution is Intuitive and Automatic

with Texture Stimuli. Scientific Reports, 5, 15628.

Stiles, N. R., Zheng, Y., & Shimojo, S. (2015). Length and orientation constancy learning in 2-

dimensions with auditory sensory substitution: the importance of self-initiated

movement. Frontiers in Psychology, 6, 842, 1-13.

Striem-Amit, E., Dakwar, O., Reich, L., & A, A. (2012). The large-scale organization of

“visual” streams emerges without visual experience. Cerebral Cortex, 22(7), 1698-

1709.

70

Striem-Amit, E., Dakwar, O., Reich, L., & Amedi, A. (2012). The large-scale organization of

“visual” streams emerges without visual experience. Cerebral Cortex, 22(7), 1698-

1709.

Striem-Amit, E., Ovadia-Caro, S., Caramazza, A., Margulies, D. S., Villringer, A., & Amedi, A.

(2015). Functional connectivity of visual cortex in the blind follows retinotopic

organization principles. Brain, 138(6), 16-1695.

Stronks, H. C., Nau, A. C., Ibbotson, M. R., & Barnes, N. (2015). The role of visual deprivation

and experience on the performance of sensory substitution devices. Brain Research,

1624, 140-152.

Stronks, H., Nau, A., Ibbotson, M., & Barnes, N. (2015). The role of visual deprivation and

experience on the performance of sensory substitution devices. Brain Research, 1624,

140-152.

Tang, H., & Beebe, D. J. (2006). An oral tactile interface for blind navigation. IEEE

Transactions on Neural Systems and Rehabilitation Engineering, 14, pp. 116-123.

Team, R. C. (2015). R: A language and environment for statistical computing. Retrieved from R

Foundation for Statistical Computing, Vienna, Austria: http://www.R-project.org

Tee, Z., Ang, L. M., Seng, K. P., Kong, J. H., Lo, R., & Khor, M. Y. (2010). SmartGuide

system to assist visually impaired people in a university environment. lete technical

review, 27, pp. 455-464.

Van Boven, R. W., Hamilton, R. H., Kauffman, T., Keenan, J. P., & Pascual-Leone. (2000). A

tactile spatial resolution in blind braille readers. Neurology, 54, 2230-2236.

Van Erp, J. B. (2005). Vibrotactile spatial acuity on the torso: effects of location and timing

parameters. Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for

Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005 (pp. 80-85).

First Joint: IEEE.

Van Erp, J. B., Van Veen, H. A., Jansen, C., & Dobbins, T. (2005). Waypoint navigation with a

vibrotactile waist belt. ACM Transactions on Applied Perception, 2(2), 106-117.

Varo, J. J., Martínez-González, M., de Irala-Estévez, J., Kearney, J., Gibney, M., & Martín, J.

A. (2003). Distribution and determinants of sedentary lifestyles in the European Union.

International Journal of Epidemiology, 32(1), 138-146.

Veraart, C. D., Wanet-Defalque, M. C., Bol, A., Michel, C., & Goffinet, A. M. (1990). Glucose

utilization in human visual cortex is abnormally elevated in blindness of early onset but

decreased in blindness of late onset. Brain research, 510(1), 115-121.

Verrillo, R., & Gescheider, G. A. (1975). Enhancement and summation in the perception of two

successive vibrotactile stimuli. Perception & Psychophysics, 18(2), 128-136.

Vierordt. (1870). Abhängigkeit der Ausbildung des Raumsinnes der Haut von der

Beweglichkeit der Körpertheile. Zeitschrift für Biologie, 6, 53-72.

71

Visual Impairment and Blindness. (2014, August). Retrieved December 29, 2016, from World

Health Organization: http://www.who.int/mediacentre/factsheets/fs282/en/

Voss, P., Lassonde, M., Gougoux, F., Madeleine, F., & Guellemot, J. P. (2004). Early- and late-

onset blind individuals show supra-normal auditory abilities in far-space. Current

Biology, 14(19), 1734-1738.

Vroom, J., & de Gelder, B. (2000). Sound enhances visual perception: Cross-modal effects of

auditory organization on visual perception. Journal of experimental psychology: Human

perception and performance, 26(5), 1583.

Wan, C. Y., Wood, A. G., Reutens, D. C., & Wilson, S. J. (2010). Early but not late-blindness

leads to enhanced auditory perception. Neuropsychologia, 48(1), 344-348.

Ward, J., & Meijer, P. (2010). Visual experiences in the blind induced by an auditory sensory

substitution device. Consciousness and Cognition, 19(1), 492-500.

Weaver, K. E., Richards, T. L., Saenz, M., Petropoulos, H., & Fine, I. (2013). Neurochemical

changes within human early blind occipital cortex. Neuroscience, 252, 222-233.

Weber, E. H. (1834). De Tactu. Leipzig: Koehler.

Weber, E. H. (1846). Der Tastsinn und das Gemeingefül . Leipzig: Engelmann, W.

Weber, E. H., Ross, H. E., & Murray, D. J. (1996). E. H. Weber on the tactile senses.

Psychology Press.

Weeks, R., Horwitz, B., Aziz-Sultan, A., Tian, B., Wessinger, C. M., Cohen, L. G., . . .

Rauschecker, J. P. (2000). A positron emission tomographic study of auditory

localization in the congenitally blind. Journal of Neuroscience, 20, 2664-2672.

Weinstein, S. (1968). Intensive and extensive aspects of tactile sensitivity as a function of body

part, sex and laterality. (D. R. Kenshalo, Ed.) The skin senses, 195-222.

Weinstein, S. (1968). Intensive and extensive aspects of tactile sensitivity as a function of body

part, sex, and laterality. In D. R. Kenshalo (Ed.), The Skin Senses (pp. 195-218).

Springfield, Illinois: Thomas.

White, B., Saunders, F. A., Scadden, L., Bach-y-Rita, P., & Collins, C. (1970). Seeing with the

skin. Perception & Psychophysis, 7(1), 23-27.

World Health Organization (2014). Visual Impairment and Blindness.

http://www.who.int/mediacentre/factsheets/fs282/en/

Wurtz, R. H., & Kandel, E. R. (2000). Central Visual Pathways. In E. Kandel, J. H. Schwartz, &

T. Jessell (Eds.), Principles of neural science (pp. 523-547). New York: McGraw-Hill.

Vigdís Vala Valgeirsdóttir - Skemman · 2016). One of the best examples of SS is the use of...

Documents

Transcript of Vigdís Vala Valgeirsdóttir - Skemman · 2016). One of the best examples of SS is the use of...