Alba Acoustics “See Your Sounds!”

Biomimetic Biomaterials Final Project Caroline Pisani, Julia Taussig, and Andy Tomaschke

May 4th, 2016

Alba Acoustics “See Your Sounds!”

Caroline Pisani, Julia Taussig, and Andy Tomaschke INTRODUCTION AND BACKGROUND Hearing is a luxury that can sometimes be taken for granted. It allows individuals to communicate through speech, enjoy music, and perform tasks more efficiently with the assistance of timers and notifications. On its most basic level, hearing provides individuals with an awareness of their environment and potentially hazardous situations. Furthermore, an individual’s ability to localize these sounds provides them with the time they need to react accordingly. Whether it’s knowing not to cross a busy street because of an oncoming ambulance, or knowing to vacate a dangerous area in the opposite direction of gunshots and explosions, sound localization can be a key factor in survival. Unfortunately, approximately 70 million individuals in the world lack this important ability as a result of either being born deaf or developing deafness later in life [1]. While it is costly and difficult to restore fully functional hearing in individuals, assuming a patient is eligible for a cochlear implant, it is much less difficult to restore the ability to localize sound. The obvious approach to restoring this ability is to simply place a microphone in either ear; but the best approach is found, as usual, in nature. Out of all the organisms on this planet that rely on sound localization to survive, none is more evolved than the Tyto alba, more commonly known as the barn owl. A barn owl’s ability to localize sound is so advanced that it is not only a tool for survival, but a weapon that maintains their place in the food chain. In a study performed by Roger S. Payne in the 1960s, he found that the barn owl can locate prey in total darkness using only the sense of hearing, with an error of less than one degree in both the vertical and horizontal planes [2]. The barn owl’s remarkable acuity at localizing sounds is attributed to two major factors: the heart shaped face which act as reflectors, and the asymmetry of the ear openings [2]. The right ear opening, according to Payne, is approximately 15 degrees higher than the left ear opening, which can be see in Figure 1 below. This asymmetry creates both intensity disparities and an inter­aural time difference [2].

1

Figure 1. Barn Owl Anatomy [3].

Inter­aural time differences occur when sounds arriving from one direction arrive at the right one ear first since it is closer in proximity to the origin of the sound. This causes a time difference between the two ears [4]. This cue is especially easy to use with abrupt sounds, as the onset of the sounds in each ear can be compared. This phenomenon is depicted in Figure 2 below. Additionally, if a sound is played at a position off to one side, sound intensities will be slightly different in the two ears. This is due to the fact that the sound paths are of different length since sound has to travel past the head to get to the far ear, and sound intensity decreases with distance, as 1/d2 (where d is the distance to the sound source) [4]. Additionally, the head interferes with the sound­wave, casting the auditory equivalent of a shadow on the far ear. This can be seen in Figure 3 below. It is these inter­aural time differences and intensity disparities created by the asymmetry of the ear openings that will be captured in the sound localization prototype.

Figure 2. Inter­Aural Time Difference Between Ears [4].

2

Figure 3. Distance from Sound Source Creates Intensity Disparities Between Ears [4].

Thus, the goal of this project is to create a sound visualization device inspired by the barn owl’s asymmetric ear openings in order to provide deaf individuals with an awareness of their environment. To achieve this goal, audio signals will be read in real­time from microphones positioned in an asymmetric array. Once the sensors have detected the signal and the inputs have been compared in the microprocessor, LEDs will illuminate on the front plane of the glasses in a location corresponding to the sound’s position. This design can be utilized to create a wearable, cost­effective device to restore sound localization for deaf and hard­of­hearing individuals. METHODS AND DESIGN In order to meet the goals of this project, design criteria were established and can be seen in Table 2. This table contains the priority and descriptions of each of the criteria for the sound localization device. One of the most essential design criteria for the device was for the sensors to be sensitive to sounds above a threshold decibel level. The sensors should only alert the user of potential hazards and “tune out” other normal sounds such as traffic passing by when walking by a road. Ideally, the user should be able to set a threshold value for the appropriate environment (e.g., for a concert or walking outdoors). Decibel levels of common sounds can be seen in Figure 4 below. Additionally, the device should illuminate the appropriate LEDs based on the direction of the sound and be bright enough for the user to see. These characteristics were deemed “Essential” for this project, and are listed as Criteria 1, 2, and 3 within the table.

3

Figure 4. Decibel Levels [5].

The final essential design criterion was in regards to user safety and is outlined as Criterion 4 below. The device must be safe for the user if water is present; thus, it was determined that the electrical components would need to be encased in a water­protective sealant while still being able to detect sounds accurately.

4

Another important aspect of the project was to mimic the sound localization of the North American Barn Owl. This project aimed to mimic their precise localization of sound by placing four microphones in an asymmetric array similar to that of the Barn Owl. This criterion was established as “High” priority, and is listed as Criterion 5. It is also important to mimic how the human brain utilizes time and amplitude difference of sounds arriving in each ear to determine location of sounds. The signal from each of the four microphones will be read in real­time, and the direction of the sound will be determined based on which sensors detect the highest value of intensity. Once the sensors have detected the signal, LEDs will illuminate on the front plane of the glasses in a location corresponding to the sound’s position. The microphone array will collect signals and transmit them to the Arduino microprocessor, which will process the signals and illuminate the appropriate LEDs. Additionally, the project aims to mimic the human auditory response time, as indicated in Criterion 6. The sensors should be able to detect sound and illuminate LEDs fast enough for the user to sense the sound and respond rapidly. It takes approximately 10 milliseconds for sound to pass one ear and to reach the other, so the time for a sound to travel from the outer ear to the aural nerve is most likely less than 10 milliseconds [6]. This device can mimic the average time to perceive a sound by minimizing the time for the sensors to detect a sound, transmit a signal to the processor, process the information, and illuminate appropriate LEDs. The brain can process images in as little as 13 milliseconds [7]. Additionally, it takes people about 200 milliseconds to consciously combine sight and sound [8]. Thus, the device must detect a sound and transmit a signal (in the form of LED light illumination that can be processed in about 13 milliseconds) to the user by lighting LED lights within 150 to 200 milliseconds in order to allow the user to detect a hazard as fast as a person with healthy vision and hearing. The final three criteria are related to user friendliness and wearability, as described in Criteria 7, 8, and 9. The device must be small and light as to not be cumbersome to the user. Further, the device should contain a power source that is able to operate for 12 hours before requiring additional charging. Finally, the device will have low manufacturing costs, which will translate to low costs for the user while also being high quality, high performing, durable, and easy to maintain.

5

Table 1. Design Criteria

# Criterion Priority Description

1 Detect sounds greater than 100 dB or a user­set threshold decibel level

Essential A 100 dB threshold was chosen as this will make the user aware of sounds such as sirens and gunshots without concerning them with sounds such as a TV or a nearby conversation [5].

2 The LEDs that illuminate when loud sounds are detected are easy for the user to see and indicate the direction of the sound

Essential The direction of the sound will be determined based on which sensors detect the signal first and second. Once the sensors have detected the signal, LEDs will illuminate on the front plane of the glasses in a location corresponding to the sound’s position.

3 The LED lights are bright enough to alert the user but do not irritate the user’s eyes

Essential According to Dr. Gregory Good, short visible light in the “blue­light” portion of the visible spectrum (380 nm to 500 nm) can cause damage to retinal structures, so LED lights that emit potentially damaging short visible light will not be used [9].

4 The electronics are safe if there is water present (the user is safe from electrical hazards).

Essential The electrical components will need to be encased in a water­protective sealant while the sensors can still detect sounds accurately and the lights can still illuminate brightly.

5 Asymmetry of the sensors to mimic the asymmetry of barn owl openings to their ear canals

High The barn owl’s left canal opening is shifted below an imaginary plane drawn through the center of the eyes, and the right canal opening is shifted just above [3]; therefore, the sensors will be placed as follows: the left sensor will be below the imaginary plane drawn through the center of the eyes, and the right sensor will be above the center plane [2].

6 Mimic how the human brain utilizes time difference and amplitude difference of sounds arriving in each ear to determine location of sounds

High The ear closer to the sound receives the sound slightly earlier, and the brain can detect differences of as little as 10 microseconds to determine the location of the sound [10]. The human brain can also detect differences in sound amplitude due to the “acoustic shadow” that the human head creates, causing different amplitude sound waves to arrive at each of the ears [4].

7 Wearable: The sensors, signal­processor, and power source are small and light enough to not be

High Specialists will need to help the user learn to use the glasses and to maintain a proper posture to allow for the sensors to operate optimally.

6

cumbersome to the glasses­wearer

8 The sensor power operates well for over 12 hours without being re­charged

Medium The end­user would be more likely to consistently use the Alba Acoustics device if it had its own power source so that he/she would not need to rely on power plugs and sockets several times throughout the day.

9 Inexpensive to manufacture, low cost for the user yet high quality, high performing, durable, and easy to maintain

Medium If the glasses, electronics, manufacturing, and transportation costs are minimized, the device can be sold for a lower price while earning a profit. This increases the availability of the device to a broader population.

RESULTS An image of the prototype is shown below in Figure 5, with each of the four sensors labeled. Additionally, the Bill of Materials can be seen in Table 2.

Table 2. Bill of Materials

Part Cost

Arduino Microprocessor $24.95

Breadboard $4.00

LEDs (4) $8.00

Sound Sensors (4) $40.00

Wire $2.50

Balsa Wood and Bolts $10.00

TOTAL $89.45

7

Figure 5. Prototype Schematic with Sensor Arrangement and LED Legend

In order for the direction of sound to be determined, the Arduino microprocessor compares sound intensities from the four sensors. The intensity comparisons made in the processing code are outlined in Table 3 below.

Table 3. Sensor Intensity Comparisons and Corresponding Sound Directions

First Sensor Second Sensor Direction of Sound

1 2 Front and Above

2 1 Front and Below

1 or 3 3 or 1 Right

2 or 4 4 or 2 Left

3 or 4 4 or 3 Behind

In order to test the prototype sound­localization device, the tester clapped his/her hands at a relatively consistent sound level one foot from the device in different directions for each trial. The tester visually inspected whether the appropriate LED lights illuminated for five seconds in five directions in the perspective of the user: left, right, front­and­above, front­and­below, and behind. The prototype successfully illuminated the appropriate LED lights for loud sounds in each direction tested (see Table 4 below).

8

Table 4. LED Illumination

Direction (Perspective of User) LED(s) that Illuminated (Color) for Five Seconds

Left Red

Right Green

Front and Above Yellow

Front and Below Blue

Behind Red, green, yellow, and blue

Figure 6 outlines the experimental set­up for testing the sound­localization device.

Figure 6. Results of Testing: a) Front and Below b) Front and Above c) Left d) Right e) Rear

9

DISCUSSION The prototype successfully illuminated the appropriate LED lights for loud sounds in each direction tested. However, the experiments conducted to test the functionality of the prototype were not well­controlled. In the future, tests can be conducted in a more controlled environment such as a sound­proof room with standard sounds at set decibel levels played via a speaker. Additionally, several sounds at or above the user­defined decibel threshold can be selected for use in testing. Each of the sounds can be tested in various locations (the locations listed in Table 4, as well as additional locations such as in front, at level to the user) at different distances from the device to determine if the appropriate LEDs illuminate for five seconds in response to the sound. Arduino software can also be used to test variances in sound intensities that cause LEDs to illuminate as well as the amount of time it takes to detect a sound and then illuminate the LED. The prototype, as it currently stands, is not a finished consumer product, and several improvements need to be made to turn the proof of concept into a user­friendly wearable. The first modification that needs to be made is the size and sensitivity of the sensors. A smaller form factor for the sensors will allow them to be aesthetically integrated into a wearable while the increased sensitivity will allow the programming to detect differences in intensity even with the sensors moved closer together. Following this, the next important modification would be to replace the Arduino Uno with a processing device that has an analog­to­digital converter for each sensor. Due to the fact that the Arduino Uno only has one analog­to­digital converter, it is only capable of reading one sensor at a time with a 100 microsecond delay in­between. This results in a slight inaccuracy in intensity readings and an inability to use the inter­aural time difference in signal processing. A device with an analog to­digital­converter for each sensor will allow the sensors to be read simultaneously, resulting in more accurate values of intensities and the ability to use the inter­aural time difference as a double­check in the code. The third modification is to add wireless communication between the sensors and processing device or to develop an on­device logic board. Wireless communication allows the processing device to be stand­alone, allowing the user to place the processing device in a pocket or handbag. It also allows the use of separate power supplies: one on the wearable for the sensors and LED and one for the processing device. An on­device logic board makes the wearable an all­in­one device, but will most likely suffer in power duration. The fourth modification is the power supply. For the device to be truly wearable, the user cannot be required to maintain connection to an outlet. Thus, a power source will need to be developed that can power the device for 12 hours without needing to be recharged. The last modification that needs to be made is to reduce the four LED indicators to two color­changing LEDs or a small display. Four LEDs simply takes up too much of the user’s view, so reducing the number of LEDs to two or replacing them with a small display results in less obstruction of the user’s view.

10

CONCLUSIONS The Alba Acoustics device functioned well and fulfilled most of the essential and high priority design criteria. The device was able to detect sounds greater than 100 dB or a user­set threshold decibel level (Criterion 1) because the testers were able to change the threshold intensity above which the device would illumine LED lights. When the LED lights illuminated, they were easily understood by the testers because different colors indicate different directions, helping the user easily localize the source of the sound (Criterion 2). The LED lights were bright enough (while not irritating the user) to alert the tester, and remained lit for five seconds to ensure that the user sees the lights before they turn off (Criterion 3). The Alba Acoustics device prototype was built with asymmetry of the sensors in order to mimic the asymmetry of barn owl openings to their ear canals (Criterion 5). The prototype also utilized the bioinspiration of how the human brain utilizes amplitude difference (intensity) of sounds arriving in each ear to determine location of sounds (Criterion 6). The prototype materials cost $89.45 (see Table 2), and thus it is likely that the final product will be inexpensive to manufacture and low­cost for the user, yet high quality, high­performing, durable, and easy to maintain (Criterion 9). The Alba Acoustics device can still be improved because it does not yet fulfill some of the criteria. For example, the device still needs to mimic how the human brain utilizes time difference of sounds arriving in each ear to determine locations of sounds (Criterion 6), the electronics are not yet safe if there is water present (Criterion 4), the device is not yet wearable (Criterion 7), and the sensor power does not yet operate well because it depends on a laptop using a USB connection as a power source (Criterion 8). The Alba Acoustics device could still be improved, but the prototype’s performance can be seen as a success in the product development perspective. The Alba Acoustics development team learned about the challenges of trying to fulfill all of the design criteria when building a prototype. The essential design criteria were prioritized, and it was accepted that more time and funds would be necessary to fulfill all of the design criteria. The team learned that it is difficult to send analog­to­digital data from four sensors to Arduino software at the same time. The team tried to analyze time differences between when each sensor of the system detected the sound in order to localize the sound and to analyze the performance of the system (minimum time differences that the device could use to detect locations of sounds), but this data could not be recorded by the Arduino. Analog inputs were therefore used to determine which sensors detected higher intensities which allowed the team to create a prototype that illuminated LEDs that located sources of sounds. Multiple Arduino microprocessors are most likely required in order to detect and record the differences in time between when each detector detected sounds. Overall, this project was a success seeing as the team was able to develop a prototype that demonstrated the principle of operation and biomimetic inspiration, while meeting a majority of the design criteria.

11

REFERENCES

1. "FAQ ­ WFD | World Federation of the Deaf." WFD World Federation of the Deaf. WFD World Federation of the Deaf. Web. 08 Mar. 2016.

2. Payne, Roger. "ACOUSTIC LOCATION OF PREY BY BARN OWLS." Journal of

Experimental Biology. 54 (1971): 535­73. Web. 8 Mar. 2016.

3. "Barn Owl (Tyto Alba)." Barn Owl (Tyto Alba). The Robinson Library. 16 Feb. 2015. Web. 09 Mar. 2016. <http://www.robinsonlibrary.com/science/zoology/birds/strigiformes/barn.htm>.

4. "Perception Lecture Notes: Auditory Pathways and Sound Localization.” Perception Lecture

Notes: Auditory Pathways and Sound Localization. Web. 21 Apr. 2016. <http://www.cns.nyu.edu/~david/courses/perception/lecturenotes/localization/localization.html>

5. Kinney, Chance. "V-Moda Faders VIP Could Save Your Hearing." Chip Chick. Chip

Chick Media. 11 May 2012. Web. 09 Mar. 2016. <http://www.chipchick.com/2012/05/faders-vip-save-hearing.html>.

6. “Audition: Hearing, the Ear, and Sound Localization.” Boundless Psychology. Boundless.

20 Aug. 2015. Web. 10 Apr. 2016. <https://www.boundless.com/psychology/textbooks/boundless­psychology­textbook/sensation­and­perception­5/sensory­processes­38/audition­hearing­the­ear­and­sound­localization­162­12697/>

7. Trafton, Anne. “In the Blink of an Eye.” MIT News. Massachusetts Institute of

Technology. 16 Jan. 2014. Web. 21 Apr. 2016. <http://news.mit.edu/2014/in­the­blink­of­an­eye­0116>.

8. Sanders, Laura. “How the Brain Perceives Time.” Science News. Society for Science &

the Public. 15 July 2015. Web. 21 Apr. 2016. <https://www.sciencenews.org/article/how­brain­perceives­time>.

9. Good, Gregory W. “Light and Eye Damage.” American Optometric Association.

aoa.org. 2 Dec. 2014. Web. 21 Apr. 2016. <http://www.aoa.org/Documents/CRG/Blue%20Light%20and%20Eye%20Damage.pdf>.

10. "How The Human Ear Works." How The Human Ear Works. Web. 21 Apr. 2016.

<http://web.mit.edu/2.972/www/reports/ear/ear.html>

12