Category: Science

Introduction:

Goal: Sensorimotor synchronization (SMS) is the rhythmic synchronization between a timed sensory stimulus and a motor response. In this experiment we will have you performing an SMS task of hearing/watching a tempo changing recurrent auditory/visual stimulus and response to it by generating a corresponding sequence of click sounds.
Task: You are supposed to generate a one-to-one response in synchronization with the auditory/visual sequence of clicks/flashes. Your sensory act is to listen or watch the sequence generated by a computer application and your motor response will be your hand-clapping, finger-tapping or hitting a given weight against a pad. Your results will be analyzed to discover how humans synchronize with a perceived changing rhythm under different circumstances. We are particularly seeking if different subjects act considerably different given their individual properties (musical background, age, gender, etc).

Anonymity: Your results with remain anonymous and will not be reported, published anywhere with your name; however, you can ask the test holders to give you your own analyzed plots in case that you want to know more about your characteristics of rhythm-adaptation.
Safety: Basically there will be no harm caused by such tests, however, if you have a background in epileptic seizure or any neural disorder you may please inform the experimenters. Besides your arm/hand/fingers that you will use during the test must be in a healthy condition.

Compensation: The experiment will be done in two days. For each day, a subject needs to perform a 45 minutes trial and at the second day takes two cinema tickets for accomplishing both trials.

Instructions for the first day experiment (Auditory Rhythm Adaptation):

1. The test is done in an anechoic chamber (an acoustically dead room) that absorbs sound perfectly. The pyramid-shaped pieces on the walls and floor are covered by carbon and iron, so you may watch your clothes not to get dirty if you lean on them.
2. Please switch off your phone and take off your watch, wristband, etc. Piercing is fine. The idea is that you should feel comfortable during the test and do not attach considerable weight to your hand. Since the room’s floor is covered by a grid, it will be very hard to find your small accessories back if they are dropped from your hand or the experiment test. So you may hand them to the experimenters before you start the required task.
3. Your task is to generate a one-to-one response in synchronization with the auditory sequence of clicks you hear.
4. To make the click sound, you can choose your dominant hand, in case that you are left-handed or right-handed. But you must use the same hand during the whole session.
5. You will be blindfolded. Please relax, sit comfortably and focus on the stimuli during the each session. The test needs your 100% attention.
6. There are three different motor tasks and you will be told by the experimenters by which mean you generate the click sounds:
   • For the finger-tapping sessions, you are asked to use the wrist to lead an abrupt, pulsed action and to suddenly release the downward force on the laptop computer keyboard’s space button.
   • For the clapping sessions, you use a take a figure-to-palm position of the right hand relative to the left hand (or vice versa) forming a right angle with a natural curvature.
   • For the weight-hitting sessions, you will be given a metal cylinder (a group of them taped to each other). Your task is holding them vertically in your preferred hand and to make the click sound by hitting the weight against a metal pad. Please keep your elbow fixed on the experiment table and do not twist your wrist to hit the pad.
7. Each trial will take up to 45 minutes and has three session
8. Each session lasts 13 minutes and is divided into three stimuli sequences of 4 minutes each, with two short breaks. You need to remain seated during a session but you can take a longer break when the experimenter gets back to you, once you finish a session.
9. In each 4-minute sequence of auditory stimuli you will listen to a series of a single piano note via a microphone. You are asked to follow the changing rhythm and simultaneously generate a click sound in synchronization with any click you hear. This means that if the tempo changes you should adapt to it as soon as you can.
10. If your task is not clear, please feel free to ask the experimenter before you start.

Instructions for the second day experiment (Visual Rhythm Adaptation):

1. The test is done in an anechoic chamber (an acoustically dead room) that absorbs sound perfectly. The pyramid-shaped pieces on the walls and floor are covered by carbon and iron, so you may watch your clothes not to get dirty if you lean on them.
2. Please switch off your phone and take off your watch, wristband, etc. Piercing is fine. The idea is that you should feel comfortable during the test and do not attach considerable weight to your hand. Since the room’s floor is covered by a grid, it will be very hard to find your small accessories back if they are dropped from your hand or the experiment test. So you may hand them to the experimenters before you start the required task.
3. Your task is to generate a one-to-one response in synchronization with the visual sequence of flashes you observe.
4. To make the click sound, you can choose your dominant hand, in case that you are left-handed or right-handed. But you must use the same hand during the whole session.
5. You need to stare at the flashing screen during a session. Sit comfortably and be prepared for focusing on the stimuli. The test needs your 100% attention.
6. There are three different motor tasks and you will be told by the experimenters by which mean you generate the click sounds:
   • For the finger-tapping sessions, you are asked to use the wrist to lead an abrupt, pulsed action and to suddenly release the downward force on the laptop computer keyboard’s space button.
   • For the clapping sessions, you use a take a figure-to-palm position of the right hand relative to the left hand (or vice versa) forming a right angle with a natural curvature.
   • For the weight-hitting sessions, you will be given a metal cylinder (a group of them taped to each other). Your task is holding them vertically in your preferred hand and to make the click sound by hitting the weight against a metal pad. Please keep your elbow fixed on the experiment table and do not twist your wrist to hit the pad.
7. This trial will take up to 45 minutes and has three sessions:
8. Each session lasts 13 minutes and is divided into three stimuli sequences of 4 minutes each, with two short breaks. You need to remain seated during a session but you can take a longer break when the experimenter gets back to you, once you finish a session.
9. In each 4-minute sequence of visual stimuli you will watch a black screen that continuously flashes in white. The flashing light changes its frequency and you are asked to follow the visual rhythm and simultaneously generate a click sound in synchronization with any white flash you observe. Each white flash takes 300mS and is long enough to perceive but try to make the sound immediately when the screen becomes white, not when it turns back to black.
10. If your task is not clear, please feel free to ask the experimenter before you start.

Please contact i@nim.ir in case that you are interested in taking part in the experiment. Thanks for consideration! 🙂

Could these virtual people at some point jam together? Or could I ask the avatar if he knows the answer to this? If he knows, can he also research how we can jam with him? And then can he at some point own a TV and designs another avatar by his own?

This actually reminded me of an idea I had in childhood. A movie about people living great lives and at the end they sadly understand that they are game avatars created by more “real” creatures. Fictionists should write more on this topic.

Title: Modeling human auditory synchronization behavior based on EEG data

Student: –
Supervisor: –
Co-supervisors: Nima Darabi, PhD at Q2S / Peter Svensson, Professor, IET and Q2S
Semester: Spring 2010

An overall view: While dancing, performing or listening to a rhythmic music, we are synchronizing our reactions based on the auditory stimuli. The characteristics of such an action depend on how we use our short-term auditory memory, as the ability to recall something heard very recently. Every suggested synchronization model should take the role of this memory into account. The traditional way of understanding this memory function is analysis of the recorded behavior of synchronous cooperative subjects such as processing of their produced sound signals. In addition to this, the measurement of the electrical brain activity might be a very useful source, assigning to the relative sound signals and provided by EEG’s good temporal resolution.

The Assignment: In this suggested project, We will do experiments in which subjects passively listen to hand claps, in order to find out how that translates to EEG activity. We will set up some well-defined and simple subjective experiments with auditory stimuli and use quantitative EEG methods (mathematical measurement of aspects of the EEG signal) to analyze the provided information. The process should be passive given the circumstances such as EEG sensitivity to body movements and their results are supposed to address these questions:

• How the brain electrical activity is influenced by passive rhythm perception?
• Which temporal structures are human perceivable as rhythm?
• How fast the auditory stimuli can be traced in the electrical brain activity?
• How this can be changed by different patterns and tempos?
• How much is this individual or training dependent?
• How we can explore memory aspects of human rhythmic and musical behavior using EEG?
• when experimenting with delays between handclapping performers, and
  we ask them to keep the tempo stable, it sounds bad to them but they can keep the tempo. On average, though, people compensate not enough so the tempo always decreases. Why don’t they compensate completely?

We also need to figure out the extent to which asynchrony is perceived. We will come up with a model that describes the behavior of a clapper. One aspect is the perception of the other clapper and the perception of one’s self: EEG might be able to show when a situation is unusual which might be related to asynchrony.

In this case we just have subject listen to a bunch of pre-recorded hand claps. Do an experiment with active clapping; subsequently, compare the EEG data and find any differences. Here E(t), B(t) and A(t) stand for Ear(t), Brain(t) and Arm(t). Brain (T) comes from EEG data related to hearing of stimuli and will help us to understand more about the processing of perceived stimuli (synchrony check / prediction performed by subject), and movement of arms, respectively.

This study finally aims at defining some memory/inertia-related parameters as a measure of strategy taken by the performers in a musical collaboration/synchronization process.

E-228 presents:

Implementing computer clapper as a tool for subjective rhythmic experiment (+)

In this presentation, Jordi will show some of his last works as a developer in the field of audio visual interactivity. Furthermore, the Computer-Clapper software will be presented, an application developed for Nima Darabi which is a programmable sequencer and impulse response detector to achieve rhythm performance metrics. Jordi will introduce MAX/MSP as an interface to develop subjective auditory and visual tests. This interface not only is used to implement human-computer subjective tests, but also will later on be used to implement the computer-clapper as a serious game.

At the end Nima Darabi will finalize Jordi’s talk to show how the observed step responses gathered from the clapper software is used to model human reaction to the tempo change by e second order damped harmonic oscillator like a damped mass spring system. Some objective quality assessment metrics will also be discussed specifically for musical interaction.

Andrew Perkis has posted to all:

Here is a challenge! A friend of mines daughter took this picture of herself lying next to a mirror. As you see her eyes are open, however, in the mirror they are closed. How can this be explained? All comments appreciated!

Together with Jordi we tried to examine our theory and ended up with these two results. We tried, not so patiently though. So eventually that worked out, but not as good as the original one:

Another day we gave it a try with the newcomer colleague:

Here is another try, it still doesn’t look as extreme as the the girl on the photo. But it seems it goes into the right direction. 🙂

Said Dirk:

Jordi mentioned that it might be also an issue with the release time of the camera, which sounds quite reasonable. If the CCD sensor is quite slow (old cameras had this issue) and writes the data from left to right, the girl might have started with closed eyes and opened them fast while taking the picture. So maybe it is a mixture of both, perspective and crappy camera.

Jordi found some links supporting the possibility that more complicated stuff than static optics might contribute:

I guess the strange camera issue it is somehow related to the known “slow scan” or “photon gating” behavior in phone cameras. Here you have some links talking about this issue or new feature.

Photon gating makes for interesting cameraphone pictures

Take Distorted and Psychedelic iPhone Photos