McGurk Effect | Simply Psychology

The McGurk effect is a perceptual phenomenon which happens when a person perceives that the movement of another individual's lips do not match ... Togglenavigation TheoriesResearchMethodsMentalHealthA-levelStatistics CognitivePsychologyMcGurkEffectTheMcGurkEffectByAyeshPerera,publishedJune10,2021Take-homeMessagesTheMcGurkeffectoccurswhenapersonperceivesthatanother’slipmovementsdonotcorrespondtowhatthatindividualissaying.Cognitivepsychologists,HarryMcGurkandJohnMacDonald,introducedtheconceptoftheMcGurkEffectin1976,afteraccidentallydiscoveringthephenomenonduringanexperiment.Variousexternalfactorssuchasvisualdistraction,tactilediversion,familiarityandsyllablestructurecanimpacttheMcGurkeffect.Braindamage,Alzheimer’sdisease,SpecificLanguageImpairmentandaphasiaaresomeinternalfactorsinfluencingthephenomenon.ThemagnitudeoftheMcGurkeffectmayvaryacrosslanguages;however,suchvariationvanisheswhentheaudiostimulibecomeunintelligible.TheMcGurkeffectisaperceptualphenomenonwhichhappenswhenapersonperceivesthatthemovementofanotherindividual’slipsdonotmatchupwithwhatthatindividualisactuallysaying.Inotherwords,itisanillusionwhichoccursintheinteractionbetweenvisionandhearingintheperceptionofspeech.Herein,theopticalcomponentofonesoundiscoupledwiththeaudialcomponentofanothersound.Thispairingsubsequentlyinducestheperceptionofathirdsound.Whileacombinationofpoorauditorystimuliandhigh-qualityocularinformationmayconstituteacommonsourceoftheMcGurkeffect,amultiplicityoffactorsinfluencethemagnitudethisphenomenon.OriginandHistoryTheMcGurkeffectwasfirstintroducedin1976bythecognitivepsychologists,HarryMcGurkandJohnMacDonaldinapapertitled,HearingLipsandSeeingVoices(McGurk&MacDonald,1976).ThiseffectwasaccidentallydiscoveredwhileMcGurkandhisassistant,MacDonald,wereconductingastudyontheperceptionoflanguagebyinfantsatvariousdevelopmentalstages.Duringthestudy,theywould,inonelocation,playthevideoofamotherspeaking,andinanotherlocation,playthesoundofhervoice.Intheexperiment,theydirectedatechniciantodub,withtheauditorysyllable“ba,”avideotapewithavisualof“ga.”Whenthedubbedtapewasplayed,MacDonaldandMcGurkperceived“da”—aphonemedistinctfromboththedubbedaudioandthevisual.BothMcGurkandMacDonaldwereconfused.Eventually,however,theyrealizedthatthephenomenonstemmednotfromamistakebythetechnicianbutfromanidiosyncrasyinhumanperception.ExternalFactorsImpactingtheMcGurkEffectVisualDistractionAstudyoftheroleofopticattentioninaudiovisualspeechperceptionobservedtheMcGurkeffectintwodifferentsituations(Tiippana,Andersen&Sams,2004).Inthefirstinstance,thelistener’sattentionwasfocusedonthefaceofthespeakerwhereasinthesecondinstance,thelistenerignoredthefacebypayingattentiontoaleafwhichmovedacrossthetalker’sface.TheresultsdemonstratedthattheMcGurkeffectwasweakerinthelatterscenario.Thisoutcomewasattributedtothemodulationofaudiovisualspeechperceptionbyvisualattention.Themodulationherein,couldhavetranspiredeitheratapremature,unisensorystateofprocessing,oronaccountofalterationsattheintegrationstageofvisualandauditoryinformation.TactileDiversionResearchhasrecentlychallengedthepopularassumptionthataudiovisualpairingtranspiresinanattention-freemode(Alsius,Navarra&Soto-Faraco,2007).Astudyinvestigatedwhetheraudiovisualspeechintegrationishinderedbythedepletionofopticandaudialattentionalresources,asattentionaldemandsareimposedonthetactiledomainwhichisnotdirectlyassociatedwithspeechperception.TheMcGurkeffectwasmeasuredinadualtaskmodelwhichinvolvedanexactingtactileassignment.Theresultsindicatedthattheproportionofvisuallyswayedresponsestoaudiovisualinformationabatedasattentionwasdeflectedtothetactiletask.Thisoutcomewasattributedtothemodulatoryimpactonaudiovisualbindingofspeechwhichismediatedbythelimitationsofsupramodalattention.Thesefindingsseemtosimultaneouslyprovideaglimpseofthedynamismandtheextensivenessoftheinteractionsbetweencrossmodalbindingmechanismsandtheattentionalsystem.FamiliarityAnexperimentwasconductedtoexaminetheclaimsfortheindependenceoffacialspeechprocessingandfacialidentity(Walker,Bruce&O'Malley,1995).Inthestudy,thefacesusedtocreatetheMcGurk-effectstimuliweremanipulatedsothatwerealientosomeparticipantsbutfamiliartoothers.Moreover,thevoicesandthefacesutilizedwereeithercongruent(belongedtothesameindividual)orincongruent(belongedtodifferentindividuals).ThedistinctparticipantgroupswerecomparedtoeachothertogaugetheirsusceptibilitytotheMcGurkillusion.Theresultsindicatedthatwhenthevoicesandthefaceswereincongruent,theparticipantswhowereacquaintedwiththefaceswerelesssusceptiblethanthosewhowereunacquaintedwiththefaces.Thisoutcomeimpliesthatfacialspeechandfacialidentityarefarfromindependent,andthatthosewhoarefamiliarwiththefacesofthespeakersarelesslikely,thanthosewhoareunfamiliar,tobeswayedbytheMcGurkeffect.SyllableStructureOneresearchstudyconductedfourexperimentstodecidewhetheropticinformationinfluencesthejudgmentsofacoustically-specifiednonspeecheventsandspeechevents(Brancazio,Best&Fowler,2006).ThestudyemployedclicksoundswhichareperceivedasnonspeechbymanyEnglishlisteners,butfunctionasconsonantsincertainAfricanlanguages.TheresultsdemonstratedasignificantMcGurkeffectforisolatedclicks.Thiseffecthowever,wasnotablysmallerthanthatforthestop-consonant-vowelsyllables.Moreover,forclick-vowelsyllables,strongMcGurkeffectswerediscoveredwhichweresimilartothoseforEnglishsyllables.Ontheotherhand,forexcisedreleaseburstsofstopconsonantsinisolation,weakMcGurkeffectswerefound;theseweresimilartotheeffectsforisolatedclicks.ThisoutcomeshowsthattheMcGurkeffectmayoccureveninnon-speechsettings.Moreover,whilephonologicalsignificanceisnotaprerequisitefortheMcGurkeffect,itdoesseemtointensifyit.InternalFactorsImpactingtheMcGurkEffectBrainDamageThehemispheresofthebraincooperatetointegratespeechinformationreceivedviatheopticandauralsenses(Baynes,Fummell&Fowler,1994).Right-handedindividuals,forwhomwordshaveprivilegedaccesstothelefthemisphereandthefacetotherighthemisphere,aremorelikelytoexperienceaMcGurkeffect.ResearchalsoshowsthattheMcGurkeffect,thoughpresent,issignificantlyslowerforthosewhohaveundergonecallosotomy.Moreover,visualstimulistronglyimpactspeechperceptioninindividualswithlesionstothelefthemisphere,andtheydemonstratealargerMcGurkeffectthantheaverageperson(Schmid,Thielmann&Ziegler,2009).However,theywouldbelesslikelytoexperiencetheMcGurkeffectifthedamagetothelefthemispherehadcompromisedtheirvisualsegmentspeechperception(Nicholson,Baum,Cuddy&Munhall,2002).Ontheotherhand,individualswhoserightbrainhemispherehasbeendamaged,demonstrateimpairmentonvisual-onlyaswellasaudio-visualintegrationfunctions.Moreover,thoughtheseindividuals’integrationoftheinformationcanproduceaMcGurkeffect,suchintegrationappearsonlyifopticstimuliareutilizedtoenhanceperformancewhentheauralsignalispoor.Thus,althoughpersonswhoserighthemisphereisdamagedmayexhibitaMcGurkeffect,itisnotstrongasitwouldbeinanormalgroup.Alzheimer’sDiseaseAstudywhichinvestigatedbrainconnectivityassociatedwithAlzheimer’sdisease(AD)viatheevaluationofacrossmodaleffect,examinedtheMcGurkeffectinthoseafflictedwiththediseaseandmatchedthecontrolparticipants(Delbeuck,Collette&VanderLinden,2007).TheresultsunveiledtheimpairmentofcrossmodalintegrationinspeechperceptionforAD.However,thephenomenonwasnotassociatedwithinterruptionsinthedistinctprocessingofvisualandauditoryspeechstimuli.Thisoutcomeimpliesthatthespecificaural-opticintegrationdeficitinADpatientsmightbetheresultofaconnectivitybreakdown.SpecificLanguageImpairmentAstudywhichexamined28preschoolerswithSpecificLanguageImpairment(SLI)and28preschoolerswithoutSLIsoughttoanalyzetheircapacityforauditory-visualintegration(Norrix,Plante,Vance&Boliek,2007).Whileboththegroupsperformedequivalentlyincongruentaudio-visualmodalities,intheincongruentaudio-visualcondition,thechildrenwithSLIdemonstratedaMcGurkeffectwhichwasweakerthanthatobservedforthosewithoutSLI.TheresultsindicatethatthosewithSLIexperienceasignificantlylowerMcGurkeffectthanthosewithoutSLI.Whiletheymaypaylessattentiontoarticulatorygesturesandemploylessvisualinformationinperceivingspeech,theyencounternosignificantchallengesinperceivingexclusivelyauralcues.AphasiaAninvestigationoftheabilityofapersonafflictedwithmildaphasiatorecognizetokensofferedinvisualonly,auditoryonlyandaudio-visualconditionsyieldedresultsmeritingattention(Youse,Cienkowski&Coelho,2004).Thehypothesiswasthatinthebimodalcondition,performancewouldbeideal,andthattheMcGurkeffectwouldexhibitintegrationofspeechinformation.Theresults,however,didnotsupportthehypotheses,butsuggestedthataperseverativeresponsepatternislimitingthesuccessfulintegrationofaudio-visualspeechinformation,andtheutilizationofbisensoryspeechstimulimaybecompromisedinadultsafflictedwithaphasia.TheMcGurkEffectinDifferentLanguagesRegardlessofthelanguagebeingused,listenersgenerallydepend,tosomedegree,onvisualinformationinspeechperception.However,theintensityoftheMcGurkeffectvariesacrosslanguages.Forinstance,Spanish,Italian,Turkish,English,DutchandGermanlistenersexperienceastrongerMcGurkeffectthanChineseandJapaneselisteners(Sekiyama,1997;Bavo,Ciorba,Prosser&Martini,2009;Erdener,2015).Theculturalpracticeofavoidingeyecontact,andtonicandsyllabiclinguisticstructuresmightaccountforthisdiminishedeffectamongtheJapaneseaswellastheChinese.Researchalsoshowsthat,unlikeEnglishchildren,Japanesechildrendonotdemonstrateadevelopmentaladvancementinvisualinfluencefollowingagesix(Sekiyama&Burnham,2008;Hisanaga,Sekiyama,Igasaki&Murayama,2009).However,JapaneselistenersrecognizetheincompatibilitybetweenauralandopticstimulibetterthanEnglishlisteners—perhapsonaccountofJapanese’swantofconsonantclusters(Sekiyama&Tohkura,1991).Notwithstandingtheaforementionedmanifestdifferences,listenersofalllanguagesarecompelledtorelyonopticstimuliwhenaudiostimuliareunintelligible.Whenthisoccurs,variationacrosslanguagesdisappearsandtheMcGurkeffectisappliedequally.Howtoreferencethisarticle:Prera,A(2021,June10).ThemcGurkfffect.SimplyPsychology.www.simplypsychology.org/mcGurkeffect.htmlAPAStyleReferencesAlsius,A.,Navarra,J.,&Soto-Faraco,S.(2007).Attentiontotouchweakensaudiovisualspeechintegration. ExperimentalBrainResearch, 183(3),399-404.Bovo,R.,Ciorba,A.,Prosser,S.,&Martini,A.(2009).TheMcGurkphenomenoninItalianlisteners. ActaOtorhinolaryngologicaItalica, 29(4),203.Baynes,K.,Funnell,M.G.,&Fowler,C.A.(1994).Hemisphericcontributionstotheintegrationofvisualandauditoryinformationinspeechperception. Perception&Psychophysics, 55(6),633-641.Boersma,P.(2012).Aconstraint-basedexplanationoftheMcGurkeffect. PhonologicalArchitecture:Empirical,TheoreticalandConceptualIssues,299-312.Brancazio,L.,Best,C.T.,&Fowler,C.A.(2006).Visualinfluencesonperceptionofspeechandnonspeechvocal-tractevents. Languageandspeech, 49(1),21-53.Delbeuck,X.,Collette,F.,&VanderLinden,M.(2007).IsAlzheimer'sdiseaseadisconnectionsyndrome?:Evidencefromacrossmodalaudio-visualillusoryexperiment. Neuropsychologia, 45(14),3315-3323.Erdener,D.(2015).TheMcGurkillusioninTurkish.TurkishJournalofPsychology.30(76):19–31.Hisanaga,S.,Sekiyama,K.,Igasaki,T.,&Murayama,N.(2009).AudiovisualspeechperceptioninJapaneseandEnglish:inter-languagedifferencesexaminedbyevent-relatedpotentials.In AVSP (pp.38-42).Massaro,D.W.,&Stork,D.G.(1998).Speechrecognitionandsensoryintegration:a240-year-oldtheoremhelpsexplainhowpeopleandmachinescanintegrateauditoryandvisualinformationtounderstandspeech. AmericanScientist, 86(3),236-244.McGurk,H.,&MacDonald,J.(1976).Hearinglipsandseeingvoices. Nature, 264(5588),746-748.Nath,A.R.,&Beauchamp,M.S.(2012).AneuralbasisforinterindividualdifferencesintheMcGurkeffect,amultisensoryspeechillusion. Neuroimage, 59(1),781-787.Nicholson,K.G.,Baum,S.,Cuddy,L.L.,&Munhall,K.G.(2002).Acaseofimpairedauditoryandvisualspeechprosodyperceptionafterrighthemispheredamage. Neurocase, 8(4),314-322.Norrix,L.W.,Plante,E.,Vance,R.,&Boliek,C.A.(2007).Auditory-visualintegrationforspeechbychildrenwithandwithoutspecificlanguageimpairment.JournalofSpeech,Language,andHearingResearch,50(6),1639–1651.Schmid,G.,Thielmann,A.,&Ziegler,W.(2009).Theinfluenceofvisualandauditoryinformationontheperceptionofspeechandnon‐speechoralmovementsinpatientswithlefthemispherelesions. Clinicallinguistics&phonetics, 23(3),208-221.Sekiyama,K.(1997).Culturalandlinguisticfactorsinaudiovisualspeechprocessing:TheMcGurkeffectinChineseparticipants. Perception&psychophysics, 59(1),73-80.Sekiyama,K.,&Burnham,D.(2008).Impactoflanguageondevelopmentofauditory‐visualspeechperception. Developmentalscience, 11(2),306-320.Sekiyama,K.,&Tohkura,Y.I.(1991).McGurkeffectinnon‐Englishlisteners:FewvisualeffectsforJapaneseparticipantshearingJapanesesyllablesofhighauditoryintelligibility. TheJournaloftheAcousticalSocietyofAmerica, 90(4),1797-1805.Tiippana,K.,Andersen,T.S.,&Sams,M.(2004).Visualattentionmodulatesaudiovisualspeechperception. EuropeanJournalofCognitivePsychology, 16(3),457-472.Walker,S.,Bruce,V.,&O’Malley,C.(1995).Facialidentityandfacialspeechprocessing:FamiliarfacesandvoicesintheMcGurkeffect. Perception&Psychophysics, 57(8),1124-1133.Youse,K.M.,Cienkowski,K.M.,&Coelho,C.A.(2004).Auditory-visualspeechperceptioninanadultwithaphasia. Braininjury, 18(8),825-834.Howtoreferencethisarticle:Prera,A(2021,June10).ThemcGurkfffect.SimplyPsychology.www.simplypsychology.org/mcGurkeffect.html Home|AboutUs|PrivacyPolicy|Advertise|ContactUsBacktotopThisworkislicensedunderaCreativeCommonsAttribution-Noncommercial-NoDerivativeWorks3.0UnportedLicense.CompanyRegistrationno:10521846reportthisadreportthisadreportthisadxx