What is Deep Learning and How Does It Work? - TechTarget

At its simplest, deep learning can be thought of as a way to automate predictive analytics. While traditional machine learning algorithms are linear, deep ... Home AItechnologies Businesssoftware deeplearning TechAccelerator In-depthguidetomachinelearningintheenterprise Prev Next supervisedlearning Whatisaneuralnetwork?Explanationandexamples Downloadthisguide1 Sharethisitemwithyournetwork: By EdBurns KateBrush Whatisdeeplearning? Deeplearningisatypeofmachinelearningandartificialintelligence(AI)thatimitatesthewayhumansgaincertaintypesofknowledge.Deeplearningisanimportantelementofdatascience,whichincludesstatisticsandpredictivemodeling.Itisextremelybeneficialtodatascientistswhoaretaskedwithcollecting,analyzingandinterpretinglargeamountsofdata;deeplearningmakesthisprocessfasterandeasier. Atitssimplest,deeplearningcanbethoughtofasawaytoautomatepredictiveanalytics.Whiletraditionalmachinelearningalgorithmsarelinear,deeplearningalgorithmsarestackedinahierarchyofincreasingcomplexityandabstraction. Tounderstanddeeplearning,imagineatoddlerwhosefirstwordisdog.Thetoddlerlearnswhatadogis--andisnot--bypointingtoobjectsandsayingtheworddog.Theparentsays,"Yes,thatisadog,"or,"No,thatisnotadog."Asthetoddlercontinuestopointtoobjects,hebecomesmoreawareofthefeaturesthatalldogspossess.Whatthetoddlerdoes,withoutknowingit,isclarifyacomplexabstraction--theconceptofdog--bybuildingahierarchyinwhicheachlevelofabstractioniscreatedwithknowledgethatwasgainedfromtheprecedinglayerofthehierarchy. Howdeeplearningworks Computerprogramsthatusedeeplearninggothroughmuchthesameprocessasthetoddlerlearningtoidentifythedog.Eachalgorithminthehierarchyappliesanonlineartransformationtoitsinputanduseswhatitlearnstocreateastatisticalmodelasoutput.Iterationscontinueuntiltheoutputhasreachedanacceptablelevelofaccuracy.Thenumberofprocessinglayersthroughwhichdatamustpassiswhatinspiredthelabeldeep. Thisarticleispartof In-depthguidetomachinelearningintheenterprise Whichalsoincludes: LearnthebusinessvalueofAI'svarioustechniques 10commonusesformachinelearningapplicationsinbusiness 6waystoreducedifferenttypesofbiasinmachinelearning Download1 DownloadthisentireguideforFREEnow! Intraditionalmachinelearning,thelearningprocessissupervised,andtheprogrammerhastobeextremelyspecificwhentellingthecomputerwhattypesofthingsitshouldbelookingfortodecideifanimagecontainsadogordoesnotcontainadog.Thisisalaboriousprocesscalledfeatureextraction,andthecomputer'ssuccessratedependsentirelyupontheprogrammer'sabilitytoaccuratelydefineafeaturesetfordog.Theadvantageofdeeplearningistheprogrambuildsthefeaturesetbyitselfwithoutsupervision.Unsupervisedlearningisnotonlyfaster,butitisusuallymoreaccurate. Initially,thecomputerprogrammightbeprovidedwithtrainingdata--asetofimagesforwhichahumanhaslabeledeachimagedogornotdogwithmetatags.Theprogramusestheinformationitreceivesfromthetrainingdatatocreateafeaturesetfordogandbuildapredictivemodel.Inthiscase,themodelthecomputerfirstcreatesmightpredictthatanythinginanimagethathasfourlegsandatailshouldbelabeleddog.Ofcourse,theprogramisnotawareofthelabelsfourlegsortail.Itwillsimplylookforpatternsofpixelsinthedigitaldata.Witheachiteration,thepredictivemodelbecomesmorecomplexandmoreaccurate. Unlikethetoddler,whowilltakeweeksorevenmonthstounderstandtheconceptofdog,acomputerprogramthatusesdeeplearningalgorithmscanbeshownatrainingsetandsortthroughmillionsofimages,accuratelyidentifyingwhichimageshavedogsinthemwithinafewminutes. Toachieveanacceptablelevelofaccuracy,deeplearningprogramsrequireaccesstoimmenseamountsoftrainingdataandprocessingpower,neitherofwhichwereeasilyavailabletoprogrammersuntiltheeraofbigdataandcloudcomputing.Becausedeeplearningprogrammingcancreatecomplexstatisticalmodelsdirectlyfromitsowniterativeoutput,itisabletocreateaccuratepredictivemodelsfromlargequantitiesofunlabeled,unstructureddata.Thisisimportantastheinternetofthings(IoT)continuestobecomemorepervasivebecausemostofthedatahumansandmachinescreateisunstructuredandisnotlabeled. Deeplearningmethods Variousmethodscanbeusedtocreatestrongdeeplearningmodels.Thesetechniquesincludelearningratedecay,transferlearning,trainingfromscratchanddropout. Learningratedecay.Thelearningrateisahyperparameter--afactorthatdefinesthesystemorsetconditionsforitsoperationpriortothelearningprocess--thatcontrolshowmuchchangethemodelexperiencesinresponsetotheestimatederroreverytimethemodelweightsarealtered.Learningratesthataretoohighmayresultinunstabletrainingprocessesorthelearningofasuboptimalsetofweights.Learningratesthataretoosmallmayproducealengthytrainingprocessthathasthepotentialtogetstuck. Thelearningratedecaymethod--alsocalledlearningrateannealingoradaptivelearningrates--istheprocessofadaptingthelearningratetoincreaseperformanceandreducetrainingtime.Theeasiestandmostcommonadaptationsoflearningrateduringtrainingincludetechniquestoreducethelearningrateovertime. Transferlearning.Thisprocessinvolvesperfectingapreviouslytrainedmodel;itrequiresaninterfacetotheinternalsofapreexistingnetwork.First,usersfeedtheexistingnetworknewdatacontainingpreviouslyunknownclassifications.Onceadjustmentsaremadetothenetwork,newtaskscanbeperformedwithmorespecificcategorizingabilities.Thismethodhastheadvantageofrequiringmuchlessdatathanothers,thusreducingcomputationtimetominutesorhours. Trainingfromscratch.Thismethodrequiresadevelopertocollectalargelabeleddatasetandconfigureanetworkarchitecturethatcanlearnthefeaturesandmodel.Thistechniqueisespeciallyusefulfornewapplications,aswellasapplicationswithalargenumberofoutputcategories.However,overall,itisalesscommonapproach,asitrequiresinordinateamountsofdata,causingtrainingtotakedaysorweeks. Dropout.Thismethodattemptstosolvetheproblemofoverfittinginnetworkswithlargeamountsofparametersbyrandomlydroppingunitsandtheirconnectionsfromtheneuralnetworkduringtraining.Ithasbeenproventhatthedropoutmethodcanimprovetheperformanceofneuralnetworksonsupervisedlearningtasksinareassuchasspeechrecognition,documentclassificationandcomputationalbiology. Deeplearningneuralnetworks? Atypeofadvancedmachinelearningalgorithm,knownasanartificialneuralnetwork,underpinsmostdeeplearningmodels.Asaresult,deeplearningmaysometimesbereferredtoasdeepneurallearningordeepneuralnetworking. Neuralnetworkscomeinseveraldifferentforms,includingrecurrentneuralnetworks,convolutionalneuralnetworks,artificialneuralnetworksandfeedforwardneuralnetworks,andeachhasbenefitsforspecificusecases.However,theyallfunctioninsomewhatsimilarways--byfeedingdatainandlettingthemodelfigureoutforitselfwhetherithasmadetherightinterpretationordecisionaboutagivendataelement. Neuralnetworksinvolveatrial-and-errorprocess,sotheyneedmassiveamountsofdataonwhichtotrain.It'snocoincidenceneuralnetworksbecamepopularonlyaftermostenterprisesembracedbigdataanalyticsandaccumulatedlargestoresofdata.Becausethemodel'sfirstfewiterationsinvolvesomewhateducatedguessesonthecontentsofanimageorpartsofspeech,thedatausedduringthetrainingstagemustbelabeledsothemodelcanseeifitsguesswasaccurate.Thismeans,thoughmanyenterprisesthatusebigdatahavelargeamountsofdata,unstructureddataislesshelpful.Unstructureddatacanonlybeanalyzedbyadeeplearningmodelonceithasbeentrainedandreachesanacceptablelevelofaccuracy,butdeeplearningmodelscan'ttrainonunstructureddata. Deeplearningexamples Becausedeeplearningmodelsprocessinformationinwayssimilartothehumanbrain,theycanbeappliedtomanytaskspeopledo.Deeplearningiscurrentlyusedinmostcommonimagerecognitiontools,naturallanguageprocessing(NLP)andspeechrecognitionsoftware.Thesetoolsarestartingtoappearinapplicationsasdiverseasself-drivingcarsandlanguagetranslationservices. Usecasestodayfordeeplearningincludealltypesofbigdataanalyticsapplications,especiallythosefocusedonNLP,languagetranslation,medicaldiagnosis,stockmarkettradingsignals,networksecurityandimagerecognition. Specificfieldsinwhichdeeplearningiscurrentlybeingusedincludethefollowing: Customerexperience(CX).Deeplearningmodelsarealreadybeingusedforchatbots.And,asitcontinuestomature,deeplearningisexpectedtobeimplementedinvariousbusinessestoimproveCXandincreasecustomersatisfaction. Textgeneration.Machinesarebeingtaughtthegrammarandstyleofapieceoftextandarethenusingthismodeltoautomaticallycreateacompletelynewtextmatchingtheproperspelling,grammarandstyleoftheoriginaltext. Aerospaceandmilitary.Deeplearningisbeingusedtodetectobjectsfromsatellitesthatidentifyareasofinterest,aswellassafeorunsafezonesfortroops. Industrialautomation.Deeplearningisimprovingworkersafetyinenvironmentslikefactoriesandwarehousesbyprovidingservicesthatautomaticallydetectwhenaworkerorobjectisgettingtooclosetoamachine. Addingcolor.Colorcanbeaddedtoblack-and-whitephotosandvideosusingdeeplearningmodels.Inthepast,thiswasanextremelytime-consuming,manualprocess. Medicalresearch.Cancerresearchershavestartedimplementingdeeplearningintotheirpracticeasawaytoautomaticallydetectcancercells. Computervision.Deeplearninghasgreatlyenhancedcomputervision,providingcomputerswithextremeaccuracyforobjectdetectionandimageclassification,restorationandsegmentation. Limitationsandchallenges Thebiggestlimitationofdeeplearningmodelsistheylearnthroughobservations.Thismeanstheyonlyknowwhatwasinthedataonwhichtheytrained.Ifauserhasasmallamountofdataoritcomesfromonespecificsourcethatisnotnecessarilyrepresentativeofthebroaderfunctionalarea,themodelswillnotlearninawaythatisgeneralizable. Theissueofbiasesisalsoamajorproblemfordeeplearningmodels.Ifamodeltrainsondatathatcontainsbiases,themodelwillreproducethosebiasesinitspredictions.Thishasbeenavexingproblemfordeeplearningprogrammersbecausemodelslearntodifferentiatebasedonsubtlevariationsindataelements.Often,thefactorsitdeterminesareimportantarenotmadeexplicitlycleartotheprogrammer.Thismeans,forexample,afacialrecognitionmodelmightmakedeterminationsaboutpeople'scharacteristicsbasedonthingslikeraceorgenderwithouttheprogrammerbeingaware. Thelearningratecanalsobecomeamajorchallengetodeeplearningmodels.Iftherateistoohigh,thenthemodelwillconvergetooquickly,producingaless-than-optimalsolution.Iftherateistoolow,thentheprocessmaygetstuck,anditwillbeevenhardertoreachasolution. Thehardwarerequirementsfordeeplearningmodelscanalsocreatelimitations.Multicorehigh-performinggraphicsprocessingunits(GPUs)andothersimilarprocessingunitsarerequiredtoensureimprovedefficiencyanddecreasedtimeconsumption.However,theseunitsareexpensiveanduselargeamountsofenergy.Otherhardwarerequirementsincluderandomaccessmemoryandaharddiskdrive(HDD)orRAM-basedsolid-statedrive(SSD). Otherlimitationsandchallengesincludethefollowing: Deeplearningrequireslargeamountsofdata.Furthermore,themorepowerfulandaccuratemodelswillneedmoreparameters,which,inturn,requiremoredata. Oncetrained,deeplearningmodelsbecomeinflexibleandcannothandlemultitasking.Theycandeliverefficientandaccuratesolutionsbutonlytoonespecificproblem.Evensolvingasimilarproblemwouldrequireretrainingthesystem. Anyapplicationthatrequiresreasoning--suchasprogrammingorapplyingthescientificmethod--long-termplanningandalgorithmlikedatamanipulationiscompletelybeyondwhatcurrentdeeplearningtechniquescando,evenwithlargedata. Deeplearningvs.machinelearning Deeplearningisasubsetofmachinelearningthatdifferentiatesitselfthroughthewayitsolvesproblems.Machinelearningrequiresadomainexperttoidentifymostappliedfeatures.Ontheotherhand,deeplearningunderstandsfeaturesincrementally,thuseliminatingtheneedfordomainexpertise.Thismakesdeeplearningalgorithmstakemuchlongertotrainthanmachinelearningalgorithms,whichonlyneedafewsecondstoafewhours.However,thereverseistrueduringtesting.Deeplearningalgorithmstakemuchlesstimetorunteststhanmachinelearningalgorithms,whosetesttimeincreasesalongwiththesizeofthedata. Furthermore,machinelearningdoesnotrequirethesamecostly,high-endmachinesandhigh-performingGPUsthatdeeplearningdoes. Intheend,manydatascientistschoosetraditionalmachinelearningoverdeeplearningduetoitssuperiorinterpretability,ortheabilitytomakesenseofthesolutions.Machinelearningalgorithmsarealsopreferredwhenthedataissmall. Instanceswheredeeplearningbecomespreferableincludesituationswherethereisalargeamountofdata,alackofdomainunderstandingforfeatureintrospection,orcomplexproblems,suchasspeechrecognitionandNLP. Thisimageillustratesthedeeplearningprocess. Formoreaboutrelatedtopics,likesupervisedlearning,unsupervisedlearningandreinforcementlearning,followthelinkstoourotherdefinitionsofthoseconcepts. ThiswaslastupdatedinMarch2021 ContinueReadingAboutdeeplearning 9typesofmachinelearningalgorithmswithacheatsheet QlikSenseBusinessimprovesQlik'scloud,AIcapabilities HowautomatedmachinelearningtoolspavethewaytoAI OpenText'sMarkBarrenecheatalksGoogle,AI,futureofCMS Deeplearningpowersamotion-trackingrevolution RelatedTerms intelligentsystem Anintelligentsystemisanadvancedcomputersystemthatcangather,analyzeandrespondtothedataitcollectsfromits... See complete definition machinevision Machinevisionistheabilityofacomputertosee;itemploysoneormorevideocameras,analog-to-digitalconversion(ADC)and... See complete definition quantum Aquantum(plural:quanta)isthesmallestdiscreteunitofaphenomenon. See complete definition DigDeeperonAItechnologies cognitivecomputing By:Alexander Gillis SambaNovaunveilsGPTBankingforfinancialindustry By:Esther Ajao Howneuralnetworktrainingmethodsaremodeledafterthehumanbrain By:Kathleen Walch recurrentneuralnetworks By:Nicole Laskowski SponsoredNews ThreeInnovativeAIUseCasesforNaturalLanguageProcessing –DellTechnologies 5BestPracticesToSecureRemoteWorkers –Microsoft SeeMore VendorResources AComputerWeeklybuyer'sguidetodeeplearning –ComputerWeekly.com AItoolsforsoftwaredevelopers:PartTwo –ComputerWeekly.com LatestTechTargetresources BusinessAnalytics CIO DataManagement ERP SearchBusinessAnalytics Automation,moresecurityandgovernancenextbigBItrends Asmanydatapreparationandmanagementtasksgetautomated,insightgenerationandactionarethenextlikelytargetsfor... 5businessanalyticstrendsthatshapedthestartof2022 WhileAIandmachinelearningcapabilitiescontinuetoevolve,anothermajormovementinBIsofarin2022hasbeentoenable... TibcoaddsSaaSversionofMDMplatform,updatesTCI TheanalyticsvendorunveiledCloudEBXasitcontinuestoaddcloud-nativeversionsofitstools.Inaddition,itlaunchedan... SearchCIO Technologycostsriseasinflationhitshardware,services InflationisaffectingtheCIOmarketbasket,influencingpurchasing.Governmentdatashowedasharpincreaseincostforservers... Biden'snomineeforOSTPdirectorbringsexperiencetorole DuringanominationhearingforAratiPrabhakar,U.S.senatorsfocusedonherexperienceandwhatshewouldbringtotheWhite... EnvironmentalsustainabilityconsiderationsforITteams Withcustomers,employeesandinvestorspressuringcompaniestogogreen,ITneedstotakealeadonsustainability.Learnsome... SearchDataManagement ApacheBeamusersdetailstreamdataprocessingapplications AtaBeamconference,usersdetailhowthey'reusingtheopensourcedataprocessingtechnologyatlargescaleinsupportofa... On-premisesvs.clouddatawarehouses:Prosandcons Datawarehousesincreasinglyarebeingdeployedinthecloud.Butbothon-premisesandclouddatawarehouseshaveplusesand... OracleDatabaseServiceforAzureconnectsMicrosofttoOCI OracleandMicrosoftareextendingtheirpartnershiptoenableMicrosoftclouduserstodirectlyprovisionandaccessOracle... SearchERP AcumaticaCEObringscloudERPcustomerperspectivetorole InthisQ&A,AcumaticaCEOJohnCasediscusseshowhisexperienceasacustomerinformsissueslikethedegreeofstandardization... Bigbangvs.phasedERPimplementation:Whichisbest? Selectingthebigbangapproachvs.thephasedapproachforanERPimplementationisacrucialchoiceforprojectleaders.Learn... IFSexpandsEAMsoftwareportfoliowithUltimoacquisition IFS'acquisitionofcloud-basedEAMvendorUltimoprovidesitwithmidmarketoptionstoaddtoitsenterpriselevelplatformina... Close