Your First Deep Learning Project in Python with Keras Step-By ...

2024-09-29

文章推薦指數： 80 %

投票人數：10人

Your First Deep Learning Project in Python with Keras Step-By-Step · 1. Load Data · 2. Define Keras Model · 3. Compile Keras Model · 4. Fit Keras ... Navigation Home MainMenuGetStarted Blog Topics DeepLearning(keras) ComputerVision NeuralNetTimeSeries NLP(Text) GANs LSTMs BetterDeepLearning Calculus IntrotoAlgorithms CodeAlgorithms IntrotoTimeSeries Python(scikit-learn) EnsembleLearning ImbalancedLearning DataPreparation R(caret) Weka(nocode) LinearAlgebra Statistics Optimization Probability XGBoost PythonforMachineLearning EBooks FAQ About Contact ReturntoContent ByJasonBrownleeonJune18,2022inDeepLearning Tweet Tweet Share Share LastUpdatedonJune20,2022 Kerasisapowerfulandeasy-to-usefreeopensourcePythonlibraryfordevelopingandevaluatingdeeplearningmodels. Itispartofthe TensorFlowlibraryandallowsyoutodefineandtrainneuralnetworkmodelsinjustafewlinesofcode. Inthistutorial,youwilldiscoverhowtocreateyourfirstdeeplearningneuralnetworkmodelinPythonusingKeras. Kick-startyourprojectwithmynewbookDeepLearningWithPython,includingstep-by-steptutorialsandthePythonsourcecodefilesforallexamples. Let’sgetstarted. UpdateFeb/2017:UpdatedpredictionexamplesoroundingworksinPython2and3. UpdateMar/2017:UpdatedexampleforthelatestversionsofKerasandTensorFlow. UpdateMar/2018:Addedalternatelinktodownloadthedataset. UpdateJul/2019:Expandedandaddedmoreusefulresources. UpdateSep/2019:UpdatedforKerasv2.2.5API. UpdateOct/2019:UpdatedforKerasv2.3.0APIandTensorFlowv2.0.0. UpdateAug/2020:UpdatedforKerasv2.4.3andTensorFlowv2.3. UpdateOct/2021:Deprecatedpredict_classsyntax UpdateJun/2022:UpdatedtomodernTensorFlowsyntax DevelopYourFirstNeuralNetworkinPythonWithKerasStep-By-StepPhotobyPhilWhitehouse,somerightsreserved. KerasTutorialOverview Thereisnotalotofcoderequired,butwearegoingtostepoveritslowlysothatyouwillknowhowtocreateyourownmodelsinthefuture. Thestepsyouaregoingtocoverinthistutorialareasfollows: LoadData. DefineKerasModel. CompileKerasModel. FitKerasModel. EvaluateKerasModel. TieItAllTogether. MakePredictions ThisKerastutorialhasafewrequirements: YouhavePython2or3installedandconfigured. YouhaveSciPy(includingNumPy)installedandconfigured. YouhaveKerasandabackend(TheanoorTensorFlow)installedandconfigured. Ifyouneedhelpwithyourenvironment,seethe tutorial: HowtoSetupaPythonEnvironmentforDeepLearning Createanewfilecalledkeras_first_network.pyandtypeorcopy-and-pastethecodeintothefileasyougo. NeedhelpwithDeepLearninginPython? Takemyfree2-weekemailcourseanddiscoverMLPs,CNNsandLSTMs(withcode). Clicktosign-upnowandalsogetafreePDFEbookversionofthecourse. StartYourFREEMini-CourseNow 1.LoadData Thefirststepistodefinethefunctionsandclassesweintendtouseinthistutorial. WewillusetheNumPylibrarytoloadourdatasetandwewillusetwoclassesfromtheKeraslibrarytodefineourmodel. Theimportsrequiredarelistedbelow. #firstneuralnetworkwithkerastutorial fromnumpyimportloadtxt fromtensorflow.keras.modelsimportSequential fromtensorflow.keras.layersimportDense ... 12345 #firstneuralnetworkwithkerastutorialfromnumpyimportloadtxtfromtensorflow.keras.modelsimportSequentialfromtensorflow.keras.layersimportDense... Wecannowloadourdataset. InthisKerastutorial,wearegoingtousethePimaIndiansonsetofdiabetesdataset.ThisisastandardmachinelearningdatasetfromtheUCIMachineLearningrepository.ItdescribespatientmedicalrecorddataforPimaIndiansandwhethertheyhadanonsetofdiabeteswithinfiveyears. Assuch,itisabinaryclassificationproblem(onsetofdiabetesas1ornotas0).Alloftheinputvariablesthatdescribeeachpatientarenumerical.Thismakesiteasytousedirectlywithneuralnetworksthatexpectnumericalinputandoutputvalues,andidealforourfirstneuralnetworkinKeras. Thedatasetisavailablefromhere: DatasetCSVFile(pima-indians-diabetes.csv) DatasetDetails Downloadthedatasetandplaceitinyourlocalworkingdirectory,thesamelocationasyourpythonfile. Saveitwiththefilename: pima-indians-diabetes.csv 1 pima-indians-diabetes.csv Takealookinsidethefile,youshouldseerowsofdatalikethefollowing: 6,148,72,35,0,33.6,0.627,50,1 1,85,66,29,0,26.6,0.351,31,0 8,183,64,0,0,23.3,0.672,32,1 1,89,66,23,94,28.1,0.167,21,0 0,137,40,35,168,43.1,2.288,33,1 ... 123456 6,148,72,35,0,33.6,0.627,50,11,85,66,29,0,26.6,0.351,31,08,183,64,0,0,23.3,0.672,32,11,89,66,23,94,28.1,0.167,21,00,137,40,35,168,43.1,2.288,33,1... WecannowloadthefileasamatrixofnumbersusingtheNumPyfunctionloadtxt(). Thereareeightinputvariablesandoneoutputvariable(thelastcolumn).Wewillbelearningamodeltomaprowsofinputvariables(X)toanoutputvariable(y),whichweoftensummarizeasy=f(X). Thevariablescanbesummarizedasfollows: InputVariables(X): Numberoftimespregnant Plasmaglucoseconcentrationa2hoursinanoralglucosetolerancetest Diastolicbloodpressure(mmHg) Tricepsskinfoldthickness(mm) 2-Hourseruminsulin(muU/ml) Bodymassindex(weightinkg/(heightinm)^2) Diabetespedigreefunction Age(years) OutputVariables(y): Classvariable(0or1) OncetheCSVfileisloadedintomemory,wecansplitthecolumnsofdataintoinputandoutputvariables. Thedatawillbestoredina2Darraywherethefirstdimensionisrowsandtheseconddimensioniscolumns,e.g.[rows,columns]. WecansplitthearrayintotwoarraysbyselectingsubsetsofcolumnsusingthestandardNumPysliceoperatoror“:”Wecanselectthefirst8columnsfromindex0toindex7viatheslice0:8.Wecanthenselecttheoutputcolumn(the9thvariable)viaindex8. ... #loadthedataset dataset=loadtxt('pima-indians-diabetes.csv',delimiter=',') #splitintoinput(X)andoutput(y)variables X=dataset[:,0:8] y=dataset[:,8] ... 1234567 ...#loadthedatasetdataset=loadtxt('pima-indians-diabetes.csv',delimiter=',')#splitintoinput(X)andoutput(y)variablesX=dataset[:,0:8]y=dataset[:,8]... Wearenowreadytodefineourneuralnetworkmodel. Note,thedatasethas9columnsandtherange0:8willselectcolumnsfrom0to7,stoppingbeforeindex8.Ifthisisnewtoyou,thenyoucanlearnmoreaboutarrayslicingandrangesinthispost: HowtoIndex,SliceandReshapeNumPyArraysforMachineLearninginPython 2.DefineKerasModel ModelsinKerasaredefinedasasequenceoflayers. WecreateaSequentialmodelandaddlayersoneatatimeuntilwearehappywithournetworkarchitecture. Thefirstthingtogetrightistoensuretheinputlayerhastherightnumberofinputfeatures.Thiscanbespecifiedwhencreatingthefirstlayerwiththeinput_shapeargumentandsettingitto(8,)forpresentingthe8inputvariablesasavector. Howdoweknowthenumberoflayersandtheirtypes? Thisisaveryhardquestion.Thereareheuristicsthatwecanuseandoftenthebestnetworkstructureisfoundthroughaprocessoftrialanderrorexperimentation(Iexplainmoreaboutthishere).Generally,youneedanetworklargeenoughtocapturethestructureoftheproblem. Inthisexample,wewilluseafully-connectednetworkstructurewiththreelayers. FullyconnectedlayersaredefinedusingtheDenseclass.Wecanspecifythenumberofneuronsornodesinthelayerasthefirstargument, andspecifytheactivationfunctionusingtheactivationargument. WewillusetherectifiedlinearunitactivationfunctionreferredtoasReLUonthefirsttwolayersandtheSigmoidfunctionintheoutputlayer. ItusedtobethecasethatSigmoidandTanhactivationfunctionswerepreferredforalllayers.Thesedays,betterperformanceisachievedusingtheReLUactivationfunction.Weuseasigmoidontheoutputlayertoensureournetworkoutputisbetween0and1andeasytomaptoeitheraprobabilityofclass1orsnaptoahardclassificationofeitherclasswithadefaultthresholdof0.5. Wecanpieceitalltogetherbyaddingeachlayer: Themodelexpectsrowsofdatawith8variables(theinput_shape=(8,)argument) Thefirsthiddenlayerhas12nodesandusesthereluactivationfunction. Thesecondhiddenlayerhas8nodesandusesthereluactivationfunction. Theoutputlayerhasonenodeandusesthesigmoidactivationfunction. ... #definethekerasmodel model=Sequential() model.add(Dense(12,input_shape=(8,),activation='relu')) model.add(Dense(8,activation='relu')) model.add(Dense(1,activation='sigmoid')) ... 1234567 ...#definethekerasmodelmodel=Sequential()model.add(Dense(12,input_shape=(8,),activation='relu'))model.add(Dense(8,activation='relu'))model.add(Dense(1,activation='sigmoid'))... Note,themostconfusingthinghereisthattheshapeoftheinputtothemodelisdefinedasanargumentonthefirsthiddenlayer.ThismeansthatthelineofcodethataddsthefirstDenselayerisdoing2things,definingtheinputorvisiblelayerandthefirsthiddenlayer. 3.CompileKerasModel Nowthatthemodelisdefined,wecancompileit. Compilingthemodelusestheefficientnumericallibrariesunderthecovers(theso-calledbackend)suchasTheanoorTensorFlow.Thebackend automaticallychoosesthebestwaytorepresentthenetworkfortrainingandmakingpredictionstorunonyourhardware,suchasCPUorGPUorevendistributed. Whencompiling,wemustspecifysomeadditionalpropertiesrequiredwhentrainingthenetwork.Remembertraininganetworkmeansfindingthebestsetofweightstomapinputstooutputsinourdataset. Wemustspecifythelossfunctiontousetoevaluateasetofweights,theoptimizerisusedtosearchthroughdifferentweightsforthenetworkandanyoptionalmetricswewouldliketocollectandreportduringtraining. Inthiscase,wewillusecrossentropyasthelossargument.ThislossisforabinaryclassificationproblemsandisdefinedinKerasas“binary_crossentropy“.Youcanlearnmoreaboutchoosinglossfunctionsbasedonyourproblemhere: HowtoChooseLossFunctionsWhenTrainingDeepLearningNeuralNetworks Wewilldefinetheoptimizerastheefficientstochasticgradientdescentalgorithm“adam“.Thisisapopularversionofgradientdescentbecauseitautomaticallytunesitselfandgivesgoodresultsinawiderangeofproblems.TolearnmoreabouttheAdamversionofstochasticgradientdescentseethepost: GentleIntroductiontotheAdamOptimizationAlgorithmforDeepLearning Finally,becauseitisaclassificationproblem,wewillcollectandreporttheclassificationaccuracy,definedviathemetricsargument. ... #compilethekerasmodel model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) ... 1234 ...#compilethekerasmodelmodel.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])... 4.FitKerasModel Wehavedefinedourmodelandcompileditreadyforefficientcomputation. Nowitistimetoexecutethemodelonsomedata. Wecantrainorfitourmodelonourloadeddatabycallingthefit()functiononthemodel. Trainingoccursoverepochsandeachepochissplitintobatches. Epoch:Onepassthroughalloftherowsinthetrainingdataset. Batch:Oneormoresamplesconsideredbythemodelwithinanepochbeforeweightsareupdated. Oneepochiscomprisedofoneormorebatches,basedonthechosenbatchsizeandthemodelisfitformanyepochs.Formoreonthedifferencebetweenepochsandbatches,seethepost: WhatistheDifferenceBetweenaBatchandanEpochinaNeuralNetwork? Thetrainingprocesswillrunforafixednumberofiterationsthroughthedatasetcalledepochs,thatwemustspecifyusingtheepochsargument.Wemustalsosetthenumberofdatasetrowsthatareconsideredbeforethemodelweightsareupdatedwithineachepoch,calledthebatchsizeandsetusingthebatch_sizeargument. Forthisproblem,wewillrunforasmallnumberofepochs(150)andusearelativelysmallbatchsizeof10. Theseconfigurationscanbechosenexperimentallybytrialanderror.Wewanttotrainthemodelenoughsothatitlearnsagood(orgoodenough)mappingofrowsofinputdatatotheoutputclassification.Themodelwillalwayshavesomeerror,buttheamountoferrorwillleveloutaftersomepointforagivenmodelconfiguration.Thisiscalledmodelconvergence. ... #fitthekerasmodelonthedataset model.fit(X,y,epochs=150,batch_size=10) ... 1234 ...#fitthekerasmodelonthedatasetmodel.fit(X,y,epochs=150,batch_size=10)... ThisiswheretheworkhappensonyourCPUorGPU. NoGPUisrequiredforthisexample,butifyou’reinterestedinhowtorunlargemodelsonGPUhardwarecheaplyinthecloud,seethispost: HowtoSetupAmazonAWSEC2GPUstoTrainKerasDeepLearningModels 5.EvaluateKerasModel Wehavetrainedourneuralnetworkontheentiredatasetandwecanevaluatetheperformanceofthenetworkonthesamedataset. Thiswillonlygiveusanideaofhowwellwehavemodeledthedataset(e.g.trainaccuracy),butnoideaofhowwellthealgorithmmightperformonnewdata.Wehavedonethisforsimplicity,butideally,youcouldseparateyourdataintotrainandtestdatasetsfortrainingandevaluationofyourmodel. Youcanevaluateyourmodelonyourtrainingdatasetusingtheevaluate()functiononyourmodelandpassitthesameinputandoutputusedtotrainthemodel. Thiswillgenerateapredictionforeachinputandoutputpairandcollectscores,includingtheaveragelossandanymetricsyouhaveconfigured,suchasaccuracy. Theevaluate()functionwillreturnalistwithtwovalues.Thefirstwillbethelossofthemodelonthedatasetandthesecondwillbetheaccuracyofthemodelonthedataset.Weareonlyinterestedinreportingtheaccuracy,sowewillignorethelossvalue. ... #evaluatethekerasmodel _,accuracy=model.evaluate(X,y) print('Accuracy:%.2f'%(accuracy*100)) 1234 ...#evaluatethekerasmodel_,accuracy=model.evaluate(X,y)print('Accuracy:%.2f'%(accuracy*100)) 6.TieItAllTogether YouhavejustseenhowyoucaneasilycreateyourfirstneuralnetworkmodelinKeras. Let’stieitalltogetherintoacompletecodeexample. #firstneuralnetworkwithkerastutorial fromnumpyimportloadtxt fromtensorflow.keras.modelsimportSequential fromtensorflow.keras.layersimportDense #loadthedataset dataset=loadtxt('pima-indians-diabetes.csv',delimiter=',') #splitintoinput(X)andoutput(y)variables X=dataset[:,0:8] y=dataset[:,8] #definethekerasmodel model=Sequential() model.add(Dense(12,input_shape=(8,),activation='relu')) model.add(Dense(8,activation='relu')) model.add(Dense(1,activation='sigmoid')) #compilethekerasmodel model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) #fitthekerasmodelonthedataset model.fit(X,y,epochs=150,batch_size=10) #evaluatethekerasmodel _,accuracy=model.evaluate(X,y) print('Accuracy:%.2f'%(accuracy*100)) 123456789101112131415161718192021 #firstneuralnetworkwithkerastutorialfromnumpyimportloadtxtfromtensorflow.keras.modelsimportSequentialfromtensorflow.keras.layersimportDense#loadthedatasetdataset=loadtxt('pima-indians-diabetes.csv',delimiter=',')#splitintoinput(X)andoutput(y)variablesX=dataset[:,0:8]y=dataset[:,8]#definethekerasmodelmodel=Sequential()model.add(Dense(12,input_shape=(8,),activation='relu'))model.add(Dense(8,activation='relu'))model.add(Dense(1,activation='sigmoid'))#compilethekerasmodelmodel.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])#fitthekerasmodelonthedatasetmodel.fit(X,y,epochs=150,batch_size=10)#evaluatethekerasmodel_,accuracy=model.evaluate(X,y)print('Accuracy:%.2f'%(accuracy*100)) YoucancopyallofthecodeintoyourPythonfileandsaveitas“keras_first_network.py”inthesamedirectoryasyourdatafile“pima-indians-diabetes.csv“.YoucanthenrunthePythonfileasascriptfromyourcommandline(commandprompt)asfollows: pythonkeras_first_network.py 1 pythonkeras_first_network.py Runningthisexample,youshouldseeamessageforeachofthe150epochsprintingthelossandaccuracy,followedbythefinalevaluationofthetrainedmodelonthetrainingdataset. Ittakesabout10secondstoexecuteonmyworkstationrunningontheCPU. Ideally,wewouldlikethelosstogotozeroandaccuracytogoto1.0(e.g.100%).Thisisnotpossibleforanybutthemosttrivialmachinelearningproblems.Instead,wewillalwayshavesomeerrorinourmodel.Thegoalistochooseamodelconfigurationandtrainingconfigurationthatachievethelowestlossandhighestaccuracypossibleforagivendataset. ... 768/768[==============================]-0s63us/step-loss:0.4817-acc:0.7708 Epoch147/150 768/768[==============================]-0s63us/step-loss:0.4764-acc:0.7747 Epoch148/150 768/768[==============================]-0s63us/step-loss:0.4737-acc:0.7682 Epoch149/150 768/768[==============================]-0s64us/step-loss:0.4730-acc:0.7747 Epoch150/150 768/768[==============================]-0s63us/step-loss:0.4754-acc:0.7799 768/768[==============================]-0s38us/step Accuracy:76.56 123456789101112 ...768/768[==============================]-0s63us/step-loss:0.4817-acc:0.7708Epoch147/150768/768[==============================]-0s63us/step-loss:0.4764-acc:0.7747Epoch148/150768/768[==============================]-0s63us/step-loss:0.4737-acc:0.7682Epoch149/150768/768[==============================]-0s64us/step-loss:0.4730-acc:0.7747Epoch150/150768/768[==============================]-0s63us/step-loss:0.4754-acc:0.7799768/768[==============================]-0s38us/stepAccuracy:76.56 Note,ifyoutryrunningthisexampleinanIPythonor Jupyternotebookyoumaygetanerror. Thereasonisthe outputprogressbarsduringtraining.Youcaneasilyturntheseoffbysettingverbose=0inthecalltothefit()andevaluate()functions,forexample: ... #fitthekerasmodelonthedatasetwithoutprogressbars model.fit(X,y,epochs=150,batch_size=10,verbose=0) #evaluatethekerasmodel _,accuracy=model.evaluate(X,y,verbose=0) ... 123456 ...#fitthekerasmodelonthedatasetwithoutprogressbarsmodel.fit(X,y,epochs=150,batch_size=10,verbose=0)#evaluatethekerasmodel_,accuracy=model.evaluate(X,y,verbose=0)... Note:Yourresultsmayvarygiventhestochasticnatureofthealgorithmorevaluationprocedure,ordifferencesinnumericalprecision.Considerrunningtheexampleafewtimesandcomparetheaverageoutcome. Whatscoredidyouget? Postyourresultsinthecommentsbelow. Neuralnetworksareastochasticalgorithm,meaningthatthesamealgorithmonthesamedatacantrainadifferentmodelwithdifferentskilleachtimethecodeisrun.Thisisafeature,notabug.Youcanlearnmoreaboutthisinthepost: EmbraceRandomnessinMachineLearning Thevarianceintheperformanceofthemodelmeansthattogetareasonableapproximationofhowwellyourmodelisperforming,youmayneedtofititmanytimesandcalculatetheaverageoftheaccuracyscores.Formoreonthisapproachtoevaluatingneuralnetworks,seethepost: HowtoEvaluatetheSkillofDeepLearningModels Forexample,belowaretheaccuracyscoresfromre-runningtheexample5times: Accuracy:75.00 Accuracy:77.73 Accuracy:77.60 Accuracy:78.12 Accuracy:76.17 12345 Accuracy:75.00Accuracy:77.73Accuracy:77.60Accuracy:78.12Accuracy:76.17 Wecanseethatallaccuracyscoresarearound77%andtheaverageis76.924%. 7.MakePredictions ThenumberonequestionIgetaskedis: AfterItrainmymodel,howcanIuseittomakepredictionsonnewdata? Greatquestion. Wecanadapttheaboveexampleanduseitto generatepredictionsonthetrainingdataset,pretendingitisanewdatasetwehavenotseenbefore. Makingpredictionsisaseasyascallingthepredict()functiononthemodel.Weareusingasigmoidactivationfunctionontheoutputlayer,sothepredictionswillbeaprobabilityintherangebetween0and1.Wecaneasilyconvertthemintoacrispbinarypredictionforthisclassificationtaskbyroundingthem. Forexample: ... #makeprobabilitypredictionswiththemodel predictions=model.predict(X) #roundpredictions rounded=[round(x[0])forxinpredictions] 12345 ...#makeprobabilitypredictionswiththemodelpredictions=model.predict(X)#roundpredictionsrounded=[round(x[0])forxinpredictions] Alternately,wecanconverttheprobabilityinto0or1topredictcrispclassesdirectly,forexample: ... #makeclasspredictionswiththemodel predictions=(model.predict(X)>0.5).astype(int) 123 ...#makeclasspredictionswiththemodelpredictions=(model.predict(X)>0.5).astype(int) Thecompleteexamplebelowmakespredictionsforeachexampleinthedataset,thenprintstheinputdata,predictedclassandexpectedclassforthefirst5examplesinthedataset. #firstneuralnetworkwithkerasmakepredictions fromnumpyimportloadtxt fromtensorflow.keras.modelsimportSequential fromtensorflow.keras.layersimportDense #loadthedataset dataset=loadtxt('pima-indians-diabetes.csv',delimiter=',') #splitintoinput(X)andoutput(y)variables X=dataset[:,0:8] y=dataset[:,8] #definethekerasmodel model=Sequential() model.add(Dense(12,input_shape=(8,),activation='relu')) model.add(Dense(8,activation='relu')) model.add(Dense(1,activation='sigmoid')) #compilethekerasmodel model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) #fitthekerasmodelonthedataset model.fit(X,y,epochs=150,batch_size=10,verbose=0) #makeclasspredictionswiththemodel predictions=(model.predict(X)>0.5).astype(int) #summarizethefirst5cases foriinrange(5): print('%s=>%d(expected%d)'%(X[i].tolist(),predictions[i],y[i])) 1234567891011121314151617181920212223 #firstneuralnetworkwithkerasmakepredictionsfromnumpyimportloadtxtfromtensorflow.keras.modelsimportSequentialfromtensorflow.keras.layersimportDense#loadthedatasetdataset=loadtxt('pima-indians-diabetes.csv',delimiter=',')#splitintoinput(X)andoutput(y)variablesX=dataset[:,0:8]y=dataset[:,8]#definethekerasmodelmodel=Sequential()model.add(Dense(12,input_shape=(8,),activation='relu'))model.add(Dense(8,activation='relu'))model.add(Dense(1,activation='sigmoid'))#compilethekerasmodelmodel.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])#fitthekerasmodelonthedatasetmodel.fit(X,y,epochs=150,batch_size=10,verbose=0)#makeclasspredictionswiththemodelpredictions=(model.predict(X)>0.5).astype(int)#summarizethefirst5casesforiinrange(5): print('%s=>%d(expected%d)'%(X[i].tolist(),predictions[i],y[i])) Runningtheexampledoesnotshowtheprogressbarasbeforeaswehavesettheverboseargumentto0. Afterthemodelisfit,predictionsaremadeforallexamplesinthedataset,andtheinputrowsandpredictedclassvalueforthefirst5examplesisprintedandcomparedtotheexpectedclassvalue. Wecanseethatmostrowsarecorrectlypredicted.Infact,wewouldexpectabout76.9%oftherowstobecorrectlypredictedbasedonourestimatedperformanceofthemodelintheprevioussection. [6.0,148.0,72.0,35.0,0.0,33.6,0.627,50.0]=>0(expected1) [1.0,85.0,66.0,29.0,0.0,26.6,0.351,31.0]=>0(expected0) [8.0,183.0,64.0,0.0,0.0,23.3,0.672,32.0]=>1(expected1) [1.0,89.0,66.0,23.0,94.0,28.1,0.167,21.0]=>0(expected0) [0.0,137.0,40.0,35.0,168.0,43.1,2.288,33.0]=>1(expected1) 12345 [6.0,148.0,72.0,35.0,0.0,33.6,0.627,50.0]=>0(expected1)[1.0,85.0,66.0,29.0,0.0,26.6,0.351,31.0]=>0(expected0)[8.0,183.0,64.0,0.0,0.0,23.3,0.672,32.0]=>1(expected1)[1.0,89.0,66.0,23.0,94.0,28.1,0.167,21.0]=>0(expected0)[0.0,137.0,40.0,35.0,168.0,43.1,2.288,33.0]=>1(expected1) IfyouwouldliketoknowmoreabouthowtomakepredictionswithKerasmodels,seethepost: HowtoMakePredictionswithKeras KerasTutorialSummary Inthispost, youdiscoveredhowtocreateyourfirstneuralnetworkmodelusingthepowerfulKerasPythonlibraryfordeeplearning. Specifically,youlearnedthesixkeystepsinusingKerastocreateaneuralnetworkordeeplearningmodel,step-by-stepincluding: Howtoloaddata. HowtodefineaneuralnetworkinKeras. HowtocompileaKerasmodelusingtheefficientnumericalbackend. Howtotrainamodelondata. Howtoevaluateamodelondata. Howtomakepredictionswiththemodel. DoyouhaveanyquestionsaboutKerasoraboutthistutorial? AskyourquestioninthecommentsandIwilldomybesttoanswer. KerasTutorialExtensions Welldone,youhavesuccessfullydevelopedyourfirstneuralnetworkusingtheKerasdeeplearninglibraryinPython. Thissectionprovidessomeextensionstothistutorialthatyoumightwanttoexplore. TunetheModel.Changetheconfigurationofthemodelortrainingprocessandseeifyoucanimprovetheperformanceofthemodel,e.g.achievebetterthan76%accuracy. SavetheModel.Updatethetutorialtosavethemodeltofile,thenloaditlateranduseittomakepredictions(seethistutorial). SummarizetheModel.Updatethetutorialtosummarizethemodelandcreateaplotofmodellayers(seethistutorial). SeparateTrainandTestDatasets.Splittheloadeddatasetintoatrainandtestset(splitbasedonrows)anduseonesettotrainthemodelandtheothersettoestimatetheperformanceofthemodelonnewdata. PlotLearningCurves.Thefit()functionreturnsahistoryobjectthatsummarizesthelossandaccuracyattheendofeachepoch.Createlineplotsofthisdata,calledlearningcurves(seethistutorial). LearnaNewDataset.Updatethetutorialtouseadifferenttabulardataset,perhapsfromtheUCIMachineLearningRepository. UseFunctionalAPI.UpdatethetutorialtousetheKerasFunctionalAPIfordefiningthemodel(seethistutorial). FurtherReading AreyoulookingforsomemoreDeepLearning tutorialswithPythonand Keras? Takealookatsomeofthese: RelatedTutorials 5StepLife-CycleforNeuralNetworkModelsinKeras Multi-ClassClassificationTutorialwiththeKerasDeepLearningLibrary RegressionTutorialwiththeKerasDeepLearningLibraryinPython HowtoGridSearchHyperparametersforDeepLearningModelsinPythonWithKeras Books DeepLearning(Textbook),2016. DeepLearningwithPython(mybook). APIs KerasDeepLearningLibraryHomepage KerasAPIDocumentation Howdidyougo?Doyouhaveanyquestionsaboutdeeplearning? PostyourquestionsinthecommentsbelowandIwilldomybesttohelp. DevelopDeepLearningProjectswithPython! WhatIfYouCouldDevelopANetworkinMinutes ...withjustafewlinesofPython DiscoverhowinmynewEbook: DeepLearningWithPython Itcoversend-to-endprojectsontopicslike: MultilayerPerceptrons, ConvolutionalNetsand RecurrentNeuralNets,andmore... FinallyBringDeepLearningTo YourOwnProjects SkiptheAcademics.Just Results. SeeWhat'sInside Tweet Tweet Share Share MoreOnThisTopicTensorFlow2Tutorial:GetStartedinDeepLearning…HowtoGetStartedwithDeepLearningforNatural…HowtoDevelopaCNNFromScratchforCIFAR-10Photo…WhatisDeepLearning?AppliedDeepLearninginPythonMini-CourseMulti-LabelClassificationofSatellitePhotosof… AboutJasonBrownlee JasonBrownlee,PhDisamachinelearningspecialistwhoteachesdevelopershowtogetresultswithmodernmachinelearningmethodsviahands-ontutorials. ViewallpostsbyJasonBrownlee→ UsingNormalizationLayerstoImproveDeepLearningModels HowtoSaveandLoadYourKerasDeepLearningModel 1,146ResponsestoYourFirstDeepLearningProjectinPythonwithKerasStep-By-Step Saurav May27,2016at11:08pm # Theinputlayerdoesn’thaveanyactivationfunction,butstillactivation=”relu”ismentionedinthefirstlayerofthemodel.Why? Reply JasonBrownlee May28,2016at6:32am # HiSaurav, Thefirstlayerinthenetworkhereistechnicallyahiddenlayer,henceithasanactivationfunction. Reply samJohnson December21,2016at2:44am # Whyhaveyoumadeitahiddenlayerthough?theinputlayerisnotusuallyrepresentedasahiddenlayer? Reply JasonBrownlee December21,2016at8:41am # Hisam, Notethisline: model.add(Dense(12,input_dim=8,init='uniform',activation='relu')) 1 model.add(Dense(12,input_dim=8,init='uniform',activation='relu')) Itdoesafewthings. Itdefinestheinputlayerashaving8inputs. Itdefinesahiddenlayerwith12neurons,connectedtotheinputlayerthatusereluactivationfunction. Itinitializesallweightsusingasampleofuniformrandomnumbers. Doesthathelp? Reply Pavidevi May17,2017at2:31am # HiJason, Uhaveusedtwodifferentactivationfunctionssohowcanweknowwhichactivationfunctionfitthemodel? JasonBrownlee May17,2017at8:38am # Sorry,Idon’tunderstandthequestion. MarcoCheung August23,2017at12:51am # HiJason, Iaminterestedindeeplearningandmachinelearning.Youmentioned“Itdefinesahiddenlayerwith12neurons,connectedtotheinputlayerthatusereluactivationfunction.”Iwonderhowcanwedeterminethenumberofneuronsinordertoachieveahighaccuracyrateofthemodel? Thanksalot!!! JasonBrownlee August23,2017at6:55am # Usetrialanderror.Wecannotspecifythe“best”numberofneuronsanalytically.Wemusttest. RamzanShahid November10,2017at4:32am # Sir,thanksforyourtutorial.WouldyouliketomaketutorialonstockDataPredictionthroughNeuralNetworkModelandtrainingthisonanystockdata.Ifyouhaveonthissopleasesharethelink.Thanks JasonBrownlee November10,2017at10:39am # Iamreticenttoposttutorialsonstockmarketpredictiongiventherandomwalkhypothesisofsecurityprices: https://machinelearningmastery.com/gentle-introduction-random-walk-times-series-forecasting-python/ DharaBhavsar August28,2019at9:54pm # Hi, Iwouldliketoknowmoreaboutactivationfunction.Howitisworking?Howmanyactivationfunctions?UsingdifferentactivationfunctionHowmuchaffecttheoutputofthemodel? IwouldliketoalsoknowabouttheHiddenLayer.Howthesizeofthehiddenlayeraffectthemodel? JasonBrownlee August29,2019at6:09am # Inthistutorial,weusereluinthehiddenlayers,learnmorehere: https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-learning-neural-networks/ Thesizeofthelayerimpactsthecapacityofthemodel,learnmorehere: https://machinelearningmastery.com/how-to-control-neural-network-model-capacity-with-nodes-and-layers/ dhani June28,2018at2:44am # hihowusecnnforpixelclassificationonmhdimages Reply JasonBrownlee June28,2018at6:22am # Whatispixelclassification?Whataremhdimages? Reply TanmayKulkarni February11,2020at5:50am # Hello!Iwanttoknowifthere’sawaytoknowthevaluesofallweightsaftereachupdation? Reply JasonBrownlee February11,2020at5:53am # Yes,youcansavethemtofileorreviewthemmanually. Oftensavingisachievedusingacheckpoint: https://machinelearningmastery.com/check-point-deep-learning-models-keras/ Reply BlackBookKeeper August18,2018at10:15pm # runfile(‘C:/Users/Owner/Documents/untitled1.py’,wdir=’C:/Users/Owner/Documents’) Traceback(mostrecentcalllast): File“”,line1,in runfile(‘C:/Users/Owner/Documents/untitled1.py’,wdir=’C:/Users/Owner/Documents’) File“C:\Users\Owner\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py”,line705,inrunfile execfile(filename,namespace) File“C:\Users\Owner\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py”,line102,inexecfile exec(compile(f.read(),filename,‘exec’),namespace) File“C:/Users/Owner/Documents/untitled1.py”,line13,in model.add(Dense(12,input_dim=8,activation=’relu’)) File“C:\Users\Owner\Anaconda3\lib\site-packages\keras\engine\sequential.py”,line160,inadd name=layer.name+‘_input’) File“C:\Users\Owner\Anaconda3\lib\site-packages\keras\engine\input_layer.py”,line177,inInput input_tensor=tensor) File“C:\Users\Owner\Anaconda3\lib\site-packages\keras\legacy\interfaces.py”,line91,inwrapper returnfunc(*args,**kwargs) File“C:\Users\Owner\Anaconda3\lib\site-packages\keras\engine\input_layer.py”,line86,in__init__ name=self.name) File“C:\Users\Owner\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py”,line515,inplaceholder x=tf.placeholder(dtype,shape=shape,name=name) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\ops\array_ops.py”,line1530,inplaceholder returngen_array_ops._placeholder(dtype=dtype,shape=shape,name=name) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\ops\gen_array_ops.py”,line1954,in_placeholder name=name) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\op_def_library.py”,line767,inapply_op op_def=op_def) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\ops.py”,line2508,increate_op set_shapes_for_outputs(ret) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\ops.py”,line1894,inset_shapes_for_outputs output.set_shape(s) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\ops.py”,line443,inset_shape self._shape=self._shape.merge_with(shape) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\tensor_shape.py”,line550,inmerge_with stop=key.stop File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\tensor_shape.py”,line798,inas_shape “””ReturnsthisshapeasaTensorShapeProto.””” File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\tensor_shape.py”,line431,in__init__ sizeforoneormoredimension.e.g.TensorShape([None,256]) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\tensor_shape.py”,line376,inas_dimension other=as_dimension(other) File“C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\tensor_shape.py”,line32,in__init__ ifvalueisNone: TypeError:int()argumentmustbeastring,abytes-likeobjectoranumber,not‘TensorShapeProto’ thiserroroccurswhen{model.add(Dense(12,input_dim=8,activation=’relu’))}thiscommandisrun anyhelp? Reply JasonBrownlee August19,2018at6:20am # Saveallcodeintoafileandrunitasfollows: https://machinelearningmastery.com/faq/single-faq/how-do-i-run-a-script-from-the-command-line Reply Penchalaiah December8,2019at6:24pm # Fantastictutorial.Theexplanationissimpleandprecise.Thanksalot Reply JasonBrownlee December9,2019at6:47am # Thanks! Reply Loc June29,2022at1:00pm # greatarttist Reply Geoff May29,2016at6:18am # Canyouexplainhowtoimplementweightregularizationintothelayers? Reply JasonBrownlee June15,2016at5:50am # Yep,seehere: http://keras.io/regularizers/ Reply afthab October5,2018at8:32pm # heyyo!!!howurstartcodinginpython Reply JasonBrownlee October6,2018at5:43am # Starthere: https://machinelearningmastery.com/faq/single-faq/how-do-i-get-started-with-python-programming Reply KWC June14,2016at12:08pm # Importstatementsifothersneedthem: fromkeras.modelsimportSequential fromkeras.layersimportDense,Activation Reply JasonBrownlee June15,2016at5:49am # Thanks. IhadtheminPart6,butIhavealsoaddedthemtoPart1. Reply Shiran January20,2020at11:30am # Greatpost! Isitpossibletotrainaneuralnetworkthatreceivesasinputavectorxandtriestopredictanothervectorywherebothxandyarefloats? Reply JasonBrownlee January20,2020at2:07pm # Yes,thisiscalledregression: https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/ Reply AakashNain June29,2016at6:00pm # Ifthereare8inputsforthefirstlayerthenwhywehavetakenthemas’12’inthefollowingline: model.add(Dense(12,input_dim=8,init=’uniform’,activation=’relu’)) Reply JasonBrownlee June30,2016at6:47am # HiAakash. Theinputlayerisdefinedbytheinput_dimparameter,heresetto8. Thefirsthiddenlayerhas12neurons. Reply Joshua July2,2016at12:04am # Iranyourprogramandihaveanerror: ValueError:couldnotconvertstringtofloat: whatcouldbethereasonforthis,andhowmayIsolveit. thanks. greatpostbytheway. Reply JasonBrownlee July2,2016at6:20am # Itmightbeacopy-pasteerror.Perhapstrytocopyandrunthewholeexamplelistedinsection6? Reply Akash September28,2018at11:12am # Hellosir,IamfacingthesameproblemvalueError:couldnotconvertstringtofloat:‘”6’ alsoIamrunningtheexamplefromsection6. Reply JasonBrownlee September28,2018at3:00pm # Ihavesomesuggestionshere: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply yashu October5,2018at8:28pm # jasoncanuplzzhelpmehowtocode Reply JasonBrownlee October6,2018at5:42am # Sorry,Icannothelpyoutowritecode. Reply KeyChy July3,2019at5:45pm # Maybewhenyousetallparametersinanextracolumninyour*.csvfile.Thanyouschouldreplacethedelimiterfrom,to;like: dataset=numpy.loadtxt(“pima-indians-diabetes.csv”,delimiter=”;”) ThissolvedtheProblemforme. Reply JasonBrownlee July4,2019at7:40am # Thanksforsharing. Reply cheikhbrahim July5,2016at7:40pm # thankyouforyoursimpleandusefulexample. Reply JasonBrownlee July6,2016at6:22am # You’rewelcomecheikh. Reply NikhilThakur July6,2016at6:39pm # HelloSir,IamtryingtouseKerasforNLP,specificallysentenceclassification.Ihavegiventhemodelbuildingpartbelow.It’stakingquitealottimetoexecute.IamusingPycharmIDE. batch_size=32 nb_filter=250 filter_length=3 nb_epoch=2 pool_length=2 output_dim=5 hidden_dims=250 #Buildthemodel model1=Sequential() model1.add(Convolution1D(nb_filter,filter_length,activation=’relu’,border_mode=’valid’, input_shape=(len(embb_weights),dim),weights=[embb_weights])) model1.add(Dense(hidden_dims)) model1.add(Dropout(0.2)) model1.add(Activation(‘relu’)) model1.add(MaxPooling1D(pool_length=pool_length)) model1.add(Dense(output_dim,activation=’sigmoid’)) sgd=SGD(lr=0.1,decay=1e-6,momentum=0.9,nesterov=True) model1.compile(loss=’mean_squared_error’, optimizer=sgd, metrics=[‘accuracy’]) Reply JasonBrownlee July7,2016at7:31am # Youmaywantalargernetwork.YoumayalsowanttouseastandardrepeatingstructurelikeCNN->CNN->Pool->Dense. SeethispostonusingaCNN: http://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-networks-python-keras/ Later,youmayalsowanttotrysomestackedLSTMs. Reply AndreNorman July15,2016at10:40am # HiJason,thanksfortheawesomeexample.Giventhattheaccuracyofthismodelis79.56%.Fromhereon,whatstepswouldyoutaketoimprovetheaccuracy? GivenmynascentunderstandingofMachineLearning,myinitialapproachwouldhavebeen: Implementforwardpropagation,thencomputethecostfunction,thenimplementbackpropagation,usegradientcheckingtoevaluatemynetwork(disableafteruse),thenusegradientdescent. However,thisapproachseemsarduouscomparedtousingKeras.Thanksforyourresponse. Reply JasonBrownlee July15,2016at10:52am # HiAndre,indeedKerasmakesworkingwithneuralnetssomucheasier.Funeven! Wemaybemaxingoutonthisproblem,buthereissomegeneraladviceforliftingperformance. –dataprep–trylotsofdifferentviewsoftheproblemandseewhichisbestatexposingthestructureoftheproblemtothelearningalgorithm(datatransforms,featureengineering,etc.) –algorithmselection–trylotsofalgorithmsandseewhichoneorfewarebestontheproblem(tryonallviews) –algorithmtuning–tunewellperformingalgorithmstogetthemostoutofthem(gridsearchorrandomsearchhyperparametertuning) –ensembles–combinepredictionsfrommultiplealgorithms(stacking,boosting,bagging,etc.) Forneuralnets,therearealotofthingstotune,Ithinktherearebiggainsintryingdifferentnetworktopologies(layersandnumberofneuronsperlayer)inconcertwithtrainingepochsandlearningrate(biggernetsneedmoretraining). Ihopethathelpsasastart. Reply AndreNorman July18,2016at7:19am # Awesome!ThanksJason=) Reply JasonBrownlee July18,2016at8:03am # You’rewelcomeAndre. Reply quentin August7,2017at8:41pm # Someinterestingstuffhere https://youtu.be/vq2nnJ4g6N0 Reply JasonBrownlee August8,2017at7:49am # Thanksforsharing.Whatdidyoulikeaboutit? Reply RomillyCocking July21,2016at12:31am # HiJason,it’sagreatexamplebutifanyonerunsitinanIPython/JupyternotebooktheyarelikelytoencounteranI/Oerrorwhenrunningthefitstep.ThisisduetoaknownbuginIPython. Thesolutionistosetverbose=0likethis #Fitthemodel model.fit(X,Y,nb_epoch=40,batch_size=10,verbose=0) Reply JasonBrownlee July21,2016at5:36am # Great,thanksforsharingRomilly. Reply Anirban July23,2016at10:20pm # Greatexample.Haveaquerythough.HowdoInowgiveainputandgettheoutput(0or1).Canyouplsgivethecmdforthat. Thanks Reply JasonBrownlee July24,2016at6:53am # Youcancallmodel.predict()togetpredictionsandroundoneachvaluetosnaptoabinaryvalue. Forexample,belowisacompleteexampleshowingyouhowtoroundthepredictionsandprintthemtoconsole. #CreatefirstnetworkwithKeras fromkeras.modelsimportSequential fromkeras.layersimportDense importnumpy #fixrandomseedforreproducibility seed=7 numpy.random.seed(seed) #loadpimaindiansdataset dataset=numpy.loadtxt("pima-indians-diabetes.csv",delimiter=",") #splitintoinput(X)andoutput(Y)variables X=dataset[:,0:8] Y=dataset[:,8] #createmodel model=Sequential() model.add(Dense(12,input_dim=8,init='uniform',activation='relu')) model.add(Dense(8,init='uniform',activation='relu')) model.add(Dense(1,init='uniform',activation='sigmoid')) #Compilemodel model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) #Fitthemodel model.fit(X,Y,nb_epoch=150,batch_size=10,verbose=2) #calculatepredictions predictions=model.predict(X) #roundpredictions rounded=[round(x)forxinpredictions] print(rounded) 1234567891011121314151617181920212223242526 #CreatefirstnetworkwithKerasfromkeras.modelsimportSequentialfromkeras.layersimportDenseimportnumpy#fixrandomseedforreproducibilityseed=7numpy.random.seed(seed)#loadpimaindiansdatasetdataset=numpy.loadtxt("pima-indians-diabetes.csv",delimiter=",")#splitintoinput(X)andoutput(Y)variablesX=dataset[:,0:8]Y=dataset[:,8]#createmodelmodel=Sequential()model.add(Dense(12,input_dim=8,init='uniform',activation='relu'))model.add(Dense(8,init='uniform',activation='relu'))model.add(Dense(1,init='uniform',activation='sigmoid'))#Compilemodelmodel.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])#Fitthemodelmodel.fit(X,Y,nb_epoch=150,batch_size=10, verbose=2)#calculatepredictionspredictions=model.predict(X)#roundpredictionsrounded=[round(x)forxinpredictions]print(rounded) Reply Debanjan March27,2017at12:04pm # Hi,Whyyouarenotusinganytestset?Youarepredictingfromthetrainingset,Ithink. Reply JasonBrownlee March28,2017at8:19am # Correct,itisjustanexampletogetyoustartedwithKeras. Reply David June26,2017at12:24am # Jason,I’mnotquiteunderstandinghowthepredictedvalues([1.0,0.0,1.0,0.0,1.0,…)maptotherealworldproblem.Forinstance,whatdoesthatfirst“1.0”intheresultsindicate? Igetthatit’sapredictionof‘true’fordiabetes…buttowhichpatientisitpredictingthat—thefirstinthelist?Sothenthesecondresult,“0.0,”isthepredictionforthesecondpatient/rowinthedataset? Reply JasonBrownlee June26,2017at6:08am # Remembertheoriginalfilehas0and1valuesinthefinalclasscolumnwhere0isnoonsetofdiabetesand1isanonsetofdiabetes. Wearepredictingnewvaluesinthiscolumn. Wearemakingpredictionsforspecialrows,wepassintheirmedicalinfoandpredicttheonsetofdiabetes.Wejusthappentodothisforanumberofrowsatatime. Reply ami July16,2018at4:30pm # hellojason iamgettingthiserrorwhilecalculatingthepredictions. #calculatepredictions predictions=model.predict(X) #roundpredictions rounded=[round(x)forxinpredictions] print(rounded) ————————————————————————— TypeErrorTraceback(mostrecentcalllast) in() 2predictions=model.predict(X) 3#roundpredictions —->4rounded=[round(x)forxinpredictions] 5print(rounded) in(.0) 2predictions=model.predict(X) 3#roundpredictions —->4rounded=[round(x)forxinpredictions] 5print(rounded) TypeError:typenumpy.ndarraydoesn’tdefine__round__method JasonBrownlee July17,2018at6:09am # Tryremovingthecalltoround(). Rachel June28,2017at8:28pm # HiJason, CanIaskwhyyouusethesamedataXyoufitthemodeltodotheprediction? #Fitthemodel model.fit(X,Y,epochs=150,batch_size=10,verbose=2) #calculatepredictions predictions=model.predict(X) Rachel Reply JasonBrownlee June29,2017at6:34am # ItisallIhaveathand.Xmeansdatamatrix. ReplaceXinpredict()withXprimeorwhateveryoulike. Reply jitendra March27,2018at7:20pm # hii,howwillifeedtheinput(8,125,96,0,0,0.0,0.232,54)togetouroutput. predictions=model.predict(X) imeaninseadofXiwanttogetoutputof8,125,96,0,0,0.0,0.232,54. Reply JasonBrownlee March28,2018at6:24am # Wrapyourinputinanarray,n-columnswithonerow,thenpassthattothemodel. Doesthathelp? Reply Roman October5,2018at11:22pm # Hello,tryingtousepredictionsonsimilarneuralnetworkbutkeepgettingerrorsthatinputdimensionhasothershape. Canyousayhowarraymustlookonexampledneuralnetwork? JasonBrownlee October6,2018at5:45am # ForanMLP,datamustbeorganizedintoa2darrayofsamplesxfeatures Anirban July23,2016at10:52pm # Iamnotabletogettothelastepoch.Gettingerrorbeforethat: Epoch11/150 390/768[==============>……………]Traceback(mostrecentcalllast):.6921 ValueError:I/Ooperationonclosedfile Icouldresolvethisbyvaryingtheepochandbatchsize. Nowtopredictaunknownvalue,iloadedanewdatasetandusedpredictcmdasbelow: dataset_test=numpy.loadtxt(“pima-indians-diabetes_test.csv”,delimiter=”,”)–hasonlyonerow X=dataset_test[:,0:8] model.predict(X) ButIamgettingerror: X=dataset_test[:,0:8] IndexError:toomanyindicesforarray Canyouhelppls. Thanks Reply JasonBrownlee July24,2016at6:55am # IseeproblemslikethiswhenyourunfromanotebookorfromanIDE. Considerrunningexamplesfromtheconsoletoensuretheywork. Considertuningoffverboseoutput(verbose=0inthecalltofit())todisabletheprogressbar. Reply DavidKluszczynski July28,2016at12:42am # HiJason! Lovedthetutorial!Ihaveaquestionhowever. Isthereawaytosavetheweightstoafileafterthemodelistrainedforuses,suchaskaggle? Thanks, David Reply JasonBrownlee July28,2016at5:47am # ThanksDavid. Youcansavethenetworkweightstofilebycallingmodel.save_weights(“model.h5”) Youcanlearnmoreinthispost: http://machinelearningmastery.com/save-load-keras-deep-learning-models/ Reply AlexHopper July29,2016at5:45am # Hey,Jason!Thankyoufortheawesometutorial!I’veuseyourtutorialtolearnaboutCNN.Ihaveonequestionforyou…SupposingIwanttouseKerastoclassicateimagesandIhave3ormoreclassestoclassify,Howcouldmyalgorithmknowaboutthisclasses?Youknow,Ihavetocodewhatisacat,adogandahorse.Isthereanywaytocodethis?I’vetriedit: target_names=[‘class0(Cats)’,‘class1(Dogs)’,‘class2(Horse)’] print(classification_report(np.argmax(Y_test,axis=1),y_pred,target_names=target_names)) Butmyresultsarenotclassifyingcorrectly. precisionrecallf1-scoresupport class0(Cat)0.000.000.0017 class1(Dog)0.000.000.0014 class2(Horse)0.991.000.992526 avg/total0.980.990.982557 Reply JasonBrownlee July29,2016at6:41am # GreatquestionAlex. Thisisanexampleofamulti-classclassificationproblem.Youmustuseaonehotencodingontheoutputvariabletobeabletomodelitwithaneuralnetworkandspecifythenumberofclassesasthenumberofoutputsonthefinallayerofyournetwork. Iprovideatutorialwiththefamousirisdatasetthathas3outputclasseshere: http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/ Reply AlexHopper August1,2016at1:22am # Thankyou. I’llcheckit. Reply JasonBrownlee August1,2016at6:25am # NoproblemAlex. Reply Anonymouse August2,2016at11:28pm # Thiswasreallyuseful,thankyou I’musingkeras(withCNNs)forsentimentclassificationofdocumentsandI’dliketoimprovetheperformance,butI’mcompletelyatalosswhenitcomestotuningtheparametersinanon-arbitraryway.Couldyoumaybepointmesomewherethatwillhelpmegoaboutthisinamoresystematicfashion?Theremustbesomeheuristicsorrules-of-thumbthatcouldguideme. Reply JasonBrownlee August3,2016at8:09am # Ihaveatutorialcomingoutsoon(nextweek)thatprovidelotsofexamplesoftuningthehyperparametersofaneuralnetworkinKeras,butlimitedtoMLPs. ForCNNs,Iwouldadvisetuningthenumberofrepeatinglayers(conv+maxpool),thenumberoffiltersinrepeatingblock,andthenumberandsizeofdenselayersatthepredictingpartofyournetwork.Alsoconsiderusingsomefixedlayersfrompre-trainedmodelsasthestartofyournetwork(e.g.VGG)andtryjusttrainingsomeinputandoutputlayersarounditforyourproblem. Ihopethathelpsasastart. Reply Shopon August14,2016at5:04pm # HelloJason,MyAccuracyis:0.0104,butyoursis0.7879andmylossis:-9.5414.Isthereanyproblemwiththedataset?Idownloadedthedatasetfromadifferentsite. Reply JasonBrownlee August15,2016at12:36pm # Ithinktheremightbesomethingwrongwithyourimplementationoryourdataset.Yournumbersarewayout. Reply mohamed August15,2016at9:30am # aftertraining,howicanusethetrainedmodelonnewsample Reply JasonBrownlee August15,2016at12:36pm # Youcancallmodel.predict() Seeanabovecommentforaspecificcodeexample. Reply OmachiOkolo August16,2016at10:21pm # HiJason, i’mastudentconductingaresearchonhowtouseartificialneuralnetworktopredictthebusinessviabilityofpotentialsoftwareprojects. Iintendtousepythonasaprogramminglanguage.TheapplicationofANNfascinatesmebuti’mnewtomachinelearningandpython.Canyouhelpsuggesthowtogoaboutthis. Manythanks Reply JasonBrownlee August17,2016at9:51am # Considergettingagoodgroundinginhowtoworkthroughamachinelearningproblemendtoendinpythonfirst. Hereisagoodtutorialtogetyoustarted: http://machinelearningmastery.com/machine-learning-in-python-step-by-step/ Reply Agni August17,2016at6:23am # DearJeson,thisisagreattutorialforbeginners.Itwillsatisfytheneedofmanystudentswhoarelookingfortheinitialhelp.ButIhaveaquestion.Couldyoupleaselightonafewthings:i)howtotestthetrainedmodelusingtestdataset(i.e.,loadingoftestdatasetandappliedthemodelandsupposethetestfilenameistest.csv)ii)printtheaccuracyobtainedontestdatasetiii)theo/phasmorethan2class(suppose4-classclassificationproblem). Pleaseshowthewholeprogramtoovercomeanyconfusion. Thanksalot. Reply JasonBrownlee August17,2016at10:03am # Iprovideanexampleelsewhereinthecomments,youcanalsoseehowtomakepredictionsonnewdatainthispost: http://machinelearningmastery.com/5-step-life-cycle-neural-network-models-keras/ Foranexampleofmulti-classclassification,youcanseethistutorial: http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/ Reply DoronVetlzer August17,2016at9:29am # IamtryingtobuildaNeuralNetworkwithsomerecursiveconnectionsbutnotafullrecursivelayer,howdoIdothisinKeras? Reply DoronVetlzer August17,2016at9:31am # IcouldprintadiagramofthenetworkbutwhatIwantBasicallyisthateachneuroninthecurrenttimeframetoknowonlyitsownpreviousoutputandnottheoutputofalltheneuronsintheoutputlayer. Reply JasonBrownlee August17,2016at10:04am # Idon’tknowoffhandDoron. Reply DoronVeltzer August23,2016at2:28am # Thanksforreplyingthough,haveagoodday. Reply sairam August30,2016at8:49am # HelloJason, Thisisagreattutorial.Thanksforsharing. Iamhavingadatasetof100fingerprintsandiwanttoextractminutiaeof100fingerprintsusingpython(Keras).Canyoupleaseadvisewheretostart?Iamreallyconfused. Reply JasonBrownlee August31,2016at8:43am # Ifyourfingerprintsareimages,youmaywanttoconsiderusingconvolutionalneuralnetworks(CNNs)thataremuchbetteratworkingimagedata. Seethistutorialondigitrecognitionforastart: http://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-networks-python-keras/ Reply padmashri July6,2017at10:12pm # HiJason Thanksforthisgreattutorial,iamnewtomachinelearningiwentthroughyourbasictutorialonkerasandalsohandwritten-digit-recognition.Iwouldliketounderstandhowicantrainasetofimagedata,foreg.thesetofimagedatacanbesomethinglikesquare,circle,pyramid. pl.letmeknowhowtheinputdataneedstofedtotheprogramandhowweneedtoexportthemodel. Reply JasonBrownlee July9,2017at10:30am # Startbypreparingahigh-qualitydataset. Reply CM September1,2016at4:23pm # HiJason, Thanksforthegreatarticle.ButIhad1query. ArethereanyinbuiltfunctionsinkerasthatcangivemethefeatureimportancefortheANNmodel? Ifnot,canyousuggestatechniqueIcanusetoextractvariableimportancefromthelossfunction?IamconsideringanapproachsimilartothatusedinRFwhichinvolvespermutingthevaluesoftheselectedvariableandcalculatingtherelativeincreaseinloss. Regards, CM Reply JasonBrownlee September2,2016at8:07am # Idon’tbelievesoCM. Iwouldsuggestusingawrappermethodandevaluatesubsetsoffeaturestodevelopafeatureimportance/featureselectionreport. Italkalotmoreaboutfeatureselectioninthispost: http://machinelearningmastery.com/an-introduction-to-feature-selection/ Iprovideanexampleoffeatureselectioninscikit-learnhere: http://machinelearningmastery.com/feature-selection-machine-learning-python/ Ihopethathelpsasastart. Reply MineshJethva May15,2017at7:49pm # haveyoudevelopanyprogressforthisapproach?Ialsohavesameproblem. Reply Kamal September7,2016at2:09am # DearJason,IamnewtoDeeplearning.Beinganovice,Iamaskingyouatechnicalquestionwhichmayseemsilly.Myquestionisthat-canweusefeatures(forexamplelengthofthesentenceetc.)ofasentencewhileclassifyingasentence(supposetheo/pare+vesentenceand-vesentence)usingdeepneuralnetwork? Reply JasonBrownlee September7,2016at10:27am # GreatquestionKamal,yesyoucan.Iwouldencourageyoutoincludeallsuchfeaturesandseewhichgiveyouabumpinperformance. Reply Saurabh September11,2016at12:42pm # Hi,HowwouldIusethisonadatasetthathasmultipleoutputs?ForexampleadatasetwithoutputAandBwhereAcouldbe0or1andBcouldbe3or4? Reply JasonBrownlee September12,2016at8:30am # Youcouldusetwoneuronsintheoutputlayerandnormalizetheoutputvariablestobothbeintherangeof0to1. Thistutorialonmulti-classclassificationmightgiveyousomeideas: http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/ Reply Tom_P September17,2016at1:47pm # HiJason, ThetutoriallooksreallygoodbutunfortunatelyIkeepgettinganerrorwhenimportingDensefromkeras.layers,Igettheerror:AttributeError:module‘theano’hasnoattribute‘gof’ IhavetriedreinstallingTheanobutithasnotfixedtheissue. Bestwishes Tom Reply JasonBrownlee September18,2016at7:57am # HiTom,sorrytohearthat.Ihavenotseenthisproblembefore. Haveyousearchedgoogle?Icanseeafewpostsanditmightberelatedtoyourversionofscipyorsimilar. Letmeknowhowyougo. Reply shudhan September21,2016at5:54pm # HeyJason, Canyoupleasemakeatutorialonhowtoaddadditionaltraindataintothealreadytrainedmodel?Thiswillbehelpfulforthebiggerdatasets.Ireadthatwarmstartisusedforrandomforest.Butnotsurehowtoimplementasalgorithm.Ageneralisedversionofhowtoimplementwouldbegood.ThankYou! Reply JasonBrownlee September22,2016at8:08am # GreatquestionShudhan! Yes,youcouldsaveyourweights,loadthemlaterintoanewnetworktopologyandstarttrainingonnewdataagain. I’llworkoutanexampleincomingweeks,timepermitting. Reply Joanna September22,2016at1:09am # HiJason, firstofallcongratulationsforthisamazingworkthatyouhavedone! Hereismyquestion: Whataboutifmy.csvfileincludesalsobothnominalandnumericalattributes? ShouldIchangemynominalvaluestonumerical? Thankyouinadvance Reply JasonBrownlee September22,2016at8:19am # HiJoanna,yes. Youcanusealabelencodertoconvertnominaltointeger,andthenevenconverttheintegertoonehotencoding. Thispostwillgiveyoucodeyoucanuse: http://machinelearningmastery.com/data-preparation-gradient-boosting-xgboost-python/ Reply ATM October2,2016at5:47am # Asmallbug:- Line25:rounded=[round(x)forxinpredictions] shouldhavenumpy.roundinstead,forthecodetorun! Greattutorial,regardless.Thebesti’veseenforintrotoANNinpython.Thanks! Reply JasonBrownlee October2,2016at8:20am # Perhapsit’syourversionofPythonorenvironment? InPython2.7theround()functionisbuilt-in. Reply AC January14,2017at2:11am # Ifthereiscommentforpython3,shouldbebetter. #useunmpy.roundinstead,ifusingpython3, Reply JasonBrownlee January15,2017at5:24am # ThanksforthenoteAC. Reply Ash October9,2016at1:36am # Thisissimpletograsp!Greatpost!Howcanweperformdropoutinkeras? Reply JasonBrownlee October9,2016at6:49am # ThanksAsh. YoucanlearnaboutdropoutwithKerashere: http://machinelearningmastery.com/dropout-regularization-deep-learning-models-keras/ Reply HomagniSaha October14,2016at4:15am # HelloJason, Youareusingmodel.predictintheendtopredicttheresults.Isitpossibletosavethemodelsomewhereintheharddiskandtransferittoanothermachine(turtlebotrunningonROSformyinstance)andthenusethemodeldirectlyonturtlebottopredicttheresults? Pleasetellmehow Thankingyou HomagniSaha Reply JasonBrownlee October14,2016at9:07am # HiHomagni,greatquestion. Absolutely! LearnexactlyhowinthistutorialIwrote: http://machinelearningmastery.com/save-load-keras-deep-learning-models/ Reply Rimi October16,2016at8:21pm # HiJason, Iimplementedyoucodetobeginwith.ButIamgettinganaccuracyof45.18%withthesameparametersandeverything. Cantfigureoutwhy. Thanks Reply JasonBrownlee October17,2016at10:29am # TheredoessoundlikeaproblemthereRimi. Confirmthecodeanddatamatchexactly. Reply Ankit October26,2016at8:12pm # HiJason, Iamlittleconfusedwithfirstlayerparameters.Yousaidthatfirstlayerhas12neuronsandexpects8inputvariables. Whythereisadifferencebetweennumberofneurons,input_dimforfirstlayer. Regards, Ankit Reply JasonBrownlee October27,2016at7:45am # HiAnkit, Theproblemhas8inputvariablesandthefirsthiddenlayerhas12neurons.Inputsarethecolumnsofdata,thesearefixed.TheHiddenlayersingeneralarewhateverwedesignbasedonwhatevercapacitywethinkweneedtorepresentthecomplexityoftheproblem.Inthiscase,wehavechosen12neuronsforthefirsthiddenlayer. Ihopethatisclearer. Reply Tom October27,2016at3:04am # Hi, Ihaveadata,IRISlikedatabutwithmorecolmuns. IwanttouseMLPandDBN/CNNClassifier(oranyotherDeepLearningclassificaitonalgorithm)onmydatatoseehowcorrectlyitdoesclassifiedinto6groups. PreviouslyusingDEEPLEARNINGFORJ,todayfirsttimeseeKERAS. doesKERAShasexamples(codeexamples)ofDLClassificationalgorithms? Kindly, Tom Reply JasonBrownlee October27,2016at7:48am # YesTom,theexampleinthispostisanexampleofaneuralnetwork(deeplearning)appliedtoaclassificationproblem. Reply Rumesa October30,2016at1:57am # Ihaveinstalledtheanobutitgivesmetheerroroftensorflow.isitmendatorytoinstallbothpackages?becausetensorflowisnotsupportedonwndows.theonlywaytogetitonwindowsistoinstallvirtualmachine Reply JasonBrownlee October30,2016at8:57am # KeraswillworkjustfinewithTheano. JustinstallTheano,andconfigureKerastousetheTheanobackend. MoreinformationaboutconfiguringtheKerasbackendhere: http://machinelearningmastery.com/introduction-python-deep-learning-library-keras/ Reply Rumesa October31,2016at4:36am # heyjasonIhaverunyourcodebutgotthefollowingerror.AlthoughIhaveareadyinstalledtheanobackend.helpmeout.Ijuststuck. UsingTensorFlowbackend. Traceback(mostrecentcalllast): File“C:\Users\pc\Desktop\first.py”,line2,in fromkeras.modelsimportSequential File“C:\Users\pc\Anaconda3\lib\site-packages\keras\__init__.py”,line2,in from.importbackend File“C:\Users\pc\Anaconda3\lib\site-packages\keras\backend\__init__.py”,line64,in from.tensorflow_backendimport* File“C:\Users\pc\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py”,line1,in importtensorflowastf ImportError:Nomodulenamed‘tensorflow’ >>> Reply JasonBrownlee October31,2016at5:34am # ChangethebackendusedbyKerasfromTensorFlowtoTheano. YoucandothiseitherbyusingthecommandlineswitchorchangingtheKerasconfigfile. SeethelinkIpostedinthepreviouspostforinstructions. Reply Maria January6,2017at1:05pm # HelloRumesa! Haveyousolvedyourproblem?Ihavethesameone.Everywhereisthesameanswerwithkeras.jsonfileorenvirinmentvariablebutitdoesn’twork.Canyoutellmewhathaveworkedforyou? Reply JasonBrownlee January7,2017at8:20am # Interesting. Maybethereisanissuewiththelatestversionandatightcouplingtotensorflow?Ihavenotseenthismyself. PerhapsitmightbeworthtestingpriorversionsofKeras,suchas1.1.0? Trythis: pipinstall--upgrade--no-depskeras==1.1.0 1 pipinstall--upgrade--no-depskeras==1.1.0 Reply Alexon November1,2016at6:54am # HiJason, Firstoff,thankssomuchforcreatingtheseresources,Ihavebeenkeepinganeyeonyournewsletterforawhilenow,andIfinallyhavethefreetimetostartlearningmoreaboutitmyself,soyourworkhasbeenreallyappreciated. Myquestionis:HowcanIset/gettheweightsofeachhiddennode? Iamplanningtocreateseveralarraysrandomizedweights,thenuseageneticalgorithmtoseewhichweightarrayperformsthebestandimproveovergenerations.Howwouldbethebestwaytogoaboutthis,andifIusea“relu”activationfunction,amIrightinthinkingtheserandomlygeneratedweightsshouldbebetween0and0.05? Manythanksforyourhelp🙂 Alexon Reply JasonBrownlee November1,2016at8:05am # ThanksAlexon, Youcangetandsettheweightsfromanetwork. Youcanlearnmoreabouthowtodothisinthecontextofsavingtheweightstofilehere: http://machinelearningmastery.com/save-load-keras-deep-learning-models/ Ihopethathelpsasastart,I’dlovetohearhowyougo. Reply Alexon November6,2016at6:36am # Thatsgreat,thanksforpointingmeintherightdirection. I’dbehappytoletyouknowhowitgoes,butmighttakeawhileasthisisverymucha“whenIcanfindthetime”projectbetweenjobs🙂 Cheers! Reply ArnaldoGunzi November2,2016at10:17pm # Niceintroduction,thanks! Reply JasonBrownlee November3,2016at7:59am # I’mgladyoufounditusefulArnaldo. Reply Abbey November14,2016at11:05pm # Goodday Ihaveaquestion,howcanIrepresentacharacterasavectorthatcouldbeaninputfortheneuralnetworktopredictthewordmeaningandtrainedusingLSTM Forinstance,IhavebftopredictboyfriendorbestfriendandsimilarlyIhave2mortopredicttomorrow.Ineedtoencodealltheinputasacharacterrepresentedasvector,sothatitcanbetrainwithRNN/LSTMtopredicttheoutput. Thankyou. KindRegards Reply JasonBrownlee November15,2016at7:54am # HiAbbey,Youcanmapcharacterstointegerstogetintegervectors. Reply Abbey November15,2016at6:17pm # ThankyouJason,ifimapcharacterstointegersvaluetogetvectorsusingEnglishAlphabets,numbersandspecialcharacters ThequestionishowwillLSTMpredictthecharacter.Pleaseexampleinmoredetailsforme. Regards Reply JasonBrownlee November16,2016at9:27am # HiAbbey, Ifyouroutputvaluesarealsocharacters,youcanmapthemontointegers,andreversethemappingtoconvertthepredictionsbacktotext. Reply Abbey November16,2016at8:39pm # Theoutputvalueofthecharactersencodingwillbetext Abbey November15,2016at6:22pm # Thankyou,Jason,ifImapcharacterstointegersvaluetogetvectorsrepresentationoftheinformaltextusingEnglishAlphabets,numbersandspecialcharacters ThequestionishowwillLSTMpredictthecharacterorwordsthathaveclosemeaningtotheinputvalue.Pleaseexampleinmoredetailsforme.IunderstandhowRNN/LSTMworkbasedonyourtutorialexamplebutthelogicindesigningprocessingiswhatIamstresswith. Regards Reply Ammar November27,2016at10:35am # hiJason, iamtryingtoimplementCNNonedimentiononmydata.so,ibluitmynetwork. theissueis: deftrain_model(model,X_train,y_train,X_test,y_test): X_train=X_train.reshape(-1,1,41) X_test=X_test.reshape(-1,1,41) numpy.random.seed(seed) model.fit(X_train,y_train,validation_data=(X_test,y_test),nb_epoch=100,batch_size=64) #Finalevaluationofthemodel scores=model.evaluate(X_test,y_test,verbose=0) print(“Accuracy:%.2f%%”%(scores[1]*100)) thismethodabovedoesnotworkanddoesnotgivemeanyerrormessage. couldyouhelpmewiththisplease? Reply JasonBrownlee November28,2016at8:40am # HiAmmar,I’msurprisedthatthereisnoerrormessage. Perhapsrunfromthecommandlineandaddsomeprint()statementstoseeexactlywhereitstops. Reply KK November28,2016at6:55pm # HiJason Greatwork.Ihaveanotherdoubt.Howcanweapplythistotextmining.Ihaveacsvfilecontainingreviewdocumentandlabel.Iwanttoapplyclassifythedocumentsbasedonthetextavailable.CanUdothisfavor. Reply JasonBrownlee November29,2016at8:48am # IwouldrecommendconvertingthecharstointsandthenusinganEmbeddinglayer. Reply AlexM November30,2016at10:52pm # MrJason,thisisgreattutorialbutIamstackwithsomeerrors. FirstIcan’tloaddatasetcorrectly,triedtocorrecterrorbutcan’tmakeit.(FileNotFoundError:[Errno2]Nosuchfileordirectory:‘pima-indians-diabetes.csv’). Second:Whiletryingtoevaluatethemodelitsays(Xisnotdefined)Maybethisisbecauseuploadingfailed. Thanks! Reply JasonBrownlee December1,2016at7:29am # YouneedtodownloadthefileandplaceitinyourcurrentworkingdirectoryAlex. Doesthathelp? Reply AlexM December1,2016at6:45pm # Sir,itisnowsuccessful…. Thanks! Reply JasonBrownlee December2,2016at8:15am # GladtohearitAlex. Reply Bappaditya December2,2016at7:35pm # HiJason, Firstofallaspecialthankstoyouforprovidingsuchagreattutorial.Iamverynewtomachinelearningandtrulyspeakingihadnobackgroundindatascience.TheconceptofMLoverwhelmedmeandnowihaveadesiretobeanexpertofthisfield.Ineedyouradvicetostartfromascratch.AlsoiamaPhDstudentinComputerEngineering(computerhardware)andiwanttoapplyitasatoolforfaultdetectionandtestingforICs.Canyouprovidemesomereferencesonthisfield? Reply JasonBrownlee December3,2016at8:29am # HiBappaditya, Mybestadviceforgettingstartedishere: http://machinelearningmastery.com/start-here/#getstarted Ibelievemachinelearninganddeeplearningaregoodtoolsforuseonproblemsinfaultdetection.Agoodplacetofindreferencesisherehttp://scholar.google.com Bestofluckwithyourproject. Reply AlexM December3,2016at8:00pm # Wellasusualinourdailycodinglifeerrorshappen,nowIhavethiserrorhowcanIcorrectit?Thanks! ”————————————————————————— NoBackendErrorTraceback(mostrecentcalllast) in() 16importlibrosa.display 17audio_path=(‘/Users/MA/PythonNotebook/OK.mp3’) —>18y,sr=librosa.load(audio_path) C:\Users\MA\Anaconda3\lib\site-packages\librosa\core\audio.pyinload(path,sr,mono,offset,duration,dtype) 107 108y=[] –>109withaudioread.audio_open(os.path.realpath(path))asinput_file: 110sr_native=input_file.samplerate 111n_channels=input_file.channels C:\Users\MA\Anaconda3\lib\site-packages\audioread\__init__.pyinaudio_open(path) 112 113#Allbackendsfailed! –>114raiseNoBackendError() NoBackendError: ” ThatistheerrorIamgettingjustwhentryingtoloadasongintolibrosa… Thanks!!@JasonBrownlee Reply JasonBrownlee December4,2016at5:30am # Sorry,thislookslikeanissuewithyourlibrosalibrary,notamachinelearningissue.Ican’tgiveyouexpertadvice,sorry. Reply AlexM December4,2016at10:30pm # ThanksIhavemanagedtocorrecttheerror… HappySundaytoyouall…… Reply JasonBrownlee December5,2016at6:49am # GladtohearitAlex. Reply ayush June19,2018at3:27am # howdidyousolvedtheproblem? Reply Lei December4,2016at10:52pm # Hi,Jason,thankyouforyouramazingexamples. Irunthesamecodeonmylaptop.ButIdidnotgetthesameresults.Whatcouldbethepossiblereasons? Iamusingwindows8.164bit+eclipse+anaconda4.2+theano0.9.4+CUDA7.5 Igotresultslikefollows. …… Epoch145/150 10/768[…………………………]–ETA:0s–loss:0.3634–acc:0.8000 80/768[==>………………………]–ETA:0s–loss:0.4066–acc:0.7750 150/768[====>…………………….]–ETA:0s–loss:0.4059–acc:0.8067 220/768[=======>………………….]–ETA:0s–loss:0.4047–acc:0.8091 300/768[==========>……………….]–ETA:0s–loss:0.4498–acc:0.7867 380/768[=============>…………….]–ETA:0s–loss:0.4595–acc:0.7895 450/768[================>………….]–ETA:0s–loss:0.4568–acc:0.7911 510/768[==================>………..]–ETA:0s–loss:0.4553–acc:0.7882 580/768[=====================>……..]–ETA:0s–loss:0.4677–acc:0.7776 660/768[========================>…..]–ETA:0s–loss:0.4697–acc:0.7788 740/768[===========================>..]–ETA:0s–loss:0.4611–acc:0.7838 768/768[==============================]–0s–loss:0.4614–acc:0.7799 Epoch146/150 10/768[…………………………]–ETA:0s–loss:0.3846–acc:0.8000 90/768[==>………………………]–ETA:0s–loss:0.5079–acc:0.7444 170/768[=====>……………………]–ETA:0s–loss:0.4500–acc:0.7882 250/768[========>…………………]–ETA:0s–loss:0.4594–acc:0.7840 330/768[===========>………………]–ETA:0s–loss:0.4574–acc:0.7818 400/768[==============>……………]–ETA:0s–loss:0.4563–acc:0.7775 470/768[=================>…………]–ETA:0s–loss:0.4654–acc:0.7723 540/768[====================>………]–ETA:0s–loss:0.4537–acc:0.7870 620/768[=======================>……]–ETA:0s–loss:0.4615–acc:0.7806 690/768[=========================>….]–ETA:0s–loss:0.4631–acc:0.7739 750/768[============================>.]–ETA:0s–loss:0.4649–acc:0.7733 768/768[==============================]–0s–loss:0.4636–acc:0.7734 Epoch147/150 10/768[…………………………]–ETA:0s–loss:0.3561–acc:0.9000 90/768[==>………………………]–ETA:0s–loss:0.4167–acc:0.8556 170/768[=====>……………………]–ETA:0s–loss:0.4824–acc:0.8059 250/768[========>…………………]–ETA:0s–loss:0.4534–acc:0.8080 330/768[===========>………………]–ETA:0s–loss:0.4679–acc:0.7848 400/768[==============>……………]–ETA:0s–loss:0.4590–acc:0.7950 460/768[================>………….]–ETA:0s–loss:0.4619–acc:0.7913 530/768[===================>……….]–ETA:0s–loss:0.4562–acc:0.7868 600/768[======================>…….]–ETA:0s–loss:0.4497–acc:0.7883 680/768[=========================>….]–ETA:0s–loss:0.4525–acc:0.7853 760/768[============================>.]–ETA:0s–loss:0.4568–acc:0.7803 768/768[==============================]–0s–loss:0.4561–acc:0.7812 Epoch148/150 10/768[…………………………]–ETA:0s–loss:0.4183–acc:0.9000 80/768[==>………………………]–ETA:0s–loss:0.3674–acc:0.8750 160/768[=====>……………………]–ETA:0s–loss:0.4340–acc:0.8250 240/768[========>…………………]–ETA:0s–loss:0.4799–acc:0.7583 320/768[===========>………………]–ETA:0s–loss:0.4648–acc:0.7719 400/768[==============>……………]–ETA:0s–loss:0.4596–acc:0.7775 470/768[=================>…………]–ETA:0s–loss:0.4475–acc:0.7809 540/768[====================>………]–ETA:0s–loss:0.4545–acc:0.7778 620/768[=======================>……]–ETA:0s–loss:0.4590–acc:0.7742 690/768[=========================>….]–ETA:0s–loss:0.4769–acc:0.7652 760/768[============================>.]–ETA:0s–loss:0.4748–acc:0.7658 768/768[==============================]–0s–loss:0.4734–acc:0.7669 Epoch149/150 10/768[…………………………]–ETA:0s–loss:0.3043–acc:0.9000 90/768[==>………………………]–ETA:0s–loss:0.4913–acc:0.7111 170/768[=====>……………………]–ETA:0s–loss:0.4779–acc:0.7588 250/768[========>…………………]–ETA:0s–loss:0.4794–acc:0.7640 320/768[===========>………………]–ETA:0s–loss:0.4957–acc:0.7562 370/768[=============>…………….]–ETA:0s–loss:0.4891–acc:0.7703 450/768[================>………….]–ETA:0s–loss:0.4737–acc:0.7867 520/768[===================>……….]–ETA:0s–loss:0.4675–acc:0.7865 600/768[======================>…….]–ETA:0s–loss:0.4668–acc:0.7833 680/768[=========================>….]–ETA:0s–loss:0.4677–acc:0.7809 760/768[============================>.]–ETA:0s–loss:0.4648–acc:0.7803 768/768[==============================]–0s–loss:0.4625–acc:0.7826 Epoch150/150 10/768[…………………………]–ETA:0s–loss:0.2751–acc:1.0000 100/768[==>………………………]–ETA:0s–loss:0.4501–acc:0.8100 170/768[=====>……………………]–ETA:0s–loss:0.4588–acc:0.8059 250/768[========>…………………]–ETA:0s–loss:0.4299–acc:0.8200 310/768[===========>………………]–ETA:0s–loss:0.4298–acc:0.8129 380/768[=============>…………….]–ETA:0s–loss:0.4365–acc:0.8053 460/768[================>………….]–ETA:0s–loss:0.4469–acc:0.7957 540/768[====================>………]–ETA:0s–loss:0.4436–acc:0.8000 620/768[=======================>……]–ETA:0s–loss:0.4570–acc:0.7871 690/768[=========================>….]–ETA:0s–loss:0.4664–acc:0.7783 760/768[============================>.]–ETA:0s–loss:0.4617–acc:0.7789 768/768[==============================]–0s–loss:0.4638–acc:0.7773 32/768[>………………………..]–ETA:0s 448/768[================>………….]–ETA:0sacc:79.69% Reply JasonBrownlee December5,2016at6:50am # Thereisrandomnessinthelearningprocessthatwecannotcontrolforyet. Seethispost: http://machinelearningmastery.com/randomness-in-machine-learning/ Reply Nanya December10,2016at2:55pm # HelloJasonBrownlee,Thxforsharing~ I’mnewindeeplearning.AndIamwonderingcanwhatyoudicussedhere:”Keras”beusedtobuildaCNNintensorflowandtrainsomecsvfielsforclassification.Maybethisisastupidquestion,butwaitingforyoureply.I’mworkingonmygraduationprojectforWordsensedisambiguationwithcnn,andjustcan’tmoveon.Hopeforyourheip~Besewishes! Reply JasonBrownlee December11,2016at5:22am # SorryNanya,I’mnotsureIunderstandyourquestion.Areyouabletorephraseit? Reply Anon December16,2016at12:51am # I’vejustinstalledAnacondawithKerasandamusingpython3.5. Itseemsthere’sanerrorwiththeroundingusingPy3asopposedtoPy2.Ithinkit’sbecauseofthischange:https://github.com/numpy/numpy/issues/5700 Iremovedtheroundingandjustusedprint(predictions)anditseemedtoworkoutputtingfloatsinstead. Doesthislookcorrect? … Epoch150/150 0s–loss:0.4593–acc:0.7839 [[0.79361773] [0.10443526] [0.90862554] …, [0.33652252] [0.63745886] [0.11704451]] Reply JasonBrownlee December16,2016at5:44am # Nice,itdoeslookgood! Reply FlorinClaudiuMihalache December19,2016at2:37am # HiJasonBrownlee Itriedtomodifiedyourexempleformyproblem(LetterRecognition,http://archive.ics.uci.edu/ml/datasets/Letter+Recognition). Mydatasetlooklikehttp://archive.ics.uci.edu/ml/machine-learning-databases/letter-recognition/letter-recognition.data(T,2,8,3,5,1,8,13,0,6,6,10,8,0,8,0,8).Itrytosplitthedataininputandouputlikethis: X=dataset[:,1:17] Y=dataset[:,0] butahavesomeerror(somethingrelatedthatstringsarenotrecognized). ItriedtomodifiedeachletterwhittheASCIIcode(Abecame65andsoon).Thestringerrordisappeared. Theprogramcompilesnowbuttheoutputlooklikethis: 17445/20000[=========================>….]–ETA:0s–loss:-1219.4768–acc:0.0000e+00 17605/20000[=========================>….]–ETA:0s–loss:-1219.4706–acc:0.0000e+00 17730/20000[=========================>….]–ETA:0s–loss:-1219.4566–acc:0.0000e+00 17890/20000[=========================>….]–ETA:0s–loss:-1219.4071–acc:0.0000e+00 18050/20000[==========================>…]–ETA:0s–loss:-1219.4599–acc:0.0000e+00 18175/20000[==========================>…]–ETA:0s–loss:-1219.3972–acc:0.0000e+00 18335/20000[==========================>…]–ETA:0s–loss:-1219.4642–acc:0.0000e+00 18495/20000[==========================>…]–ETA:0s–loss:-1219.5032–acc:0.0000e+00 18620/20000[==========================>…]–ETA:0s–loss:-1219.4391–acc:0.0000e+00 18780/20000[===========================>..]–ETA:0s–loss:-1219.5652–acc:0.0000e+00 18940/20000[===========================>..]–ETA:0s–loss:-1219.5520–acc:0.0000e+00 19080/20000[===========================>..]–ETA:0s–loss:-1219.5381–acc:0.0000e+00 19225/20000[===========================>..]–ETA:0s–loss:-1219.5182–acc:0.0000e+00 19385/20000[============================>.]–ETA:0s–loss:-1219.6742–acc:0.0000e+00 19535/20000[============================>.]–ETA:0s–loss:-1219.7030–acc:0.0000e+00 19670/20000[============================>.]–ETA:0s–loss:-1219.7634–acc:0.0000e+00 19830/20000[============================>.]–ETA:0s–loss:-1219.8336–acc:0.0000e+00 19990/20000[============================>.]–ETA:0s–loss:-1219.8532–acc:0.0000e+00 20000/20000[==============================]–1s–loss:-1219.8594–acc:0.0000e+00 18880/20000[===========================>..]–ETA:0sacc:0.00% Idonotunderstandwhy.Canyoupleasehelpme Reply Anon December26,2016at6:44am # WhatversionofPythonareyourunning? Reply karishmasharma December22,2016at10:03am # HiJason, Sincetheepochissetto150andbatchsizeis10,doesthetrainingalgorithmpick10trainingexamplesatrandomineachiteration,giventhatwehadonly768totalinX.Ordoesitsamplerandomlyafterithasfinishedcoveringall. Thanks Reply JasonBrownlee December23,2016at5:27am # Goodquestion, Ititeratesoverthedataset150timesandwithinoneepochitworksthrough10rowsatatimebeforedoinganupdatetotheweights.Thepatternsareshuffledbeforeeachepoch. Ihopethathelps. Reply Kaustuv January9,2017at4:57am # HiJason Thanksalotforthisblog.Itreallyhelpsmetostartlearningdeeplearningwhichwasinaplanningstateforlastfewmonths.Yoursimpleenrichblogsareawsome.Noquestionsfrommysidebeforecompletingalltutorials. Onequestionregardingavailabilityofyourbook.HowcanIbuythosebooksfromIndia? Reply JasonBrownlee January9,2017at7:53am # Allmybooksandtrainingaredigital,youcanpurchasethemfromhere: http://machinelearningmastery.com/products Reply StephenWilson January15,2017at4:00pm # HiJason,firstlyyourworkhereisafantasticresourceandIamverythankfulfortheeffortyouputin. Iamaslightly-better-than-beginneratpythonandanabsolutenoviceatML,Iwonderifyoucouldhelpmeclassifymyproblemandfindanangletoworkatitfrom. Mydataisthus: ColumnNames:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,Result Values:4,4,6,6,3,2,5,5,0,0,0,0,0,0,0,4 IwanttofindthepercentagechanceofeachColumnNamescategorybeingtheResultbasedofftheconfigurationofallthevaluespresentfrom1-15.ThenifneedbecomparetheconfigurationofValueswithanotherrowofvaluestofindthesame,Resultinginthetotalneededcalculationas: ColumnNames:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,Result Values:4,4,6,6,3,2,5,5,0,0,0,0,0,0,0,4 Values2:7,3,5,1,4,8,6,2,9,9,9,9,9,9,9 Iapologizeifmyexplanationisnotclear,andappreciateanyhelpyoucangivemethankyou. Reply JasonBrownlee January16,2017at10:39am # HiStephen, Thisprocessmighthelpyouworkthroughyourproblem: http://machinelearningmastery.com/start-here/#process Specificallythefirststepindefiningyourproblem. Letmeknowhowyougo. Reply Rohit January16,2017at10:37pm # ThanksJasonforsuchaniceandconciseexample. JustwantedtoaskifitispossibletosavethismodelinafileandportittomaybeanAndroidoriOSdevice?Ifso,whatarethelibrariesavailableforthesame? Thanks Rohit Reply JasonBrownlee January17,2017at7:38am # ThanksRohit, Here’sanexampleofsavingaKerasmodeltofile: http://machinelearningmastery.com/save-load-keras-deep-learning-models/ Idon’tknowaboutrunningKerasonanAndroidoriOSdevice.Letmeknowhowyougo. Reply zaheerkhan June16,2017at7:17pm # DearJason,Thanksforsharingthisarticle. Iamnovicetothedeeplearning,andmyapologyifmyquestionisnotclear.myquestioniscouldwecallallthatfunctionsandprogramfromany.php,.aspx,or.htmlwebpage.imeaniloadthevariablesandotherfilesselectionfromuserinterfaceandthenmaketheminputtothisfunctions. willbewaitingforyourkindreply. thanksinadvance. zaheer Reply JasonBrownlee June17,2017at7:25am # Perhaps,thissoundslikeasystemsdesignquestion,notreallymachinelearning. Iwouldsuggestyougatherrequirements,assessriskslikeanysoftwareengineeringproject. Reply Hsiang January18,2017at3:35pm # Hi,Jason Thankyouforyourblog!Itiswonderful! Iusedtensorflowasbackend,andimplementedtheproceduresusingJupyter. Idid“sourceactivatetensorflow”->“ipythonnotebook”. IcansuccessfullyuseKerasandimporttensorflow. However,itseemsthatsuchenvironmentdoesn’tsupportpandasandsklearn. Doyouhaveanywaytoincorporatepandas,sklearnandkeras? (Iwishtousesklearntorevisittheclassificationproblemandcomparetheaccuracywiththedeeplearningmethod.ButIalsowishtoputtheworkstogetherinthesameinterface.) Thanks! Reply JasonBrownlee January19,2017at7:24am # Sorry,Idonotusenotebooksmyself.Icannotofferyougoodadvice. Reply Hsiang January19,2017at12:53pm # Thanks,Jason! Actuallytheproblemisnotonnotebooks.EvenIusedtheterminalmode,i.e.doing“sourceactivatetensorflow”only.Itfailedtoimportsklearn.Doesthatmeantensorflowlibraryisnotcompatiblewithsklearn?Thanksagain! Reply JasonBrownlee January20,2017at10:17am # SorryHsiang,Idon’thaveexperienceusingsklearnandtensorflowwithvirtualenvironments. Reply Hsiang January21,2017at12:46am # Thankyou! JasonBrownlee January21,2017at10:34am # You’rewelcomeHsiang. keshavbansal January24,2017at12:45am # hellosir, Averyinformativepostindeed.Iknowmyquestionisaverytrivialonebutcanyoupleaseshowmehowtopredictonaexplicitlymentioneddatatuplesayv=[6,148,72,35,0,33.6,0.627,50] thanksforthetutorialanyway Reply JasonBrownlee January24,2017at11:04am # Hikeshav, Youcanmakepredictionsbycallingmodel.predict() Reply CATRINAWEBB January25,2017at9:06am # WhenIrerunthefile(withoutpredictions)doesitresetthemodelandweights? Reply Ericson January30,2017at8:04pm # excusemesir,iwannaaskyouaquestionaboutthisparagraph”dataset=numpy.loadtxt(“pima-indians-diabetes.csv”,delimiter=’,’)”,iusedthemacanddownloadedthedataset,theniexchangedthetextintocsvfile.Runningtheprogram ,henigot:{Python2.7.13(v2.7.13:a06454b1afa1,Dec172016,12:39:47) [GCC4.2.1(AppleInc.build5666)(dot3)]ondarwin Type“copyright”,“credits”or“license()”formoreinformation. >>> ============RESTART:/Users/luowenbin/Documents/database_test.py============ UsingTensorFlowbackend. Traceback(mostrecentcalllast): File“/Users/luowenbin/Documents/database_test.py”,line9,in dataset=numpy.loadtxt(“pima-indians-diabetes.csv”,delimiter=’,’) File“/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/npyio.py”,line985,inloadtxt items=[conv(val)for(conv,val)inzip(converters,vals)] File“/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/npyio.py”,line687,infloatconv returnfloat(x) ValueError:couldnotconvertstringtofloat:book >>>} Howcanisolvethisproblem?givemeahandthankyou! Reply JasonBrownlee February1,2017at10:22am # HiEricson, Confirmthatthecontentsof“pima-indians-diabetes.csv”meetyourexpectationofalistofCSVlines. Reply Sukhpal February7,2017at9:00pm # excusemesir,whenirunthiscodeformydataset,Iencounterthisproblem…pleasehelpmefindingsolutiontothisproblem runfile(‘C:/Users/sukhpal/.spyder/temp.py’,wdir=’C:/Users/sukhpal/.spyder’) UsingTensorFlowbackend. Traceback(mostrecentcalllast): File“”,line1,in runfile(‘C:/Users/sukhpal/.spyder/temp.py’,wdir=’C:/Users/sukhpal/.spyder’) File“C:\Users\sukhpal\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py”,line866,inrunfile execfile(filename,namespace) File“C:\Users\sukhpal\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py”,line87,inexecfile exec(compile(scripttext,filename,‘exec’),glob,loc) File“C:/Users/sukhpal/.spyder/temp.py”,line1,in fromkeras.modelsimportSequential File“C:\Users\sukhpal\Anaconda2\lib\site-packages\keras\__init__.py”,line2,in from.importbackend File“C:\Users\sukhpal\Anaconda2\lib\site-packages\keras\backend\__init__.py”,line67,in from.tensorflow_backendimport* File“C:\Users\sukhpal\Anaconda2\lib\site-packages\keras\backend\tensorflow_backend.py”,line1,in importtensorflowastf ImportError:Nomodulenamedtensorflow Reply JasonBrownlee February8,2017at9:34am # Thisisachangewiththemostrecentversionoftensorflow,Iwillinvestigateandchangetheexample. Fornow,considerinstallingandusinganolderversionoftensorflow. Reply Will February14,2017at5:33am # Greattutorial!Amazingamountofworkyou’veputinandgreatmarketingskills(Ialsohaveanemaillist,ebooksandsequence,etc).IranthisinJupyternotebook…Inoticedthe144thepoch(acc.7982)hadmoreaccuracythanat150.Whyisthat? P.S.ididthisfortheprint:print(numpy.round(predictions)) Itseemstoavoidalistofarrayswhichwhenprintingincludesthedtype(messy) Reply JasonBrownlee February14,2017at10:07am # ThanksWill. Themodelwillfluctuateinperformancewhilelearning.Youcanconfiguretriggeredcheckpointstosavethemodelif/whenconditionslikeadecreaseintrain/validationperformanceisdetected.Here’sanexample: http://machinelearningmastery.com/check-point-deep-learning-models-keras/ Reply Sukhpal February14,2017at3:50pm # Pleasehelpmetofindoutthiserror runfile(‘C:/Users/sukhpal/.spyder/temp.py’,wdir=’C:/Users/sukhpal/.spyder’)ERROR:executionaborted Reply JasonBrownlee February15,2017at11:32am # I’mnotsureSukhpal. Considergettingcodeworkingfromthecommandline,Idon’tuseIDEsmyself. Reply Kamal February14,2017at5:15pm # pleasehelpmetofindthiserrorfindthiserror Epoch194/195 195/195[==============================]–0s–loss:0.2692–acc:0.8667 Epoch195/195 195/195[==============================]–0s–loss:0.2586–acc:0.8667 195/195[==============================]–0s Traceback(mostrecentcalllast): Reply JasonBrownlee February15,2017at11:32am # WhatwastheerrorexactlyKamal? Reply Kamal February15,2017at3:24pm # sirwhenirunthecodeonmydataset thenitdoesnotshowoverallaccuracyalthoughitshowstheaccuracyandlossforthewholeiterations Reply JasonBrownlee February16,2017at11:06am # I’mnotsureIunderstandyourquestionKamal,pleaseyoucouldrestateit? Reply Val February15,2017at9:00pm # HiJason,imjuststartingdeeplearninginpythonusingkerasandtheano.Ihavefollowedtheinstallationinstructionswithoutahitch.Testedsomeexamplesbutwhenirunthisonelinebylineigetalotofexceptionsanderrorsonceirunthe“model.fit(X,Y,nb_epochs=150,batch_size=10” Reply JasonBrownlee February16,2017at11:06am # Whaterrorsareyougetting? Reply CrisH February17,2017at8:12pm # Hi,howdoIknowwhatnumbertouseforrandom.seed()?Imeanyouuse7,isthereanyreasonforthat?Alsoisitenoughtouseitonlyonce,inthebeginningofthecode? Reply JasonBrownlee February18,2017at8:38am # YoucanuseanynumberCrisH.Thefixedrandomseedmakestheexamplereproducible. Youcanlearnmoreaboutrandomnessandrandomseedsinthispost: http://machinelearningmastery.com/randomness-in-machine-learning/ Reply kk February18,2017at1:53am # amnewtodeeplearningandfoundthisgreattutorial.keepitupandlookforward!! Reply JasonBrownlee February18,2017at8:41am # Thanks! Reply IqraAmeer February21,2017at5:20am # HI,Ihaveaprobleminexecutiontheaboveexampleasit.Itseemsthatit’snotrunningproperlyandstopsatUsingTensorFlowbackend. Epoch147/150 768/768[==============================]–0s–loss:0.4709–acc:0.7878 Epoch148/150 768/768[==============================]–0s–loss:0.4690–acc:0.7812 Epoch149/150 768/768[==============================]–0s–loss:0.4711–acc:0.7721 Epoch150/150 768/768[==============================]–0s–loss:0.4731–acc:0.7747 32/768[>………………………..]–ETA:0sacc:76.43% Iamnewinthisfield,couldyoupleaseguidemeaboutthiserror. Ialsoexecutedonanotherdataset,itstopswiththesamebehavior. Reply JasonBrownlee February21,2017at9:39am # Whatistheerrorexactly?Theexamplehangs? MaybetrytheTheanobackendandseeifthatmakesadifference.Alsomakesureallofyourlibrariesareuptodate. Reply IqraAmeer February22,2017at5:47am # DearJason, Thankyousomuchforyourvaluablesuggestions.ItriedTheanobackendandalsoupdatedallmylibraries,butagainithangedat: 768/768[==============================]–0s–loss:0.4656–acc:0.7799 Epoch149/150 768/768[==============================]–0s–loss:0.4589–acc:0.7826 Epoch150/150 768/768[==============================]–0s–loss:0.4611–acc:0.7773 32/768[>………………………..]–ETA:0sacc:78.91% Reply JasonBrownlee February22,2017at10:05am # I’msorrytohearthat,Ihavenotseenthisissuebefore. PerhapsaRAMissueoraCPUoverheatingissue?Areyouabletotrydifferenthardware? Reply frd March8,2017at2:50am # Hi! Wereyouabletofindasolutionforthat? I’mhavingexactlythesameproblem (…) Epoch149/150 768/768[==============================]–0s–loss:0.4593–acc:0.7773 Epoch150/150 768/768[==============================]–0s–loss:0.4586–acc:0.7891 32/768[>………………………..]–ETA:0sacc:76.69% Reply Bhanu February23,2017at1:51pm # Hellosir, iwanttoaskwetherwecanconvertthiscodetodeeplearningwidincreasingnumberoflayers.. Reply JasonBrownlee February24,2017at10:12am # Sureyoucanincreasethenumberoflayers,tryitandsee. Reply AnanyaMohapatra February28,2017at6:40pm # hellosir, couldyoupleasetellmehowdoideterminetheno.ofneuronsineachlayer,becauseiamusingadifferentdatsetandamunabletoknowtheno.ofneuronsineachlayer Reply JasonBrownlee March1,2017at8:33am # HiAnanya,greatquestion. Sorry,thereisnogoodtheoryonhowtoconfigureaneuralnet. Youcanconfigurethenumberofneuronsinalayerbytrialanderror.Alsoconsidertuningthenumberofepochsandbatchsizeatthesametime. Reply AnanyaMohapatra March1,2017at4:42pm # thankyousomuchsir.Itworked!🙂 Reply JasonBrownlee March2,2017at8:11am # GladtohereitAnanya. Reply JayantSahewal February28,2017at8:11pm # HiJason, reallyhelpfulblog.Ihaveaquestionabouthowmuchtimedoesittaketoconverge? Ihaveadatasetwitharound4000records,3inputcolumnsand1outputcolumn.Icameupwiththefollowingmodel defcreate_model(dropout_rate=0.0,weight_constraint=0,learning_rate=0.001,activation=’linear’): #createmodel model=Sequential() model.add(Dense(6,input_dim=3,init=’uniform’,activation=activation,W_constraint=maxnorm(weight_constraint))) model.add(Dropout(dropout_rate)) model.add(Dense(1,init=’uniform’,activation=’sigmoid’)) #Optimizer optimizer=Adam(lr=learning_rate) #Compilemodel model.compile(loss=’binary_crossentropy’,optimizer=optimizer,metrics=[‘accuracy’]) returnmodel #createmodel model=KerasRegressor(build_fn=create_model,verbose=0) #definethegridsearchparameters batch_size=[10] epochs=[100] weight_constraint=[3] dropout_rate=[0.9] learning_rate=[0.01] activation=[‘linear’] param_grid=dict(batch_size=batch_size,nb_epoch=epochs,dropout_rate=dropout_rate,\ weight_constraint=weight_constraint,learning_rate=learning_rate,activation=activation) grid=GridSearchCV(estimator=model,param_grid=param_grid,n_jobs=-1,cv=5) grid_result=grid.fit(X_train,Y_train) Ihavea32coremachinewith64GBRAManditdoesnotconvergeeveninmorethananhour.Icanseeallthecoresbusy,soitisusingallthecoresfortraining.However,ifIchangetheinputneuronsto3thenitconvergesinaround2minutes. Kerasversion:1.1.1 Tensorflowversion:0.10.0rc0 theanoversion:0.8.2.dev-901275534cbfe3fbbe290ce85d1abf8bb9a5b203 It’susingTensorflowbackend.Canyouhelpmeunderstandwhatisgoingonorpointmeintherightdirection?Doyouthinkswitchingtotheanowillhelp? Best, Jayant Reply JasonBrownlee March1,2017at8:36am # Thispostmighthelpyoutuneyourdeeplearningmodel: http://machinelearningmastery.com/improve-deep-learning-performance/ Ihopethathelpsasastart. Reply AnimeshMohanty March1,2017at9:21pm # hellosir, couldyoupleasetellmehowcaniplottheresultsofthecodeonagraph.Imadeafewadjustmentstothecodesoastorunitonadifferentdataset. Reply JasonBrownlee March2,2017at8:16am # WhatdoyouwanttoplotexactlyAnimesh? Reply AnimeshMohanty March2,2017at4:56pm # Accuracyvsno.ofneuronsintheinputlayerandtheno.ofneuronsinthehiddenlayer Reply param March2,2017at12:15am # sircanuplzexplain thedifferentattributesusedinthisstatement print(“%s:%.2f%%”%(model.metrics_names[1],scores[1]*100)) Reply param March2,2017at12:16am # precisely,whatismodel.metrics_names Reply JasonBrownlee March2,2017at8:22am # model.metrics_namesisalistofnamesofthemetricscollectedduringtraining. Moredetailshere: https://keras.io/models/sequential/ Reply JasonBrownlee March2,2017at8:20am # Hiparam, Itisusingstringformatting.%sformatsastring,%.2fformatsafloatingpointvaluewith2decimalplaces,%%includesapercentsymbol. Youcanlearnmoreabouttheprintfunctionhere: https://docs.python.org/3/library/functions.html#print Moreinfoonstringformattinghere: https://pyformat.info/ Reply VijinKP March2,2017at4:01am # HiJason, Itwasanawesomepost.CouldyoupleasetellmehowtowedecidethefollowinginaDNN1.numberofneuronsinthehiddenlayers 2.numberofhiddenlayers Thanks. Vijin Reply JasonBrownlee March2,2017at8:22am # GreatquestionVijin. Generally,trialanderror.Therearenogoodtheoriesonhowtoconfigureaneuralnetwork. Reply VijinKP March3,2017at5:23am # Wedocrossvalidation,gridsearchetctofindthehyperparametersinmachinealgorithms.Similarlycanwedoanythingtoidentifytheaboveparameters?? Reply JasonBrownlee March3,2017at7:46am # Yes,wecanusegridsearchandtuningforneuralnets. Thestochasticnatureofneuralnetsmeansthateachexperiment(setofconfigs)willhavetoberunmanytimes(30?100?)sothatyoucantakethemeanperformance. Moregeneralinfoontuningneuralnetshere: http://machinelearningmastery.com/improve-deep-learning-performance/ Moreonrandomnessandstochasticalgorithmshere: http://machinelearningmastery.com/randomness-in-machine-learning/ Reply Bogdan March2,2017at11:48pm # Jason,Pleasetellmeabouttheselinesinyourcode: seed=7 numpy.random.seed(seed) Whatdotheydo?Andwhydotheydoit? OnemorequestioniswhydoyoucallthelastsectionBonus:Makeaprediction? IthoughtthiswhatANNwascreatedfor.Whatthepointifyournetwork’soutputisjustwhatyouhavealreadyknow? Reply JasonBrownlee March3,2017at7:44am # Theyseedtherandomnumbergeneratorsothatitproducesthesamesequenceofrandomnumberseachtimethecodeisrun.Thisistoensureyougetthesameresultasme. I’mnotconvinceditworkswithKerasthough. Moreonrandomnessinmachinelearninghere: http://machinelearningmastery.com/randomness-in-machine-learning/ Iwasshowinghowtobuildandevaluatethemodelinthistutorial.Thepartaboutstandalonepredictionwasanadd-on. Reply Sounaksahoo March3,2017at7:39pm # whatexactlyistheworkof“seed”intheneuralnetworkcode?whatdoesitdo? Reply JasonBrownlee March6,2017at10:44am # Seedreferstoseedingtherandomnumbergeneratorsothatthesamesequenceofrandomnumbersisgeneratedeachtimetheexampleisrun. Theaimistomaketheexamples100%reproducible,butthisishardwithsymbolicmathlibslikeTheanoandTensorFlowbackends. Formoreonrandomnessinmachinelearning,seethispost: http://machinelearningmastery.com/randomness-in-machine-learning/ Reply PriyaSundari March3,2017at10:19pm # hellosir couldyouplztellmewhatistheroleofoptimizerandbinary_crossentropyexactly?itiswrittenthatoptimizerisusedtosearchthroughtheweightsofthenetworkwhichweightsarewetalkingaboutexactly? Reply JasonBrownlee March6,2017at10:48am # HiPriya, Youcanlearnmoreaboutthefundamentalsofneuralnetshere: http://machinelearningmastery.com/neural-networks-crash-course/ Reply Bogdan March3,2017at10:23pm # IfIamnotmistaken,thoselinesIcommentedaboutusedwhenwewrite init=‘uniform’ ? Reply Bogdan March3,2017at10:44pm # Couldyouexplaininmoredetailswhatisthebatchsize? Reply JasonBrownlee March6,2017at10:50am # HiBogdan, Batchsizeishowmanypatternstoshowtothenetworkbeforetheweightsareupdatedwiththeaccumulatederrors.Thesmallerthebatch,thefasterthelearning,butalsothemorenoisythelearning(highervariance). Tryexploringdifferentbatchsizesandseetheeffectonthetrainandtestperformanceovereachepoch. Reply Mohammad March7,2017at6:50am # DearJason Firstly,thanksforyourgreattutorials. Iamtryingtoclassifycomputernetworkspacketsusingfirst500bytesofeverypackettoidentifyitsprotocol.Iamtryingtouse1dconvolution.forsimplertask,Ijustwanttodobinaryclassificationandthentacklemultilabelclassificationfor10protocols.Hereismycodebuttheaccuracywhichislike.63.howcanIimprovetheperformance?shouldIUseRNNs? ######## model=Sequential() model.add(Convolution1D(64,10,border_mode=’valid’, activation=’relu’,subsample_length=1,input_shape=(500,1))) #model.add(Convolution2D(32,5,5,border_mode=’valid’,input_shape=(1,28,28),)) model.add(MaxPooling1D(2)) model.add(Flatten()) model.add(Dense(200,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’)) model.compile(loss=’binary_crossentropy’, optimizer=’adam’,metrics=[‘accuracy’]) model.fit(train_set,y_train, batch_size=250, nb_epoch=30, show_accuracy=True) #x2=get_activations(model,0,xprim) #score=model.evaluate(t,y_test,show_accuracy=True,verbose=0) #print(score[0]) Reply JasonBrownlee March7,2017at9:37am # Thispostlistssomeideastotryanliftperformance: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply Damiano March7,2017at10:13pm # HiJason,thankyousomuchforthisawesometutorial.Ihavejuststartedwithpythonandmachinelearning. Iamjokingwiththecodedoingfewchanges,forexampleihavechanged.. this: #createmodel model=Sequential() model.add(Dense(250,input_dim=8,init=’uniform’,activation=’relu’)) model.add(Dense(200,init=’uniform’,activation=’relu’)) model.add(Dense(200,init=’uniform’,activation=’relu’)) model.add(Dense(1,init=’uniform’,activation=’sigmoid’)) andthis: model.fit(X,Y,nb_epoch=250,batch_size=10) theniwouldliketopasssomearraysforpredictionso… new_input=numpy.array([[3,88,58,11,54,24.8,267,22],[6,92,92,0,0,19.9,188,28], [10,101,76,48,180,32.9,171,63], [2,122,70,27,0,36.8,0.34,27],[5,121,72,23,112,26.2,245,30]]) predictions=model.predict(new_input) printpredictions#[1.0,1.0,1.0,0.0,1.0] isthiscorrect?Inthisexampleiusedthesameseriesoftraining(thathave0class),butiamgettingwrongresults.Onlyonearrayiscorrectlypredicted. Thankyousomuch! Reply JasonBrownlee March8,2017at9:41am # Looksgood.Perhapsyoucouldtrychangingtheconfigurationofyourmodeltomakeitmoreskillful? Seethispost: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply ANJI March13,2017at8:48pm # hellosir, couldyoupleasetellmetorectifymyerrorbelowitisraisedwhilemodelistraining: str(array.shape)) ValueError:Errorwhencheckingmodelinput:expectedconvolution2d_input_1tohave4dimensions,butgotarraywithshape(68,28,28). Reply JasonBrownlee March14,2017at8:17am # ItlookslikeyouareworkingwithCNN,notrelatedtothistutorial. ConsidertryingthistutorialtogetfamiliarwithCNNs: http://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-networks-python-keras/ Reply Rimjhim March14,2017at8:21pm # Iwantaneuralthatcanpredictsinvalues.Furtherfromagivendatasetineedtodeterminethefunction(forexampleifthedataisoftanorcos,thenhowtodeterminethatdataisoftanonlyorcosonly) Thanksinadvance Reply Sudarshan March15,2017at11:19pm # KerasjustupdatedtoKeras2.0.Ihaveanupdatedversionofthiscodehere:https://github.com/sudarshan85/keras-projects/tree/master/mlm/pima_indians Reply JasonBrownlee March16,2017at7:59am # Nicework. Reply subhasish March16,2017at5:09pm # hellosir, canweusePSO(particleswarmoptimisation)inthis?ifsocanyoutellhow? Reply JasonBrownlee March17,2017at8:25am # Sorry,Idon’thaveanexampleofPSOforfittingneuralnetworkweights. Reply AnanyaMohapatra March16,2017at10:03pm # hellosir, whattypeofneuralnetworkisusedinthiscode?asthereare3typesofNeuralnetworkthatare…feedforward,radialbasisfunctionandrecurrentneuraknetwork. Reply JasonBrownlee March17,2017at8:28am # Amultilayerperceptron(MLP)neuralnetwork.Aclassictypefromthe1980s. Reply Diego March17,2017at3:58am # gotthiserrorwhilecompiling.. sigmoid_cross_entropy_with_logits()gotanunexpectedkeywordargument‘labels’ Reply JasonBrownlee March17,2017at8:30am # Perhapsconfirmthatyourlibrariesarealluptodate(Keras,TheanoorTensorFlow)? Reply Rohan March20,2017at5:20am # HiJason! Iamtryingtousetwooddframesofavideotopredicttheevenone.ThusIneedtogivetwoimagesasinputtothenetworkandgetoneimageasoutput.Canyouhelpmewiththesyntaxforthefirstmodel.add()?IhaveX_trainofdimension(190,2,240,320,3)where190arethenumberofoddpairs,2arethetwooddimages,and(240,320,3)arethe(height,width,depth)ofeachimage. Reply HerliMenezes March21,2017at8:33am # Hello,Jason, Thanksforyourgoodtutorial.Howeverifoundsomeissues: Warningslikethese: 1–Warning(fromwarningsmodule): File“/usr/lib/python2.7/site-packages/keras/legacy/interfaces.py”,line86 ‘calltotheKeras2API:'+signature) UserWarning:UpdateyourDensecalltotheKeras2API:Dense(12,activation=”relu”,kernel_initializer=”uniform”,input_dim=8) 2-Warning(fromwarningsmodule): File"/usr/lib/python2.7/site-packages/keras/legacy/interfaces.py",line86 'calltotheKeras2API:‘+signature) UserWarning:UpdateyourDensecalltotheKeras2API:Dense(8,activation="relu",kernel_initializer="uniform") 3–Warning(fromwarningsmodule): File“/usr/lib/python2.7/site-packages/keras/legacy/interfaces.py”,line86 ‘calltotheKeras2API:'+signature) UserWarning:UpdateyourDensecalltotheKeras2API:Dense(1,activation=”sigmoid”,kernel_initializer=”uniform”) 3-Warning(fromwarningsmodule): File"/usr/lib/python2.7/site-packages/keras/models.py",line826 warnings.warn('Thenb_epochargumentinfit' UserWarning:Thenb_epochargumentinfithasbeenrenamedepochs`. Ithinktheseareduetosomepackageupdate.. But,theoutputofpredictionswasanarrayofzeros… suchas:[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,….0.0] IamrunninginaLinuxMachine,Fedora24, Python2.7.13(default,Jan122017,17:59:37) [GCC6.3.120161221(RedHat6.3.1-1)]onlinux2 Why? Thankyou! Reply JasonBrownlee March21,2017at8:45am # TheselooklikewarningsrelatedtotherecentKeras2.0release. Theylooklikejustwarningandthatyoucanstillruntheexample. Idonotknowwhyyouaregettingallzeros.Iwillinvestigate. Reply AnanyaMohapatra March21,2017at6:21pm # hellosir, canyoupleasehelpmebuildarecurrentneuralnetworkwiththeabovegivendataset.iamhavingabittroubleinbuildingthelayers… Reply JasonBrownlee March22,2017at7:56am # HiAnanya, ThePimaIndiandiabetesdatasetisabinaryclassificationproblem.ItisnotappropriateforaRecurrentNeuralNetworkasthereisnosequenceinformationtolearn. Reply AnanyaMohapatra March22,2017at8:04pm # sirsocouldyoutellonwhichtypeofdatasetwouldtherecurrentneuralnetworkaccuratelywork?ihavethedatasetofEEGsignalsofepilepticpatients…willrecurrentnetworkworkonthis? Reply JasonBrownlee March23,2017at8:49am # Itmayifitisregularenough. LSTMsareexcellentatsequenceproblemsthathaveregularityorclearsignalstodetect. Reply Shane March22,2017at5:18am # HiJason,IhaveaquickquestionrelatedtoanerrorIamreceivingwhenrunningthecodeinthetutorial… WhenIrun #Compilemodel model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) Pythonreturnsthefollowingerror: sigmoid_cross_entropy_with_logits()gotanunexpectedkeywordargument‘labels’ Reply JasonBrownlee March22,2017at8:09am # Sorry,IhavenotseenthiserrorShane. Perhapscheckthatyourenvironmentisuptodatewiththelatestversionsofthedeeplearninglibraries? Reply Tejes March24,2017at1:04am # HiJason, Thanksforthisawesomepost. Iranyourcodewithtensorflowbackend,justoutofcuriosity.TheaccuracyreturnedwasdifferenteverytimeIranthecode.Thatdidn’thappenwithTheano.Canyoutellmewhy? Thanksinadvance! Reply JasonBrownlee March24,2017at7:56am # Youwillgetdifferentaccuracyeachtimeyourunthecodebecauseneuralnetworksarestochastic. Thisisnotrelatedtothebackend(Iexpect). Moreonrandomnessinmachinelearninghere: http://machinelearningmastery.com/randomness-in-machine-learning/ Reply SaurabhBhagvatula March27,2017at9:49pm # HiJason, I’mnewtodeeplearningandlearningitfromyourtutorials,whichpreviouslyhelpedmeunderstandMachineLearningverywell. Inthefollowingcode,Iwanttoknowwhythenumberofneuronsdifferfrominput_diminfirstlayerofNueralNet. #createmodel model=Sequential() model.add(Dense(12,input_dim=8,init=’uniform’,activation=’relu’)) model.add(Dense(8,init=’uniform’,activation=’relu’)) model.add(Dense(1,init=’uniform’,activation=’sigmoid’)) Reply JasonBrownlee March28,2017at8:22am # Youcanspecifythenumberofinputsvia“input_dim”,youcanspecifythenumberofneuronsinthefirsthiddenlayerasthefirstparametertoDense(). Reply SaurabhBhagvatula March28,2017at4:15pm # Thanxalot. Reply JasonBrownlee March29,2017at9:05am # You’rewelcome. Reply Nalini March29,2017at2:52am # HiJason whilerunningthiscodeforkfoldcrossvalidationitisnotworking.pleasegivethecodeforkfoldcrossvalidationinbinaryclass Reply JasonBrownlee March29,2017at9:10am # Generallyneuralnetsaretooslow/largefork-foldcrossvalidation. Nevertheless,youcanuseasklearnwrapperforakerasmodelanduseitwithanysklearnresamplingmethod: http://machinelearningmastery.com/evaluate-performance-machine-learning-algorithms-python-using-resampling/ Reply trangtruong March29,2017at7:04pm # HiJason,whyiusefunctionevaluatetogetaccuracyscoremymodelwithtestdataset,itreturnresult>1,ican’tunderstand. Reply enixon April3,2017at3:08am # HeyJason,thanksforthisgreatarticle!Igetthefollowingerrorwhenrunningthecodeabove: TypeError:Receivedunknownkeywordarguments:{‘epochs’:150} Anyideasonwhythatmightbe?Ican’tget‘epochs’,nb_epochs,etctowork… Reply JasonBrownlee April4,2017at9:07am # YouneedtoupdatetoKerasversion2.0orhigher. Reply AnanyaMohapatra April5,2017at9:30pm # defbaseline_model(): #createmodel model=Sequential() model.add(Dense(10,input_dim=25,init=’normal’,activation=’softplus’)) model.add(Dense(3,init=’normal’,activation=’softmax’)) #Compilemodel model.compile(loss=’mean_squared_error’,optimizer=’adam’,metrics=[‘accuracy’]) returnmodel sirheremean_square_errorhasbeenusedforlosscalculation.IsitthesameasLMSalgorithm.Ifnot,canweuseLMS,NLMSorRLStocalculatetheloss? Reply AhmadHijazi April5,2017at10:19pm # HelloJason,thankyoualotforthisexample. Myquestionis,afterItrainedthemodelandanaccuracyof79.2%forexampleisobtainedsuccessfully,howcanItestthismodelonnewdata? forexampleifanewpatientwithnewrecordsappear,Iwanttoguesstheresult(0or1)forhim,howcanIdothatinthecode? Reply JasonBrownlee April9,2017at2:36pm # Youcanfityourmodelonallavailabletrainingdatathenmakepredictionsonnewdataasfollows: yhat=model.predict(X) 1 yhat=model.predict(X) Reply PerickFlaus April6,2017at12:16am # ThanksJason,howcanwetestifnewpatientwillbediabeticorno(0or1)? Reply JasonBrownlee April9,2017at2:36pm # Fitthemodelonalltrainingdataandcall: yhat=model.predict(X) 1 yhat=model.predict(X) Reply Gangadhar April12,2017at1:28am # DrJason, Incompilingthemodeligotbelowerror TypeError:compile()gotanunexpectedkeywordargument‘metrics’ unabletoresolvethebelowerror Reply JasonBrownlee April12,2017at7:53am # EnsureyouhavethelatestversionofKeras,v2.0orhigher. Reply OmogbehinAzeez April13,2017at1:48am # Hellosir, Thankyouforthepost.Aquickquestion,mydatasethas24inputand1binaryoutput(170instances,100epoch,hiddenlayer=6and10batch,kernel_initializer=’normal’).IadaptedyourcodeusingTensorflowandkeras.Iamhavinganaccuracyof98to100percent.Iamscaredofover-fittinginmymodel.Ineedyourcandidadvice.Kindregardssir Reply JasonBrownlee April13,2017at10:07am # Yes,evaluateyourmodelusingk-foldcross-validationtoensureyouarenottrickingyourself. Reply OmogbehinAzeez April14,2017at1:08am # Thankyousir Reply SethuBaktha April13,2017at5:19am # HiJason, IfIwanttousethediabetesdataset(NOTPima)https://archive.ics.uci.edu/ml/datasets/DiabetestopredictBloodGlucosewhichtutorialsande-booksofyourswouldIneedtostartwith….Also,thedatainitscurrentformatwithtime,codeandvalueisitusableasisordoIneedtoconvertthedatainanotherformattobeabletouseit. Thanksforyourhelp Reply JasonBrownlee April13,2017at10:13am # Thisprocesswillhelpyouframeandworkthroughyourdataset: http://machinelearningmastery.com/start-here/#process Ihopethathelpsasastart. Reply SethuBaktha April13,2017at10:25am # Dr.Jason, Thedataistimeseries(timebaseddata)withcategorical(20)withtwonumbersoneforinsulinlevelandanotherforbloodsugarlevel…Eachtimeseriesdatadoesnothaveeverycategoricaldata…Forexampleonecategoryisbloodsugarbeforebreakfast,anothercategoryisbloodsugarafterbreakfast,beforelunchandafterlunch…Sometimessomeofthesecategorydataismissing…Ireadthroughtheabovelink,butdoesnottalkabouttimeseries,categoricaldatawithsomecategoryofdatamissingwhattodointhosecases….Pleaseletmeknowifanyofyourbookswillhelpclarifythesepoints? Reply JasonBrownlee April14,2017at8:43am # HiSethu, Ihavemanypostsontimeseriesthatwillhelp.Getstartedhere: http://machinelearningmastery.com/start-here/#timeseries Withcategoricaldata,Iwouldrecommendanintegerencodingperhapsfollowedbyaone-hotencoding.Youcanlearnmoreabouttheseencodingshere: http://machinelearningmastery.com/data-preparation-gradient-boosting-xgboost-python/ Ihopethathelps. Reply OmogbehinAzeez April14,2017at9:49am # Hellosir, IsitcompulsorytonormalizethedatabeforeusingANNmodel.IreaditsomewhereIwhichtheauthorinsistedthateachattributebecomparableonthescaleof[0,1]forameaningfulmodel.Whatisyourtakeonthatsir.Kindregards. Reply JasonBrownlee April15,2017at9:29am # Yes.Youmustscaleyourdatatotheboundsoftheactivationused. Reply shiva April14,2017at10:38am # HiJason,Youaresimplyawesome.I’moneofthemanywhogotbenefitedfromyourbook“machinelearningmasterywithpython”.I’mworkingwithamedicalimageclassificationproblem.Ihavetwoclassesofmedicalimages(eachclasshaving1000imagesof32*32)tobeworkeduponbytheconvolutionalneuralnetworks.Couldyouguidemehowtoloadthisdatatothekerasdataset?Orhowtousemydatawhilefollowingyoursimplesteps?kindlyhelp. Reply JasonBrownlee April15,2017at9:30am # LoadthedataasnumpyarraysandthenyoucanuseitwithKeras. Reply OmogbehinAzeez April18,2017at12:09am # Hellosir, IadaptedyourcodewiththecrossvalidationpipelinedwithANN(Keras)formymodel.Itgaveme100%still.IgotthedatafromUCI(ChronicKidneyDisease).Itwas400instances,24inputattributesand1binaryattribute.WhenIremovedtherowswithmissingdataIwasleftwith170instances.Ismydatasettoosmallfor(24inputlayer,24hiddenlayerand1outputlayerANN,usingadamandkernelinitializerasuniform)? Reply JasonBrownlee April18,2017at8:32am # Itisnottoosmall. Generally,thesizeofthetrainingdatasetreallydependsonhowyouintendtousethemodel. Reply OmogbehinAzeez April18,2017at11:10pm # Thankyousirfortheresponse,IguessIhavetocontendwiththeover-fittingofmymodel. Reply PadmanabhanKrishnamurthy April19,2017at6:26pm # HiJason, Greattutorial.Lovethesite🙂 Justaquickquery:whyhaveyouusedadamasanoptimizeroversgd?Moreover,whendoweusesgdoptimization,andwhatexactlydoesitinvolve? Thanks Reply JasonBrownlee April20,2017at9:23am # ADAMseemstoconsistentlyworkwellwithlittleornocustomization. SGDrequiresconfigurationofatleastthelearningrateandmomentum. Tryafewmethodsandusetheonethatworksbestforyourproblem. Reply PadmanabhanKrishnamurthy April20,2017at4:32pm # Thanks🙂 Reply OmogbehinAzeez April25,2017at8:13am # Hellosir, Gooddaysir,howcanIgetalltheweightsandbiasesofthekerasANN.Kindregards. Reply JasonBrownlee April26,2017at6:19am # Youcansavethenetworkweights,seethispost: http://machinelearningmastery.com/save-load-keras-deep-learning-models/ YoucanalsousetheAPItoaccesstheweightsdirectly. Reply Shiva April27,2017at5:43am # HiJason, IamcurrentlyworkingwiththeIMDBsentimentanalysisproblemasmentionedinyourbook.AmusingAnaconda3withPython3.5.2.Inanattempttosummarizethereviewlengthasyouhavementionedinyourbook,Whenitrytoexecutethecommand: result=map(len,X) print(“Mean%.2fwords(%f)”%(numpy.mean(result),numpy.std(result))) itreturnstheerror:unsupportedoperandtype(s)for/:‘map’and‘int’ kindlyhelpwiththemodifiedsyntax.lookingforward… Reply JasonBrownlee April27,2017at8:47am # I’msorrytohearthat.Perhapscommentoutthatline? Orchangeittoremovetheformattingandjustprinttherawmeanandstdevvaluesforyoutoreview? Reply Elikplim May1,2017at1:58am # Hello,quitenewtoPython,NumpyandKeras(backgroundinPHP,MYSQLetc).Ifthereare8inputvariablesand1outputvarable(9total),andtheArrayindexingstartsfromzero(fromwhatI’vegatheredit’saNumpyArray,whichisbuiltonPythonlists)andtheorderis[rows,columns],thenshouldn’tourinputvariable(X)beX=dataset[:,0:7](whereweselectfromthe1stto8thcolumns,ie.0thto7thindices)andoutputvariable(Y)beY=dataset[:,8](wherewethe9thcolumn,ie.8thindex)? Reply JasonBrownlee May1,2017at5:59am # Youcanlearnmoreaboutarrayindexinginnumpyhere: https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html Reply JackieLee May1,2017at12:47pm # I’mhavingtroubleswiththepredictionspart.ItsavesValueError:Errorwhencheckingmodelinput:expecteddense_1_inputtohaveshape(None,502)butgotarraywithshape(170464,502) ###MAKEPREDICTIONS### testset=numpy.loadtxt(“right_stim_FD1.csv”,delimiter=”,”) A=testset[:,0:502] B=testset[:,502] probabilities=model.predict(A,batch_size=10,verbose=1) predictions=float(round(a)forainprobabilities) accuracy=numpy.mean(predictions==B) #roundpredictions #rounded=[round(x[0])forxinpredictions] print(predictions) print(“PredictionAccuracy:%.2f%%”%(accuracy*100)) Reply JasonBrownlee May2,2017at5:55am # Itlookslikeyoumightbegivingtheentiredatasetastheoutput(y)ratherthanjusttheoutputvariable. Reply AnastasiosSelalmazidis May2,2017at12:27am # Hithere, Ihaveaquestionregardingdeeplearning.InthistutorialwebuildaMLPwithKeras.IsthisDeepLearningorisitjustaMLPBackpropagation? Reply JasonBrownlee May2,2017at5:59am # DeeplearningisMLPbackpropthesedays: http://machinelearningmastery.com/what-is-deep-learning/ Generally,deeplearningreferstoMLPswithlotsoflayers. Reply EricT May2,2017at8:59pm # Hi, WouldyoumindifIusethiscodeasanexampleofasimplenetworkinaschoolprojectofmine? Needtoaskbeforeusingit,sinceIcannotfindanywhereinthistutorialthatyouareOKwithanyoneusingthecode,andtheethicsmomentofmycourserequiresmetoask(andofcoursegivecreditwherecreditisdue). Kindregards EricT Reply JasonBrownlee May3,2017at7:35am # Yesit’sfinebutItakenoresponsibilityandyoumustcreditthesource. IanswerthisquestioninmyFAQ: http://machinelearningmastery.com/start-here/#faq Reply BinhLN May7,2017at3:11am # HiJason Ihaveaproblem MyDatasethave500record.ButMyteacherwantmydatasethave100.000record.Imusthaveanewalgorithmfordatageneration.Pleasehelpme Reply Dp May11,2017at2:26am # Canyougiveadeepcnncodewhichincludes25layers,inthefirstconvlayerthefiltersizsshouldbe39×39wothatotallf64filters,inthe2ndconvlayer,21×21with32filters,inthe3rdconvlayer11×11with64filters,4thConvlayer7×7with32layers.Forainputsizeofimage256×256.ImCompetelynewinthisDeeplearningThingbutifyoucancodethatformeitwouldbeagreathelp.Thanks Reply JasonBrownlee May11,2017at8:33am # Considerusinganoff-the-shelfmodellikeVGG: https://keras.io/applications/ Reply Maple May13,2017at12:58pm # Ihavetofollowwiththefacebookmetrics.Buttheresultisverylow.Helpme. Ichangedtheinputbutdidnotimprove http://archive.ics.uci.edu/ml/datasets/Facebook+metrics Reply JasonBrownlee May14,2017at7:24am # Ihavealistofsuggestionsthatmayhelpasastart: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply Alessandro May14,2017at1:01am # HiJason, GreatTutorialandthanksforyoureffort. Ihaveaquestion,sinceIambeginnerwithkerasandtensorflow. Ihaveinstalledbothofthem,kerasandtensorflow,thelatestversionandIhaverunyourexamplebutIgetalwaysthesameerror: Traceback(mostrecentcalllast): File“CNN.py”,line18,in model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) File“/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/models.py”,line777,incompile **kwargs) File“/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/engine/training.py”,line910,incompile sample_weight,mask) File“/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/engine/training.py”,line436,inweighted score_array=fn(y_true,y_pred) File“/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/losses.py”,line51,inbinary_crossentropy returnK.mean(K.binary_crossentropy(y_pred,y_true),axis=-1) File“/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py”,line2771,inbinary_crossentropy logits=output) TypeError:sigmoid_cross_entropy_with_logits()gotanunexpectedkeywordargument‘labels’ Couldyouhelp?Thanks Alessandro Reply JasonBrownlee May14,2017at7:30am # Ouch,Ihavenotseenthiserrorbefore. Someideas: –Considertryingthetheanobackendandseeifthatmakesadifference. –Trysearching/postingonthekerasusergroupandslackchannel. –Trysearching/postingonstackoverfloworcrossvalidated. Letmeknowhowyougo. Reply Alessandro May14,2017at9:44am # HiJason, Ifoundtheissue.Thetensorflowinstallationwasoutdated;soIhaveupdateditandeverything isworkingnicely. Goodnight, Alessandro Reply JasonBrownlee May15,2017at5:50am # I’mgladtohearitAlessandro. Reply SheikhRafiulIslam May25,2017at3:36pm # ThankyouMr.Brownleeforyourwonderfuleasytounderstandexplanation Reply JasonBrownlee June2,2017at11:41am # Thnaks. Reply WAZED May29,2017at12:31am # HiJason, Thankyouverymuchforyourwonderfultutorial.Ihaveaquestionregardingthemetrices.Istheredefaultwaytodeclaremetrices“Precision”and“Recall”inaddtionwiththe“Accurace”. Br WAZED Reply JasonBrownlee June2,2017at12:15pm # Yes,seehere: https://keras.io/metrics/ Reply chiranjibkonwar May29,2017at4:30am # HiJason, pleasesendmeasmallnotecontainingresourcesfromwhereicanlearndeeplearningfromscratch.thanksforthewonderfulreadyouhadprepared. Thanksinadvance yes,myemailidis[email protected] Reply JasonBrownlee June2,2017at12:16pm # Here: http://machinelearningmastery.com/start-here/#deeplearning Reply Jeff June1,2017at11:48am # WhytheNNhavemistakesmanytimes? Reply JasonBrownlee June2,2017at12:54pm # Whatdoyoumeanexactly? Reply kevin June2,2017at5:53pm # HiJason, Iseemtobegettinganerrorwhenapplyingthefitmethod: ValueError:Errorwhencheckinginput:expecteddense_1_inputtohaveshape(None,12)butgotarraywithshape(767,8) Ilookedthisupandthemostprominentsuggestionseemedtobeupgradekerasandtheno,whichIdid,butthatdidn’tresolvetheproblem. Reply JasonBrownlee June3,2017at7:24am # Ensureyouhavecopiedthecodeexactlyfromthepost. Reply HemanthKumarK June3,2017at2:15pm # hiJason, Iamstuckwithanerror TypeError:sigmoid_cross_entropy_with_logits()gotanunexpectedkeywordargument‘labels’ mytensorflowandkerasvirsionsare keras:2.0.4 Tensorflow:0.12 Reply JasonBrownlee June4,2017at7:46am # I’msorrytohearthat,Ihavenotseenthaterrorbefore.Perhapsyoucouldpostaquestiontostackoverfloworthekerasusergroup? Reply xena June4,2017at6:36pm # cananyonetellmewhichneuralnetworkisbeingusedhere?IsitMLP?? Reply JasonBrownlee June5,2017at7:40am # Yes,itisamultilayerperceptron(MLP)feedforwardneuralnetwork. Reply NirmeshShah June9,2017at11:00pm # HiJason, IhaverunthiscodesuccessfullyonPCwithCPU. IfIhavetorunthesamecodenanotherPCwhichcontainsGPU,WhatlineshouldIaddtomakeitsurethatitrunsontheGPU Reply JasonBrownlee June10,2017at8:24am # Thecodewouldstaythesame,yourconfigurationoftheKerasbackendwouldchange. PleaserefertoTensorFloworTheanodocumentation. Reply Prachi June12,2017at7:30pm # WhatifIwanttotrainmyneuralwhichshoulddetectwhethertheluggageisabandonedornot?Howdoiproceedforit? Reply JasonBrownlee June13,2017at8:18am # Thisprocesswillhelpyouworkthroughyourpredictivemodelingproblemendtoend: http://machinelearningmastery.com/start-here/#process Reply Ebtesam June14,2017at11:15pm # Hi Iwasbuildneuralmachinetranslationmodelbutthescoreiwasgetis0iamnotsurewhy Reply JasonBrownlee June15,2017at8:45am # Hereisagoodlistofthingstotry: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply SarvottamPatel June20,2017at7:31pm # HHeyJason,firstofallthankyouverymuchfromthecoreofmyhearttomakemeunderstandthisperfectly,Ihaveanerroraftercompleting150iteration. File“keras_first_network.py”,line53,in print(“\n%s:%.2f”%(model.metrics_names[1]*100)) TypeError:notenoughargumentsforformatstring Reply SarvottamPatel June20,2017at8:05pm # SorrySirmybad,actuallyIwroteitwrongly Reply JasonBrownlee June21,2017at8:12am # Confirmthatyouhavecopiedthelineexactly: print("\n%s:%.2f%%"%(model.metrics_names[1],scores[1]*100)) 1 print("\n%s:%.2f%%"%(model.metrics_names[1],scores[1]*100)) Reply Joydeep June30,2017at4:15pm # HiDrJason, ThanksforthetutorialtogetstartedusingKeras. IusedthebelowsnippettodirectlyloadthedatasetfromtheURLratherthandownloadingandsavingasthismakesthecodemorestreamlinedwithouthavingtonavigateelsewhere. #loadpimaindiansdataset datasource=numpy.DataSource().open(“http://archive.ics.uci.edu/ml/machine-learning-databases/pima-indians-diabetes/pima-indians-diabetes.data”) dataset=numpy.loadtxt(datasource,delimiter=”,”) Reply JasonBrownlee July1,2017at6:28am # Thanksforthetip. Reply Yvette July7,2017at9:01pm # Thanksforthishelpfulresource! Reply JasonBrownlee July9,2017at10:38am # I’mgladithelped. Reply Andeep July10,2017at1:14am # HiDrBrownlee, thankyouverymuchforthisgreattutorial! Iwouldbegrateful,ifyoucouldanswersomequestions: 1.Whatdoesthe7in“numpy.random.seed(7)”means? 2.InmycaseIhave3inputneuronsand2outputneurons.Isthecorrectnotation: X=dataset[:,0:3] Y=dataset[:,3:4]? 3.Thebatchsizemeanshowmanytrainingdataareusedinoneepoch,amIright? Ihavethoughtwehavetousethewholetrainingdatasetforthetraining.InthiscaseIwoulddeterminethebatchsizeasthenumberoftrainingdatapairsIhaveachievedthroughexperimentsetc..Inyourexample,doesthebatch(sized10)meansthatthecomputeralwaysusesthesame10trainingdataineveryepochorarethe10trainingdatarandomlychosenamongalltrainingdatabeforeeveryepoch? 4.Whenevaluatingthemodelwhatdoesthelossmeans(e.g.inloss:0.5105–acc:0.7396)? Isitthesumofvaluesoftheerrorfunction(e.g.mean_squared_error)oftheoutputneurons? Reply JasonBrownlee July11,2017at10:19am # Youcanuseanyrandomseedyoulike,morehere: http://machinelearningmastery.com/reproducible-results-neural-networks-keras/ Youarereferringtothecolumnsinyourdata.Yournetworkwillalsoneedtobeconfiguredwiththecorrectnumberofinputsandoutputs(e.g.inputandoutputlayers). Batchsizeisthenumberofsamplesinthedatasettoworkthroughbeforeupdatingnetworkweights.Oneepochiscomprisedofoneormorebatches. Lossisthetermbeingoptimizedbythenetwork.Hereweuselogloss: https://en.wikipedia.org/wiki/Cross_entropy Reply Andeep July16,2017at7:43am # Thankyouforyourresponse,DrBrownlee!! Reply JasonBrownlee July16,2017at8:00am # Ihopeithelps. Reply PatrickZawadzki July11,2017at5:35am # Isthereanywaytoseetherelationshipbetweentheseinputs?Essentiallyunderstandwhichinputsaffecttheoutputthemost,orperhapswhichpairsofinputsaffecttheoutputthemost? Maybepairingthiswithunsuperviseddeeplearning?Iwanttohavelessofa“blackbox”forthedevelopednetworkifatallpossible.Thankyouforyourgreatcontent! Reply JasonBrownlee July11,2017at10:34am # Yes,tryandRFE: http://machinelearningmastery.com/feature-selection-machine-learning-python/ Reply Bernt July13,2017at10:12pm # HiJason, Thankyouforsharingyourskillsandcompetence. Iwanttostudythechangeinweightsandpredictionsbetweeneachepochrun. Havetriedtousethemodel.train_on_batchmethodandthemodel.fitmethodwithepoch=1andbatch_sizeequalallthesamples. Butitseemslikethemodeldoesn’tsavethenewupdatedweights. IprintpredictionsbeforeandafterIdontseeachangeintheevaluationscores. Partsofthecodeisprintedbelow. Anyidea? Thanks. #Compilemodel model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) #evaluatethemodel scores=model.evaluate(X,Y) print(“\n%s:%.2f%%”%(model.metrics_names[1],scores[1]*100)) #RunoneupdateofthemodeltrainedrunwithXandcomparedwithY model.train_on_batch(X,Y) #Fitthemodel model.fit(X,Y,epochs=1,batch_size=768) scores=model.evaluate(X,Y) print(“\n%s:%.2f%%”%(model.metrics_names[1],scores[1]*100)) Reply JasonBrownlee July14,2017at8:29am # Sorry,IhavenotexploredevaluatingaKerasmodelthisway. Perhapsitisafault,IwouldrecommendpreparingthesmallestpossibleexamplethatdemonstratestheissueandposttotheKerasGitHubissues. Reply iman July18,2017at11:18pm # Hi,Itriedtoapplythistothetitanicdataset,howeverthepredictionswereall0.4.Whatdoyousuggestfor: #createmodel model=Sequential() model.add(Dense(12,input_dim=4,activation=’relu’)) model.add(Dense(4,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’)) model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’])#’sgd’ model.fit(X,Y,epochs=15,batch_size=10) Reply JasonBrownlee July19,2017at8:26am # Thispostwillgiveyousomeideastolisttheskillofyourmodel: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply Camus July19,2017at2:14am # HiDrJason, ThisisprobablyastupidquestionbutIcannotfindouthowtodoit…andIambeginneronNeuralNetwork. Ihaverelativelysamenumberofinputs(7)andoneoutput.Thisoutputcantakenumbersbetween-3000and+3000. IwanttobuildaneuralnetworkmodelinpythonbutIdon’tknowhowtodoit. Doyouhaveanexamplewithoutputsdifferentfrom0-1. Tanksinadvance Camus Reply JasonBrownlee July19,2017at8:28am # Ensureyouscaleyourdatathenusetheabovetutorialtogetstarted. Reply KhalidHussain July21,2017at11:28pm # HiJasonBrownlee Iamusingthesamedata“pima-indians-diabetes.csv”butallpredictedvaluesarelessthen1andareinfractionwhichcouldnotdistinguishanyclass. IfIroundoffthenallbecome0. Iamusingmodel.predict(x)function YouarerequestedtokindlyguidemewhatIamdoingwrongarehowcanIachievecorrectpredictedvalue. Thankyou Reply JasonBrownlee July22,2017at8:36am # Consideryouhavecopiedallofthecodeexactlyfromthetutorial. Reply Ludo July25,2017at6:59pm # HelloJason, Thanksyouforyourgreatexample.Ihavesomecomments. –Whyyouhavechoice“12”inputshiddenlayers?andnot24/32..it’sarbitary? –Samequestionaboutepochsandbatch_size? Thisvalueareverysensible!!ihavetrywith32inputsfirstlayer,epchos=500andbatch_size=1000andtheresultisverydifferents…i’amat65%accurancy. Thxforyouhelp. Regards. Reply JasonBrownlee July26,2017at7:50am # Yes,itisarbitrary.Tunetheparametersofthemodeltoyourproblem. Reply AlmoutasemBellahRajab July25,2017at7:32pm # Wow,you’restillreplyingtocommentsmorethanayearlater!!!…you’regreat,,thanks.. Reply JasonBrownlee July26,2017at7:50am # Yep. Reply Jane July26,2017at1:23am # Thanksforyourtutorial,IfounditveryusefultogetmestartedwithKeras.I’vepreviouslytriedTensorFlow,butfounditverydifficulttoworkwith.Idohaveaquestionforyouthough.IhavebothTheanoandTensorFlowinstalled,howdoIknowwhichback-endKerasisusing?Thanksagain Reply JasonBrownlee July26,2017at8:02am # Keraswillprintwhichbackendituseseverytimeyourunyourcode. YoucanchangethebackendintheKerasconfigurationfile(~/.keras/keras.json)whichlookslike: { "image_data_format":"channels_last", "backend":"tensorflow", "epsilon":1e-07, "floatx":"float32" } 123456 { "image_data_format":"channels_last", "backend":"tensorflow", "epsilon":1e-07, "floatx":"float32"} Reply MasoodImran July28,2017at12:00am # HelloJason, MyunderstandingofMachineLearningorevaluatingdeeplearningmodelsisalmost0.But,thisarticlegivesmelotofinformation.Itisexplainedinasimpleandeasytounderstandlanguage. Thankyouverymuchforthisarticle.WouldyousuggestanygoodreadtofurtherexploreMachineLearningordeeplearningmodelsplease? Reply JasonBrownlee July28,2017at8:31am # Thanks. Yes,startrighthere: http://machinelearningmastery.com/start-here/#deeplearning Reply Peggy August3,2017at7:14pm # IfIhavetrainedpredictionmodelsorneuralnetworkfunctionscripts.HowcanIusethemtomakepredictionsinanapplicationthatwillbeusedbyendusers?IwanttousepythonbutitseemsIwillhavetoredothetraininginPythonagain.IsthereawayIcanrewritethescriptsinPythonwithoutretrainingandjustcallthefunctionofpredicting? Reply JasonBrownlee August4,2017at6:58am # Youneedtotrainandsavethefinalmodelthenloadittomakepredictions. Thispostwillmakeitclear: http://machinelearningmastery.com/train-final-machine-learning-model/ Reply Shane August8,2017at2:38pm # Jason,Iusedyourtutorialtoinstalleverythingneededtorunthistutorial.Ifollowedyourtutorialandrantheresultingprogramsuccessfully.Canyoupleasedescribewhattheoutputmeans?Iwouldliketothankyouforyourveryinformativetutorials. Reply Shane August8,2017at2:39pm # 768/768[==============================]–0s–loss:0.4807–acc:0.7826 Epoch148/150 768/768[==============================]–0s–loss:0.4686–acc:0.7812 Epoch149/150 768/768[==============================]–0s–loss:0.4718–acc:0.7617 Epoch150/150 768/768[==============================]–0s–loss:0.4772–acc:0.7812 32/768[>………………………..]–ETA:0s acc:77.99% Reply JasonBrownlee August8,2017at5:12pm # Itissummarizingthetrainingofthemodel. Thefinallineevaluatestheaccuracyofthemodel’spredictions–reallyjusttodemonstratehowtomakepredictions. Reply JasonBrownlee August8,2017at5:11pm # WelldoneShane. Whichoutput? Reply Bene August9,2017at1:02am # HelloJason,ireallylikedyourWorkandithelpedmealotwithmyfirststeps. Butiamnotreallyfamiliarwiththenumpystuff: SohereismyQuestion: dataset=numpy.loadtxt(“pima-indians-diabetes.csv”,delimiter=”,”) #splitintoinput(X)andoutput(Y)variables X=dataset[:,0:8] Y=dataset[:,8] Igetthatthenumpy.loadtxtisextractingtheinformationfromthecvsFile butwhatdoesthestuffintheBracketsmeanlikeX=dataset[:,0:8] whythe“:”andwhy,0:8 itsprobablyprettydumbbutican’tfindagoodexplanationonline😀 thanksreallymuch! Reply JasonBrownlee August9,2017at6:37am # GoodquestionBene,it’scalledarrayslicing: https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html Reply Bene August9,2017at10:59pm # ThathelpedmeouttankyouJason🙂 Reply Chen August12,2017at5:43pm # CanItranslateittoChineseandputittoInternetinordertoletotherChinesepeoplecanreadyourarticle? Reply JasonBrownlee August13,2017at9:46am # No,pleasedonot. Reply DeepLearning August12,2017at7:36pm # Itseemsthatusingthisline: np.random.seed(5) …isredundanti.e.theKerasoutputinalooprunningthesamemodelwiththesameconfigurationwillyieldasimilarvarietyofresultsregardlessifit’ssetatall,orwhichnumberitissetto.OramImissingsomething? Reply JasonBrownlee August13,2017at9:52am # Deeplearningalgorithmsarestochastic(randomwithinarange).Thatmeansthattheywillmakedifferentpredictions/learndifferentthingswhenthesamemodelistrainedonthesamedata.Thisisafeature: http://machinelearningmastery.com/randomness-in-machine-learning/ Youcanfixtherandomseedtoensureyougetthesameresult,anditisagoodideafortutorialstohelpbeginnersout: http://machinelearningmastery.com/reproducible-results-neural-networks-keras/ Whenevaluatingtheskillofamodel,Iwouldrecommendrepeatingtheexperimentntimesandtakingskillastheaverageoftheruns.Seeherefortheprocedure: http://machinelearningmastery.com/evaluate-skill-deep-learning-models/ Doesthathelp? Reply DeepLearning August14,2017at3:08am # ThanksJason🙂 Itotallygetwhatitshoulddo,butasIhadpointedout,itdoesnotdoit.Ifyourunthecodesyouhaveprovidedaboveinaloopforsay10times.First10withrandomseedsetandtheother10timeswithoutthatlineofcodealltogether.Thencomparetheresult.AtleasttheresultI’mgetting,issuggestingtheeffectisnottherei.e.bothsetsof10timeswillhavesimilarvariationintheresult. Reply JasonBrownlee August14,2017at6:26am # Itmaysuggestthatthemodelisoverprescribedandeasilyaddressesthetrainingdata. Reply DeepLearning August14,2017at3:12am # Nicepostbytheway>http://machinelearningmastery.com/evaluate-skill-deep-learning-models/ Thanksforsharingit.Beenlatelythinkingabouttheaspectofaccuracyalot,itseemsthatatthemomentit’sa“hotmess”intermsofthewaycommontoolsdoitoutofthebox.IthinkalotofnonPhD/nonexpertcrowd(mostpeople)willatleastinitiallybeeasilyconfusedandmakethekindsofmistakesyoupointoutinyourpost. Thanksforalltheamazingcontributionsyouaremakinginthisfield! Reply JasonBrownlee August14,2017at6:26am # I’mgladithelped. Reply Haneesh December7,2019at10:36pm # HiJason, i’mactuallytryingtofind“spamfilterforquoraquestions”whereihaveadatasetwithlabel-0’sand1’sandquestionscolumns.pleaseletmeknowtheapproachandpathtobuildamodelforthis. Thanks Reply JasonBrownlee December8,2019at6:10am # Soundslikeagreatproject. Thetutorialshereontextclassificationwillhelp: https://machinelearningmastery.com/start-here/#nlp Reply RATNANITINPATIL August14,2017at8:16pm # HelloJason,Thanksforawonderfultutorial. CanIuseGeneticAlgorithmforfeatureselection?? Ifyes,Couldyoupleaseprovidethelinkforit??? Thanksinadvance. Reply JasonBrownlee August15,2017at6:34am # Sure.Sorry,Idon’thaveanyexamples. Generally,computersaresofastitmightbeeasiertotestallcombinationsinanexhaustivesearch. Reply sunny1304 August15,2017at3:44pm # HiJson, Thankyouforyourawesometutorial. Ihaveaquestionforyou. Isthereanyguidelineonhowtodecideonneuronnumberforournetwork. forexampleyouused12forthr1stlayerand8forthesecondlayer. howdoyoudecideonthat? Thanks Reply JasonBrownlee August15,2017at4:58pm # No,thereisnowaytoanalyticallydeterminetheconfigurationofthenetwork. Iusetrialanderror.Youcangridsearch,randomsearch,orcopyconfigurationsfromtutorialsorpapers. Reply yihadad August16,2017at6:53pm # HiJson, Thanksforawonderfultutorial. RunamodelgeneratedbyaCNNittakeshowmuchram,cpu? Thanks Reply JasonBrownlee August17,2017at6:39am # Itdependsonthedatayouareusingtofitthemodelandthesizeofthemodel. Verylargemodelscouldbe500MBofRAMormore. Reply Ankur September1,2017at3:15am # Hi, Pleaseletmeknow,howcanivisualisethecompleteneuralnetworkinKeras………………. Iamlookingforthecompletearchitecture–likenumberofneuronsintheInputLayer,hiddenlayer,outputlayerwithweights. Pleasehavealookatthelinkpresentbelow,heresomeonehascreatedabeutifulvisualisation/architectureusingneuralnetpackageinR. Pleaseletmeknow,canwecreatesuchtypeofmodelinKERAS https://www.r-bloggers.com/fitting-a-neural-network-in-r-neuralnet-package/ Reply JasonBrownlee September1,2017at6:50am # UsetheKerasvisualizationAPI: https://keras.io/visualization/ Reply ASAD October17,2017at3:23am # HelloANKUR,,,,howareyou? youhavetryvisualizationinkeraswhichissuggestedbyJasonBrownlee? ifyouhavetriedthenpleasesendmecodeiamalsotryingbutdidnotwork.. pleaseguideme Reply Adam September3,2017at1:45am # ThankyouDr.Brownleeforthegreattutorial, Ihaveaquestionaboutyourcode: istheargumentmetrics=[‘accuracy’]necessaryinthecodeanddoesitchangetheresultsoftheneuralnetworkorisitjustforshowingmetheaccuracyduringcompiling? thankyou!! Reply JasonBrownlee September3,2017at5:48am # No,itjustprintsouttheaccuracyofthemodelattheendofeachepoch.LearnmoreaboutKerasmetricshere: https://machinelearningmastery.com/custom-metrics-deep-learning-keras-python/ Reply PottOfGold September5,2017at12:14am # HiJason, yourworkhereisreallygreat.Ithelpedmealot. IrecentlystumbledupononethingIcannotunderstand: Forthepimasdatasetyoustate: <> WhenIlookatthetableofthepimasdataset,theexamplesareinrowsandthefeaturesincolumns,soyourinputdimensionisthenumberofcolumns.AsfarasIcansee,youdon’tchangethetable. Forneuralnetworks,isn’ttheinputnormally:examples=columns,features=rows? IsthisdifferentforKeras?OrcanIusebothshapes?Anifyes,what’sthedifferenceintheconstructionofthenet? Thankyou!! Reply JasonBrownlee September7,2017at12:36pm # No,featuresarecolumns,rowsareinstancesorexamples. Reply PottOfGold September7,2017at3:35pm # Thanks!🙂 Ihadalotofdiscussionsbecauseofthat. InAndrewNgnewCourseracourseit’sexplainedasexamples=columns,features=rows,buthedoesn’tuseKerasofcourse,butprogrammstheneuralnetworksfromscratch. Reply JasonBrownlee September9,2017at11:38am # Idoubtthat,Ithinkyoumayhavemixeditup.Columnsareneverexamples. Reply PottOfGold October6,2017at6:26pm # ThatswhatIthought,butIlookeditupinthenotationforthenewcourseracourse(deeplearning.ai)andthereitsays:misthenumerofexamplesinthedatasetandnistheinputsize,whereXsuperscriptnxmistheinputmatrix… Buteitherway,youhelpedme!Thankyou.🙂 LinLi September16,2017at1:50am # HiJason,thankyousomuchforyourtutorial,ithelpsmealot.Ineedyourhelpforthequestionbelow: Icopythecodeandrunit.AlthoughIgottheclassificationresults,thereweresomewarningmessagesintheprocess.Asfollows: Warning(fromwarningsmodule): File“C:\Users\llfor\AppData\Local\Programs\Python\Python35\lib\site-packages\keras\callbacks.py”,line120 %delta_t_median) UserWarning:Methodon_batch_end()isslowcomparedtothebatchupdate(0.386946).Checkyourcallbacks. Idon’tknowwhy,andcannotfindanyanswertothisquestion.I’mlookingforwardtoyourreply.Thanksagain! Reply JasonBrownlee September16,2017at8:43am # Sorry,Ihavenotseenthismessagebefore.Itlookslikeawarning,youmightbeabletoignoreit. Reply LinLi September16,2017at12:24pm # Thanksforyourreply.I’mastart-learnerondeeplearning.I’dliketoputitasidetemporarily. Reply Sagar September22,2017at2:51pm # HiJason, Greatarticle,thumbsupforthat.IamgettingthiserrorwhenItrytorunthefileonthecommandprompt.Anysuggestions.Thanksforyouresponse. ####################################################################### C:\Work\ML>pythonkeras_first_network.py UsingTensorFlowbackend. 2017-09-2210:11:11.189829:WC:\tf_jenkins\home\workspace\rel-win\M\windows\PY\ 36\tensorflow\core\platform\cpu_feature_guard.cc:45]TheTensorFlowlibrarywasn ‘tcompiledtouseAVXinstructions,buttheseareavailableonyourmachineand couldspeedupCPUcomputations. 2017-09-2210:11:11.190829:WC:\tf_jenkins\home\workspace\rel-win\M\windows\PY\ 36\tensorflow\core\platform\cpu_feature_guard.cc:45]TheTensorFlowlibrarywasn ‘tcompiledtouseAVX2instructions,buttheseareavailableonyourmachinean dcouldspeedupCPUcomputations. 32/768[>………………………..]–ETA:0s acc:78.52% ####################################################################### Reply JasonBrownlee September23,2017at5:35am # Lookslikewarningmessagesthatyoucanignore. Reply Sagar September24,2017at3:52am # ThanksIgottoknowwhattheproblemwas.Accordingtosection6Ihadsetverboseargumentto0whilecalling“model.fit()”.Nowalltheepochsaregettingprinted. Reply JasonBrownlee September24,2017at5:17am # Gladtohearit. Reply Valentin September26,2017at6:35pm # HiJason, Thanksfortheamazingarticle.Clearandstraightforward. IhadsomeproblemsinstallingKerasbutwasadvisedtoprefix withtf.contrib.keras soIhavecodelike model=tf.contrib.keras.models.Sequential() Dense=tf.contrib.keras.layers.Dense NowItrytotrainKerasonsomesmalldatafiletoseehowthingsworkout: 1,1,0,0,8 1,2,1,0,4 1,0,0,1,5 1,0,1,0,7 0,1,0,0,8 1,4,1,0,4 1,0,2,1,1 1,0,1,0,7 Thefirst4columnsareinputsandthe5-thcolumnisoutput. Iusethesamecodefortraining(adjustnumberofinputs)asinyourarticle, butthenetworkonlygetsto12.5%accuracy. Anyadvise? Thanks, Valentin Reply JasonBrownlee September27,2017at5:40am # ThanksValentin. Ihaveagoodlistofsuggestionsforimprovingmodelperformancehere: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply Priya October3,2017at2:28pm # HiJason, Itriedreplacingthepimadatawithrandomdataasfollows: X_train=np.random.rand(18,61250) X_test=np.random.rand(18,61250) Y_train=np.array([0.0,1.0,1.0,0.0,1.0,1.0,1.0,0.0,1.0, 0.0,1.0,0.0,0.0,1.0,0.0,0.0,0.0,1.0,]) Y_test=np.array([1.0,0.0,0.0,1.0,1.0,0.0,1.0,1.0,1.0, 1.0,0.0,0.0,0.0,1.0,0.0,0.0,1.0,0.0,]) _,input_size=X_train.shape#putthisininput_diminthefirstdenselayer Itooktheround()offofthepredictionssoIcouldseethefullvalueandtheninsertedmyrandomtestdatainmodel.fit(): predictions=model.predict(X_test) preds=[x[0]forxinpredictions] print(preds) model.fit(X_train,Y_train,epochs=100,batch_size=10,verbose=2,validation_data=(X_test,Y_test)) Ifoundsomethingslightlyodd;Iexpectedthepredictedvaluestobearound0.50,plusorminussome,butinstead,Igotthis: [0.49525392,0.49652839,0.49729034,0.49670222,0.49342978,0.49490061,0.49570397,0.4962129,0.49774086,0.49475089,0.4958384,0.49506786,0.49696651,0.49869373,0.49537542,0.49613148,0.49636957,0.49723724] whichisnear0.50butalwayslessthan0.50.Iranthisafewtimeswithdifferentrandomseeds,soit’snotcoincidental.Wouldyouhaveanyexplanationforwhyitdoesthis? Thanks, Priya Reply JasonBrownlee October3,2017at3:46pm # Perhapscalculatethemeanofyourtrainingdataandcompareittothepredictedvalue.Itmightbesimplesamplingerror. Reply Priya October4,2017at1:02am # IfoundoutIwasdoingpredictionsbeforefittingthemodel.(Isupposethatwouldmeanthenetworkhadn’tadjustedtothedata’sdistributionyet.) Reply Saurabh October7,2017at5:59am # HelloJason, Itriedtotrainthismodelonmylaptop,itisworkingfine.ButItriedtotrainthismodelongoogle-cloudwiththesameinstructionsasinyourexample-5.Butitisfailing. Canyoujustletmeknow,whichchangesaretorequiredforthemodel,sothatIcantrainthisoncloud. Reply JasonBrownlee October7,2017at7:37am # Sorry,Idon’tknowaboutgooglecloud. IhaveinstructionshereforrunningonAWS: https://machinelearningmastery.com/develop-evaluate-large-deep-learning-models-keras-amazon-web-services/ Reply tobegit3hub October12,2017at6:40pm # Greatpost.Thanksforsharing. Reply JasonBrownlee October13,2017at5:45am # You’rewelcome. Reply Manoj October12,2017at11:43pm # HiJason, Isthereawaytostorethemodel,onceitiscreatedsothatIcanuseitfordifferentinputdatasetsasandwhenneeded. Reply JasonBrownlee October13,2017at5:48am # Yes,youcansaveittofile.Seethistutorial: https://machinelearningmastery.com/save-load-machine-learning-models-python-scikit-learn/ Reply Cam October23,2017at6:11pm # Igetasyntaxerrorforthe model.fit()lineinthisexample.Isitduetolibraryconflictswiththeanoandtensorflowifihavebothinstalled? Reply JasonBrownlee October24,2017at5:28am # Perhapsensureyourenvironmentisuptodateandthatyoucopiedthecodeexactly. Thistutorialcanhelpwithsettingupyourenvironment: http://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/ Reply Cam October24,2017at2:11pm # Thanks,fixed! Reply JasonBrownlee October24,2017at4:01pm # Gladtohearit. Reply DiegoQuintana October25,2017at7:37am # HiJason,thanksfortheexample. HowwouldyoupredictasingleelementfromX?X[0]raisesaValueError ValueError:Errorwhenchecking:expecteddense_1_inputtohaveshape(None,8)butgotarraywithshape(8,1) Thanks! Reply JasonBrownlee October25,2017at3:56pm # Youcanreshapeittohave1rowand8columns: X=X.reshape((1,8)) 1 X=X.reshape((1,8)) Thispostwillgiveyoufurtheradvice: https://machinelearningmastery.com/index-slice-reshape-numpy-arrays-machine-learning-python/ Reply harald April10,2019at8:26pm # Shoulditbe:X[0].reshape((1,8))? Reply JasonBrownlee April11,2019at6:35am # Yep! Reply ShahbazWasti October28,2017at1:30pm # DearSir, Ihaveinstalledandconfiguredtheenvironmentaccordingtoyourdirectionsbutwhilerunningtheprogramihavefollowingerror “fromkeras.utilsimportnp_utils” Reply JasonBrownlee October29,2017at5:50am # Whatistheerrorexactly? Reply Zhengping October30,2017at12:12am # HiJason,thanksforthegreattutorials.Ijustlearntandrepeatedtheprograminyour“YourFirstMachineLearningProjectinPythonStep-By-Step”withoutproblem.Nowtryingthisone,gettingstuckattheline“model=Sequential()”whentheInteractivewindowthrows:NameError:name‘Sequential’isnotdefined.triedtogoogle,can’tfindasolution.IdidimportSequentialfromkeras.modelsasinurexamplecode.copypastedasitis.Thanksinadvanceforyourhelp. Reply Zhengping October30,2017at12:14am # I’mrunningurexamplesinAnaconda4.4.0environmentinvisualstudiocommunityversion.relevantpackageshavebeeninstalledasinurearliertutorialsinstructed. Reply Zhengping October30,2017at12:18am # >>#createmodel …model=Sequential() … Traceback(mostrecentcalllast): File“”,line2,in NameError:name‘Sequential’isnotdefined >>>model.add(Dense(12,input_dim=8,init=’uniform’,activation=’relu’)) Traceback(mostrecentcalllast): File“”,line1,in AttributeError:‘SVC’objecthasnoattribute‘add’ Reply JasonBrownlee October30,2017at5:39am # Thisdoesnotlookgood.Perhapsposttheerrortostackexchangeorotherkerassupport.Ihavealistofkerassupportsiteshere: https://machinelearningmastery.com/get-help-with-keras/ Reply JasonBrownlee October30,2017at5:38am # LookslikeyouneedtoinstallKeras.Ihaveatutorialhereonhowtodothat: https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/ Reply Akhil October30,2017at5:04pm # HoJason, Thanksalotforthiswonderfultutorial. Ihaveaquestion: Iwanttouseyourcodetopredicttheclassification(1or0)ofunknownsamples.ShouldIcreateonecommoncsvfilehavingthetrain(known)aswellasthetest(unknown)data.Whereasthe‘classification’columnfortheknowndatawillhaveaknownvalue,1or0,fortheunknowndata,shouldIleavethecolumnempty(andletthecodedecidetheoutcome)? Thanksalot Reply JasonBrownlee October31,2017at5:29am # Greatquestion. No,youonlyneedtheinputsandthemodelcanpredicttheoutputs,callmodel.predict(X). Also,thispostwillgiveageneralideaonhowtofitafinalmodel: https://machinelearningmastery.com/train-final-machine-learning-model/ Reply Guilherme November3,2017at1:26am # HiJason, Thisisreallycool!Iamblownaway!Thankssomuchformakingitsosimpleforabeginnertohavesomehandson.Ihaveacouplequestions: 1)wherearetheweights,canIsaveand/orretrievethem? 2)ifIwanttotrainimageswithdogsandcatsandlaterasktheneuralnetworkwhetheranewimagehasacatoradog,howdoIgetmyinputimagetopassasanarrayandmyoutputresulttobe“cat”or“dog”? Thanksagainandgreatjob! Reply JasonBrownlee November3,2017at5:20am # Theweightsareinthemodel,youcansavethem: https://machinelearningmastery.com/save-load-keras-deep-learning-models/ Yes,youwouldsaveyourmodel,thencallmodel.predict()onthenewdata. Reply Michael November5,2017at8:33am # HiJason, Areyoufamiliarwithapythontool/packagethatcanbuildneuralnetworkasinthetutorial,butsuitablefordatastreammining? Thanks, Michael Reply JasonBrownlee November6,2017at4:46am # Notreally,sorry. Reply bea November8,2017at1:58am # Hi,there.Couldyoupleaseclarifywhyexactlyyou’vebuiltyournetworkwith12neuronsinthefirstlayer? “Thefirstlayerhas12neuronsandexpects8inputvariables.Thesecondhiddenlayerhas8neuronsandfinally,theoutputlayerhas1neurontopredicttheclass(onsetofdiabetesornot)…” Should’ntithave8neuronsatthestart? Thanks Reply JasonBrownlee November8,2017at9:28am # Theinputlayerhas8,thefirsthiddenlayerhas12.Ichose12throughalittletrialanderror. Reply Guilherme November9,2017at12:54am # HiJason, Doyouhaveorelsecouldyourecommendabeginner’slevelimagesegmentationapproachthatusesdeeplearning?Forexample,Iwanttotrainsomeneuralnettoautomatically“find”aparticularfeatureoutofanimage. Thanks! Reply JasonBrownlee November9,2017at10:00am # Sorry,Idon’thaveimagesegmentationexamples,perhapsinthefuture. Reply Andy November12,2017at6:56pm # HiJason, IjuststartedmyDLtrainingafewweeksago.AccordingtowhatIlearnedincourse,inordertotraintheparametersfortheNN,weneedtoruntheForwardandBackwardpropagation;however,lookingatyourKerasexample,idon’tfindanyofthesepropagationprocesses.DoesitmeanthatKerashasitsownmechanismtofindtheparametersinsteadofusingForwardandBackwardpropagation? Thanks! Reply JasonBrownlee November13,2017at10:13am # Itisperformingthoseoperationsunderthecoversforyou. Reply Badr November13,2017at11:42am # HiJason, CanyouexplainwhyIgotthefollowingoutput: ValueErrorTraceback(mostrecentcalllast) in() —->1model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) 2model.fit(X,Y,epochs=150,batch_size=10) 3scores=model.evaluate(X,Y) 4print(“\n%s:%.2f%%”%(model.metrics_names[1],scores[1]*100)) /Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/models.pyincompile(self,optimizer,loss,metrics,sample_weight_mode,**kwargs) 545metrics=metrics, 546sample_weight_mode=sample_weight_mode, –>547**kwargs) 548self.optimizer=self.model.optimizer 549self.loss=self.model.loss /Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/engine/training.pyincompile(self,optimizer,loss,metrics,loss_weights,sample_weight_mode,**kwargs) 620loss_weight=loss_weights_list[i] 621output_loss=weighted_loss(y_true,y_pred, –>622sample_weight,mask) 623iflen(self.outputs)>1: 624self.metrics_tensors.append(output_loss) /Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/engine/training.pyinweighted(y_true,y_pred,weights,mask) 322defweighted(y_true,y_pred,weights,mask=None): 323#score_arrayhasndim>=2 –>324score_array=fn(y_true,y_pred) 325ifmaskisnotNone: 326#CastthemasktofloatXtoavoidfloat64upcastingintheano /Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/objectives.pyinbinary_crossentropy(y_true,y_pred) 46 47defbinary_crossentropy(y_true,y_pred): —>48returnK.mean(K.binary_crossentropy(y_pred,y_true),axis=-1) 49 50 /Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/backend/tensorflow_backend.pyinbinary_crossentropy(output,target,from_logits) 1418output=tf.clip_by_value(output,epsilon,1–epsilon) 1419output=tf.log(output/(1–output)) ->1420returntf.nn.sigmoid_cross_entropy_with_logits(output,target) 1421 1422 /Users/badrshomrani/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/nn_impl.pyinsigmoid_cross_entropy_with_logits(_sentinel,labels,logits,name) 147#pylint:disable=protected-access 148nn_ops._ensure_xent_args(“sigmoid_cross_entropy_with_logits”,_sentinel, –>149labels,logits) 150#pylint:enable=protected-access 151 /Users/badrshomrani/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/nn_ops.pyin_ensure_xent_args(name,sentinel,labels,logits) 1696ifsentinelisnotNone: 1697raiseValueError(“Onlycall%swith” ->1698“namedarguments(labels=…,logits=…,…)”%name) 1699iflabelsisNoneorlogitsisNone: 1700raiseValueError(“Bothlabelsandlogitsmustbeprovided.”) ValueError:Onlycallsigmoid_cross_entropy_with_logitswithnamedarguments(labels=…,logits=…,…) Reply JasonBrownlee November14,2017at10:05am # Perhapsdoublecheckyouhavethelatestversionsofthekerasandtensorflowlibrariesinstalled?! Reply Badr November14,2017at10:50am # keraswasoutdated Reply JasonBrownlee November15,2017at9:44am # Gladtohearyoufixedit. Reply Mikael November22,2017at8:20am # HiJason,thanksforyourshorttutorial,helpsalottoactuallygetyourhandsdirtywithasimpleexample. Ihavetried5differentparametersandgotsomeinterestingresultstoseewhatwouldhappen.Unfortunately,Ididntrecordrunningtime. Test1 Test2 Test3 Test4 Test5 Test6 Test7 numberoflayers 3 3 3 3 3 3 4 Trainset 768 768 768 768 768 768 768 Iterations 150 100 1000 1000 1000 150 150 Rateofupdate 10 10 10 5 1 1 5 Errors 173 182 175 139 161 169 177 Values 768 768 768 768 768 768 768 %Error 23,0000% 23,6979% 22,7865% 18,0990% 20,9635% 22,0052% 23,0469% Ican’tseemtoseeatrendhere..Thatcouldputmeontherighttracktoadjustmyhyperparameters. Doyouhaveanyadviceonthat? Reply JasonBrownlee November22,2017at11:17am # Somethingiswrong.Hereisagoodlistofthingstotry: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply Nikolaos November28,2017at10:58am # Hi,Itrytoimplementtheaboveexamplewithfer2013.csvbutIreceiveanerror,itispossibletohelpmetoimplementthiscorrectly? keras.modelsimportSequential fromkeras.layersimportDense importnumpy importnumpyasnp #fixRandomseedforreproducibility numpy.random.seed(7) Y=[] X=[] #loaddataset forlineinopen("fer2013.csv"): row=line.split(',') Y.append(int(row[0])) X.append([int(p)forpinrow[1].split()]) X,Y=np.array(X)/255.0,np.array(Y) print(Y.shape) print(X.shape) #createmodel model=Sequential() model.add(Dense(12,input_dim=(35887,2304),activation='tanh')) model.add(Dense(8,activation='tanh')) model.add(Dense(1,activation='sigmoid')) #CompileModel model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) #FitModel model.fit(X,Y,epochs=150,batch_size=1) #evaluatethemodel scores=model.evaluate(X,Y) print("\n%s:%.2f%%"%(model.metrics_names[1],scores[1]*100)) #calculatepredictions predictions=model.predict(X) #roundpredictions rounded=[round(x[0])forxinpredictions] print(rounded) 12345678910111213141516171819202122232425262728293031323334353637383940 keras.modelsimportSequentialfromkeras.layersimportDenseimportnumpyimportnumpyasnp #fixRandomseedforreproducibilitynumpy.random.seed(7)Y=[]X=[]#loaddatasetforlineinopen("fer2013.csv"): row=line.split(',') Y.append(int(row[0])) X.append([int(p)forpinrow[1].split()])X,Y=np.array(X)/255.0,np.array(Y)print(Y.shape)print(X.shape) #createmodelmodel=Sequential()model.add(Dense(12,input_dim=(35887,2304),activation='tanh'))model.add(Dense(8,activation='tanh'))model.add(Dense(1,activation='sigmoid')) #CompileModelmodel.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) #FitModelmodel.fit(X,Y,epochs=150,batch_size=1) #evaluatethemodelscores=model.evaluate(X,Y)print("\n%s:%.2f%%"%(model.metrics_names[1],scores[1]*100)) #calculatepredictionspredictions=model.predict(X)#roundpredictionsrounded=[round(x[0])forxinpredictions]print(rounded) Reply JasonBrownlee November29,2017at8:10am # Sorry,Icannotdebugyourcode. Whatistheproblemexactly? Reply Tanya December2,2017at12:06am # Hello, ihaveaabitgeneralquestion. Ihavetodoaforecastingforrestaurantsales(meaningthatIhavetopredict4mealsbasedonahistoricaldailysalesdata),weathercondition(suchastemperature,rain,etc),officialholidayandin-off-season.Ihavetoperformthatforecastingusingneuronalnetworks. Iamunfortunatelynotaveryskilledinpython.OnmycomputerIhavePython2.7andIhaveinstallanaconda.Iamtryingtolearnexercisingwithyourcodes,Mr.Brownlee.ButsomehowIcannotrunthecodeatall(inSpyder).CanyoutellmewhatkindofversionofpythonandanacondaIhavetoinstallonmycomputerandinwhichenvironment(jupiterlab,notebook,qtconsole,spyder,etc)Icanrunthecode,sotoworkandnottogiveerrorfromtheverybeginning? Iwillbeverythankfulforyourresponse KG Tanya Reply JasonBrownlee December2,2017at9:02am # Perhapsthistutorialwillhelpyousetupandconfirmyourenvironment: http://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/ IwouldalsorecommendrunningcodefromthecommandlikeasIDEsandnotebookscanintroduceandhideerrors. Reply Eliah December3,2017at10:53am # HiDr.Brownlee. IlookedoverthetutorialandIhadaquestionregardingreadingthedatafromabinaryfile?ForinstanceIworkingonsolvingtheslidingtiledn-puzzleusingneuralnetworks,butIseemtohavetroubletogettingmydatawhichisinabinaryfileanditgeneratesthenumberofmoverequiredforthen-puzzletobesolvein.Amnotsureifyouhavedealtwiththisbefore,butanyhelpwouldbeappreciated. Reply JasonBrownlee December4,2017at7:43am # Sorry,Idon’tknowaboutyourbinaryfile. Perhapsafteryouloadyourdata,youcanconvertittoanumpyarraysothatyoucanprovideittoaneuralnet? Reply Eliah December4,2017at9:28am # Thanksforthetip,I’lltryit. Reply Wafaa December7,2017at4:59pm # Thankyouveryverymuchforallyourgreattutorials. IfIwantedtoaddbatchlayeraftertheinputlayer,howshouldIdoit? CuzIappliedthistutorialonadifferentdatasetandfeaturesandIthinkIneednormalizationorstandardizationandIwanttodoittheeasiestway. Thankyou, Reply JasonBrownlee December8,2017at5:35am # Irecommendpreparingthedatapriortofittingthemodel. Reply zaheer December9,2017at3:03am # thanksforsharingsuchnicetutorials,ithelpedmealot.iwanttoprinttheconfusionmatrixfromtheaboveexample.andonemorequestion. ifihave 20-inputvariable 1-classlabel(binary) and400instances howiwouldknow,settingupthedenselayerparameterinthefirstlayerandhiddenlayerandoutputlayer.likeaboveexampleyouhaveplaced.12,8,1 Reply JasonBrownlee December9,2017at5:44am # Irecommendtrialanderrortoconfigurethenumberofneuronsinthehiddenlayertoseewhatworksbestforyourspecificproblem. Reply zaheer December9,2017at3:29am # C:\Users\zaheer\AppData\Local\Programs\Python\Python36\python.exeC:/Users/zaheer/PycharmProjects/PythonBegin/Bin-CLNCL-Copy.py UsingTensorFlowbackend. Traceback(mostrecentcalllast): File“C:/Users/zaheer/PycharmProjects/PythonBegin/Bin-CLNCL-Copy.py”,line28,in model.fit(x_train,y_train,epochs=100,batch_size=100) File“C:\Users\zaheer\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\models.py”,line960,infit validation_steps=validation_steps) File“C:\Users\zaheer\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py”,line1574,infit batch_size=batch_size) File“C:\Users\zaheer\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py”,line1407,in_standardize_user_data exception_prefix=’input’) File“C:\Users\zaheer\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py”,line153,in_standardize_input_data str(array.shape)) ValueError:Errorwhencheckinginput:expecteddense_1_inputtohaveshape(None,20)butgotarraywithshape(362,1) Reply JasonBrownlee December9,2017at5:45am # Ensuretheinputshapematchesyourdata. Reply AnamZahra December10,2017at7:40pm # DearJason!Greatjobaverysimpleguide. Iamtryingtoruntheexactcodebutthereisaneror str(array.shape)) ValueError:Errorwhencheckingtarget:expecteddense_3tohaveshape(None,1)butgotarraywithshape(768,8) HowcanIresolve. Ihavewindows10andspyder. Reply JasonBrownlee December11,2017at5:24am # Sorrytohearthat,perhapsconfirmthatyouhavethelatestversionofNumpyandKerasinstalled? Reply nazekhassouneh December11,2017at7:33am # afterrunthiscode,iwillcalculatetheaccuracy,howidid,i iwanttosplitthedatasetintotestdata,trainingdata andevaluatethemodelandcalculatetheaccuracy thankdr. Reply Suchith December21,2017at2:35pm # Inthemodelhowmanyhiddenlayersarethere? Reply JasonBrownlee December21,2017at3:35pm # Thereare2hiddenlayers,1inputlayerand1outputlayer. Reply AmareMahtesenu December22,2017at9:55am # hithere.thisblogisveryawesomeliketheAdrian’spyimagesearchblog.IhaveonequestionandthatisdoyouhaveorwillyouhaveatutorialonkerasframeworkwithSSDorYoloarchitechtures? Reply JasonBrownlee December22,2017at4:16pm # Thanksforthesuggestion,Ihopetocovertheminthefuture. Reply KyujinChae January8,2018at2:22pm # Thanksforyourawesomearticle. Iamreallyenjoying ‘MachineLearningMastery’!! Reply JasonBrownlee January8,2018at3:54pm # Thanks! Reply LuisGaldo January9,2018at8:41am # HelloJason! Thisisanawesomearticle! IamwritingareportforasubjectinuniversityandIhaveusedyourcodeduringmyimplementation,woulditbepossibletocitethispostinbibtex? Thankyou! Reply JasonBrownlee January9,2018at3:17pm # Sure,youcancitethewebpagedirectly. Reply NikhilGupta January25,2018at8:05pm # Myquestionisregardingpredict.Iusedtogetdecimalsinthepredictionarray.Suddenly,IstartedseeingonlyIntegers(0or1)intherun.Anyideawhatcouldbecausingthechange? predictions=model.predict(X2) predictions Out[3]: array([[0.], [0.], [0.], …, [0.], [0.], [0.]],dtype=float32) Reply JasonBrownlee January26,2018at5:39am # Perhapschecktheactivationfunctionontheoutputlayer? Reply NikhilGupta January28,2018at3:30am # #createmodel.FullyconnectedlayersaredefinedusingtheDenseclass model=Sequential() model.add(Dense(12,input_dim=len(x_columns),activation=’relu’))#12neurons,8inputs model.add(Dense(8,activation=’relu’))#Hiddenlayerwith8neurons model.add(Dense(1,activation=’sigmoid’))#1outputlayer.Sigmoidgive0/1 Reply joe January27,2018at1:25am # ==================RESTART:/Users/apple/Documents/deep1.py================== UsingTensorFlowbackend. Traceback(mostrecentcalllast): File“/Users/apple/Documents/deep1.py”,line20,in model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) File“/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/keras/models.py”,line826,incompile **kwargs) File“/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/keras/engine/training.py”,line827,incompile sample_weight,mask) File“/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/keras/engine/training.py”,line426,inweighted score_array=fn(y_true,y_pred) File“/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/keras/losses.py”,line77,inbinary_crossentropy returnK.mean(K.binary_crossentropy(y_true,y_pred),axis=-1) File“/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py”,line3069,inbinary_crossentropy logits=output) TypeError:sigmoid_cross_entropy_with_logits()gotanunexpectedkeywordargument‘labels’ >>> Reply JasonBrownlee January27,2018at5:58am # Ihavenotseemthiserror,sorry.Perhapstrypostingtostackoverflow? Reply Atefeh January27,2018at4:04pm # HelloMr.Janson AfterinstallingAnacondaanddeeplearninglibraries,IreadyourFreemini-courseandItriedtowritethecodeaboutthehandwrittendigitrecognition. Iwrotethecodesinjupyternotebook,amIright? ifnotwhereshouldIwritethecodes? andifIwanttouseanotherdataset(myowndataset)howcanIuseinthecode? andhowcanIseetheresult,forexampletheaccuracypercentage? Iamreallysorryformysimplequestions!Ihavewrittenalotofcodein“Matlab”butIamreallyabeginnerinPythonandAnaconda,myteacherforcemetousePythonandkerasformyproject. thankyouverymuchforyourhelp Reply JasonBrownlee January28,2018at8:22am # Anotebookisfine. YoucanwritecodeinaPythonscriptandthenrunthescriptdirectly. Reply Atefeh January28,2018at12:01am # HelloMr.Jansonagain IwrotethecodebelowfromyourFreeminicourseforhandwrittendigitrecognition,butafterrunningIfacedthesyntaxerror: fromkeras.datasetsimportmnist … (X_train,y_train),(X_test,y_test)=mnist.load_data() X_train=X_train.reshape(X_train.shape[0],1,28,28) X_test=X_test.reshape(X_test.shape[0],1,28,28) fromkeras.utilsimportnp_utils … y_train=np_utils.to_categorical(y_train) y_test=np_utils.to_categorical(y_test) model=Sequential() model.add(Conv2D(32,(3,3),padding=’valid’,input_shape=(1,28,28), activation=’relu’)) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Flatten()) model.add(Dense(128,activation=’relu’)) model.add(Dense(num_classes,activation=’softmax’)) model.compile(loss=’categorical_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) File“”,line2 2model.add(Conv2D(32,(3,3),padding=’valid’,input_shape=(1,28,28), ^ SyntaxError:invalidsyntax wouldyoupleasehelpme?! thanksalot Reply JasonBrownlee January28,2018at8:25am # This: model.add(Conv2D(32,(3,3),padding=’valid’,input_shape=(1,28,28), activation=’relu’)) 12 model.add(Conv2D(32,(3,3),padding=’valid’,input_shape=(1,28,28),activation=’relu’)) shouldbe: model.add(Conv2D(32,(3,3),padding=’valid’,input_shape=(1,28,28),activation=’relu’)) 1 model.add(Conv2D(32,(3,3),padding=’valid’,input_shape=(1,28,28),activation=’relu’)) Reply Lila January29,2018at8:04am # Thankyoufortheawsomeblogandexplanations.Ihavejustaquestion:Howcanwegetpredictedvaluesbythemodel..Manythanks Reply JasonBrownlee January29,2018at8:21am # Asfollows: X=... yhat=model.predict(X) 12 X=...yhat=model.predict(X) Reply Lila January30,2018at1:22am # Thankyouforyourpromptanswer.IamtryingtolearnhowkerasmodelsworkandIused.Itrainedthemodellikethis: model.compile(loss=’mean_squared_error’,optimizer=’sgd’,metrics=[‘MSE’]) AsoutputIhavethoselines Epoch10000/10000 10/200[>………………………..]–ETA:0s–loss:0.2489–mean_squared_error:0.2489 200/200[==============================]–0s56us/step–loss:0.2652–mean_squared_error:0.2652 andmyquestionwhatthedifferencebetweenthetwolines(MSEvalues) Reply JasonBrownlee January30,2018at9:53am # Theyshouldbethesamething.Onemaybecalculatedattheendofeachbatch,andoneattheendofeachepoch. Reply Atefeh January30,2018at4:28am # hello afterrunningagainitshowanerror: NameErrorTraceback(mostrecentcalllast) in() —->1model=Sequential() 2model.add(Conv2D(32,(3,3),padding=’valid’,input_shape=(1,28,28),activation=’relu’)) 3model.add(MaxPooling2D(pool_size=(2,2))) 4model.add(Flatten()) 5model.add(Dense(128,activation=’relu’)) NameError:name‘Sequential’isnotdefined Reply JasonBrownlee January30,2018at9:55am # Youaremissingtheimports.Ensureyoucopyallcodefromthecompleteexampleattheend. Reply Atefeh January31,2018at1:02am # fromkeras.datasetsimportmnist … (X_train,y_train),(X_test,y_test)=mnist.load_data() X_train=X_train.reshape(X_train.shape[0],1,28,28) X_test=X_test.reshape(X_test.shape[0],1,28,28) fromkeras.utilsimportnp_utils … y_train=np_utils.to_categorical(y_train) y_test=np_utils.to_categorical(y_test) model=Sequential() 2model.add(Conv2D(32,(3,3),padding=’valid’,input_shape=(1,28,28),activation=’relu’)) 3model.add(MaxPooling2D(pool_size=(2,2))) 4model.add(Flatten()) 5model.add(Dense(128,activation=’relu’)) 6model.add(Dense(num_classes,activation=’softmax’)) 7model.compile(loss=’categorical_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) Reply Atefeh February2,2018at5:01am # hello pleasetellmehowcanIfindoutthattensorflowandkerasarecorrectlyinstalledonmysystem. maybetheproblemisthat,becausenocoderunsinmyjupyter.andno“import”actswell(forexampleimportpandas) thankyou Reply JasonBrownlee February2,2018at8:23am # Seethispost: https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/ Reply Dan February3,2018at12:29am # Hi.I’mtotallynewtomachinelearningandI’mtryingtowrapmyheadaroundit. IhaveaproblemIcan’tquitesolveyet.Anddon’tknowwheretostartactually. Ihaveadictionarywithafewkey:valuepairs.Thekeyisarandom4digitnumberfrom0000to9999.Andthevalueforeachkeyissetasfollows:ifadigitinanumberiseither0,6or9thenitsweightis1,ifadigitis8thenit’sweightis2,anyotherdigithasaweightof0.Alltheweightsaresummarisedthenandhereyouhavethevalueforthekey.(example:{‘0000’:4,‘1234’:0,‘1692’:2,‘8800’:6}–andsoon). NowI’mtryingtobuildamodelthatwillpredictthecorrectvalueofagivenkey.(i.eifIgiveit2222theansweris0,ifIgiveit9011–it’s2).WhatIdidfirstiscreatedaCSVfilewith5columns,firstfourisasplit(byasingledigit)keyfrommydictionary,andthefifthcolumnisthevalueforeachkey.NextIcreatedadatasetanddefinedamodel(likethistutorialbutwithinput_dim=4).NowwhenItrainthemodeltheaccuracywon’tgohigherthen~30%.Alsoyourmodelisbasedonbinaryoutput,whereasmineshouldhaveanintegerfrom0to8.WheredoIgofromhere? Thankyouforallyoureffortinadvance!🙂 Reply JasonBrownlee February3,2018at8:42am # Thispostmighthelpyounaildownyourproblemasapredictivemodelingproblem: http://machinelearningmastery.com/how-to-define-your-machine-learning-problem/ Reply Alex February5,2018at5:22am # ThereisonethingIjustdontget. Anexampleofrowdatais6,148,72,35,0,33.6,0.627,50,1 Iguessthenumberattheendisifthepersonhasdiabetes(1)ordoesnot(0),butwhatIdontunderstandishowIknowthe‘prediction’isaboutthat0or1,teherearealotofothervariablesinthedata,andIdontsee‘diabetes’beingalabelforanyofthat. So,howdoIknoworhowdoIsetwichvariable(number)Iwanttopredict? Reply JasonBrownlee February5,2018at7:49am # Youinterpretthepredictioninyourapplicationorusage. Themodeldoesnotcarewhattheinputsandoutputsare,itdoesthebestitcan.Itdoesnotintrinsicallycareaboutdiabetes. Reply blaisexen February6,2018at9:14am # hi, @JasonBrownlee,MasterofKerasPython. I’mdevelopingafacerecognitiontesting,IsuccessfullyusedRprop,itwasgoodforstaticimagesorfacepictures,Ialsohavetestsvmresults. WhatdoyouthinkinyourexperiencedthatKerasisbetterorpowerfulthanRprop? becauseIwasalsothinkingtousedKeras(1:1)forfinalresultofRprop(1:many). orwhichdoyouthinkisbettersystem? thanksinadvancefortheadvices. IalsoheardoneoftheleaderofcommercialfacerecognizersusesPNN(useslibopenblas),soIreallydoubtwhichonetochooseformyfinalthesisandapplication. Reply JasonBrownlee February6,2018at9:29am # Whatdoyoumeanbyrprop?Ibelieveitisjustanoptimizationalgorithm,whereasKerasisadeeplearninglibrary. https://en.wikipedia.org/wiki/Rprop Reply blaisexen February17,2018at10:46am # Ok,IthinkIunderstandyou. IusedAccord.Net Rproptestingwasgood MLRtestingwasgood SVMtestingwasgood RBMtestingwasgood Iusedclassificationforfaceimages Theyareonlygoodforstaticfacepictures100×100 butifIusedanotherpicturefromthem, these4testingIhavefailed. DoyouthinkifIusedKerasinimagefacerecognitionwillhaveagoodresultorgoodprediction? becauseifKeraswillhaveagoodresultthenI’llhavetousedcesarsouzakerasc# https://github.com/cesarsouza/keras-sharp thanksforthereply. Reply JasonBrownlee February18,2018at6:45am # Tryitandsee. Reply CHIRANJEEVI February8,2018at8:52pm # Whatisthedifferencebetweentheaccuracywegetwhenwefitthemodelandtheaccuracy_score()ofsklearn.metrics,whattheymeanexactly? Reply JasonBrownlee February9,2018at9:05am # Accuracyisasummaryofthenumberofpredictionsthatweremadecorrectlyoutofallpredictionsthatweremade. Itisusedasanestimateofmodelskillonnewoutofsampledata. Reply Shinan February8,2018at9:09pm # isweatherforecastingcandoneusingRNN? Reply JasonBrownlee February9,2018at9:06am # No.Weatherforecastingisdonewithensemblesofphysicssimulationsonverylargecomputers. Reply CHIRANJEEVI February9,2018at3:56pm # wehaven’tpredictinganytingduringthefit(itsjustatraining,likemappingF(x)=Y) butstillgettingacc,whatisthisacc? Epoch1/150 768/768[==============================]–1s1ms/step–loss:0.6771–acc:0.6510 Thankyouinadvance Reply JasonBrownlee February10,2018at8:50am # Predictionsaremadeaspartofbackpropagatingerror. Reply lcy1031 February12,2018at1:00pm # HiJason, Manythankstoyouforagreattutorial.Ihavecouplequestionstoyouasfollowings. 1).HowcanIgetthescoreofPrediction? 2).HowcanIoutputtheresultofpredictruntoafileinwhichtheoutputislistedbyvertical? Iseeyoueverywheretoanswerquestionsandhelppeople.Yourtimeandpatienceweregreatlyappreciated! Charles Reply JasonBrownlee February12,2018at2:50pm # Youcanmakepredictionswithamodelasfollows: yhat=model.predict(X) Youcanthensavethenumpyarrayresulttofile. Reply Callum February21,2018at10:11am # HiI’vejustfinishedthistutorialbuttheonlyproblemiswhatareweactuallyfindingintheresultsasinwhatdoaccuracyandlossmeanandwhatweareactuallyfindingout. I’mreallynewtothewholeneuralnetworksthinganddon’treallyunderstandthemyet,I’dbeverygratefulifyou’reabletoreply ManyThanks Callum Reply JasonBrownlee February22,2018at11:12am # Accuracyisthemodelskillintermsofthenumberofcorrectpredictionsdividedbythetotalnumberofpredictions. Lossthefunctionthatthenetworkisoptimising,somethingdifferentiableandrelatabletothemetricofinterestforthemodel,inthiscaselogarithmiclossusedforclassification. Reply PedroWenner February23,2018at1:27am # HiJason, Firstofallcongratulationsforyourawesomework,IfinallygotthehangofML(hopefully,haha). So,testingsomechangesinthenumberofneuronsandbatchsize/epochs,Iachieved99.87%ofaccuracy. TheparametersIusedwere: #createmodel model=Sequential() model.add(Dense(240,input_dim=8,init=’uniform’,activation=’relu’)) model.add(Dense(160,init=’uniform’,activation=’relu’)) model.add(Dense(1,init=’uniform’,activation=’sigmoid’)) #Compilemodel model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) #Fitthemodel model.fit(X,Y,epochs=1500,batch_size=100,verbose=2) AndwhenIrunit,Ialwaysget99,87%ofaccuracy,whichIthinkit’sagoodthing,right?PleasetellmeifIdidsomethingwrongorifthisisafalsepositive. Thankyouinadvanceandsorryforthebadenglish😉 Reply JasonBrownlee February23,2018at12:00pm # thataccuracyisgreat,therewillalwaysbesomeerror. Reply Shiny March2,2018at12:56am # Theaboveexampleisverygoodsir,Iwanttodopricechangepredictionofelectronicsinonlineshoppingproject.Canyougiveanysuggestionsaboutmyproject.Youhadanyexampleofpricepredictionusingneuralnetworkpleasesendalinksir. Reply JasonBrownlee March2,2018at5:33am # Iwouldrecommendfollowingthisprocess: https://machinelearningmastery.com/start-here/#process Reply awaludin March6,2018at12:38am # Hi,veryhelpfulexample.ButIstilldon’tunderstandwhyyouload X=dataset[:,0:8] Y=dataset[:,8] IfIdo X=dataset[:,0:7]itwon’twork Reply JasonBrownlee March6,2018at6:16am # Youcanlearnmoreaboutindexingandslicingnumpyarrayshere: https://machinelearningmastery.com/index-slice-reshape-numpy-arrays-machine-learning-python/ Reply JeongKim March8,2018at1:48pm # Thankyouforthetutorial. Perhaps,someonealreadytoldyouthis.Thedatasetisnolongeravailable. Reply JasonBrownlee March8,2018at2:55pm # Thanksforthenote,I’llfixthatupASAP. Reply WesleyCampbell March9,2018at1:24am # Thanksverymuchfortheconciseexample!Asan“interestedamateur”withmoreexperiencecodingforscientificdatamanipulationthanforsoftwaredevelopment,asimple,high-levelexplanationlikethisoneismuchappreciated.Ifindsometimesthatdocumentationpagescanbeabitlow-levelformyliking,evenwithcodingexperiencemultiplelanguages.ThisarticlewasallIneededtogetstarted,andwasmuchmorehelpfulthanother“officialtutorials.” Reply JasonBrownlee March9,2018at6:24am # Thanks,I’mgladtohearthatWesley. Reply Trung March10,2018at12:55am # Thankyouforyourtutorial,butthedatasetisnotaccessible.Couldyoupleasefixit. Reply JasonBrownlee March10,2018at6:33am # Thanks,I’llfixit. Reply atefeh March16,2018at10:11pm # hello Ihavefoundacodetoconvertingmyimagedatatomnistformat.butIfacetoanerrorbelow. wouldyoupleasehelpme? importos fromPILimportImage fromarrayimport* fromrandomimportshuffle #Loadfromandsaveto Names=[[‘./training-images’,’train’],[‘./test-images’,’test’]] fornameinNames: data_image=array(‘B’) data_label=array(‘B’) FileList=[] fordirnameinos.listdir(name[0])[1:]:#[1:]Excludes.DS_StorefromMacOS path=os.path.join(name[0],dirname) forfilenameinos.listdir(path): iffilename.endswith(“.png”): FileList.append(os.path.join(name[0],dirname,filename)) shuffle(FileList)#Usefullforfurthersegmentingthevalidationset forfilenameinFileList: label=int(filename.split(‘/’)[2]) Im=Image.open(filename) pixel=Im.load() width,height=Im.size forxinrange(0,width): foryinrange(0,height): data_image.append(pixel[y,x]) data_label.append(label)#labelsstart(oneunsignedbyteeach) hexval=“{0:#0{1}x}”.format(len(FileList),6)#numberoffilesinHEX #headerforlabelarray header=array(‘B’) header.extend([0,0,8,1,0,0]) header.append(int(‘0x’+hexval[2:][:2],16)) header.append(int(‘0x’+hexval[2:][2:],16)) data_label=header+data_label #additionalheaderforimagesarray ifmax([width,height])<=256: header.extend([0,0,0,width,0,0,0,height]) else: raiseValueError('Imageexceedsmaximumsize:256×256pixels'); header[3]=3#ChangingMSBforimagedata(0x00000803) data_image=header+data_image output_file=open(name[1]+'-images-idx3-ubyte','wb') data_image.tofile(output_file) output_file.close() output_file=open(name[1]+'-labels-idx1-ubyte','wb') data_label.tofile(output_file) output_file.close() #gzipresultingfiles fornameinNames: os.system('gzip'+name[1]+'-images-idx3-ubyte') os.system('gzip'+name[1]+'-labels-idx1-ubyte') FileNotFoundErrorTraceback(mostrecentcalllast) in() 13 14FileList=[] —>15fordirnameinos.listdir(name[0])[1:]:#[1:]Excludes.DS_StorefromMacOS 16path=os.path.join(name[0],dirname) 17forfilenameinos.listdir(path): FileNotFoundError:[WinError3]Thesystemcannotfindthepathspecified:‘./training-images’ Reply JasonBrownlee March17,2018at8:37am # Lookslikethecodecannotfindyourimages.Perhapschangethepathinthecode? Reply Sayan March17,2018at4:57pm # Thanksalotsir,thiswasaverygoodandintuitivetutorial Reply JasonBrownlee March18,2018at6:01am # Thanks,I’mgladithelped. Reply NikhilGupta March19,2018at11:12pm # Igotapredictionmodelrunningsuccessfullyforfrauddetection.Mydatasetisover50millionandgrowing.Iamseeingapeculiarissue. Whentheloadeddatais10millionorless,MypredictionisOK. AssoonasIload11milliondata,Mypredictionsaturatestoaparticular(say0.48)andkeepsonrepeating.Thatisallpredictionswillbe0.48,irrespectiveoftheinput. Ihavetriedwillmultiplecombinationsofthedensemodel. #createmodel model=Sequential() model.add(Dense(32,input_dim=4,activation=’tanh’)) model.add(Dense(28,activation=’tanh’)) model.add(Dense(24,activation=’tanh’)) model.add(Dense(20,activation=’tanh’)) model.add(Dense(16,activation=’tanh’)) model.add(Dense(12,activation=’tanh’)) model.add(Dense(8,activation=’tanh’)) model.add(Dense(1,activation=’sigmoid’)) Reply JasonBrownlee March20,2018at6:21am # Perhapscheckwhetheryouneedtotrainonalldata,oftenasmallsampleissufficient. Reply NikhilGupta March22,2018at2:45am # Oh.Ibelievethatthemachinelearningaccuracywillimproveaswegetmoredataovertime. Reply ChandraSutrisnoTjhong March28,2018at4:43pm # HI, Howdoyoudefinenumberofhiddenlayersandneuronsperlayer? Reply JasonBrownlee March29,2018at6:30am # Therearenogoodheuristics,trialanderrorisagoodapproach.Discoverwhatworksbestforyourspecificdata. Reply Aravind March30,2018at12:12am # Iexecutedthecodeandgottheoutput,buthowtousethispredictionintheapplication. Reply JasonBrownlee March30,2018at6:39am # Dependsontheapplication. Reply Sabarish March30,2018at12:16am # Whatdoesthevalue1.0and0..0signifies?? Reply JasonBrownlee March30,2018at6:39am # Inwhatcontext? Reply Anand April1,2018at3:51pm # Ifnumberofinputsare8thenwhydidyouuse12neuronsininputlayer?Moreoverwhyisactivationfunctionusedininputlayer? Reply JasonBrownlee April2,2018at5:19am # Thenumberofneuronsinthefirsthiddenlayercanbedifferenttothenumberofneuronsintheinputlayer(e.g.numberofinputfeatures).Theyareonlylooselyrelated. Reply Lia April1,2018at11:49pm # HelloSir, Doestheneuralnetworkuseastandardizedindependentvariablevalues,orshouldwefeeditwithstandardizedonesinthefittingandpredictingstages.Thanks Reply JasonBrownlee April2,2018at5:23am # Trybothandseewhatworksbestforyourspecificpredictivemodelingproblem. Reply MarkLittlewood October27,2021at9:18am # HiIwasplayingwitha2inputdatasetandwhenIhadthefirstlayersetatDense(4itonlyoutputNaNfortheloss.HoweverwhenIreducedthisto3Igotmeaningfullossoutput.IstheresomethingaboutthemaximumDensvalueinrelationtotheinputsthatcausesthis? Reply AdrianTam October27,2021at12:56pm # Thereshouldnotbe.ItismorelikelyduetohowthelayersareinitializedthannumberofneuronsintheDenselayer. Reply tareknahool April4,2018at5:17am # youalwaysfantastic,it’sagreatlesson.But,franklyIdon’tknowwhatisthemeaningof “\n%s:%.2f%%”%andwhyyouusedthenumber(1)inthatcode(model.metrics_names[1],scores[1]*100)) Reply JasonBrownlee April4,2018at6:19am # ThisisPythonstringformatting: https://pyformat.info/ Reply AbhilashMenon April5,2018at6:27am # Dr.Brownlee, Whenwepredict,isitpossibletohavethepredictionsforeachrowinthetestdatasetrightnexttoitinthesamerow.IthoughtofprintingpredictionsandthencopyingitinexcelbutIamnotsureifKeraspreservesorder.Couldyoupleasehelpmeoutwiththisissue?Thankssomuchforallyourhelp! Reply JasonBrownlee April5,2018at3:05pm # Yes,theorderofpredictionsmatchestheorderofinputvalues. Doesthathelp? Reply AndreaGrandi April9,2018at6:37am # IsDeepLearningsomekindof“blackmagic”🙂? Ihadpreviouslyusedscikit-learnandMachineLearningforthesamedataset,tryingtoapplyallthetechniquesIdidlearnbothhereandonbooks,togeta76%accuracy. ItriedthisKerastutorial,usingTensorFlowasbackendandI’mgetting80%accuracyatfirsttryO_o Reply JasonBrownlee April10,2018at6:08am # No,notmagic,justdifferent. Welldonethough! Reply MannyCorrao April11,2018at8:30am # Canyoutellusthecolumnnames?Ithinkthatisimportantbecauseithelpsusunderstandwhatthenetworkisevaluatingandlearningabout. Thanks, Manny Reply JasonBrownlee April11,2018at4:11pm # Yes,theyarelistedhere: https://github.com/jbrownlee/Datasets/blob/master/pima-indians-diabetes.names Reply rachit April11,2018at7:13pm # WhileExecutingversions.py iamgettingthiserror Traceback(mostrecentcalllast): File“versions.py”,line2,in importscipy File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\scipy\__init__.py”,line61,in fromnumpyimportshow_configasshow_numpy_config File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\__init__.py”,line142,in from.importadd_newdocs File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\add_newdocs.py”,line13,in fromnumpy.libimportadd_newdoc File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\lib\__init__.py”,line8,in from.type_checkimport* File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\lib\type_check.py”,line11,in importnumpy.core.numericas_nx File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\core\__init__.py”,line74,in fromnumpy.testingimport_numpy_tester File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\testing\__init__.py”,line12,in from.importdecoratorsasdec File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\testing\decorators.py”,line6,in from.nose_tools.decoratorsimport* File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\testing\nose_tools\decorators.py”,line20,in from.utilsimportSkipTest,assert_warns File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\numpy\testing\nose_tools\utils.py”,line15,in fromtempfileimportmkdtemp,mkstemp File“C:\Users\ATITGARG\Anaconda3\lib\tempfile.py”,line45,in fromrandomimportRandomas_Random File“C:\Users\ATITGARG\random.py”,line7,in fromkeras.modelsimportSequential File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\keras\__init__.py”,line3,in from.importutils File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\keras\utils\__init__.py”,line4,in from.importdata_utils File“C:\Users\ATITGARG\Anaconda3\lib\site-packages\keras\utils\data_utils.py”,line23,in fromsix.moves.urllib.errorimportHTTPError ImportError:cannotimportname‘HTTPError’ Reply JasonBrownlee April12,2018at8:35am # Perhapsyouneedtoupdateyourenvironment? Reply Gray April14,2018at4:25am # Jason–veryimpressivework!Evenmoreimpressiveisyourdetailedanswertoeveryquestion.Iwentthroughthemallandgotalotofusefulinformation.Greatjob! Reply JasonBrownlee April14,2018at6:50am # ThanksGray! Reply octdes April14,2018at2:39pm # HelloJason, Thank’sforthegoodtuto! Howwouldyouname/describethestructureofthisneuronalnetwork? Thepointisthatifindstrangethatyoucanhaveadifferentnmberofinputandofneuronesintheinputlayer.Mostoftheneuronalnetworkdiagrammihaveseen,eachinputisdirectlyconnectedwithoneneuroneoftheinputlayer.Ihaveneverseenaneuronalnetworkdiagrammwherethenumberofinputisdifferentwiththenumberofneuronesintheinputlayer. Doyouhavecounterexampleordothereissomethingiunderstandwrong? Thankyouforyourworkandsharingyourknowledge🙂 Reply JasonBrownlee April15,2018at6:24am # Thetypeofneuralnetworkinthispostisamulti-layerperceptronoranMLPforshort. Thefirst“layer”inthecodeactuallydefinesboththeinputlayerandthefirsthiddenlayeratthesametime. Thenumberofinputsmustmatchthenumberofcolumnsintheinputdata.Thenumberofneuronsinthefirsthiddenlayercanbeanythingyouwant. Doesthathelp? Reply Ashley April16,2018at7:29am # ThankyouVERYmuchforthistutorial,Jason!ItisthebestIhavefoundontheinternet.Asapoliticalscientistpursuingcomplexoutcomeslikethisone,Iwaslookingformodelsthatallowformorecomplicatedrelationships.Yourcodeandpostaresoclearlyarticulated;IwasabletoadaptitformypurposesmoreeasilythanIthoughtwouldbepossible.Onepossibleextensionofyourwork,andpossiblythistutorial,wouldbetomapthelayersandnodesontoatheoryofthedatageneratingprocess. Reply JasonBrownlee April16,2018at2:54pm # ThanksAshley,I’mgladithelped. Thanksforthesuggestion. Reply EricMiles April20,2018at1:22am # I’mjuststartingoutworkingthroughyoursite–thanksforthegreatresource!IwantedtopointoutwhatIthinkisatypo:inthecodeblockjustbeforeSection2“DefineModel”IbelievewejustwantX=dataset[:,0:7]sothatwedon’tincludetheoutputvariablesinourinputs. Reply JasonBrownlee April20,2018at6:00am # No,itiscorrectEric. Xwillhave8columns(0-7),theoriginaldatasethas9. YoucanlearnmoreaboutarrayslicingandrangesinPythonhere: https://machinelearningmastery.com/index-slice-reshape-numpy-arrays-machine-learning-python/ Reply Rafa April28,2018at12:50am # Greattutorial,finallyIhavefoundagoodwebaboutdeeplearning(Y) Reply JasonBrownlee April28,2018at5:31am # Thanks. Reply Vivek May7,2018at8:31pm # Greattutorialthankforhelp.IhaveoneprojectinwhichihavetodoCADimages(basically3-dmechanicalimageclassification).canyoupleasegiveroadmaphowcaniproceed? Iamnewandidonthaveanyidea Reply JasonBrownlee May8,2018at6:12am # Thisismygeneralroadmapforapredictivemodelingproblem: https://machinelearningmastery.com/start-here/#process Reply Vivek May9,2018at10:03pm # Thanksalotsir.Thiswillhelpmetoproceed Reply JasonBrownlee May10,2018at6:31am # I’mgladtohearthat. Reply Rahmadars May8,2018at1:36am # Thankssirforthetutorial. Actuallyistillhavesomequestion: 1.Isthisbackpropagationneuralnetwork? 2.Howtoinitializenguyen-widrowrandomweights 3.Ihavemyowndataset,eachconsistof1×64matrix,whichisthecorrectone?Inormalizeeachcolumnofit,oreachrowofit? Thanks. Imtheonewhoaskeduinbackpropagationfromscratchpage Reply JasonBrownlee May8,2018at6:16am # Yes,itusesbackpropgationtoupdatetheweights. Sorry,Idon’tknowaboutthatinitializationmethod,youcanseethesupportedmethodshere: https://keras.io/initializers/ Tryasuiteofdatapreparationschemestoseewhatworksbestforyourspecificdatasetandchosenmodel. Reply Hussein May9,2018at10:33pm # HiJason, Thisisaveryniceintrotoadauntingbutintriguingtechnology!IwantedtoplayaroundwithyourcodeandseeifIcouldcomeupwithsomesimpledatasetandseehowthepredictionswillworkout–oneideathatoccurredtomeis,canImakeamodelthatpredictswhatcountryatelephonenumberbelongsto.Sothetrainingdatasetlookslikea2columnCSV,phonenumberandcountry…that’sbasicallyonefeature.Doyouthinkthiswouldbeeffectiveatall?Whatotherfeaturescouldbeaddedhere?I’llstillgivethisashot,butwouldappreciateanythoughts/ideas! Thanks! Reply JasonBrownlee May10,2018at6:33am # Thecountrycodewouldmakeittoosimpleaproblem–e.g.itcanbesolvedwithalook-uptable. Reply Hussein May10,2018at4:24pm # True,Ijustwantedtoseeifmachinelearningcouldbeusedto“figureout”thelookuptableasopposedtobeprovidedwithonebytheuser,givenenoughdata..notapracticaluse-case,butasalearningexercise.Asitturnsout,mydata-setofabout700phonenumberswasn’teffectiveforthis.Butagain,isthisbecausetheproblemhadtoofewfeatures,i.einmycase,justone?WhatifIincreasedthenumberoffeatures,sayphonenumber,countrycode,citythephonenumberbelongsto,maybeeventhecellphonecompanythenumberisregisteredto,doyouthinkthatwouldmakethetrainingmoreeffective? Reply JasonBrownlee May11,2018at6:33am # Ifyoucanwriteanifstatementorusealook-uptabletosolvetheproblem,thenitmightbeabadfitformachinelearning. Thispostwillhelpyouframeyourproblem: http://machinelearningmastery.com/how-to-define-your-machine-learning-problem/ Reply Hussein May11,2018at5:15pm # ThanksJasonforthatresource.I’llcheckitout.Ialsocameacrossthis(https://elitedatascience.com/machine-learning-projects-for-beginners)thatI’mreadingthrough,foranyoneelsethat’slookingforasmallMLproblemtosolveasalearningexperience. JasonBrownlee May12,2018at6:27am # Great. FrankLu May14,2018at7:44pm # Greattutorialveryhelpful,thenIhaveaquestion.Whichaccountedforthelargestproportionin8inputs?Wehave8factorsinthedatasetlikepregnancies,glucose,bloodpressureandtheothers.So,Whichfactorismostrelatedtodiabetesused?HowdoweknowthisproportionthroughMLP? Thanks！ Reply JasonBrownlee May15,2018at7:53am # Wemightnotknow.Thisisthedifferencebetweendescriptiveandpredictivemodels. Thisisreallytheissueofmodelinterpretability,Iwritemoreaboutithere: https://machinelearningmastery.com/faq/single-faq/how-do-i-interpret-the-predictions-from-my-model Reply Paolo May16,2018at7:59pm # HiJason, thanksforyourtutorials. Ihaveaquestion,doyouusekeraswithpandastoo?Inthiscase,itisbettertoimportdatawihnumpyanyway?Whatdoyousuggest? Thankyouagain, Paolo Reply JasonBrownlee May17,2018at6:31am # Yes,andyes. Reply Stefan November10,2018at1:06am # Howso?Iusuallyseepandas.readcsv()toreadfiles.Doeskerasonlyacceptnumpyarrays? Reply JasonBrownlee November10,2018at6:07am # Correct. Reply zohreh May20,2018at9:14am # Thanksforyourgreattutorial.IhaveacreditcarddatasetandIwanttodofrauddetectiononit.ithas312columns,SobeforedoingDNN,Ishoulddodimensionreduction,thenusingDNN?andanotherquestionisthatIsitpossibletodoCNNonmydatasetaswell? Thankyou Reply JasonBrownlee May21,2018at6:24am # Yes,choosethefeaturesthatbestmaptotheoutputvariable. ACNNcanbeusedifthereisaspatialrelationshipinthedata,suchasasequenceoftransactionsoverspaceortime. Reply zohreh May23,2018at6:44am # Thanksforyouranswer,SoIthinkCNNdoesn’tmakesenseformydataset, Doyouhaveanytutorialforactivelearning? thanksforyourtime. Reply JasonBrownlee May23,2018at2:37pm # Idon’tknowifitisappropriate,Iwastryingtoprovideenoughinformationforyoutomakethatcall. Ihopetocoveractivelearninginthefuture. Reply zohreh May24,2018at3:13am # yesIunderstand,Isaidaccordingtoyourprovidedinformation,thankyousomuchforyouranswersandgreattutorials. MiguelGarcía May24,2018at11:55am # Canyoushareatutorialforfirstneuralnetowrkwithmultilabelsupport? Reply JasonBrownlee May24,2018at1:51pm # Thanksforthesuggestion. Reply Sathish May24,2018at12:57pm # howtocreateconvolutionallayersandvisualizefeaturesinkeras Reply JasonBrownlee May24,2018at1:51pm # Goodquestion,sorry,Idon’thaveaworkedexample. Reply Anam May28,2018at3:52am # DearJason, Igetanerror”ValueError:couldnotconvertstringtofloat:“Kindlyhelptosolvetheissue.AndIamusingmyowndatasetwhichconsistoftextnotnumbers(likethedatasetyouhaveused). Thanks! Reply JasonBrownlee May28,2018at6:04am # Thismightgiveyousomeideas: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply Anam May29,2018at7:26am # DearJason, Iamrunningyourcodeexamplefromsection6.ButIgetanerrorinthefollowingcodesnippet: CodeSnippet: dataset=numpy.loadtxt(“pima_indians.csv”,delimiter=”,”) #splitintoinput(X)andoutput(Y)variables X=dataset[:,0:8] Y=dataset[:,8] Error: ValueError:couldnotconvertstringtofloat:“6 Kindlyguidemetosolvetheissue.Thanksforyourprecioustime. Reply JasonBrownlee May29,2018at2:49pm # I’msorrytohearthat,Ihavesomesuggestionshere: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply GautamSharma June19,2018at1:20am # DidyoufindanysolutionasIamgettingthesameerror? Reply moti June4,2018at3:34am # HiDoctor,inthispythoncodewhereshallIgetthe“keras”package? Reply JasonBrownlee June4,2018at6:34am # ThistutorialshowsyouhowtoinstallKeras: https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/ Reply AmmaraHabib June5,2018at5:13am # Hyjason,Thanksforanamazingpost.Ihaveaquestionherethatcanweusedenselayerasinputfortextclassification(e.g:sentimentclassificationofmoviereviews).Ifyesthanhowcanweconvertthetextdatasetintonumericfordenselayer. Reply JasonBrownlee June5,2018at6:47am # Youcan,althoughitiscommontoonehotencodethetextoruseanembeddinglayer. Ihaveexamplesofbothontheblog. Reply AmmaraHabib June5,2018at9:18am # Thanksforyourprecioustime.Sir,youmeanthatfirstiuseembeddinglayerasinputlayerandtheniusedenselayerasthehiddenlayer? Reply JasonBrownlee June5,2018at3:05pm # Yes. Reply LisaXie June15,2018at1:12pm # Hi,thanksforyourtutorial.Iamwonderinghowyousetthenumberneuronsandactivationfunctionsforeachlayer,eg.12neuronsforthe1stlayerand8forthesecond. Reply JasonBrownlee June15,2018at2:50pm # Iusedalittletrialanderror. Reply Marwa June18,2018at1:25am # Hijason, IdeveloppedtwoneuralnetworksusingkerasbutIhavethiserror: line1336,in_do_call raisetype(e)(node_def,op,message) ResourceExhaustedError:OOMwhenallocatingtensorwithshape[7082368,50] [[Node:training_1/Adam/Variable_14/Assign=Assign[T=DT_FLOAT,_class=[“loc:@training_1/Adam/Variable_14″],use_locking=true,validate_shape=true,_device=”/job:localhost/replica:0/task:0/device:GPU:0”](training_1/Adam/Variable_14,training_1/Adam/zeros_14)]] Haveyouanidea? Thanks. Reply JasonBrownlee June18,2018at6:42am # Sorry,Ihavenotseenthiserrorbefore.Perhapstryposting/searchingonstackoverflow? Reply prateekbhadauria June23,2018at11:38pm # sirihavearegressionrelateddatasetwhichcontainsanarrayof49999rowsand20coloumns,iwanttoimplementCNNonthisdataset, iputmycodeaspermyperceptionkindlygivemesuggestion,tocorrectitiwasstuckmainlybyputtingmydensedimensionspecially fromkeras.modelsimportSequential fromkeras.layersimportDense importnumpyasnp importtensorflowastf frommatplotlibimportpyplot fromsklearn.datasetsimportmake_regression fromsklearn.preprocessingimportMinMaxScaler fromsklearn.metricsimportmean_squared_error fromkeras.wrappers.scikit_learnimportKerasRegressor fromsklearn.preprocessingimportStandardScaler fromkeras.layersimportDense,Dropout,Flatten fromkeras.layersimportConv2D,MaxPooling2D fromkeras.optimizersimportSGD seed=7 np.random.seed(seed) fromscipy.ioimportloadmat dataset=loadmat(‘matlab2.mat’) Bx=basantix[:,50001:99999] Bx=np.transpose(Bx) Fx=fx[:,50001:99999] Fx=np.transpose(Fx) fromsklearn.cross_validationimporttrain_test_split Bx_train,Bx_test,Fx_train,Fx_test=train_test_split(Bx,Fx,test_size=0.2,random_state=0) scaler=StandardScaler()#ClassiscreateasScaler scaler.fit(Bx_train)#Thenobjectiscreatedortofitthedataintoit Bx_train=scaler.transform(Bx_train) Bx_test=scaler.transform(Bx_test) model=Sequential() defbase_model(): keras.layers.Dense(Dense(49999,input_shape=(20,),activation=’relu’)) model.add(Dense(20)) model.add(Dense(49998,init=’normal’,activation=’relu’)) model.add(Dense(49998,init=’normal’)) model.compile(loss=’mean_squared_error’,optimizer=‘adam’) returnmodel scale=StandardScaler() Bx=scale.fit_transform(Bx) Bx=scale.fit_transform(Bx) clf=KerasRegressor(build_fn=base_model,nb_epoch=100,batch_size=5,verbose=0) clf.fit(Bx,Fx) res=clf.predict(Bx) ##linebelowthrowsanerror clf.score(Fx,res) Reply JasonBrownlee June24,2018at7:33am # Sorry,Icannotdebugyourcodeforyou.Perhapspostyourcodeanderrortostackoverflow? Reply MadhavPrakash June24,2018at3:01am # HiJason, Lookingatthedataset,Icouldfindthatthereweremanyattributeswitheachofthemdifferingintermsofunits.Whyhaven’tyourescaled/normalisedthedata?butstillmanagedtogetanaccuracyof75%? Reply JasonBrownlee June24,2018at7:35am # Ideally,weshouldrescalethedata. Thereluactivationfunctionismoreflexiblewithunscaleddata. Reply MadhavPrakash June24,2018at4:23pm # Ohkay,thanks. Also,I’veimplementedaNNonadatabasesimilartothis,wheretheaccuracyvariesb/w70-75%.I’vetriedtoincreasetheaccuracybytuningvariousparametersandfunctions(learningrate,no.oflayers,neuronsperlevel,earlystopping,activationfn,initialization,optimizeretc…)butitwasnotasuccess.Myquestioniswhendoicometoknowthati’vereachedthemaximumaccuracypossibleformyimplementation?Doistaycontentwiththecurrentaccuracy? Reply JasonBrownlee June25,2018at6:19am # Whenwerunoutoftimeorideas. Ilistsomemoreideashere: http://machinelearningmastery.com/machine-learning-performance-improvement-cheat-sheet/ Andhere: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply AarronWilson July8,2018at8:19am # Firstofallthanksforthetutorial.AlsoIacknowledgethatthisnetworkismoreforeducationalpurposes.Yetthisnetworkcanbeimprovedto83-84%accuracywithstandardnormalizationalone.Alsoitcanhit93-95%accuracybyusingadeepermodel. #Standardnormalization X=StandardScaler().fit_transform(X) #andadeepermodel model=Sequential() model.add(Dense(12,input_dim=8,activation=’relu’)) model.add(Dense(12,activation=’relu’)) model.add(Dense(12,activation=’relu’)) model.add(Dense(12,activation=’relu’)) model.add(Dense(12,activation=’relu’)) model.add(Dense(8,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’)) Reply JasonBrownlee July9,2018at6:30am # Thanks,yes,normalizationisagoodideaingeneralwhenworkingwithneuralnets. Reply Alex July10,2018at3:47am # Hi,thankyouforthisgreatarticle Imaginethatinmydatasetinsteadofdiabetesbeinga0or1Ihave3results,Imean,thedatarowsarelikethis data1,data2,sickness 123,124,0 142,541,0 156,418,1 142,541,1 156,418,2 So,Ineedtocategorizefor3values,IfIusethissameexampleyougaveushowcanIdeterminetheoutput? Reply JasonBrownlee July10,2018at6:51am # TheoutputwillbesicknessAlex.PerhapsIdon’tunderstandyourquestion? Reply Alex July10,2018at7:11am # Theoutputwillbesicknessyes Reply Alex July10,2018at10:17am # SorryformyEnglish,itisnotmynataltongue,Iwillredomyquesyion.WhatImeanisthis,Iwillbehavingalabelwithmorethan2results,0isonesickness,1willbeotherand2willbeother. HowcanIusethemodelyoushowedustofitthe3results? Reply JasonBrownlee July10,2018at2:26pm # Isee,thisiscalledamulti-classclassificationproblem. Thistutorialwillhelp: https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/ Reply adsad July11,2018at1:06am # isitpossibletopredictthelotteryoutcome.ifsohow? Reply JasonBrownlee July11,2018at5:59am # No.Iexplainmorehere: https://machinelearningmastery.com/faq/single-faq/can-i-use-machine-learning-to-predict-the-lottery Reply Tom July14,2018at2:32am # HiJason,Irunyourfirstexamplecodeinthistutorial.butwhatmakesmeconfusedis: Whythefinaltrainingaccuracy(0.7656)isdifferentfromtheevaluatedscores(78.26%)inthesamedatasets(trainingset)?Ican’tfigureitout.Canyoutellmeplease?Thanksalot! Epoch150/150 768/768[==============================]–0s–loss:0.4827–acc:0.7656 32/768[>………………………..]–ETA:0s acc:78.26% Reply JasonBrownlee July14,2018at6:20am # Oneistheperformanceonthetrainingset,theotheronthevalidationset. Youcanlearnmoreaboutthedifferencehere: https://machinelearningmastery.com/difference-test-validation-datasets/ Reply Tom July14,2018at9:09pm # Thanksfortherapidreply.ButInoticedthatinyourcodethetrainingsetandvalidationsetareexactlythesamedataset.Pleasecheckitforconfirmation.Thecodeisinthepart“6.TieItAllTogether”. #Fitthemodel model.fit(X,Y,epochs=150,batch_size=10) #evaluatethemodel scores=model.evaluate(X,Y) So,myproblemisstillthesame:Whythefinaltrainingaccuracy(0.7656)isdifferentfromtheevaluatedscores(78.26%)inthesamedatasets? Thanks! Reply JasonBrownlee July15,2018at6:14am # Perhapsverboseoutputmightbeaccumulatedovereachbatchratherthansummarizingskillattheendofthetrainingepoch. Reply ami July16,2018at2:01am # HelloJason, DoyouhavesometutorialonsignalprocessingusingCNN?IhavecsvfilesofsomebiomedicalsignalslikeECGandiwanttoclassifynormalandabnormalsignalsusingdeeplearning. WithRegards Reply JasonBrownlee July16,2018at6:11am # Yes,Ihaveasuiteoftutorialsscheduledonthistopic.Theyshouldbeoutsoon. Reply EL July16,2018at7:19pm # Hi,thankyousomuchforyourtutorial.Iamtryingtomakeaneuralnetworkthatwilltakeadatasetandreturnifitissuitabletobeanalyzedbyanotherprogramihave.Isitpossibletofeedthiswithacceptabledatasetsandunacceptabledatasetsandthencallitonanewdatasetandthenreturnwhetherthisdatasetisacceptable?Thankyouforyourhelp,Iamverynewtomachinelearning. Reply JasonBrownlee July17,2018at6:14am # Tryitandseehowyougo. Reply ami July18,2018at2:37pm # Ohreally!Thankyousomuch.Canyoupleasenotifymewhenthetutorialswillbeoutbecauseiamdoingaprojectandiamstuckrightnow. WithRegards Reply Diagrams July30,2018at2:45pm # Itwouldbeveryveryhelpfulfornewcomersifyouhadadiagramofthenetwork,showingindividualnodesandgraphedges(andbiasnodesandactivationfunctions),andindicatingonitwhichpartsweregeneratedbywhichmodel.addcommands/parameters.Similartohttps://zhu45.org/posts/2017/May/25/draw-a-neural-network-through-graphviz/ I’vetriedvisualizingitwithfromkeras.utils.plot_modelandtensorboard,butneitherproduceanode-leveldiagram. Reply JasonBrownlee July31,2018at5:58am # Thanksforthesuggestion. Reply Aravind July30,2018at7:57pm # cananyonetellasimplewaytorunmyannkerastensorflowbackendinGPU.Thanks Reply JasonBrownlee July31,2018at6:00am # ThesimplestwayIknowhow: https://machinelearningmastery.com/develop-evaluate-large-deep-learning-models-keras-amazon-web-services/ Reply farli August6,2018at1:08pm # Didyouusebackpropagationhere? Reply JasonBrownlee August6,2018at2:54pm # Yes. Reply farli August13,2018at9:40am # Canyoupleasemakeatutorialonconvolutionalneuralnet?Thatwouldbereallyhelpful..:) Reply JasonBrownlee August13,2018at2:27pm # Yes,ihavemanyontheblogalready.Trytheblogsearch. Reply KarimGamal August7,2018at8:52pm # IhaveaproblemwhereIgettheresultasshownbelow Epoch146/150–0s–loss:-1.2037e+03–acc:0.0000e+00 Epoch147/150–0s–loss:-1.2037e+03–acc:0.0000e+00 Epoch148/150–0s–loss:-1.2037e+03–acc:0.0000e+00 Epoch149/150–0s–loss:-1.2037e+03–acc:0.0000e+00 Epoch150/150–0s–loss:-1.2037e+03–acc:0.0000e+00 whereinmydatasettheoutputisavaluebetween0to500notonly0and1 sohowcanIfixthisinmycode Reply JasonBrownlee August8,2018at6:18am # Soundslikearegressionproblem.Changetheactivationfunctionintheoutputlayertolinearandthelossfunctionto‘mse’. Seethistutorial: https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/ Reply Tim August15,2018at5:54am # AWESOME!!!Thankssomuchforthis. Reply JasonBrownlee August15,2018at6:11am # You’rewelcome,I’mhappyithelped. Reply tania August27,2018at8:35pm # HiJason, Thankyouforthetutorial.IamrelativelynewtoMLandIamcurrentlyworkingonaclassificationproblemthatisnonbinary. Mydatasetconsistsofanumberoflabeledsamples–allmeasuringthesamequantity/unit.Theamounttypicallyrangesfrom10to20labeledsamples/inputs.However,thefeedforwardortestingsamplewillonlycontain7ofthoseinputs(atrandom). I’mstrugglingtofindasolutiontodesigningasystemthatacceptsfewerinputsthanwhatistypicallyfoundinthetrainingset. Reply JasonBrownlee August28,2018at5:59am # Perhapstryfollowingthisprocess: https://machinelearningmastery.com/start-here/#process Reply VaibhavJaiswal September10,2018at6:28pm # Greattutorialthere!Butthemainaspectofthemodelistopredictonasample.Ifiprintthefirstpredictedvalue,itshowsmesomevaluesforallthecolumnsofcategoricalfeatures.Howtogetthepredictednumberfromthesample? Reply JasonBrownlee September11,2018at6:26am # Theorderofthepredictionsmatchestheorderoftheinputs. Reply Glen September19,2018at10:45pm # IthinkImustbedoingsomethingwrong,Ikeepgettingtheerror: File“C:\Users\glens\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py”,line519,in__exit__ c_api.TF_GetCode(self.status.status)) InvalidArgumentError:Inputtoreshapeisatensorwith10values,buttherequestedshapehas1 [[Node:training_19/Adam/gradients/loss_21/dense_64_loss/Mean_1_grad/Reshape=Reshape[T=DT_FLOAT,Tshape=DT_INT32,_class=[“loc:@training_19/Adam/gradients/loss_21/dense_64_loss/Mean_1_grad/truediv”],_device=”/job:localhost/replica:0/task:0/device:GPU:0″](training_19/Adam/gradients/loss_21/dense_64_loss/mul_grad/Sum,training_19/Adam/gradients/loss_21/dense_64_loss/Mean_1_grad/DynamicStitch/_1703)]] AreyouabletoshedanylightonwhyIwouldgetthiserror? Thankyou Reply JasonBrownlee September20,2018at7:59am # Ihavenotseenthiserror,Ihavesomesuggestionshere: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply Snehasish September19,2018at11:15pm # HiJason,thanksforthisawesometutorial.Ihaveonedoubt–whydidtheevaluationnotproduce100%accuracy?Afterall,weusedthesamedatasetforevaluationastheoneusedfortrainingitself. Reply JasonBrownlee September20,2018at8:00am # Goodquestion! Weareapproximatingachallengingmappingfunction,notmemorizingexamples.Assuch,therewillalwaysbeerror. Iexplainmorehere: https://machinelearningmastery.com/faq/single-faq/why-cant-i-get-100-accuracy-or-zero-error-with-my-model Reply MarkC September27,2018at12:49am # Howdoyoupredictsomethingyouwanttopredictsuchasnewdata.forexampleIdidaspamdetectionbutdontknowhowtopredictwhetherasentenceiwriteisspamornot. Reply JasonBrownlee September27,2018at6:01am # Youcancallmodel.predict()withafinalizedmodel.Morehere: https://machinelearningmastery.com/faq/single-faq/how-do-i-make-predictions Reply Vivek October1,2018at3:17am # HelloSir, Iamnewandunderstoodsomepartofyourcode.Ihavequestioninpredictionmodelbasicallywedivideourdataintotrainingandtestset.Intheexampleabovetheentiredatasetisusedastrainingdataset.Howcanwetrainthemodelontrainingsetuseitforthepredictionontestset? Reply JasonBrownlee October1,2018at6:28am # Greatquestion,yes,trainthemodelonallavailabledataandthenuseittostartmakingpredictions. Morehere: https://machinelearningmastery.com/train-final-machine-learning-model/ Reply Vivek35 October1,2018at7:11am # HelloSir, It’sgreattutorialtounderstand.However,Iamnewandwanttounderstandsomethingoutofit.Intheabovecodewehavetreatedentiredatasetastrainingset.Canwedividethisintotrainingsetandtestset,applymodeltotrainingsetanduseitfortestsetprediction.Howcanweachievewiththeabovecode? Reply JasonBrownlee October1,2018at2:39pm # Thanks. Yes,youcansplitthedatasetmanuallyorusescikit-learntomakethesplitforyou.Iexplainmorehere: https://machinelearningmastery.com/faq/single-faq/how-do-i-evaluate-a-machine-learning-algorithm Reply Lipi October5,2018at6:26am # HiJason, Iamtryingtopredictusingmyneuralnetwork.IhaveusedMinMaxScalerinthefeatureswhiletrainingthedata.Idon’tgetagoodpredictionifIdon’tusethesametransformfunctiononthepredictiondatasetwhichIusedonthefeatureswhiletrainingthedata.Couldyousuggestmethecorrectapproachinthissituation? Reply JasonBrownlee October5,2018at2:29pm # Youmustusethesametransformtobothpreparetrainingdataandtomakepredictionsonnewdata. Reply Lipi October5,2018at10:12pm # Thankyou! Reply neenu October6,2018at3:57pm # hiiamnewtothisiwritewfollowingcodeinspyder fromkeras.modelsimportSequential fromkeras.layersimportDense importnumpy #fixrandomseedforreproducibility numpy.random.seed(7) #loadpimaindiansdataset dataset=numpy.loadtxt(“pima-indians-diabetes.txt”,encoding=”UTF8″,delimiter=”,”) #splitintoinput(X)andoutput(Y)variables X=dataset[:,0:8] Y=dataset[:,8] #createmodel model=Sequential() model.add(Dense(12,input_dim=8,activation=’relu’)) model.add(Dense(8,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’)) #Compilemodel model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) #Fitthemodel model.fit(X,Y,epochs=150,batch_size=10) #evaluatethemodel scores=model.evaluate(X,Y) print(“\n%s:%.2f%%”%(model.metrics_names[1],scores[1]*100)) Andigotthisasoutput runfile(‘C:/Users/DELL/Anaconda3/Scripts/temp.py’,wdir=’C:/Users/DELL/Anaconda3/Scripts’) UsingTensorFlowbackend. Traceback(mostrecentcalllast): File“”,line1,in runfile(‘C:/Users/DELL/Anaconda3/Scripts/temp.py’,wdir=’C:/Users/DELL/Anaconda3/Scripts’) File“C:\Users\DELL\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py”,line668,inrunfile execfile(filename,namespace) File“C:\Users\DELL\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py”,line108,inexecfile exec(compile(f.read(),filename,‘exec’),namespace) File“C:/Users/DELL/Anaconda3/Scripts/temp.py”,line1,in fromkeras.modelsimportSequential File“C:\Users\DELL\Anaconda3\lib\site-packages\keras\__init__.py”,line3,in from.importutils File“C:\Users\DELL\Anaconda3\lib\site-packages\keras\utils\__init__.py”,line6,in from.importconv_utils File“C:\Users\DELL\Anaconda3\lib\site-packages\keras\utils\conv_utils.py”,line9,in from..importbackendasK File“C:\Users\DELL\Anaconda3\lib\site-packages\keras\backend\__init__.py”,line89,in from.tensorflow_backendimport* File“C:\Users\DELL\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py”,line5,in importtensorflowastf File“C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\__init__.py”,line22,in fromtensorflow.pythonimportpywrap_tensorflow#pylint:disable=unused-import File“C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\__init__.py”,line49,in fromtensorflow.pythonimportpywrap_tensorflow File“C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py”,line74,in raiseImportError(msg) ImportError:Traceback(mostrecentcalllast): File“C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py”,line14,inswig_import_helper returnimportlib.import_module(mname) File“C:\Users\DELL\Anaconda3\lib\importlib\__init__.py”,line126,inimport_module return_bootstrap._gcd_import(name[level:],package,level) File“”,line994,in_gcd_import File“”,line971,in_find_and_load File“”,line955,in_find_and_load_unlocked File“”,line658,in_load_unlocked File“”,line571,inmodule_from_spec File“”,line922,increate_module File“”,line219,in_call_with_frames_removed ImportError:DLLloadfailedwitherrorcode-1073741795 Duringhandlingoftheaboveexception,anotherexceptionoccurred: Traceback(mostrecentcalllast): File“C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py”,line58,in fromtensorflow.python.pywrap_tensorflow_internalimport* File“C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py”,line17,in _pywrap_tensorflow_internal=swig_import_helper() File“C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py”,line16,inswig_import_helper returnimportlib.import_module(‘_pywrap_tensorflow_internal’) File“C:\Users\DELL\Anaconda3\lib\importlib\__init__.py”,line126,inimport_module return_bootstrap._gcd_import(name[level:],package,level) ModuleNotFoundError:Nomodulenamed‘_pywrap_tensorflow_internal’ FailedtoloadthenativeTensorFlowruntime. Seehttps://www.tensorflow.org/install/install_sources#common_installation_problems forsomecommonreasonsandsolutions.Includetheentirestacktrace abovethiserrormessagewhenaskingforhelp. Reply JasonBrownlee October7,2018at7:24am # Irecommendthistutorialtohelpyousetupyourenvironment: https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/ Irecommendthatyoudon’tuseamIDEornotebook: https://machinelearningmastery.com/faq/single-faq/why-dont-use-or-recommend-notebooks Instead,Irecommendyousavecodetoa.pyfileandrunfromthecommandline: https://machinelearningmastery.com/faq/single-faq/how-do-i-run-a-script-from-the-command-line Reply kamal October15,2018at1:08am # sirpleaseprovidethepythoncodeforadaptiveneurofuzzyclassifier Reply JasonBrownlee October15,2018at7:31am # Thanksforthesuggestion. Reply RajanKumar June29,2021at3:44pm # Iamwaitingtooforit. Reply Shahbaz October24,2018at4:44am # blessedonusir, canugivemeideaaboutOCRsystem,formyfinalyearproject,plzgivemeback-endstratigyforOCR,ruhaveanycodeonOCR Reply JasonBrownlee October24,2018at6:32am # Perhapsstarthere: http://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-networks-python-keras/ Reply AndrewAgib October29,2018at10:39pm # model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) showasyntaxerroronthatsentencewhatcouldbethereason Reply JasonBrownlee October30,2018at6:02am # Ihavesomesuggestionshere: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply VASUDEVKP November3,2018at10:13pm # HelloJason, Ihavethetheanobackendinstalled.IamusingWindowsOSandduringexecutionIamgettinganerror“NomodulenamedTensorFlow”.Pleasehelp Reply JasonBrownlee November4,2018at6:27am # YoumayhavetochangetheconfigurationofKerastouseTheanoinstead. Moredetailshere: https://keras.io/backend/ Reply ImenDrs November4,2018at7:09am # HiJason, Please,howcanwecalculatetheprecisionandrecallofthisexample? Andthanks. Reply JasonBrownlee November5,2018at6:06am # Youcanusescikit-learnmetrics: http://scikit-learn.org/stable/modules/classes.html#sklearn-metrics-metrics Reply Stefan November10,2018at2:59am # Ithoughtsigmoidandsoftmaxwerequitesimilaractivationfunctions.Butwhentryingthesamemodelwithsoftmaxasactivationforthelastlayerinsteadofsigmoid,myaccuracyismuchmuchworse. Doesthatmakesensetoyou?Ifsowhy?IfeellikeIseesoftmaxmoreofteninothercodethansigmoid. Reply JasonBrownlee November10,2018at6:09am # Nope. Sigmoidfor2classes. Softmaxfor>2classes Reply AmudaKamorudeen November10,2018at4:46pm # I’mworkingonmodelthatwillpredictpropensityofcustomerthatarelikelytoterminatetheirservicewithcompany.Ihavedatasetof70000rowsand500columns,PleasehowcanIpassnumericdataasaninputtoaconvolutionalneuralnetwork(CNN). Reply JasonBrownlee November11,2018at5:59am # CNNsareonlyappropriatefordatawithaspatialrelationship,suchasimages,timeseriesandtext. Reply irfan November18,2018at3:22pm # hijason, iamusingtensorflowasbackend. fromkeras.modelsimportSequential fromkeras.layersimportDense importsys fromkerasimportlayers fromkeras.utilsimportplot_model print(model.layer()) erro. ————————————————————————— AttributeErrorTraceback(mostrecentcalllast) in 9model.add(Dense(512,activation=’relu’)) 10model.add(Dense(10,activation=’sigmoid’)) —>11print(model.layer()) 12#Compilemodel 13model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) AttributeError:‘Sequential’objecthasnoattribute‘layer’ Reply JasonBrownlee November19,2018at6:44am # Whyareyoutryingtoprintmodel.layer()? Reply Mario December2,2018at5:30am # HiJason Firstthanksforamazingtutorial,sinceyourscriptsareusinglistofvalueswhilemyinputsarelistof24×20matriceswhicharefilledoutbyvaluesinespecialorderhowtheymeasuredfor3parametersin3000cycles,howcanIfeedthistypematrice-dataorlet’ssayhowcanIfeedstreamofimagesfor3differentparametersIalreadyextractedfromrawdatasetandafterpreprocessingIconvertthemto24*20matricesor.pngimages?HowshouldIchangethisscriptsothatIcanusemydataset? Reply JasonBrownlee December2,2018at6:26am # WhenusinganMLPwithimages,youmustflatteneachmatrixofpixeldatatoasinglerowvector. Reply EvangelosArgyropoulos December18,2018at6:15am # HiJason, Thankfortutorial.1questions. Iusethealgorithmfortimeseriesprediction0=buy1=sell.Doesthismodeloverfit? Reply JasonBrownlee December18,2018at6:27am # Youcanonlyknowifyoutryfittingitandevaluatinglearningcurvesontrainandvalidationdatasets. Reply SOURAVMONDAL December28,2018at7:42am # GreattutorialSir. Isthereawaytovisualizedifferentlayerswiththeirnodesandinterconnectionsamongthem,ofamodelcreatedinkeras(imeanthebasicstructureofaneuralnetworkwithlayersofnodesandinterconnectionsamongthem). Reply JasonBrownlee December29,2018at5:46am # Yes,checkoutthistutorial: https://machinelearningmastery.com/visualize-deep-learning-neural-network-model-keras/ Reply ImenDrs December28,2018at11:29pm # Thanksforthistutorial. Ihaveaproblemwhenitrytocompileandfitmymodel.Itreturnvalueerror:ValueError:couldnotconvertstringtofloat:’24,26,99,31,623,863,77,32,362,998,1315,33,291,14123,39,8,335,2308,349,403,409,1250,417,47,1945,50,188,51,4493,3343,13419,6107,84,18292,339,9655,22498,1871,782,1276,2328,56,17633,24004,24236,1901,6112,22506,26397,816,502,352,24238,18330,7285,2160,220,511,17680,68,5137,26398,875,542,354,2045,555,2145,93,327,26399,3158,7501,26400,8215′. Canyouhelpmeplease. Reply JasonBrownlee December29,2018at5:52am # Perhapsyourdatacontainsastring? Reply ImenDrs December29,2018at7:59am # Thedatacontains”user,number_of_followers,list_of_followers,number_of_followee,list_of_followee,number_of_mentions,list_of_user_mentioned…” thevaluesinthelistareseparatedbycommas. Forexample:“36;3;52,3,87;5;63,785,22,11,6;0;“ Reply Somashekhar January2,2019at4:39am # Hi,Isthereasolutionpostedforsolvingpima-indians-diabetes.csvforpredictionusingLSTM? Reply JasonBrownlee January2,2019at6:42am # No.LSTMsareforsequentialdataonly,andthepimaindiansdatasetisnotasequencepredictionproblem. Reply ImenDrs January4,2019at9:56pm # Isthereawaytousespecificfieldsinthedatasetinsteadoftheentireuploadeddataset. Andthanks. Reply JasonBrownlee January5,2019at6:56am # Yes,fieldsarecolumnsinthedatasetmatrixandyoucanremovethosecolumnsthatyoudonotwanttouseasinputstoyourmodel. Reply Kahina January5,2019at12:43am # Thankyousomuch!It’shelpful Reply JasonBrownlee January5,2019at6:58am # I’mhappytohearthatitwashelpful. Reply Khemmarut January12,2019at11:35pm # Traceback(mostrecentcalllast): File“C:/Users/Admin/PycharmProjects/NN/nnt.py”,line119,in rounded=[round(X[:1])forxinpredictions] File“C:/Users/Admin/PycharmProjects/NN/nnt.py”,line119,in rounded=[round(X[:1])forxinpredictions] TypeError:typenumpy.ndarraydoesn’tdefine__round__method Helpmeplease Thankyou. Reply JasonBrownlee January13,2019at5:41am # Perhapsensurethatyourlibrariesareuptodate? Thismighthelp: https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/ Reply PritiPachpande January31,2019at2:50am # HiJason, Thankyoufortheamazingtutorial.Iamtryingtobuildanautoencodermodelinkerasusingbackendtensorflow. Ineedtousetensorflow(liketf.ifft,tf.fft)functionsinthemodel.CanyouguidemetowardshowcanIdoit?ItriedusinglambdalayerbuttheaccuracydecreaseswhenIuseit. Also,Imusingmodel.predict()functiontocheckthevaluesbetweentheintermediatelayers.AmIdoingitright? Also,canyouguidemetowardshowtousereshapefunctioninkeras? Thanksforyourhelp Reply JasonBrownlee January31,2019at5:36am # Sorry,Idon’tknowaboutthefunctionsyouareusing.Perhapspostonstackoverflow? Reply Crawford January31,2019at9:34pm # HiJason, Yourtutorialsarebrilliant,thanksforputtingallthistogether. Inthistutorialtheresultiseithera1or0,butwhatifyouhavedatawithmorethantwopossibleresults,e.g.0,1,2,orsimilar? CanIdosomethingwiththecodeyouhavepresentedhere,orisawholeotherapproachrequired? IhavesomewhatachievedwhatI’mtryingtodousingyour“firstmachinelearningproject”usingaknnmodel,butIhadtosimplifymydatabystrippingoutsomevariables.Ibelievethereisvalueintheseextravariables,sothoughttheneuralnetworkmightbeuseful,butlikeIsaidIhavethreeclassificationsnottwo. Thanks. Reply JasonBrownlee February1,2019at5:37am # Yes,hereisanexampleofamulti-classclassificationwithaneuralnet: https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/ Reply Crawford February1,2019at10:11pm # Brilliant,thanks. Reply Sergio February1,2019at10:18am # Hi,Imtryingtoconstructaneuralnetworkusingcomplexnumberasinputs,Ifollowedyourrecommendatinsbutigetthefollowingwarning: ` ComplexWarning:Castingcomplexvaluestorealdiscardstheimaginarypartreturnarray(a,dtype,copy=False,order=order) Thecoderunwithoutproblems,butthepredictionsis25%exact. Ispossibletousecomplexnumberinneuralnetworks..? Douhavesomeadvices? Reply JasonBrownlee February1,2019at11:06am # Idon’tthinktheKerasAPIsupportscomplexnumbersasinput. Reply Sergio February1,2019at2:17pm # Douhaveanysuggestiontodealwithcomplexnumbers? Reply JasonBrownlee February2,2019at6:06am # Notoffhand,sorry. PerhapsposttotheKerasusersgrouptoseeifanyonehastriedthisbefore: https://machinelearningmastery.com/get-help-with-keras/ Reply ArnabKumarMishra February1,2019at9:47pm # HiJason, Iamtryingtorunthecodeinthetutorialwithsomeminormodifications,butIamfacingaproblemwiththetraining. Thetraininglossandaccuracybotharestayingthesameacrossepochs(Pleasetakealookatthecodesnippetandtheoutputbelow).Thisisforadifferentdataset,notthediabetesdataset. Ihavetriedtosolvethisproblemusingthesuggestionsgiveninhttps://stackoverflow.com/questions/37213388/keras-accuracy-does-not-change Buttheproblemisstillthere. Canyoupleasetakealookatthisandhelpmesolvethisproblem?Thanks. CODEandOUTPUTSnippets: #createmodel model=Sequential() model.add(Dense(15,input_dim=9,activation=’relu’)) model.add(Dense(10,activation=’relu’)) model.add(Dense(5,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’)) #compilemodel model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) #Fitthemodel model.fit(xTrain,yTrain,epochs=500,batch_size=10) Epoch1/200 81/81[==============================]–0s177us/step–loss:-8.4632–acc:0.4691 Epoch2/200 81/81[==============================]–0s148us/step–loss:-8.4632–acc:0.4691 Epoch3/200 81/81[==============================]–0s95us/step–loss:-8.4632–acc:0.4691 Epoch4/200 81/81[==============================]–0s116us/step–loss:-8.4632–acc:0.4691 Epoch5/200 81/81[==============================]–0s106us/step–loss:-8.4632–acc:0.4691 Epoch6/200 81/81[==============================]–0s98us/step–loss:-8.4632–acc:0.4691 Epoch7/200 81/81[==============================]–0s145us/step–loss:-8.4632–acc:0.4691 Epoch8/200 81/81[==============================]–0s138us/step–loss:-8.4632–acc:0.4691 Epoch9/200 81/81[==============================]–0s105us/step–loss:-8.4632–acc:0.4691 Epoch10/200 81/81[==============================]–0s128us/step–loss:-8.4632–acc:0.4691 Epoch11/200 81/81[==============================]–0s129us/step–loss:-8.4632–acc:0.4691 Epoch12/200 81/81[==============================]–0s111us/step–loss:-8.4632–acc:0.4691 Epoch13/200 81/81[==============================]–0s106us/step–loss:-8.4632–acc:0.4691 Epoch14/200 81/81[==============================]–0s144us/step–loss:-8.4632–acc:0.4691 Epoch15/200 81/81[==============================]–0s106us/step–loss:-8.4632–acc:0.4691 Epoch16/200 81/81[==============================]–0s180us/step–loss:-8.4632–acc:0.4691 Epoch17/200 81/81[==============================]–0s125us/step–loss:-8.4632–acc:0.4691 Epoch18/200 81/81[==============================]–0s183us/step–loss:-8.4632–acc:0.4691 Epoch19/200 81/81[==============================]–0s149us/step–loss:-8.4632–acc:0.4691 Epoch20/200 81/81[==============================]–0s146us/step–loss:-8.4632–acc:0.4691 Epoch21/200 81/81[==============================]–0s206us/step–loss:-8.4632–acc:0.4691 Epoch22/200 81/81[==============================]–0s135us/step–loss:-8.4632–acc:0.4691 Epoch23/200 81/81[==============================]–0s116us/step–loss:-8.4632–acc:0.4691 Epoch24/200 81/81[==============================]–0s135us/step–loss:-8.4632–acc:0.4691 Epoch25/200 81/81[==============================]–0s121us/step–loss:-8.4632–acc:0.4691 Epoch26/200 81/81[==============================]–0s110us/step–loss:-8.4632–acc:0.4691 Epoch27/200 81/81[==============================]–0s104us/step–loss:-8.4632–acc:0.4691 Epoch28/200 81/81[==============================]–0s122us/step–loss:-8.4632–acc:0.4691 Epoch29/200 81/81[==============================]–0s117us/step–loss:-8.4632–acc:0.4691 Epoch30/200 81/81[==============================]–0s111us/step–loss:-8.4632–acc:0.4691 Epoch31/200 81/81[==============================]–0s123us/step–loss:-8.4632–acc:0.4691 Epoch32/200 81/81[==============================]–0s116us/step–loss:-8.4632–acc:0.4691 Epoch33/200 81/81[==============================]–0s120us/step–loss:-8.4632–acc:0.4691 Epoch34/200 81/81[==============================]–0s156us/step–loss:-8.4632–acc:0.4691 Epoch35/200 81/81[==============================]–0s131us/step–loss:-8.4632–acc:0.4691 Epoch36/200 81/81[==============================]–0s122us/step–loss:-8.4632–acc:0.4691 Epoch37/200 81/81[==============================]–0s110us/step–loss:-8.4632–acc:0.4691 Epoch38/200 81/81[==============================]–0s121us/step–loss:-8.4632–acc:0.4691 Epoch39/200 81/81[==============================]–0s123us/step–loss:-8.4632–acc:0.4691 Epoch40/200 81/81[==============================]–0s111us/step–loss:-8.4632–acc:0.4691 Epoch41/200 81/81[==============================]–0s115us/step–loss:-8.4632–acc:0.4691 Epoch42/200 81/81[==============================]–0s119us/step–loss:-8.4632–acc:0.4691 Epoch43/200 81/81[==============================]–0s115us/step–loss:-8.4632–acc:0.4691 Epoch44/200 81/81[==============================]–0s133us/step–loss:-8.4632–acc:0.4691 Epoch45/200 81/81[==============================]–0s114us/step–loss:-8.4632–acc:0.4691 Epoch46/200 81/81[==============================]–0s112us/step–loss:-8.4632–acc:0.4691 Epoch47/200 81/81[==============================]–0s143us/step–loss:-8.4632–acc:0.4691 Epoch48/200 81/81[==============================]–0s124us/step–loss:-8.4632–acc:0.4691 Epoch49/200 81/81[==============================]–0s129us/step–loss:-8.4632–acc:0.4691 Epoch50/200 Thesamegoesonfortherestoftheepochsaswell. Reply JasonBrownlee February2,2019at6:14am # Ihavesomesuggestionsherethatmighthelp: http://machinelearningmastery.com/improve-deep-learning-performance/ Reply Nagesh February4,2019at1:50am # HiJason, Canyoupleaseupdateme,whetherwecanplotagraph(epochvsacc)? Ifyesthenhow. Reply JasonBrownlee February4,2019at5:49am # Ishowhowhere: https://machinelearningmastery.com/display-deep-learning-model-training-history-in-keras/ Reply Nils February5,2019at1:28am # Greatstuff,thanks! Ijustwonderedthatinchapter2thereisadescriptionofthe“init”parameter,butinallsourcesitwasmissing. Iaddeditlike: model.add(Dense(12,input_dim=8,init=’uniform’,activation=’relu’)) ThenIgotthiswarning: pima_diabetes.py:25:UserWarning:UpdateyourDensecalltotheKeras2API:Dense(12,input_dim=8,activation="relu" ,kernel_initializer="uniform") model.add(Dense(12,input_dim=8,init=’uniform’,activation=’relu’)) Solutionformewastousethe“kernel_initializer”instead: model.add(Dense(12,input_dim=8,activation=”relu”,kernel_initializer=”uniform”)) RegardingthesamelineIgotonequestion:Isitcorrect,thatitaddsoneinputlayerwith8neuronsANDanotherhiddenlayerwith12neurons? So,woulditresultinthesameANNtodothis? model.add(Dense(8,input_dim=8,kernel_initializer=’uniform’)) model.add(Dense(8,activation=”relu”,kernel_initializer=’uniform’)) Reply JasonBrownlee February5,2019at8:29am # Yes,perhapsyourversionofthebookisoutofdate,emailmetogetthelatestversion? Yes,thedefinitionofthefirsthiddenlayeralsodefinestheinputlayerviaanargument. Reply Shuja February8,2019at12:00am # HiJason Iamgettingthefollowingerror (env)[email protected]:~$pythonkeras_test.py UsingTensorFlowbackend. Traceback(mostrecentcalllast): File“keras_test.py”,line8,in dataset=numpy.loadtxt(“pima-indians-diabetes.csv”,delimiter=”,”) File“/home/shuja/env/lib/python3.6/site-packages/numpy/lib/npyio.py”,line955,inloadtxt fh=np.lib._datasource.open(fname,‘rt’,encoding=encoding) File“/home/shuja/env/lib/python3.6/site-packages/numpy/lib/_datasource.py”,line266,inopen returnds.open(path,mode,encoding=encoding,newline=newline) File“/home/shuja/env/lib/python3.6/site-packages/numpy/lib/_datasource.py”,line624,inopen raiseIOError(“%snotfound.”%path) OSError:pima-indians-diabetes.csvnotfound. Reply JasonBrownlee February8,2019at7:52am # Lookslikethedatasetwasnotdownloadedandplaceinthesamedirectoryasyourscript. Reply Shubham February12,2019at4:55am # Hi,Jason Thanksforthetutorial. DoyouhavesomegoodreferenceoranexamplewhereIcanlearnaboutsettingup“AdversarialNeuralNetworks”. Shubham Reply JasonBrownlee February12,2019at8:08am # Notatthisstage,Ihopetocoverthetopicinthefuture. Reply Daniel March13,2019at8:14am # HeyJason, I’vebeenreadingyourtutorialsforawhilenowonavarietyofMLtopics,andIthinkthatyouwriteverycleanlyandconcisely.ThankyouformakingalmosteverytopicI’veencounteredunderstandable. However,onethingIhavenoticedisthatthecommentsectionsonyourpagessometimescoverthebulkofthewebpage.ThefirstcoupletimesIsawthissite,IsawhowtinymyscrollbarwasandIassumedthatthetutorialwouldbe15pageslong,onlytofindthatyourintroductionswereinfact“gentle”aspromisedandeverythingbutthefirstsliverofthepagewerepeople’sresponsesandyourresponsesback.Ithinkitwouldbeveryusefulifyoucouldsomehowcondensetheresponses(maybea“showresponses”button?)toonlyshowtheactualcontent.Notonlywouldeverythinglookbetter,butIthinkitwouldalsopreventpeoplefrominitiallythinkingyourblogwasexceptionallylong,likeIdidafewtimes. Reply JasonBrownlee March13,2019at8:26am # Greatfeedback,thanksDaniel.I’llseeiftherearesomegoodwordpresspluginsforthis. Reply ismael March22,2019at5:22am # donotworkwhy Reply JasonBrownlee March22,2019at8:39am # Sorrytohearthatyou’rehavingtrouble,whatistheproblemexactly? Reply FelixDaniel March30,2019at7:09am # Awesomeworkonmachinelearning…IwasjustthinkingonhowtostartmyjourneyintoMachineLearning,IrandomlysearchedforpeopleinMachineLearningonLinkedInthat’showIfindmyselfhere…I’mdelightedtoseethis…HereismyfinalbusstoptostartbuildingupinML.ThanksforacceptingmyconnectiononLinkedIn. IhaveaprojectthatamabouttostartbutIdon’tknowhowandtheroadMap.PleaseIneedyourdetailedguideline. Hereisthetopic HumanActivityRecognitionSystemthatControlsoverweightinChildrenandAdults. Reply JasonBrownlee March31,2019at9:22am # Soundslikeagreatproject,youcangetstartedhere: https://machinelearningmastery.com/start-here/#deep_learning_time_series Reply AkshayaE April13,2019at11:38pm # canyoupleaseexplainmewhyweuse12neuronsinthefirstlayer?8areinputsandaretherest4biases? Reply JasonBrownlee April14,2019at5:49am # No,the12referstothe12nodesinthefirsthiddenlayer,nottheinputlayer. Theinputlayerisdefinedbyainput_dimargumentonthefirsthiddenlayer. Iexplainmorehere: https://machinelearningmastery.com/faq/single-faq/how-do-you-define-the-input-layer-in-keras Reply AkshayaE April14,2019at8:09pm # thankyoufortheimmediateresponse.mydoubthasbeencleared. Reply JasonBrownlee April15,2019at7:52am # Happytohearthat. Reply Abhiram April19,2019at11:50pm # hiiJason,abovepredictionsarebetween0to1,Mylabelsare1,1,1,2,2,2,3,3,3……..36,36,36. Nowiwanttopredictclass36thenwhatshouldido?? Reply JasonBrownlee April20,2019at7:39am # Whatproblemareyouhavingexactly? Reply Akash April22,2019at12:56am # HiJason, IamlearningNLPandfacingdifficultieswithunderstandingNLPwithDeepLearning. Please,canyouhelpwithconvertingthefollowingN:NtoN:1model? Iwanttochangemyvec_yfrommax_input_words_amountlengthto1. HowshouldIdefinethelayersanduseLSTMorRNNor…? ThankYou. x=df1[‘Question’].tolist() y=df1[‘Answer’].tolist() max_input_words_amount=0 tok_x=[] foriinrange(len(x)): tokenized_q=nltk.word_tokenize(re.sub(r”[^a-z0-9]+”,”“,x[i].lower())) max_input_words_amount=max(len(tokenized_q),max_input_words_amount) tok_x.append(tokenized_q) vec_x=[] forsentintok_x: sentvec=[ft_cbow_model[w]forwinsent] vec_x.append(sentvec) vec_y=[] forsentiny: sentvec=[ft_cbow_model[sent]] vec_y.append(sentvec) fortok_sentinvec_x: tok_sent[max_input_words_amount-1:]=[] tok_sent.append(ft_cbow_model[‘_E_’]) fortok_sentinvec_x: iflen(tok_sent)pythonfirstnn.py UsingTheanobackend. Traceback(mostrecentcalllast): File“firstnn.py”,line14,in model.add(Dense(12,input_dim=8,activation=’relu’)) File“C:\Users\Roger\Anaconda3\lib\site-packages\keras\engine\sequential.py”,line165,inadd layer(x) File“C:\Users\Roger\Anaconda3\lib\site-packages\keras\engine\base_layer.py”,line431,in__call__ self.build(unpack_singleton(input_shapes)) File“C:\Users\Roger\Anaconda3\lib\site-packages\keras\layers\core.py”,line866,inbuild constraint=self.kernel_constraint) File“C:\Users\Roger\Anaconda3\lib\site-packages\keras\legacy\interfaces.py”,line91,inwrapper returnfunc(*args,**kwargs) File“C:\Users\Roger\Anaconda3\lib\site-packages\keras\engine\base_layer.py”,line249,inadd_weight weight=K.variable(initializer(shape), File“C:\Users\Roger\Anaconda3\lib\site-packages\keras\initializers.py”,line218,in__call__ dtype=dtype,seed=self.seed) File“C:\Users\Roger\Anaconda3\lib\site-packages\keras\backend\theano_backend.py”,line2600,inrandom_uniform returnrng.uniform(shape,low=minval,high=maxval,dtype=dtype) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\sandbox\rng_mrg.py”,line872,inuniform rstates=self.get_substream_rstates(nstreams,dtype) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\configparser.py”,line117,inres returnf(*args,**kwargs) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\sandbox\rng_mrg.py”,line779,inget_substream_rstates multMatVect(rval[0],A1p72,M1,A2p72,M2) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\sandbox\rng_mrg.py”,line62,inmultMatVect [A_sym,s_sym,m_sym,A2_sym,s2_sym,m2_sym],o,profile=False) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\compile\function.py”,line317,infunction output_keys=output_keys) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\compile\pfunc.py”,line486,inpfunc output_keys=output_keys) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\compile\function_module.py”,line1841,inorig_function fn=m.create(defaults) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\compile\function_module.py”,line1715,increate input_storage=input_storage_lists,storage_map=storage_map) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\link.py”,line699,inmake_thunk storage_map=storage_map)[:3] File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\vm.py”,line1091,inmake_all impl=impl)) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\op.py”,line955,inmake_thunk no_recycling) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\op.py”,line858,inmake_c_thunk output_storage=node_output_storage) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”,line1217,inmake_thunk keep_lock=keep_lock) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”,line1157,in__compile__ keep_lock=keep_lock) File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”,line1609,incthunk_factory key=self.cmodule_key() File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”,line1300,incmodule_key c_compiler=self.c_compiler(), File“C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”,line1379,incmodule_key_ np.core.multiarray._get_ndarray_c_version()) AttributeError:(‘Thefollowingerrorhappenedwhilecompilingthenode’,DotModulo(A,s,m,A2,s2,m2),‘\n’,“module‘numpy.core.multiarray’hasnoattribute‘_get_ndarray_c_version'”) Reply Roger May12,2019at1:58am # IfollowedallthestepstosetuptheenvironmentbutwhenIranthecodeIgotanattributeerror‘module‘numpy.core.multiarray’hasnoattribute‘_get_ndarray_c_version” Reply JasonBrownlee May12,2019at6:45am # Perhapstrysearching/postingonstackoverflow? Reply JasonBrownlee May12,2019at6:45am # Ouch,perhapsnumpyisnotinstalledcorrectly? Reply Roger May12,2019at8:34pm # Nonumpy1.16.2doesnotworkwiththeano1.0.3asservedupcurrentlybyAnaconda.Idowngradedtonumpy1.13.0. Reply JasonBrownlee May13,2019at6:46am # ThanksRoger. Reply Aditya May21,2019at5:02pm # HiJason, Thanksforthisamazingexample! WhatIobserveintheexampleisthedatabaseusedispurelynumeric. Mydoubtis: Howcantheexamplebemodifiedtohandlecategoricalinput? WillitworkiftheinputsareOneHotEncoded? Reply JasonBrownlee May22,2019at7:38am # Yes,youcanuseaonehotencodingforourinputcategoricalvariables. Reply Aditya May31,2019at3:41pm # CanyoupleaseprovideagoodreferencepointforOHEinpython? Thanksinadvance!🙂 Reply JasonBrownlee June1,2019at6:09am # Sure: https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/ Reply Aditya June2,2019at3:36am # Ireadthelinkanditwashelpful.Now,Ihaveadoubtspecifictomynetwork. Ihave3categoricalinputwhichhavedifferentsizes.Onehasaround15‘categories’whiletheothertwohave5.SoafterIOneHotencodeeachofthem,doIhavetomaketheirsizessamebypadding?Orit’llworkasitit? JasonBrownlee June2,2019at6:42am # Youcanencodeeachvariableandconcatenatethemtogetherintoonevector. Oryoucanhaveamodelwithoneinputforeachvariableandletthemodelconcatenatethem. Sri June17,2019at7:29pm # Hi, Ifthereisoneindependentvariable(saycountry)withmorethan100labels,howtoresolveit. Ithinkonlyonehotencodingwillnotworkincludingscaling. Isthereanyalternativeforit Reply JasonBrownlee June18,2019at6:37am # Youcantry: –integerencoding –onehotencoding –embedding Testeachandseewhatworksbestforyourspecificdataset. Reply MK June21,2019at7:05pm # Hijason, thanksalotforyourposts,helpedmealot. 1.HowcanIaddconfusionmatrix? 2.HowcanIchangelearningrate? CheersMartin Reply JasonBrownlee June22,2019at6:35am # Addaconfusionmatrix: https://machinelearningmastery.com/custom-metrics-deep-learning-keras-python/ Tunelearningrate: https://machinelearningmastery.com/understand-the-dynamics-of-learning-rate-on-deep-learning-neural-networks/ Reply Guhanpalanivel July1,2019at10:35pm # hijason, Ihavetrainedaneuralnetworkmodelwith6monthsdataanddeployedataremotesite, whenreceivingthenewdataforupcomingmonths, isthereanywaytoautomaticallyupdatethemodelwithadditionofnewtrainingdata? Reply JasonBrownlee July2,2019at7:31am # Yes,perhapstheeasiestwayistorefitthemodelonthenewdataoronallavailabledata. Reply Shubham July5,2019at8:46pm # Hijason, Iwanttoprinttheneuralnetworkscoreasafunctionofoneofthevariable.,howdoidothat? Regards Shubham Reply JasonBrownlee July6,2019at8:35am # Perhapstryalinearactivationunitandamselossfunction? Reply MahaLakshmi July17,2019at7:37pm # Sir,Iamworkingwithsklearn.neural_network.MLPClassifierinPython.nowIwanttogivemyownInitialWeightstoClassifier.howtodothat?pleasehelpme.ThanksinAdvance Reply JasonBrownlee July18,2019at8:25am # Sorry,Idon’thaveanexampleofthis. Perhapstrypostingonstackoverflow? Reply MahaLakshmi July18,2019at4:09pm # Thankyouforyourresponse Reply Ron July24,2019at8:39am # Normalizationofthedataincreasestheaccuracyinthe90’s. https://stackoverflow.com/questions/39525358/neural-network-accuracy-optimization Reply JasonBrownlee July24,2019at2:19pm # Thanksforsharing. Reply Hammad July29,2019at6:12pm # Dearsir, Iwouldliketoapplyabovesharedexampleonarraysproducedby“train_test_split”butitdoesnotwork,asthesearraysarenotintheformofnumpy. Letmegiveyouthedetails,Ihave“XYZ”dataset.Thedatasethasthefollowingspecifications: TotalImages=630 2500featureshasbeenextractedfromeachimage.Eachfeaturehasfloattype. TotalClasses=7 Now,afterprocessingthefeaturefile,Ihavegotresultsinthefollowingvariables: XData:containsfeaturesdataintwodimensionalarrayform(rows:630,columns:2500) YData:containoriginallabelsofclassesinonedimensionalarrayform(rows:630,column:1) So,byusingthefollowingcode,Isplitthedatasetintotrainandtestingdata: fromsklearn.model_selectionimporttrain_test_split x_train,x_test,y_train,y_test=train_test_split(XData,YData,stratify=YData,test_size=0.25) Now,Iwouldliketoapplythedeep-learningexamplessharedonthisblogonmydatasetwhichisnowintheformarrays,andgenerateoutputaspredictionoftestingdataandaccuracy. Canyoupleaseletmeknowaboutit,whichcanworkontheabovearrays? Reply JasonBrownlee July30,2019at6:05am # Yes,theKerasmodelcanoperateonnumpyarraysdirectly. PerhapsIdon’tfollowtheproblemthatyou’rehavingexactly? Reply Hammad July30,2019at6:01pm # Dearsir, Thanks,Iconvertedmyarraysintonumpyformat. Now,Ihavefollowedyourtutorialonmulti-classificationproblem(https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/)andusethefollowingcode: ############################################################ importpandas fromkeras.modelsimportSequential fromkeras.layersimportDense fromkeras.wrappers.scikit_learnimportKerasClassifier fromkeras.utilsimportnp_utils fromsklearn.model_selectionimportcross_val_score fromsklearn.model_selectionimportKFold fromsklearn.preprocessingimportLabelEncoder fromsklearn.pipelineimportPipeline fromsklearn.metricsimportaccuracy_score seed=5 totalclasses=7#ClassLabelsare:‘p1’,‘p2’,‘p3’,‘p4’,‘p5’,‘p6’,‘p7′ totalimages=630 totalfeatures=2500#featuresgeneratedfromimages #Datahasbeenimportedfromfeaturefile,whichresultstwoarraysXDataandYData #XDatacontainsfeaturesdatasetwithoutnumpyarrayform #YDatacontainslabelswithoutnumpyarrayform #encodeclassvaluesasintegers encoder=LabelEncoder() encoder.fit(YData) encoded_Y=encoder.transform(YData) #convertintegerstodummyvariables(i.e.onehotencoded) dummy_y=np_utils.to_categorical(encoded_Y) #definebaselinemodel defbaseline_model(): #createmodel model=Sequential() model.add(Dense(8,input_dim=totalfeatures+1,activation=’relu’)) model.add(Dense(totalclasses,activation=’softmax’)) #Compilemodel model.compile(loss=’categorical_crossentropy’,optimizer=’adam’,metrics= ‘accuracy’]) returnmodel estimator=KerasClassifier(build_fn=baseline_model,nb_epoch=200,batch_size=5,verbose=0) x_train,x_test,y_train,y_test=train_test_split(XData,dummy_y,test_size=0.25,random_state=seed) x_train=np.array(x_train) x_test=np.array(x_test) y_train=np.array(y_train) y_test=np.array(y_test) estimator.fit(x_train,y_train) predictions=estimator.predict(x_test) print(predictions) print(encoder.inverse_transform(predictions)) ######################################################## Thecodegeneratesnosyntaxerror. Now,Iwouldliketoask: 1.DoesIhaveappliedthedeeplearning(NeuralNetworkModel)inarightway? 2.HowcouldIcalculatetheaccuracy,confusionmatrix,andclassification_report? 3.CanyoupleasesuggestwhatothertypeofdeeplearningalgorithmscouldIapplyonthistypeofproblem? Afterapplyingdifferentdeeplearningalgorithm,Iwouldliketocomparetheiraccuraciessuchas,youdidintutorialhttps://machinelearningmastery.com/machine-learning-in-python-step-by-step/,byplottinggraphs. Reply JasonBrownlee July31,2019at6:46am # Sorry,Idon’thavethecapacitytoreviewyourcode. Thispostshowshowtocalculatemetrics: https://machinelearningmastery.com/how-to-calculate-precision-recall-f1-and-more-for-deep-learning-models/ Irecommendtestingasuiteofmethodsinordertodiscoverwhatworksbestforyourspecificdataset: https://machinelearningmastery.com/faq/single-faq/what-algorithm-config-should-i-use Reply Tyson September3,2019at10:07pm # HiJason, Greattutorial.IamnowtryingnewdatasetsfromtheUCIarchive.HoweverIamrunningintoproblemswhenthedataisincomplete.Ratherthananumberthereisa‘?’indicatingthatthedataismissingorunknown.SoIamgetting ValueError:couldnotconvertstringtofloat:‘?’ Isthereawaytoignorethatdata?Iamsuremanydatasetshavethisissuewherepiecesaremissing. Thanksinadvance! Reply JasonBrownlee September4,2019at5:58am # Yes,youcanreplacemissingdatawiththemeanormedianofthevariable–atleastasastartingpoint. Reply Srinu September10,2019at9:07pm # CanyouprovideGUIcodeforthesamedatalikecallingtheANNmodelfromawebsiteorfromandroidapplication. Reply JasonBrownlee September11,2019at5:33am # Idon’tseewhynot. Reply HemanthKumar September20,2019at12:58pm # dearsir ValueError:Errorwhencheckinginput:expectedconv2d_5_inputtohave4dimensions,butgotarraywithshape(250,250,3) Iamgettingthiserror whatstepsIdid original_image->resizedtosameresolution->convertedtonumpyarray->savedandloadedtox_train->fedintonetworkmodel->modal.fit(x_train..gettingthiserror Reply JasonBrownlee September20,2019at1:42pm # Perhapsstartwiththistutorialforimageclassification: https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-to-classify-photos-of-dogs-and-cats/ Reply HemanthKumar September20,2019at3:14pm # thanksforresponsesir🙂 afterthatIamgettinglistindexoutofrangeerroratmodel.fit Reply JasonBrownlee September21,2019at6:43am # I’msorrytohearthat,Ihavesomesuggestionsherethatmayhelp: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply AnthonyTheKoala September26,2019at2:58am # DearDrJason, Thankyouforthistutorial. Ihavebeenplayingaroundwiththenumberoflayersandthenumberofneurons. Inthecurrentcode model=Sequential() model.add(Dense(12,input_dim=8,activation='relu')) model.add(Dense(8,activation='relu')) model.add(Dense(1,activation='sigmoid')) 1234 model=Sequential()model.add(Dense(12,input_dim=8,activation='relu'))model.add(Dense(8,activation='relu'))model.add(Dense(1,activation='sigmoid')) Ihaveplayedaroundwithincreasingthenumbersinthefirstlayer: model=Sequential() model.add(Dense(100,input_dim=8,activation='relu')) model.add(Dense(8,activation='relu')) model.add(Dense(1,activation='sigmoid')) 1234 model=Sequential()model.add(Dense(100,input_dim=8,activation='relu'))model.add(Dense(8,activation='relu'))model.add(Dense(1,activation='sigmoid')) Theresultisthattheaccuracydidn’timprovemuch. Therewasanimprovementintheadditionoflayers. Wheneachlayerhadsayalargenumberofneurons,theaccuracyimproved. Thisisnottheonlyexample,butplayingaroundwiththefollowingcode: model=Sequential() model.add(Dense(200,input_dim=8,activation='relu')) model.add(Dense(800,activation='relu')) model.add(Dense(200,activation='relu')) model.add(Dense(400,activation='relu')) model.add(Dense(200,activation='relu')) model.add(Dense(1,activation='sigmoid')) 1234567 model=Sequential()model.add(Dense(200,input_dim=8,activation='relu'))model.add(Dense(800,activation='relu'))model.add(Dense(200,activation='relu'))model.add(Dense(400,activation='relu'))model.add(Dense(200,activation='relu'))model.add(Dense(1,activation='sigmoid')) Theaccuracyachievedwas91.1% Iaddedtwomorelayers model=Sequential() model.add(Dense(200,input_dim=8,activation='relu')) model.add(Dense(800,activation='relu')) model.add(Dense(200,activation='relu')) model.add(Dense(400,activation='relu')) model.add(Dense(200,activation='relu')) model.add(Dense(400,activation='relu')) model.add(Dense(800,activation='relu')) model.add(Dense(1,activation='sigmoid')) 123456789 model=Sequential()model.add(Dense(200,input_dim=8,activation='relu'))model.add(Dense(800,activation='relu'))model.add(Dense(200,activation='relu'))model.add(Dense(400,activation='relu'))model.add(Dense(200,activation='relu'))model.add(Dense(400,activation='relu'))model.add(Dense(800,activation='relu'))model.add(Dense(1,activation='sigmoid')) Theaccuracydroppedslightlyto88% Fromthesebriefexperiments,increasingthenumberofneuronsasinyourfirstexampledidnotincreaseaccuracy. Howeveraddingmorelayersespeciallywithalargenumberofneuronsdidincreasetheaccuracytoabout91% BUTiftherearetoomanylayersthereisaslightdropinaccuracyto88%. Myquestionisthereawaytoincreasetheaccuracyanyfurtherthan91%? Thankyou, AnthonyofSydney Reply JasonBrownlee September26,2019at6:45am # Ifthisisthepimaindiansdataset,thenthebestaccuracyisabout78%via10-foldcrossvalidation,anythingmoreisprobablyoverfitting. Yes,Ihavetonsoftutorialsondiagnosingissueswithmodelsandliftingperformance,youcanstarthere: https://machinelearningmastery.com/start-here/#better Reply AnthonyTheKoala September26,2019at6:05am # DearDrJason, Furtherexperimentation,Iplayedwiththefollowingcode model=Sequential() model.add(Dense(25,input_dim=8,activation='relu')) model.add(Dense(89,activation='relu')) model.add(Dense(377,activation='relu')) model.add(Dense(233,activation='relu')) model.add(Dense(55,activation='relu')) model.add(Dense(1,activation='sigmoid')) 1234567 model=Sequential()model.add(Dense(25,input_dim=8,activation='relu'))model.add(Dense(89,activation='relu'))model.add(Dense(377,activation='relu'))model.add(Dense(233,activation='relu'))model.add(Dense(55,activation='relu'))model.add(Dense(1,activation='sigmoid')) Iobtainedanaccuracyof95%byplayingaroundwiththenumberofneuronsincreasingthendecreasing. Icannotworkoutasystematicwayofimprovingtheaccuracy. Thankyou, AnthonyofSydney Reply JasonBrownlee September26,2019at6:46am # Haha,yes.Thatisthegreatopenproblemwithneuralnets(nogoodtheoriesforhowtoconfigurethem)andwhywemustuseempiricalmethods. Reply AnthonyTheKoala September26,2019at1:57pm # DearDrJason, thankyouforthosereplies. Yes,itwasthePimaIndiandatasetthatiscoveredinthistutorial. BeforeIindulgeinfurtherreadingson10-foldcrossvalidation,pleasebrieflyanswer: *whatisthemeaningofoverfit. *whyisanaccuracyof96%regardedasoverfit. Todo: Playaroundwithsimplefunctionsandplayaroundwiththistutorialandthenlookatoverfitting: Forexamplesupposewehavex=0,1,2,3,4,5andf(x)=x^2 x:0,1,2,3,4,5 f(x):0,1,4,9,16,25 12 x :0,1,2,3,4,5f(x) :0,1,4,9,16,25 Theaim: *toseeifthereisanaccuratemappingofthefunctionofxandf(x)forx=0..5 *toseewhathappenswhenwepredictforx=6,7,8.Willitbe36,49,64? *weaskifthereissuchathingasoverfittingthemodelexists. Thankyou, AnthonyofSydney Reply JasonBrownlee September27,2019at7:43am # Overfitmeansbetterperformanceonthetrainingsetatthecostofperformingworseonthetestset. Itcanalsomeanbetterperformanceonatest/validationsetatthecostofworseperformanceonnewdata. Iknowfromexperiencethatthelimitonthatdatasetis77-78%afterhavingworkedwithitintutorialsforabout20years. Reply Andrey September29,2019at8:32pm # HiJason, Iseethedataisnotdividedforthatoftrainingandforthetest.Whyisthat?Whatdoespredictionmeaninthiscase? Andrey Reply JasonBrownlee September30,2019at6:07am # Itmightmeanthattheresultisalittleoptimistic. Ididthattokeepthisexampleverysimpleandeasytofollow. Reply AnthonyTheKoala September29,2019at9:00pm # DearDrJason, Itriedtodothesameforadeterministicmodelofxandfxwherex=[0,1,2,3,4,5]andfx=x**2 Iwanttoseehowmachinelearningoperateswithadeterministicfunction. HoweverIamonlygetting16.67%accuracy. Hereisthecodebasedonthethistutorial fromkeras.modelsimportSequential fromkeras.layersimportDense importnumpyasnp #Aimistoseehowadeterministicfunctionwilloperateusingmachinelearning #Inyear7algebrawehavexandy.yisknownasf(x). #Sohereweaimtohaveastructoreof[indepvar,depvar] #thatis[x,fx] #Makinga2D(like)list x=[iforiinrange(6)];#havealistofx=[0,1,2,3,4,5] #x=np.array(x) fx=[x**2forxinx];#havealistoffx=[0,1,4,9,16,25] #fx=np.array(fx) model=Sequential() model.add(Dense(100,input_dim=1,activation='relu')) model.add(Dense(200,activation='relu')) model.add(Dense(1,activation='relu')) model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) model.fit(x,fx,epochs=150,batch_size=2,verbose=0) _,accuracy=model.evaluate(x,fx) print('Accuracy:%.2f'%(accuracy*100)) 123456789101112131415161718192021222324252627 fromkeras.modelsimportSequentialfromkeras.layersimportDenseimportnumpyasnp #Aimistoseehowadeterministicfunctionwilloperateusingmachinelearning#Inyear7algebrawehavexandy.yisknownasf(x). #Sohereweaimtohaveastructoreof[indepvar,depvar]#thatis[x,fx] #Makinga2D(like)listx=[iforiinrange(6)];#havealistofx=[0,1,2,3,4,5]#x=np.array(x) fx=[x**2forxinx];#havealistoffx=[0,1,4,9,16,25]#fx=np.array(fx) model=Sequential()model.add(Dense(100,input_dim=1,activation='relu'))model.add(Dense(200,activation='relu'))model.add(Dense(1,activation='relu')) model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])model.fit(x,fx,epochs=150,batch_size=2,verbose=0)_,accuracy=model.evaluate(x,fx)print('Accuracy:%.2f'%(accuracy*100)) Weknowthatfx=x**2ispredictable.WhatdoIneedtodo. Thankyou, AnthonyofSydney Reply JasonBrownlee September30,2019at6:10am # Perhapsyouneedhundredsofthousandsofexamples? Andperhapsthemodelwillneedtobetunedforyourproblem,e.g.perhapsusingmselossandalinearactivationfunctionintheoutputlayerbecauseitisaregressionproblem. Reply AnthonyTheKoala October1,2019at5:15am # DearDrJason, Itriedwithmse-lossandlinearactivationfunctionandstillonlyobtained1%accuracy. fromkeras.modelsimportSequential fromkeras.layersimportDense importnumpyasnp #Aimistoseehowadeterministicfunctionwilloperateusingmachinelearning #Inyear7algebrawehavexandy.yisknownasf(x). x=[iforiinrange(100)];#havealistofx=[0,1,2,3,4,5] x=np.array(x) fx=[x**2forxinx];#havealistoffx=[0,1,4,9,16,25] fx=np.array(fx) model=Sequential() model.add(Dense(12,input_dim=1,activation='linear')) model.add(Dense(33,activation='linear')) model.add(Dense(1,activation='linear')) #model.compile(loss='mean_squared_error',optimizer='softmax',metrics=['accuracy']) model.compile(loss='mean_squared_error',optimizer='sgd') #model.compile(loss='mean_squared_error') model.fit(x,fx,epochs=10,batch_size=1,verbose=0) _,accuracy=model.evaluate(x,fx) print('Accuracy:%.2f'%(accuracy*100)) 1234567891011121314151617181920212223242526 fromkeras.modelsimportSequentialfromkeras.layersimportDenseimportnumpyasnp #Aimistoseehowadeterministicfunctionwilloperateusingmachinelearning#Inyear7algebrawehavexandy.yisknownasf(x). x=[iforiinrange(100)];#havealistofx=[0,1,2,3,4,5]x=np.array(x) fx=[x**2forxinx];#havealistoffx=[0,1,4,9,16,25]fx=np.array(fx) model=Sequential()model.add(Dense(12,input_dim=1,activation='linear'))model.add(Dense(33,activation='linear'))model.add(Dense(1,activation='linear')) #model.compile(loss='mean_squared_error',optimizer='softmax',metrics=['accuracy'])model.compile(loss='mean_squared_error',optimizer='sgd')#model.compile(loss='mean_squared_error') model.fit(x,fx,epochs=10,batch_size=1,verbose=0)_,accuracy=model.evaluate(x,fx)print('Accuracy:%.2f'%(accuracy*100)) HoweverIgetthis: 32/100[========>.....................]-ETA:0s 100/100[==============================]-0s312us/step Traceback(mostrecentcalllast): File"C:\Python36\deterministicII.py",line25,in _,accuracy=model.evaluate(x,fx) TypeError:'float'objectisnotiterable 123456 32/100[========>.....................]-ETA:0s100/100[==============================]-0s312us/stepTraceback(mostrecentcalllast): File"C:\Python36\deterministicII.py",line25,in _,accuracy=model.evaluate(x,fx)TypeError:'float'objectisnotiterable Iwanttomapadeterministicfunctiontoseeifmachinelearningwillworkoutf(x)withouttheformula. Reply JasonBrownlee October1,2019at7:00am # Accuracyisnotavalidmetricforregressionproblems: https://machinelearningmastery.com/faq/single-faq/how-do-i-calculate-accuracy-for-regression Youareveryclose! Also,tryamuchlargerdatasetofexamples.Hundredsorthousands. Reply AnthonyTheKoala October1,2019at10:18am # DearDrJason, Iremovedthemodel.evaluatefromtheprogram.BUTstillIhavenotgotasatisfactorymatchoftheexpectedandactualvalues. fromkeras.modelsimportSequential fromkeras.layersimportDense importnumpyasnp #Aimistoseehowadeterministicfunctionwilloperateusingmachinelearning #Inyear7algebrawehavexandy.yisknownasf(x). x=[iforiinrange(100)];#havealistofx=[0,1,2,3,4,5] x=np.array(x) fx=[x**2forxinx];#havealistoffx=[0,1,4,9,16,25] fx=np.array(fx) model=Sequential() model.add(Dense(100,input_dim=1,activation='linear')) model.add(Dense(100,activation='linear')) model.add(Dense(1,activation='linear')) model.compile(loss='mean_squared_error',optimizer='adam') model.fit(x,fx,epochs=1000,batch_size=1000,verbose=0) #Removingthemodel.evaluatecode predictions=model.predict_classes(x) foriinrange(6): print('%s=>%d(expected%d)'%(x[i],predictions[i],fx[i])) 1234567891011121314151617181920212223242526 fromkeras.modelsimportSequentialfromkeras.layersimportDenseimportnumpyasnp #Aimistoseehowadeterministicfunctionwilloperateusingmachinelearning#Inyear7algebrawehavexandy.yisknownasf(x). x=[i foriinrange(100)];#havealistofx=[0,1,2,3,4,5]x=np.array(x) fx=[x**2 forxinx];#havealistoffx=[0,1,4,9,16,25]fx=np.array(fx) model=Sequential()model.add(Dense(100,input_dim=1,activation='linear'))model.add(Dense(100,activation='linear'))model.add(Dense(1,activation='linear'))model.compile(loss='mean_squared_error',optimizer='adam') model.fit(x,fx,epochs=1000,batch_size=1000,verbose=0) #Removingthemodel.evaluatecode predictions=model.predict_classes(x)foriinrange(6): print('%s=>%d(expected%d)'%(x[i],predictions[i],fx[i])) Output 0=>0(expected0) 1=>0(expected1) 2=>0(expected4) 3=>0(expected9) 4=>1(expected16) 5=>1(expected25) 123456 0=>0(expected0)1=>0(expected1)2=>0(expected4)3=>0(expected9)4=>1(expected16)5=>1(expected25) Notyetgettingamatchoftheexpectedandtheactualvalues Thankyou, AnthonyofSydney Reply JasonBrownlee October1,2019at2:17pm # Perhapsthemodelarchitecture(layersandnodes)needstuning? Perhapsthelearningrateneedstuning? Perhapsyouneedmoretrainingexamples? Perhapsyouneedmoreorfewerepochs? … Moreideashere: https://machinelearningmastery.com/start-here/#better Reply AnthonyTheKoala October2,2019at7:41am # DearDrJason, Icannotfindasystematicwaytofindawayforamachinelearningalgorithmtouseittocomputeadeterministicequationsuchasy=f(x)wheref(x)=x**2. Iamstillhavingtrouble.Iwillbepostingthisonthepage.Essentiallyis(i)adding/droppinglayers,(ii)adjustingthenumberofepochs,(iii)adjustingthebatch_size.ButIhaven’tcomecloseyet. Alsousingthefunctionmodel.predictratherthanmodel.predict_classes. Hereistheprogramwithmostofthecommentedoutlinesdeleted. fromkeras.modelsimportSequential fromkeras.layersimportDense importnumpyasnp #Aimistoseehowadeterministicfunctionwilloperateusingmachinelearning #Inyear7algebrawehavexandy.yisknownasf(x).Herey=f(x)=x**2 x=[iforiinrange(100)];#havealistofx=[0,1,2,3,4,5,.....,99] x=np.array(x) fx=[x**2forxinx];#havealistoffx=x**2=[0,1,4,9,16,25,...,9801] fx=np.array(fx) model=Sequential() model.add(Dense(55,input_dim=1,activation='linear')) model.add(Dense(34,activation='linear')) model.add(Dense(21,activation='linear')) model.add(Dense(13,activation='linear')) model.add(Dense(1,activation='linear')) model.compile(loss='mean_squared_error',optimizer='adam') model.fit(x,fx,epochs=89,batch_size=144,verbose=0) predictions=model.predict(x);#Thisseemstoworkinsteadofmodel.predict_classes print("x,predicted,expected") foriinrange(6): print('%s=>%d(expected%d)'%(x[i],predictions[i],fx[i])) 1234567891011121314151617181920212223242526272829 fromkeras.modelsimportSequentialfromkeras.layersimportDenseimportnumpyasnp #Aimistoseehowadeterministicfunctionwilloperateusingmachinelearning#Inyear7algebrawehavexandy.yisknownasf(x).Herey=f(x)=x**2 x=[i foriinrange(100)];#havealistofx=[0,1,2,3,4,5,.....,99]x=np.array(x) fx=[x**2 forxinx];#havealistoffx=x**2=[0,1,4,9,16,25,...,9801]fx=np.array(fx) model=Sequential()model.add(Dense(55,input_dim=1,activation='linear'))model.add(Dense(34,activation='linear'))model.add(Dense(21,activation='linear'))model.add(Dense(13,activation='linear'))model.add(Dense(1,activation='linear')) model.compile(loss='mean_squared_error',optimizer='adam') model.fit(x,fx,epochs=89,batch_size=144,verbose=0) predictions=model.predict(x);#Thisseemstoworkinsteadofmodel.predict_classes print("x,predicted,expected")foriinrange(6): print('%s=>%d(expected%d)'%(x[i],predictions[i],fx[i])) Theoutputis: x,predicted,expected 0=>29(expected0) 1=>110(expected1) 2=>191(expected4) 3=>272(expected9) 4=>353(expected16) 5=>434(expected25) 1234567 x,predicted,expected0=>29(expected0)1=>110(expected1)2=>191(expected4)3=>272(expected9)4=>353(expected16)5=>434(expected25) NomatterhowmuchIadjustthenumberofneuronsperlayer,thenumberoflayers,thenoofepochsandthebatchsize,the“predicted”appearslikeanarithmeticprogression,notageometricprogression. Notethetermstn+1–tnis81forallthepredictedvaluesinthemachinelearningmodel. BUTweknowthatthedifferencebetweensuccessivetermsiny=f(x)isnotthesame. Forexample,innonlinearrelationsuchasf(x)=x**2,f(x)=0,1,2,4,9,16,25,36,thedifferencebetweenthetermsis:1,1,2,5,7,9,11,thatistn+1–tn!=tn+2–tn+1. Sostillhavingtroubleworkingouthowtogetamachinelearningalgorithmevaluatef(x)withouttheformula. Reply JasonBrownlee October2,2019at8:15am # Hereisthesolution,hopeithelps #fitanmlponxvsx^2 fromsklearn.preprocessingimportMinMaxScaler fromkeras.modelsimportSequential fromkeras.layersimportDense fromnumpyimportasarray frommatplotlibimportpyplot #definedata x=asarray([iforiinrange(1000)]) y=asarray([a**2forainx]) #reshapeintorowsandcols x=x.reshape((len(x),1)) y=y.reshape((len(y),1)) #scaledata x_s=MinMaxScaler() x=x_s.fit_transform(x) y_s=MinMaxScaler() y=y_s.fit_transform(y) #fitamodel model=Sequential() model.add(Dense(10,input_dim=1,activation='relu')) model.add(Dense(1)) model.compile(loss='mse',optimizer='adam') model.fit(x,y,epochs=150,batch_size=10,verbose=0) mse=model.evaluate(x,y,verbose=0) print(mse) #predict yhat=model.predict(x) #plotrealvspredicted pyplot.plot(x,y,label='y') pyplot.plot(x,yhat,label='yhat') pyplot.legend() pyplot.show() 1234567891011121314151617181920212223242526272829303132 #fitanmlponxvsx^2fromsklearn.preprocessingimportMinMaxScalerfromkeras.modelsimportSequentialfromkeras.layersimportDensefromnumpyimportasarrayfrommatplotlibimportpyplot#definedatax=asarray([iforiinrange(1000)])y=asarray([a**2forainx])#reshapeintorowsandcolsx=x.reshape((len(x),1))y=y.reshape((len(y),1))#scaledatax_s=MinMaxScaler()x=x_s.fit_transform(x)y_s=MinMaxScaler()y=y_s.fit_transform(y)#fitamodelmodel=Sequential()model.add(Dense(10,input_dim=1,activation='relu'))model.add(Dense(1))model.compile(loss='mse',optimizer='adam')model.fit(x,y,epochs=150,batch_size=10,verbose=0)mse=model.evaluate(x,y,verbose=0)print(mse)#predictyhat=model.predict(x)#plotrealvspredictedpyplot.plot(x,y,label='y')pyplot.plot(x,yhat,label='yhat')pyplot.legend()pyplot.show() Iguessyoucouldalsodoaninverse_transform()onthepredictedvaluestogetbacktooriginalunits. Reply AnthonyTheKoala October2,2019at9:05am # DearDrJason, Thankyouverymuchforyourreply.Igotanmseintheorderof3x10**-6. Despitethis,Iwillbestudyingtheprogramandlearnmyselfabout(i)theMinMaxScalerandwhyweuseit,(ii)fit_transform(y)and(iii)onehiddenlayerof10neurons,and(iii)Iwillstillhavetolearnaboutthechoiceofactivationfunctionandlossfunctions.Thekeraswebsitehasasectiononlossfunctionsathttps://keras.io/losses/buthavingalookatthePython“IDLE”program,alookatfromkerasimportlosses,therearemanymorelossfunctionswhicharenecessarytocompileamodel. Inaddition,thepredictedvalueswillhavetobere-computedtoitsunscaledvalues.SoIwillalsolookup‘rescaling’. Thankyouagain, Anthony,SydneyNSW Reply JasonBrownlee October2,2019at10:10am # Yes,youcanuseinverse_transformtounscalethepredictions,asImentioned. Reply AnthonyTheKoala October3,2019at6:26am # DearDrJason, Iknowhowtousetheinverse_transformfunction: FirstapplytheMinMaxScalertoscaleto0to1 x_s=MinMaxScaler() x=x_s.fit_transform(x) y_s=MinMaxScaler() y=y_s.fit_transform(y) 1234 x_s=MinMaxScaler()x=x_s.fit_transform(x)y_s=MinMaxScaler()y=y_s.fit_transform(y) Ifwewanttoreconstitutexandy,itissimpleto: x_original=x_s.inverse_transform(x);#wherexwastransformed/scaled y_original=y_s.inverse_transform(y);#whereywastransformed/scaled 12 x_original=x_s.inverse_transform(x);#wherexwastransformed/scaledy_original=y_s.inverse_transform(y);#whereywastransformed/scaled x_sandy_shastheminandmaxvaluesstoredoftheoriginalpre-transformeddata. BUThowdoyoutransformyhattoitsoriginalscalewhenitwasnotsubjecttotheinverse_transformfunction. IfIreliedonthey_s.inverse_transform(yhat),whereyougetthis: yhat_restored=y_s.inverse_transform(yhat);#usingthevaluesofyminandymaxoforiginaldata yhat_restored[0:10] array([[6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43]],dtype=float32) 123456789101112 yhat_restored=y_s.inverse_transform(yhat);#usingthevaluesofyminandymaxoforiginaldatayhat_restored[0:10]array([[6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43]],dtype=float32) Iwas‘hoping’forsomethingclosetotheoriginal: >>>y_restored=y_s.inverse_transform(y) >>>y_restored[0:10] array([[0.], [1.], [4.], [9.], [16.], [25.], [36.], [49.], [64.], [81.]]) 123456789101112 >>>y_restored=y_s.inverse_transform(y)>>>y_restored[0:10]array([[0.], [1.], [4.], [9.], [16.], [25.], [36.], [49.], [64.], [81.]]) BUTyhatdoesnotusetheMinMaxScaleratthestart. DoIhavetorewritemyownfunction? Thanks, AnthonyofSydneyNSW Reply JasonBrownlee October3,2019at6:54am # Themodelpredictsscaledvalues,applytheinversetransformonyhatdirectly. Reply AnthonyTheKoala October3,2019at2:39pm # DearDrJason, Ididthatapplytheinversetransformofyhatdirectly,BUTGOTthese Cutdownversionofcode y_s=MinMaxScaler() y=y_s.fit_transform(y);#y_sstorestheminandmaxvaluesaccordingtothesklearndoc #Notetheaboveisfory.WEDON'TKNOWyhat(min)&yhat(max yhat=model.predict(x);#wehavethescaledestimate. x_original=x_s.inverse_transform(x);#thisprintedokay #Printoutofyhattransformed #Calculateyhatscaledusingtheminandmaxvaluesoff(x)=y yhat_restored=y_s.inverse_transform(yhat) #Printyhat print(yhat_restored[0:10]) array([[6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43]],dtype=float32) 123456789101112131415161718192021222324252627 y_s=MinMaxScaler()y=y_s.fit_transform(y);#y_sstorestheminandmaxvaluesaccordingtothesklearndoc #Notetheaboveisfory.WEDON'TKNOWyhat(min)&yhat(max yhat=model.predict(x); #wehavethescaledestimate. x_original=x_s.inverse_transform(x);#thisprintedokay #Printoutofyhattransformed#Calculateyhatscaledusingtheminandmaxvaluesoff(x)=y yhat_restored=y_s.inverse_transform(yhat) #Printyhatprint(yhat_restored[0:10])array([[6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43], [6838.43]],dtype=float32) Don’tunderstandhowtogetaninversetransformofyhatwhenIdon’tknowthe‘untransformed’valuebecauseIhavenotestimatedit. Thankyou, AnthonyofSydney Reply JasonBrownlee October4,2019at5:39am # Youcaninversetransformyandyhatandplotboth. Reply AnthonyTheKoala October4,2019at3:20am # DearDrJason, Itrieditagaintoillustratethatdespitethepredictedfittingaparabolaforscaledpredictedandexpectedvaluesoff(x)theresultingvalueswhen‘unscaled’backtotheoriginaldoesseemsquiteabsurd. Code–relevant #plotrealvspredicted pyplot.plot(x,y,label='y') pyplot.plot(x,yhat,label='yhat') pyplot.legend() print("Thegraphofthe(x,predictedf(x)and(x,f(x)isonaseparatewindow") pyplot.show() y_predicted=y_s.inverse_transform(yhat) y_expected=y_s.inverse_transform(y) x_original=x_s.inverse_transform(x) #print(y_predicted[0:10,].tolist(),x_original[0:10,].tolist()) print("Printingthefirst10,predicted,expected,andx") foriinrange(10): print(y_predicted[i],y_expected[i],x_original[i]) print("let'strysomeotherarbitrarysection,say10:20") #print(y_predicted[9:21,].tolist(),y_predicted[9:21,].tolist(),x_original[9:21,].tolist()) print("printing10thto20th,predicted,expected,andx") foriinrange(10): print(y_predicted[i+10],y_expected[i+10],x_original[i+10]) 123456789101112131415161718192021 #plotrealvspredictedpyplot.plot(x,y,label='y')pyplot.plot(x,yhat,label='yhat')pyplot.legend()print("Thegraphofthe(x,predictedf(x)and(x,f(x)isonaseparatewindow")pyplot.show()y_predicted=y_s.inverse_transform(yhat)y_expected=y_s.inverse_transform(y) x_original=x_s.inverse_transform(x)#print(y_predicted[0:10,].tolist(),x_original[0:10,].tolist())print("Printingthefirst10,predicted,expected,andx")foriinrange(10): print(y_predicted[i],y_expected[i],x_original[i]) print("let'strysomeotherarbitrarysection,say10:20")#print(y_predicted[9:21,].tolist(),y_predicted[9:21,].tolist(),x_original[9:21,].tolist())print("printing10thto20th,predicted,expected,andx")foriinrange(10): print(y_predicted[i+10],y_expected[i+10],x_original[i+10]) Theresultingoutput: 5.472537487406726e-06 Printingthefirst10,predicted,expected,andx [1030.0833][0.][0.] [1030.0833][1.][1.] [1030.0833][4.][2.] [1030.0833][9.][3.] [1030.0833][16.][4.] [1030.0833][25.][5.] [1030.0833][36.][6.] [1030.0833][49.][7.] [1030.0833][64.][8.] [1030.0833][81.][9.] let'strysomeotherarbitrarysection,say10:20 printing10thto20th,predicted,expected,andx [1030.0833][100.][10.] [1030.0833][121.][11.] [1030.0833][144.][12.] [1030.0833][169.][13.] [1030.0833][196.][14.] [1030.0833][225.][15.] [1030.0833][256.][16.] [1030.0833][289.][17.] [1030.0833][324.][18.] [1030.0833][361.][19.] 123456789101112131415161718192021222324 5.472537487406726e-06Printingthefirst10,predicted,expected,andx[1030.0833][0.][0.][1030.0833][1.][1.][1030.0833][4.][2.][1030.0833][9.][3.][1030.0833][16.][4.][1030.0833][25.][5.][1030.0833][36.][6.][1030.0833][49.][7.][1030.0833][64.][8.][1030.0833][81.][9.]let'strysomeotherarbitrarysection,say10:20printing10thto20th,predicted,expected,andx[1030.0833][100.][10.][1030.0833][121.][11.][1030.0833][144.][12.][1030.0833][169.][13.][1030.0833][196.][14.][1030.0833][225.][15.][1030.0833][256.][16.][1030.0833][289.][17.][1030.0833][324.][18.][1030.0833][361.][19.] WhenIplotted(x,yhat)and(x,f(x)),theplotwasasexpected.BUTwhenIrescaledtheyhatback,allthevaluesofunscaledyhatwere1030.0833whichisquiteodd. Why? Thankyou, AnthonyofSydneyNSW Reply AnthonyTheKoala October4,2019at3:31am # DearDrJason, Iprintedtheyhat,andtheywereallthesame. Thisisdespitethattheplotofthescaledvalues(x,yhat)lookedlikeaparabola Note:thisispriortoscaling. #plotrealvspredicted pyplot.plot(x,y,label='y') pyplot.plot(x,yhat,label='yhat') pyplot.legend() print("thegraphisprintedonanotherwindow") pyplot.show() print("Printingtheoutputofthescaledvaluesofyhat,f(x)andx") print("printingthefirst10") foriinrange(10): print(yhat[i],y[i],x[i]) print("printingthe10thto20th") foriinrange(10): print(yhat[i+10],y[i+10],x[i+10]) 1234567891011121314 #plotrealvspredictedpyplot.plot(x,y,label='y')pyplot.plot(x,yhat,label='yhat')pyplot.legend()print("thegraphisprintedonanotherwindow")pyplot.show() print("Printingtheoutputofthescaledvaluesofyhat,f(x)andx")print("printingthefirst10")foriinrange(10): print(yhat[i],y[i],x[i])print("printingthe10thto20th")foriinrange(10): print(yhat[i+10],y[i+10],x[i+10]) Yetdespitetheexpectedplotsofscaledvalues(x,yhat),and(x,y),yhat’svaluesarethesame Printingtheoutputofthescaledvalues printingthefirst10 [0.00117336][0.][0.] [0.00117336][1.002003e-06][0.001001] [0.00117336][4.00801202e-06][0.002002] [0.00117336][9.01802704e-06][0.003003] [0.00117336][1.60320481e-05][0.004004] [0.00117336][2.50500751e-05][0.00500501] [0.00117336][3.60721081e-05][0.00600601] [0.00117336][4.90981472e-05][0.00700701] [0.00117336][6.41281923e-05][0.00800801] [0.00117336][8.11622433e-05][0.00900901] printingthe10thto20th [0.00117336][0.0001002][0.01001001] [0.00117336][0.00012124][0.01101101] [0.00117336][0.00014429][0.01201201] [0.00117336][0.00016934][0.01301301] [0.00117336][0.00019639][0.01401401] [0.00117336][0.00022545][0.01501502] [0.00117336][0.00025651][0.01601602] [0.00117336][0.00028958][0.01701702] [0.00117336][0.00032465][0.01801802] [0.00117336][0.00036172][0.01901902] 1234567891011121314151617181920212223 Printingtheoutputofthescaledvaluesprintingthefirst10[0.00117336][0.][0.][0.00117336][1.002003e-06][0.001001][0.00117336][4.00801202e-06][0.002002][0.00117336][9.01802704e-06][0.003003][0.00117336][1.60320481e-05][0.004004][0.00117336][2.50500751e-05][0.00500501][0.00117336][3.60721081e-05][0.00600601][0.00117336][4.90981472e-05][0.00700701][0.00117336][6.41281923e-05][0.00800801][0.00117336][8.11622433e-05][0.00900901]printingthe10thto20th[0.00117336][0.0001002][0.01001001][0.00117336][0.00012124][0.01101101][0.00117336][0.00014429][0.01201201][0.00117336][0.00016934][0.01301301][0.00117336][0.00019639][0.01401401][0.00117336][0.00022545][0.01501502][0.00117336][0.00025651][0.01601602][0.00117336][0.00028958][0.01701702][0.00117336][0.00032465][0.01801802][0.00117336][0.00036172][0.01901902] Idon’tgetit.Youwouldexpectasimilarityofyhatandf(x). Iwouldappreciatearesponse Thankyou, AnthonyofSydney Reply JasonBrownlee October4,2019at5:49am # Sorry,Idon’thavethecapacitytodebugyourexamplesfurther.Ihopethatyoucanunderstand. Reply AnthonyTheKoala October4,2019at6:36am # DearDrJason, Iaskedthequestionathttps://datascience.stackexchange.com/questions/61223/reconstituting-estimated-predicted-values-to-original-scale-from-minmaxscalerandhopethatthereisananswer. Thanks AnthonyOfSydney Reply JasonBrownlee October4,2019at8:35am # Hereisthesolution #fitanmlponxvsx^2 fromsklearn.preprocessingimportMinMaxScaler fromkeras.modelsimportSequential fromkeras.layersimportDense fromnumpyimportasarray frommatplotlibimportpyplot #definedata x=asarray([iforiinrange(1000)]) y=asarray([a**2forainx]) #reshapeintorowsandcols x=x.reshape((len(x),1)) y=y.reshape((len(y),1)) #scaledata x_s=MinMaxScaler() x=x_s.fit_transform(x) y_s=MinMaxScaler() y=y_s.fit_transform(y) #fitamodel model=Sequential() model.add(Dense(10,input_dim=1,activation='relu')) model.add(Dense(1)) model.compile(loss='mse',optimizer='adam') model.fit(x,y,epochs=150,batch_size=10,verbose=0) mse=model.evaluate(x,y,verbose=0) print(mse) #predict yhat=model.predict(x) #inversetransforms x=x_s.inverse_transform(x) y=y_s.inverse_transform(y) yhat=y_s.inverse_transform(yhat) #plotrealvspredicted pyplot.plot(x,y,label='y') pyplot.plot(x,yhat,label='yhat') pyplot.legend() pyplot.show() 123456789101112131415161718192021222324252627282930313233343536 #fitanmlponxvsx^2fromsklearn.preprocessingimportMinMaxScalerfromkeras.modelsimportSequentialfromkeras.layersimportDensefromnumpyimportasarrayfrommatplotlibimportpyplot#definedatax=asarray([iforiinrange(1000)])y=asarray([a**2forainx])#reshapeintorowsandcolsx=x.reshape((len(x),1))y=y.reshape((len(y),1))#scaledatax_s=MinMaxScaler()x=x_s.fit_transform(x)y_s=MinMaxScaler()y=y_s.fit_transform(y)#fitamodelmodel=Sequential()model.add(Dense(10,input_dim=1,activation='relu'))model.add(Dense(1))model.compile(loss='mse',optimizer='adam')model.fit(x,y,epochs=150,batch_size=10,verbose=0)mse=model.evaluate(x,y,verbose=0)print(mse)#predictyhat=model.predict(x)#inversetransformsx=x_s.inverse_transform(x)y=y_s.inverse_transform(y)yhat=y_s.inverse_transform(yhat)#plotrealvspredictedpyplot.plot(x,y,label='y')pyplot.plot(x,yhat,label='yhat')pyplot.legend()pyplot.show() Thethreemissinglineswere: #inversetransforms x=x_s.inverse_transform(x) y=y_s.inverse_transform(y) yhat=y_s.inverse_transform(yhat) 1234 #inversetransformsx=x_s.inverse_transform(x)y=y_s.inverse_transform(y)yhat=y_s.inverse_transform(yhat) Reply AnthonyTheKoala October4,2019at9:59am # DearDrJason, IamcomingtotheconclusionthattheremustbeabugNOTinyoursolutionandneitherinmysolution.Ithinkitiscomingfromabuginthelowerimplementationofthelanguage. Iprintedthescaledversionofyhat,f(x)actualandxandgotthis. NOTEthevaluesarethesameforthescaledversionofyhat. Thatis: model.fit(x,y,epochs=277,batch_size=200,verbose=0) mse=model.evaluate(x,y,verbose=0) print("thevalueofthemse") print(mse) #predict yhat=model.predict(x) 123456 model.fit(x,y,epochs=277,batch_size=200,verbose=0)mse=model.evaluate(x,y,verbose=0)print("thevalueofthemse")print(mse)#predictyhat=model.predict(x) DESPITEthesuccessfulplotof(x,yhat)and(x,f(x), theresultingoutputofthefirst10ofthescaledoutputofyhatisthesame, ThatiswewouldgetaFLATLINEifweplotted(x,yhat),BUTTHEPLOTWASAPARABOLA. [0.00161531][0.][0.] [0.00161531][1.002003e-06][0.001001] [0.00161531][4.00801202e-06][0.002002] [0.00161531][9.01802704e-06][0.003003] [0.00161531][1.60320481e-05][0.004004] [0.00161531][2.50500751e-05][0.00500501] [0.00161531][3.60721081e-05][0.00600601] [0.00161531][4.90981472e-05][0.00700701] [0.00161531][6.41281923e-05][0.00800801] [0.00161531][8.11622433e-05][0.00900901] 12345678910 [0.00161531][0.][0.][0.00161531][1.002003e-06][0.001001][0.00161531][4.00801202e-06][0.002002][0.00161531][9.01802704e-06][0.003003][0.00161531][1.60320481e-05][0.004004][0.00161531][2.50500751e-05][0.00500501][0.00161531][3.60721081e-05][0.00600601][0.00161531][4.90981472e-05][0.00700701][0.00161531][6.41281923e-05][0.00800801][0.00161531][8.11622433e-05][0.00900901] Whenwedidthefollowingtransforms: x=x_s.inverse_transform(x) y=y_s.inverse_transform(y) yhat=y_s.inverse_transform(yhat) 123 x=x_s.inverse_transform(x)y=y_s.inverse_transform(y)yhat=y_s.inverse_transform(yhat) WESTILLGOTTHESAMEFAULTFORTHEUNSCALEDVALUESofyhat.The2ndcolumnisf(x)andthirdcolumnisx. [1612.0857][0.][0.] [1612.0857][1.][1.] [1612.0857][4.][2.] [1612.0857][9.][3.] [1612.0857][16.][4.] [1612.0857][25.][5.] [1612.0857][36.][6.] [1612.0857][49.][7.] [1612.0857][64.][8.] [1612.0857][81.][9.] 12345678910 [1612.0857][0.][0.][1612.0857][1.][1.][1612.0857][4.][2.][1612.0857][9.][3.][1612.0857][16.][4.][1612.0857][25.][5.][1612.0857][36.][6.][1612.0857][49.][7.][1612.0857][64.][8.][1612.0857][81.][9.] Conclusion:Itisnotaprogrammaticalbugineitheryoursolutionormysolution.Ibelieveitmaybealowerimplementationproblem. WhyamI‘persistent’inthismatter:becauseincaseIhavemorecomplexmodelsIwanttoseethepredicted/yhatvaluesthatarere-scaled. Idon’tknowiftherearepeopleatstackexchangewhomayhaveaninsight. Iappreciateyourtime,manyblessingstoyou, AnthonyofSydney Reply JasonBrownlee October6,2019at8:05am # Ibelieveiscorrect,giventhatitisanexponential,themodelhasdecidedthatitcangiveupcorrectnessatthelowendforcorrectnessatthehighend–giventhereductioninMSE. Considerchangingthenumberofexamplesfrom1Kto100,thenreviewall100valuesmanually–you’llseewhatImean. Allofthisisagoodexercise,welldone. Reply AnthonyTheKoala October13,2019at10:57pm # DearDrJason, Ididthisproblemagainandgotverygoodresults! IcannotexplainwhyIgotaccurateresults,whenIexpectedtogetaccurateresults,BUTtheyarecertainlyanimprovement. TherescaledoriginalandfittedvaluesproducedanRMSof0.0. Hereisthecodewithvariablenameschangedslightly. fromsklearn.preprocessingimportMinMaxScaler fromkeras.modelsimportSequential fromkeras.layersimportDense fromnumpyimportsqrt frommatplotlibimportpyplot x=asarray([iforiinrange(100)]) y=asarray([i**2foriinx]) x=x.reshape((len(x),1)) y=y.reshape((len(y),1)) x_s=MinMaxScaler() xscaled=x_s.fit_transform(x) y_s=MinMaxScaler() yscaled=y_s.fit_transform(y) model=Sequential() model.add(Dense(100,input_dim=1,activation='relu') model.add(Dense(1)) model.compile(loss='mse',optimizer='adam') model.fit(xscaled,yscaled,epochs=150,batch_size=10,verbose=0) mse=model.evaluate(xscaled,yscaled,verbose=0) mse 2.9744908551947447e-05 yhat=model.predict(x) yhat_original=y_s.inverse_transform(yscaled) #Firstfiveelementsofpredictedvalues yhat_original[:5].T array([[0.,1.,4.,9.,16.]]) #Firstfiveelementsoforiginaly y[:5].T array([[0,1,4,9,16]]) #Lastfiveelementsoftheoriginalseries. y[-5:].T array([[9025,9216,9409,9604,9801]]) #Lastfiveelementsofpredictedvalues yhat_original[-5:].T array([[9025.,9216.,9409.,9604.,9801.]]) #NowdeterminingtheRMSofthepredictedandoriginalvaluesofy cum_sum=0 foriinrange(len(yhat_original)): cum_sum+=(yoriginal[i]-yhat_original[i])**2/len(yhat_original) rms=sqrt(cum_sum) rms[0] 0.0 #Plottingtherescaledoriginalandrescaledyhat pyplot.plot(xoriginal,yoriginal,label='y') pyplot.plot(xoriginal,yhat_original,label='fitted') pyplot.legend() pyplot.show() 1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556 fromsklearn.preprocessingimportMinMaxScalerfromkeras.modelsimportSequentialfromkeras.layersimportDensefromnumpyimportsqrtfrommatplotlibimportpyplot x=asarray([iforiinrange(100)])y=asarray([i**2foriinx])x=x.reshape((len(x),1))y=y.reshape((len(y),1)) x_s=MinMaxScaler()xscaled=x_s.fit_transform(x)y_s=MinMaxScaler()yscaled=y_s.fit_transform(y) model=Sequential()model.add(Dense(100,input_dim=1,activation='relu')model.add(Dense(1))model.compile(loss='mse',optimizer='adam')model.fit(xscaled,yscaled,epochs=150,batch_size=10,verbose=0) mse=model.evaluate(xscaled,yscaled,verbose=0)mse2.9744908551947447e-05 yhat=model.predict(x)yhat_original=y_s.inverse_transform(yscaled) #Firstfiveelementsofpredictedvaluesyhat_original[:5].Tarray([[0., 1., 4., 9.,16.]])#Firstfiveelementsoforiginalyy[:5].Tarray([[0, 1, 4, 9,16]]) #Lastfiveelementsoftheoriginalseries.y[-5:].Tarray([[9025,9216,9409,9604,9801]])#Lastfiveelementsofpredictedvaluesyhat_original[-5:].Tarray([[9025.,9216.,9409.,9604.,9801.]]) #NowdeterminingtheRMSofthepredictedandoriginalvaluesofycum_sum=0foriinrange(len(yhat_original)): cum_sum+=(yoriginal[i]-yhat_original[i])**2/len(yhat_original)rms=sqrt(cum_sum)rms[0]0.0 #Plottingtherescaledoriginalandrescaledyhatpyplot.plot(xoriginal,yoriginal,label='y')pyplot.plot(xoriginal,yhat_original,label='fitted')pyplot.legend()pyplot.show() Itworks,therescaledyhatisasexpectedbutcannotexplainwhyitwas“cuckoo”,intheprevious.Moreexperimentationonthis. Nevertheless,mynextprojectisk-foldssamplingonadeterministicfunctiontoseeifthegapsintheresampleddatafoldwillgiveusanaccuratepredictiondespitetherandomsamplingineachfold. Thankyou, AnthonyofSydney Reply AnthonyTheKoala October13,2019at11:44pm # DearDrJason, Apologies,IthoughttheRMSwas‘unrealistic’.Ihadaprogrammingerror. Nevertheless,Ididitagain,andstillproducedresultswhichlookedpleasing. x=x.reshape((len(x),1)) y=y.reshape((len(y),1)) x_s=MinMaxScaler() y_s=MinMaxScaler() x_scaled=x_s.fit_transform(x) y_scaled=y_s.fit_transform(y) model=Sequential() model.add(Dense(100,input_dim=1,activation='relu')) model.add(Dense(1)) model.compile(loss='mse',optimizer='adam') model.fit(x_scaled,y_scaled,epochs=100,batch_size=10,verbose=0) mse=model.evaluate(x_scaled,y_scaled,verbose=0) mse 1.0475558547113905e-05 yhat=model.predict(x_scaled) yhat_original=y_s.inverse_transform(yhat) #Firstfiveofyhat_original(yhatrescaled) yhat_original[:5].T array([[11.835742,11.835742,11.835742,11.835742,11.835742]] #comparedtofirstoriginal5elementsofy=0,1,4,9,16 #Lastfiveofyhat_original(yhatrescaled) yhat_original[-5:].T array([[8985.839,9154.454,9323.067,9491.684,9660.3] #comparedtolastoriginal5elementsofy=9025,9216,9409,9604,9801 #NowdeterminetheRMSofthepredictedandoriginalvalues cum_sum=0 foriinrange(len(yhat_original)): cum_sum+=(y[i]-yhat_original[i])**2/len(yhat_original) mse=sqrt(cum_sum) mse array([31.72189417]) pyplot.plot(x,y,label='y') pyplot.plot(x,yhat_original,label='estimated') pyplot.legend() pyplot.show() 1234567891011121314151617181920212223242526272829303132333435363738394041 x=x.reshape((len(x),1))y=y.reshape((len(y),1))x_s=MinMaxScaler()y_s=MinMaxScaler()x_scaled=x_s.fit_transform(x)y_scaled=y_s.fit_transform(y)model=Sequential()model.add(Dense(100,input_dim=1,activation='relu'))model.add(Dense(1)) model.compile(loss='mse',optimizer='adam')model.fit(x_scaled,y_scaled,epochs=100,batch_size=10,verbose=0) mse=model.evaluate(x_scaled,y_scaled,verbose=0)mse1.0475558547113905e-05 yhat=model.predict(x_scaled)yhat_original=y_s.inverse_transform(yhat) #Firstfiveofyhat_original(yhatrescaled)yhat_original[:5].Tarray([[11.835742,11.835742,11.835742,11.835742,11.835742]]#comparedtofirstoriginal5elementsofy=0,1,4,9,16 #Lastfiveofyhat_original(yhatrescaled)yhat_original[-5:].Tarray([[8985.839,9154.454,9323.067,9491.684,9660.3 ]#comparedtolastoriginal5elementsofy=9025,9216,9409,9604,9801 #NowdeterminetheRMSofthepredictedandoriginalvaluescum_sum=0foriinrange(len(yhat_original)): cum_sum+=(y[i]-yhat_original[i])**2/len(yhat_original)mse=sqrt(cum_sum)msearray([31.72189417])pyplot.plot(x,y,label='y')pyplot.plot(x,yhat_original,label='estimated')pyplot.legend()pyplot.show() Insum,therescaledyhatproducedresultsclosertotheoriginalvalues.Thelowervaluesofyhatrescaledappeartobeodd. Despitethatthevaluesneedtobemorerealisticatthebottomendeventhoughtheplotoftherescaledx&rescaledy,andrescaledxandrescaledyhatlookclose. Moreinvestigationsneededonthebatchsize,epochsandoptimizers. Next,todok-foldssamplingonadeterministicfunctiontoseeifthegapsintheresampleddatafoldwillgiveusanaccuratepredictiondespitetherandomsamplingineachfold. Againapologiesforthemistakeinthepreviouspost. AnthonyofSydney Reply JasonBrownlee October14,2019at8:08am # Welldone. Reply AnthonyTheKoala November21,2019at5:03am # DearDrJason, Aperson‘Serali’aparticlephysicistreliedtomeat“StackExchange”repliedandsuggestedthatIshuffletheoriginaldata.Theshufflingofdatainthiscontexthasnothingtodowiththeshufflingink-folds.Accordingtothecontributor,theresultsshouldimprove.Sourcehttps://datascience.stackexchange.com/questions/61223/reconstituting-estimated-predicted-values-to-original-scale-from-minmaxscaler ThecodeisexactlythesameaswhatIwasexperimentingwith.SoIwillshowthenecessarycodetoshuffeatthestartandde-shuffleattheend. Shufflingcodeatthebeginning: fromsklearn.preprocessingimportMinMaxScaler fromkeras.modelsimportSequential fromkeras.layersimportDense fromnumpyimportasarray fromnumpyimportsqrt frommatplotlibimportpyplot fromnumpy.randomimportseed fromnumpy.randomimportshuffle fromnumpy.randomimportsample importnumpyasnp #Wewillwanttoreshufflethedata x=[iforiinrange(100)] y=[i**2foriinx] xfx=np.vstack((x,y)).T xy=xfx shuffle(xy) #x=asarray([iforiinrange(100)]) #y=asarray([i**2foriinx]) #x=asarray(xy[:,0]);#x.reshape((len(x),1)) x=np.reshape(xy[:,0],(100,1)) #print('debug,sizex=%d'+str(np.shape(x))) y=np.reshape(xy[:,1],(100,1)) #y=asarray(xy[:,1]);#y.reshape((len(y),1)) #print('debug,sizey=%d'+str(np.shape(y))) x_s=MinMaxScaler() y_s=MinMaxScaler() x_scaled=x_s.fit_transform(x) y_scaled=y_s.fit_transform(y) #Therestisfedintomodel ...... ....... yhat=model.predict(x_scaled) yhat_original=y_s.inverse_transform(yhat) 123456789101112131415161718192021222324252627282930313233343536373839 fromsklearn.preprocessingimportMinMaxScalerfromkeras.modelsimportSequentialfromkeras.layersimportDensefromnumpyimportasarrayfromnumpyimportsqrtfrommatplotlibimportpyplot fromnumpy.randomimportseedfromnumpy.randomimportshufflefromnumpy.randomimportsample importnumpyasnp #Wewillwanttoreshufflethedatax=[iforiinrange(100)]y=[i**2foriinx] xfx=np.vstack((x,y)).Txy=xfxshuffle(xy) #x=asarray([iforiinrange(100)])#y=asarray([i**2foriinx]) #x=asarray(xy[:,0]);#x.reshape((len(x),1))x=np.reshape(xy[:,0],(100,1))#print('debug,sizex=%d'+str(np.shape(x)))y=np.reshape(xy[:,1],(100,1))#y=asarray(xy[:,1]);#y.reshape((len(y),1))#print('debug,sizey=%d'+str(np.shape(y)))x_s=MinMaxScaler()y_s=MinMaxScaler()x_scaled=x_s.fit_transform(x)y_scaled=y_s.fit_transform(y)#Therestisfedintomodel.............yhat=model.predict(x_scaled)yhat_original=y_s.inverse_transform(yhat) Theendcodewas‘unshuffled’/sortedinordertodisplaythedifferencebetweentheactualandpredicted. #Plottingdotsinsteadoflineplototherwisewegetazig-zagplot pyplot.plot(x,y,'r.',label='y') pyplot.plot(x,yhat_original,'b.',label='estimated') #printingthefirstvalues-wehavetosortthevaluesinordertoseethemin #theirpropercontext. xy=np.vstack((x[:,0],y[:,0])).T xyhat=np.vstack((x[:,0],yhat_original[:,0])).T xyy=np.sort(xy,axis=0) xyhatt=np.sort(xyhat,axis=0) print("printingx,y,yhat") forloopinrange(10): print(xyy[loop,0],xyy[loop,1],xyhatt[loop,1]) pyplot.legend() pyplot.show() 12345678910111213141516171819 #Plottingdotsinsteadoflineplototherwisewegetazig-zagplotpyplot.plot(x,y,'r.',label='y')pyplot.plot(x,yhat_original,'b.',label='estimated') #printingthefirstvalues-wehavetosortthevaluesinordertoseethemin#theirpropercontext.xy=np.vstack((x[:,0],y[:,0])).Txyhat=np.vstack((x[:,0],yhat_original[:,0])).T xyy=np.sort(xy,axis=0)xyhatt=np.sort(xyhat,axis=0) print("printingx,y,yhat")forloopinrange(10): print(xyy[loop,0],xyy[loop,1],xyhatt[loop,1]) pyplot.legend()pyplot.show() Hereisalistingofx,f(x)andyhat printingx,y,yhat 001.4915295839309692 112.66086745262146 244.75526237487793 399.125076293945312 41615.723174095153809 52524.287418365478516 63635.04938507080078 74947.73912811279297 86462.95930480957031 98180.16889190673828 1234567891011 printingx,y,yhat001.4915295839309692112.66086745262146244.75526237487793399.12507629394531241615.72317409515380952524.28741836547851663635.0493850708007874947.7391281127929786462.9593048095703198180.16889190673828 Thingstoimprove: *adjustingthenumberoflayers. *adjustinghowmanyneuronsineachlayer *adjustingthebatchsize *adjustingtheepochsize Inaddition *lookatk-foldsforfurthermodelrefinement. Thankyou AnthonyofSydney AnthonyTheKoala November24,2019at4:12pm # DearDrJason, Hereisanevenimprovedversionwithverycloseresults. InsteadofMinMaxScaler,Itookthelogs(tothebasee)oftheinputsxandf(x)appliedmymodel,thenretransformedmymodeltoitsoriginalvalues. Snippetsofcodetransformingthedata #Wewillwanttoreshufflethedata x=[iforiinrange(100)] y=[i**2foriinx] xfx=np.vstack((x,y)).T xy=xfx seed(1) shuffle(xy) x=np.reshape(xy[:,0],(100,1)) print('debug,sizex=%d'+str(np.shape(x)))#shapeis(100,1) y=np.reshape(xy[:,1],(100,1)) print('debug,sizey=%d'+str(np.shape(y))) #x_s=MinMaxScaler() #x_s=MinMaxScaler(feature_range=(0,200)) #y_s=MinMaxScaler() #x_scaled=x_s.fit_transform(x) #y_scaled=y_s.fit_transform(y) x_scaled=np.log(x+1);#weadd1soasnottohaveanerroraslog(0)producesanerror y_scaled=np.log(y+1);#weadd1soasnottohaveanerroraslog(0)producesanerror model=Sequential() ...#themodelisappliedonthetransformeddata 123456789101112131415161718192021222324 #Wewillwanttoreshufflethedatax=[iforiinrange(100)]y=[i**2foriinx] xfx=np.vstack((x,y)).Txy=xfxseed(1)shuffle(xy)x=np.reshape(xy[:,0],(100,1))print('debug,sizex=%d'+str(np.shape(x)))#shapeis(100,1)y=np.reshape(xy[:,1],(100,1)) print('debug,sizey=%d'+str(np.shape(y)))#x_s=MinMaxScaler()#x_s=MinMaxScaler(feature_range=(0,200))#y_s=MinMaxScaler()#x_scaled=x_s.fit_transform(x)#y_scaled=y_s.fit_transform(y)x_scaled=np.log(x+1);#weadd1soasnottohaveanerroraslog(0)producesanerrory_scaled=np.log(y+1);#weadd1soasnottohaveanerroraslog(0)producesanerror model=Sequential()...#themodelisappliedonthetransformeddata The #Weneedtoresortthenumbers #inordertoprintthefirst10values xy=np.vstack((x[:,0],y[:,0])).T xyhat=np.vstack((x[:,0],yhat_original[:,0])).T xyy=np.sort(xy,axis=0) xyhatt=np.sort(xyhat,axis=0) print("printingx,y,yhat") forloopinrange(10): print(xyy[loop,0],xyy[loop,1],xyhatt[loop,1]) #wanttopredictforthevalues100and200 Xnew=np.reshape([100,200],(2,1)) print("let'spredictforvalues100and200") print("thevaluesofx=Xnewbeforetransform%s,%s"%(Xnew[0],Xnew[1])) Xnew=np.log(Xnew+1) print("valuesofscaledxnewtoputintothemodel%s,%s"%(Xnew[0],Xnew[1])) ynew=model.predict(Xnew) #Re-transformtheoriginalvalues ynew=np.exp(ynew)-1 print("ThevaluesofXnewanditspredictedyhat") forloopinrange(len(Xnew)): print("Xnew[%s]=%s,ynew[%s]=%s"%(loop,Xnew[loop],loop,ynew[loop])) 1234567891011121314151617181920212223242526 #Weneedtoresortthenumbers#inordertoprintthefirst10valuesxy=np.vstack((x[:,0],y[:,0])).Txyhat=np.vstack((x[:,0],yhat_original[:,0])).T xyy=np.sort(xy,axis=0)xyhatt=np.sort(xyhat,axis=0)print("printingx,y,yhat")forloopinrange(10): print(xyy[loop,0],xyy[loop,1],xyhatt[loop,1]) #wanttopredictforthevalues100and200Xnew=np.reshape([100,200],(2,1)) print("let'spredictforvalues100and200")print("thevaluesofx=Xnewbeforetransform%s,%s"%(Xnew[0],Xnew[1])) Xnew=np.log(Xnew+1)print("valuesofscaledxnewtoputintothemodel%s,%s"%(Xnew[0],Xnew[1]))ynew=model.predict(Xnew) #Re-transform theoriginalvaluesynew=np.exp(ynew)-1print("ThevaluesofXnewanditspredictedyhat")forloopinrange(len(Xnew)): print("Xnew[%s]=%s,ynew[%s]=%s"%(loop,Xnew[loop],loop,ynew[loop])) Theresultingoutput:Notehowclosetheactualf(x)istothepredictedf(x) printingx,y,yhat 000.00208890438079834 110.9818048477172852 244.111057281494141 399.025933265686035 41615.918327331542969 52524.944564819335938 63636.00426483154297 74949.05435562133789 86463.969764709472656 98180.93276977539062 let'spredictforvalues100and200 thevaluesofx=Xnewbeforetransform[100],[200] valuesofscaledxnewtoputintothemodel[4.61512052],[5.30330491] ThevaluesofXnewanditspredictedyhat Xnew[0]=[100.],ynew[0]=[10008.037] Xnew[1]=[200.],ynew[1]=[40082.062] 12345678910111213141516171819 printingx,y,yhat000.00208890438079834110.9818048477172852244.111057281494141399.02593326568603541615.91832733154296952524.94456481933593863636.0042648315429774949.0543556213378986463.96976470947265698180.93276977539062 let'spredictforvalues100and200thevaluesofx=Xnewbeforetransform[100],[200]valuesofscaledxnewtoputintothemodel[4.61512052],[5.30330491] ThevaluesofXnewanditspredictedyhatXnew[0]=[100.],ynew[0]=[10008.037]Xnew[1]=[200.],ynew[1]=[40082.062] JasonBrownlee November25,2019at6:21am # Nicework. kamu October6,2019at7:51pm # HiJason, Thankyouverymuchfor“YourFirstDeepLearningProjectinPythonwithKerasStep-By-Step”tutorial.Itisveryusefulforme.Iwanttoaskyou: CanIcode: model.add(Dense(8))#inputlayer model.add(Dense(12,activation=’relu’))#firsthiddenlayer Insteadof: model.add(Dense(12,input_dim=8,activation=’relu’))#inputlayerandfirsthiddenlayer Sincerely. Reply JasonBrownlee October7,2019at8:29am # No. Theinput_dimargumentdefinestheinputlayer. Reply keryums October17,2019at1:22am # HiJason,isitnotnecessarytousethekerasutilility‘to_categorical’toconvertyouryvectorintoamatrixbeforefittingthemodel? Reply JasonBrownlee October17,2019at6:37am # Youcan,oryoucanusethesklearntoolstodothesamething. Reply AquillaSetiawanKanadi October17,2019at6:35am # HiJason, Thanksalotforyourtutorialaboutdeeplearningproject,itreallyhelpmealotinmyjourneytolearnmachinelearning. Ihaveaquestionaboutthedatasplittingincodeabove,howisthesplittingworkbetweendatafortrainingandthedataforvalidatethetrainingdata?I’vetriedtoreadyourtutorialaboutthedatasplittingbutihavenoideasaboutthedatasplittingworkabove. Thankyou, Aquilla Reply JasonBrownlee October17,2019at6:47am # Wedidnotsplitthedata,wefitandevaluatedononeset.Wedidthisforbrevity. Reply Loveyourwork! October17,2019at11:43am # HiJason, Ijustwantedtothankyou.Thistutorialisincrediblyclearandwellpresented.Unlikemanyotheronlinetutorialsyouexplainveryeloquentlytheintuitionbehindthelinesofcodeandwhatisbeingaccomplishedwhichisveryuseful.AssomeonejuststartingoutwithKerasIhadbeenfindingsomeofthecoding,aswellashowKerasandTensorflowinteract,confusing.AfteryourexplanationsKerasseemsincrediblybasic.I’vebeenlookingoversomeofmyrecentcodefromotherKerastutorialsandInowunderstandhoweverythingworks. Thanksagain! Reply JasonBrownlee October17,2019at1:50pm # Welldoneonyourprogressandthanksforyoursupport! Reply Ahmed October19,2019at6:16am # DearJason.Iamdeeplygratefultothisamazingwork.Everythingworkswellsofar.KingRegards Reply JasonBrownlee October19,2019at6:55am # Thanks,welldoneonyourprogress! Reply JAMESJONAH October28,2019at10:56am # Pleaseineedhelp,whichalgorithmsisthebestincyberthreatdetectionandhowtoimplementit.thanks Reply JasonBrownlee October28,2019at1:18pm # ThisisacommonquestionthatIanswerhere: https://machinelearningmastery.com/faq/single-faq/what-algorithm-config-should-i-use Reply shivan October29,2019at7:01am # hellosir doyouhaveanimplementationabout(medicalimageanalysiswithdeeplearning). ineedtostartwithmedicalimageNOTrealworldimage thanksforyourhelp. Reply JasonBrownlee October29,2019at1:47pm # Notreally,sorry. Reply shivan October31,2019at9:18am # so,whatdoyourecommendmeaboutit thanks. Reply JasonBrownlee October31,2019at1:36pm # Perhapsstartbycollectingadataset. Thenconsiderreviewingtheliteraturetoseewhattypesofdataprepandmodelsotherhaveusedforsimilardata. Reply NasirShah October30,2019at7:27am # Sir.iamnewtoneuralnetwork.sofromwhereistartit.orwhichtutorialiwatch.ididn’thaveanyideaaboutit. Reply JasonBrownlee October30,2019at1:55pm # Yes,youcanstarthere: https://machinelearningmastery.com/start-here/#deeplearning Reply himahansi November3,2019at1:35pm # hellosir,I’mnewtothisfield.I’mgoingtodevelopmonophonicmusicalinstrumentclassificationsystemusingpythonandKeras.sir,Iwanttofindmonophonicdataset,howcanIfindit. Itrytogetpianomusicfromyoutubeandconvertitto.wawfileandsplittingit.Isitagoodorbad?oranothermethodsavailabletogetfreedatasetontheweb..giveyoursuggestionsplease?? Reply JasonBrownlee November4,2019at6:37am # Perhapsthiswillhelp: https://machinelearningmastery.com/faq/single-faq/where-can-i-get-a-dataset-on-___ Reply MonaAhmed November20,2019at3:14am # igotscore76.69 Reply JasonBrownlee November20,2019at6:20am # Welldone! Reply NiallXie November26,2019at8:26am # Hello,IjustwanttosaythatIamelatedtouseyourtutorial.So,IamworkingonagroupprojectwithmyteamandIuseddatasetsrepresentingheartdisease,diabetesandbreastcancerforthistutorial.However,thiscodeexamplewillgiveanerrorwhenthecellcontainsastringvalue,inthiscase…titlenameslikeclump_thickessand?willproduceanerror.howdoIfixthis? Reply JasonBrownlee November26,2019at1:28pm # Thanks. Perhapstryencodingyourcategoriesusingaonehotencodingfirst: https://machinelearningmastery.com/how-to-prepare-categorical-data-for-deep-learning-in-python/ Reply Mohamed November28,2019at10:46pm # thankyousirforthisarticle,wouldyoupleasesuggestanexamplewithtestingdata? Reply JasonBrownlee November29,2019at6:49am # SorryIdon’tunderstandyourquestion,canyouelaborate? Reply Chris December3,2019at10:49pm # Ibelievethereissomethingwrongwiththe(150/10)15updatestothemodelweights.Theinternalcoefficientsareupdatedaftereverysinglebatch.Ourdataiscomprisedof768samples.Sincebatch_size=10,weobtain77batches(76with10samplesandonewith8).Therefore,ateachepochweshouldsee77updatesofweightsandcoefficientsandnot15.Moreover,thetotalnumberofupdatesmustbe:150*77=11550.AmImissingsomethingimportant? Reallygoodjobandverywell-writtenarticle(allyourarticles).Keepupthegoodjob.Cheers Reply JasonBrownlee December4,2019at5:37am # You’reright.NotsurewhatIwasthinkingthere.Simplified. Reply Justine December14,2019at9:58am # Thanks!Thisismyfirstforayintokeras,andthetutorialwentswimmingly.Amnowtrainingonmyowndata.Itisnotperformingworsethanonmyothermachinelearningmodels(that’sawin:). Reply JasonBrownlee December15,2019at6:02am # Welldone! Reply x December17,2019at8:37am # Hi，Jason.Thankssomuchforyouranswer.NowmyquestioniswhyIcan’tfoundmydirectoryinJupyterandputthe‘pima-indians-diabetes.csv’init. OSErrorTraceback(mostrecentcalllast) in 4fromkeras.layersimportDense 5#loadthedataset —->6dataset=loadtxt(‘pima-indians-diabetes.csv’,delimiter=’,’) 7#splitintoinput(X)andoutput(y)variables 8X=dataset[:,0:8] D:\anaconda\lib\site-packages\numpy\lib\npyio.pyinloadtxt(fname,dtype,comments,delimiter,converters,skiprows,usecols,unpack,ndmin,encoding,max_rows) 966fname=os_fspath(fname) 967if_is_string_like(fname): –>968fh=np.lib._datasource.open(fname,‘rt’,encoding=encoding) 969fencoding=getattr(fh,‘encoding’,‘latin1’) 970fh=iter(fh) D:\anaconda\lib\site-packages\numpy\lib\_datasource.pyinopen(path,mode,destpath,encoding,newline) 267 268ds=DataSource(destpath) –>269returnds.open(path,mode,encoding=encoding,newline=newline) 270 271 D:\anaconda\lib\site-packages\numpy\lib\_datasource.pyinopen(self,path,mode,encoding,newline) 621encoding=encoding,newline=newline) 622else: –>623raiseIOError(“%snotfound.”%path) 624 625 OSError:pima-indians-diabetes.csvnotfound. Reply JasonBrownlee December17,2019at1:36pm # Perhapstryrunningthecodefilefromthecommandline,asfollows: https://machinelearningmastery.com/faq/single-faq/how-do-i-run-a-script-from-the-command-line Reply ManoharNookala December22,2019at9:32pm # Hisir, Mynameismanohar.itrainedadeeplearningmodeloncarpriceprediction.igot loss:nan–acc:0.0000e+00.ifyougivemeyouremailIDtheniwillsendyou.youcantellmetheproblem.pleasedothishelpbecauseiamabeginner. Reply JasonBrownlee December23,2019at6:48am # Perhapsyouneedtoscalethedatapriortofitting? Perhapsyouneedtousereluactivation? Perhapsyouneedsometypeofregularization? Perhapsyouneedalargerorsmallermodel? Reply ShoneXu January5,2020at1:24am # HiJason, thanksanditisagreattutorial.just1question.dowehavetotrainthemodelby“model.fit(x,y,epochs=150,batch_size=10)”everytimebeforemakingthepredictionbecauseittakesaverylongtimetotrainthemodel.Iamjustwonderingwhetheritispossibletosavethetrainedmodelandgostraighttothepredictionskippingthemodel.fit(eg:pickle)? manythanksforyouradviceinadvance cheers Reply JasonBrownlee January5,2020at7:06am # No,youcanfitthemodelonce,thensaveit: https://machinelearningmastery.com/save-load-keras-deep-learning-models/ Thenlaterloaditandmakepredictions. Reply ShoneXu January7,2020at2:16pm # Thanksandwillcheckitout Reply ustengg January8,2020at7:40pm # ThankyousomuchforthistutorialsirbutHowcanIusethemodeltopredictusingdataoutsidethedataset? Reply JasonBrownlee January9,2020at7:24am # Callmodel.predict()withthenewinputs. Seethe“MakePredictions”section. Reply ustengg January9,2020at4:07pm # Nice!Thankyousomuch,Sir.Ifigureditoutusingthelinkonthe“Makepredictions”section.I’velearnedalotfromyourtutorials.You’rethebest! Reply JasonBrownlee January10,2020at7:22am # Nicework! Thanks. Reply monica January23,2020at4:00am # HiJason, Thanksforsharingthispost. Ihaveaquestion,whenItriedtosplitthedataset (X=dataset[:,0:8] y=dataset[:,8]) itgivesmeanerror:TypeError:‘(slice(None,None,None),slice(0,8,None))’isaninvalidkey howcanIfixit? Thanks, monica Reply JasonBrownlee January23,2020at6:41am # Sorrytohearthat,thismighthelp: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply SamSarjant January23,2020at9:11pm # Thanksforthetutorial!Thisisawonderful‘HelloWorld’toDeepLearning Reply JasonBrownlee January24,2020at7:51am # Thanks,I’mhappyitwashelpful. Reply Keerthan January24,2020at4:01pm # HelloJason!hopeyouaredoinggood. Iamactuallydoingaprojectonclassificationofthyroiddiseaseusingbackpropagationwithstocasticgradientdescentmethod,canyouhelpmeoutwiththecodealittlebit? Reply JasonBrownlee January25,2020at8:31am # Perhapsstartbyadaptingthecodeintheabovetutorial? Reply Shakir January25,2020at1:29am # DearSir Iwanttopredictairpollutionusingdeeplearningtechniquespleasesuggesthowtogoaboutwithmydatasets Reply JasonBrownlee January25,2020at8:39am # Starthere: https://machinelearningmastery.com/start-here/#deep_learning_time_series Reply Yared February7,2020at4:36pm # AttributeError:module‘tensorflow’hasnoattribute‘get_default_graph’AttributeError:module‘tensorflow’hasnoattribute‘get_default_graph’ Reply JasonBrownlee February8,2020at7:05am # PerhapsconfirmyouareusingTF2andKeras2.3. Reply Yared February7,2020at4:41pm # IwenttodetectagreementerrorsinasentenceusingLSTMtechniquespleasesuggesthowtogoaboutwithmydatasets Reply JasonBrownlee February8,2020at7:05am # YoucangetstartedwithNLPproblemshere: https://machinelearningmastery.com/start-here/#nlp Reply PavitraNayak February29,2020at2:55pm # HelloJason Iamusingthiscodeformyproject.Itworksperfectlyforyourdataset.ButIhaveadatasetwhichhastoomany0’sand1’s.SoIamgettingthewrongprediction.WhatcanIdotosolvethisproblem? Reply JasonBrownlee March1,2020at5:22am # Herearesomesuggestions: https://machinelearningmastery.com/improve-deep-learning-performance/ Reply nurul March6,2020at5:50pm # hi.Iwannaask.ihadfollowallthestepsbuti’mstuckatthefitthemodel.Thiserroroccured.HowcanIsolvethisproblem? Reply kiki March6,2020at6:44pm # Ihavealreadytriedthisstepandstuckatthefitphaseandgotthiserror.Doyouhaveanysolutionformyproblem? ————————————————————————— ValueErrorTraceback(mostrecentcalllast) in 1#fitthekerasmodelonthedataset —->2model.fit(x,y,batch_size=10,epochs=150) ~\Anaconda4\lib\site-packages\keras\engine\training.pyinfit(self,x,y,batch_size,epochs,verbose,callbacks,validation_split,validation_data,shuffle,class_weight,sample_weight,initial_epoch,steps_per_epoch,validation_steps,validation_freq,max_queue_size,workers,use_multiprocessing,**kwargs) 1152sample_weight=sample_weight, 1153class_weight=class_weight, ->1154batch_size=batch_size) 1155 1156#Preparevalidationdata. ~\Anaconda4\lib\site-packages\keras\engine\training.pyin_standardize_user_data(self,x,y,sample_weight,class_weight,check_array_lengths,batch_size) 577feed_input_shapes, 578check_batch_axis=False,#Don’tenforcethebatchsize. –>579exception_prefix=’input’) 580 581ifyisnotNone: ~\Anaconda4\lib\site-packages\keras\engine\training_utils.pyinstandardize_input_data(data,names,shapes,check_batch_axis,exception_prefix) 143‘:expected‘+names[i]+‘tohaveshape‘+ 144str(shape)+‘butgotarraywithshape‘+ –>145str(data_shape)) 146returndata 147 ValueError:Errorwhencheckinginput:expecteddense_133_inputtohaveshape(16,)butgotarraywithshape(17,) Reply JasonBrownlee March7,2020at7:15am # Perhapsthiswillhelpyoucopythecodefromthetutorial: https://machinelearningmastery.com/faq/single-faq/how-do-i-copy-code-from-a-tutorial Reply JasonBrownlee March7,2020at7:13am # I’msorrytohearthat,perhapsthiswillhelp: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply kiki March9,2020at12:18pm # Thanksfortheanswerjason Reply JasonBrownlee March10,2020at5:34am # You’rewelcome. Reply laz March7,2020at2:59pm # Hey,Jason! Again…Thanksforyourawesometutorialsandforgivingyourknowledgetothepublic!>800commentsandnearlyallanswered,you’regreat.Ican’tunderstandhowyoumanageallthat,writinggreatcontent,domlstuff,teach,learn,greatrespect! 2generalquestions: Question(1): Whyandwhendoweneedtoflatten()inputsandinwhichcasesnot? Forexample4numericinputs,alagof2ofeveryinputmeans4*2=8valuesperbatch: Ialwaysdothis,nomatterhowmanyinputsorlags,igivethatasflatarraytotheinput: 1set/batch:[[1.0,1.1,2.0,2.1,3.0,3.1,4.0,4.1]] Input(shape=(8,))#kerasfuncapi Doesitmakesensetoinputastructurelikethis,ifso–why/when? Better?[[[1.0,1.1],[2.0,2.1],[3.0,3.1],[4.0,4.1]]] Question(2): AreyoustillusingTheano?Astheydonotupdateit,itbecomesolder,butnotworse;).ItriedTensorflowalot–butalwayswithlowerperformanceintermsofspeed.Theanoismuchfaster(factor3-10)forme.Butusingmorethan1coreisalwaysslowerforme,inboththeanoandtf.Didyouexperiencedsimilarthings?Ialsotriedtorch,nicebutitwasalsoslowerasthegoodoldtheano.Anyideasoralternatives(ican’tusegpu/external/aws)? Iwouldbehappytoseeyoudoingsomedeepreinforcementlearning(DRL)stuff,whatdoyouthink?Areyou? Regards,keepitup😉 Reply JasonBrownlee March8,2020at6:07am # Youneedtoflattenwhentheoutputshapeofonelayerdoesnotmatchtheinputshapeofanother,e.g.CNNoutputtoaDense. No.Iuseandrecommendtensorflowandhaveforyears.Tensorflowusedtonotworkforwindowsusers,soIrecommendtheanoforthem–andstilldoiftheyhavetrouble.Theanoworksfineandwillcontinuetoworkfineformostapplications. No,RLisnotpractical/useful: https://machinelearningmastery.com/faq/single-faq/do-you-have-tutorials-on-deep-reinforcement-learning Reply laz March8,2020at11:29am # DearJason,thanksforyouranswer;)… “flattenwhentheoutputshapeofonelayerdoesnotmatchtheinputshapeofanother,e.g.CNNoutputtoaDense.” Thanks.Thequestionaboutthe“flatten”operationwasnotabouttheflatten()betweenlayers,itwasabouthowtopresentinputstotheinputlayer.Sorryforbeingvague.MaybeImisunderstoodsomething,arethereusecaseswheretheFEATURES/INPUTS/LAGSarenotflattened? “RLisnotpractical/useful” Isthisstatementbasedonyourexperienceordoyoutaketheopinionofotherswithoutcheckingityourselfhere;)?Pleasedonotmisunderstand,youaretheexperthere.However,icanrefutesomeargumentsagainstRL. Rewardsarehardtocreate:dependsonyourenvironment Unstable:dependsonyourenvironment,code,setup IstartedexperimentingwithasimpleDQN,IexpandeditstepbystepandnowIhavea“DuelingDoubleDQN”.Itlearnswellandquick.Iadmit–onsimpledata.Butitdoesitrepeatableandreproducible!Soiwouldsay:Ingeneral,itworks. Ihavetoseehowitworkswithmorecomplicateddata.ThatiswhyIemphasizedthattheperformanceofthismethodstronglydependsontheareaofapplication. Butthereisahugeproblem,mostpublicsourcescontainincorrectcodeorincorrectimplementations.Ihaveneverreportedorfoundsomanybugsonanysubject.Theseerrorsarecopiedagainandagainandintheendmanythinkthattheyarecorrect.Ihavecollectedtonsoflinksandpdffilestounderstandanddebugthisbeast. Nomatter,youhavetodecideforyourself.Ifyouwanttotakealookatit,takeasimpleexample,eventheDQN(withoutduelingordouble)isabletolearn–ifthecodeiscorrect.AndalthoughI’mnotamathematician:tounderstandhowitworksandwhatpossibilitiesitoffers–mademesmile😉… Reply JasonBrownlee March9,2020at7:14am # FormoreontheinputshapeofLSTMs/1dCNNs,seethis: https://machinelearningmastery.com/faq/single-faq/what-is-the-difference-between-samples-timesteps-and-features-for-lstm-input Idon’tyetseeanROIfor“developersatwork”incoveringRLasdescribedinthelink. Reply laz March8,2020at10:44pm # Interestingread: “WeuseadoubledeepQ-learningnetwork(DDQN)tofindtherightmaterialtypeandtheoptimalgeometricaldesignformetasurfacehologramstoreachhighefficiency.TheDDQNactslikeanintelligentsweepandcouldidentifytheoptimalresultsin~5.7billionstatesafteronly2169steps.Theoptimalresultswerefoundbetween23differentmaterialtypesandvariousgeometricalpropertiesforathree-layerstructure.Thecomputedtransmissionefficiencywas32%forhigh-qualitymetasurfaceholograms;thisistwotimesbiggerthanthepreviouslyreportedresultsunderthesameconditions.” https://www.nature.com/articles/s41598-019-47154-z Reply JasonBrownlee March9,2020at7:16am # Thanksforsharing. Reply YzN March11,2020at4:01am # Literallythebest“firstneuralnetworktutorial” Got85.68accbyaddinglayersanddecreasingbatchsize Reply JasonBrownlee March11,2020at5:29am # Thanks. Welldone! Reply Neha March14,2020at12:05am # HelloJason, Ihaveaquickquestion. Iamtryingtobuildjust1sigmoidneuronforabinaryclassificationtask,basicallyIamimplyingthisishow1sigmoidmodelis: model=Sequential() model.add(Dense(1,activation=’sigmoid’)) Myinputsareimagesofsize=(39*39*3) IamunsureastohowtoinputtheseimagestomyDenselayer(whichistheonlylayerIamusing) Iamcurrentlyusingbelowforinputtingmyimages: train_generator=train_datagen.flow_from_directory(train_data_dir, target_size=(39,39), batch_size=batch_size) class_mode=’binary’) ButsomehowDenselayercannotacceptinputshape(39,39,3). Somyquestionis,howdoIinputmyimagesdatatotheDenselayer? Reply JasonBrownlee March14,2020at8:13am # YoucanflattentheinputoruseaCNNastheinputinsteadthatisdesignedfor3dinputsamples. Reply BertrandBru March29,2020at12:38am # HiJason, Thankyouverymuchforyourtutorial. Iamnewintheworldofdeepleraning.IhavebeenabletomodifyyourcodeandmakeitworkforasetofdataIrecordedwitha3axisaccelerometer.MygoalwastodetectifIwaswalkingorrunning.Irecordedaround50trialsofeachactivities.Fromthesignal,Icalculatedspecificparametersthatenablethecodetodifferenciatethetwoactivities.Amongsttheparameters,Icalculatedforallaxis,themean,minandmaxvalues,andsomeparametersinthedomainfrequencies(the3firstpeakofthepowerspectrumandtheirrespectiveposition). ItworksverywellandIamabletoeasilydetectifIamrunningorwalking. Ithendecidedtoaddathridactivities:standing.Ialsorecorded50trialsofthisactivity.IfItrainmymodelwithstandingandrunning,Icanidentifythetwoactivity.SameifItrainitwithstandingandwalkingorwithwalkingandrunning. ItismorecomplicatedifItrainmymodelwiththethreeactivities.Infact,itcan’tdoit.Itcanonlyrecgonisethefirsttwoactivities.Soforexampleifstanding,walkingandrunninghavethefollowingID:0,1and2,thenitcanonlydetect0and1(standingandwalking).Itthinksthatallrunningtrialsarewalkingtrials.Ifstanding,runningandwalinkinghavethefollowingID:0,1and2,thenitcanonlydetect0and1(standingandrunning).Itthinksthatallwalkingtrialsarerunningtrials. Sohereismyquestion:Assumingyouhavethedataset,ifyouneededtoadaptyourcodesoitcandetectifpeopleare0:notdiabetic,1:peoplearediabetictype1,and2:peoplearediabetictype2,howwouldyoumodifyyourscript? Thankyouverymuchforyourhelp. Reply JasonBrownlee March29,2020at6:00am # You’rewelcome. Welldone. Thisiscalledmulti-classclassification,thistutorialwillhelp: https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/ Reply BertrandBru March29,2020at7:15am # Thankyousomuchforcomingtomesoquickly. ThisisexactlywhatIwaslookingfor. Cheers, Reply JasonBrownlee March30,2020at5:27am # You’rewelcome. I’mhappytohearthat. Reply DipakKambale March31,2020at10:16pm # HiJason, Igotaccuracy75.52.Isitok??pleaseletmeknow Reply JasonBrownlee April1,2020at5:49am # Welldone.Tryrunningtheexampleafewtimes. Reply islamuddin April1,2020at6:20pm # hellosirjason. sirhowtosatiableaccuracyrunthecodonegivenoutforexample86%nexttime82%howtosolvethis! #import fromnumpyimportloadtxt fromkeras.modelsimportSequential fromkeras.layersimportDense #loadthedataset dataset=loadtxt(‘E:/ms/impotnt/iwp1.csv’,delimiter=’,’) #splitintoinput(X)andoutput(y)variables X=dataset[:,0:8] y=dataset[:,8] #definethekerasmodel #model=Sequential() model=Sequential() #model.add(Dense(25,input_dim=8,init=’uniform’,activation=’relu’)) model.add(Dense(30,input_dim=8,activation=’relu’)) model.add(Dense(95,activation=’relu’)) model.add(Dense(377,activation=’relu’)) model.add(Dense(233,activation=’relu’)) model.add(Dense(55,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’)) #compilethekerasmodel model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) #fitthekerasmodelonthedataset model.fit(X,y,epochs=150,batch_size=10) #evaluatethekerasmodel _,accuracy=model.evaluate(X,y) print(‘Accuracy:%.2f’%(accuracy*100)) output 0.1153–accuracy:0.9531 Epoch149/150 768/768[==============================]–0s278us/step–loss:0.1330–accuracy:0.9401 Epoch150/150 768/768[==============================]–0s277us/step–loss:0.1468–accuracy:0.9375 768/768[==============================]–0s41us/step Accuracy:94.01 Reply JasonBrownlee April2,2020at5:46am # ThisisacommonquestionthatIanswerhere: https://machinelearningmastery.com/faq/single-faq/can-you-read-review-or-debug-my-code Reply MHusnainAliNasir April3,2020at2:55am # Traceback(mostrecentcalllast): File“keras_first_network.py”,line7,in dataset=loadtxt(‘pima-indians-diabetes.csv’,delimiter=’,’) File“C:\Users\Hussnain\anaconda3\lib\site-packages\numpy\lib\npyio.py”,line1159,inloadtxt forxinread_data(_loadtxt_chunksize): File“C:\Users\Hussnain\anaconda3\lib\site-packages\numpy\lib\npyio.py”,line1087,inread_data items=[conv(val)for(conv,val)inzip(converters,vals)] File“C:\Users\Hussnain\anaconda3\lib\site-packages\numpy\lib\npyio.py”,line1087,in items=[conv(val)for(conv,val)inzip(converters,vals)] File“C:\Users\Hussnain\anaconda3\lib\site-packages\numpy\lib\npyio.py”,line794,infloatconv returnfloat(x) ValueError:couldnotconvertstringtofloat:‘”6’ IAMHAVINTHEABOVEERRORWHILERUNNINGITPLEaSEHELP.IamusingAnaconda3,Python3.7,tensorflow,keras Reply JasonBrownlee April3,2020at6:57am # Sorrytohearthat,thiswillhelp: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply MadhawaAkalanka April9,2020at6:22pm # (base)C:\Users\MadhawaAkalanka\pythoncodes>pythonkeras_first_network.py UsingTensorFlowbackend. 2020-04-0913:42:28.003791:Itensorflow/core/platform/cpu_feature_guard.cc:142] YourCPUsupportsinstructionsthatthisTensorFlowbinarywasnotcompiledto use:AVXAVX2 2020-04-0913:42:28.014066:Itensorflow/core/common_runtime/process_util.cc:147 ]Creatingnewthreadpoolwithdefaultinteropsetting:2.Tuneusinginter_op _parallelism_threadsforbestperformance. Traceback(mostrecentcalllast): File“keras_first_network.py”,line12,in model.fix(X,Y,epochs=150,batch_size=10) AttributeError:‘Sequential’objecthasnoattribute‘fix’ Ihadthiserrorwhileit’sbeingrun.pleasehelp. Reply JasonBrownlee April10,2020at8:25am # Sorrytohearthat,seethis: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply RahimDehkharghani April14,2020at2:05am # DearJason Thanksforyourwonderfulwebsiteandbooks.IamaPhDholderandoneofyourfansinDeepLearning.SometimesIgetdisappointedbecauseIcannotachievemygoalinthisarea.Mygoalistodiscoversomethingnewandpublishit.AlthoughIunderstandyourcodesmostlybuthavingcontributioninthisfieldisdifficultandrequiresunderstandingthewholetheorywhichIhavenotbeenabletodosofar.Canyoupleasegivemesometipstocontinue?Thanksalot Reply JasonBrownlee April14,2020at6:25am # You’rewelcome. Keepworkingoniteveryday.That’smybestadvice. Reply MattGurney April16,2020at10:52pm # Thereisatypo“inputtothemodellisdefined” Reply JasonBrownlee April17,2020at6:21am # Thanks!Fixed. Reply MattGurney April16,2020at11:28pm # UsingthelatestlibrariestodayIgetanumberofwarningsduetolatestnumpy:1.18.1notbeingcompatiblewithlatestTensorFlow:1.13.1. i.e: FutureWarning:Passing(type,1)or‘1type’…(6times) to_int32(fromtensorflow.python.ops.math_ops)isdeprecated Optionsaretoreverttoanoldernumpyorsuppressthewarnings,Itookthesuppressroutewiththiscode: #firstneuralnetworkwithkerastutorial #SuppresswarningsduetoTF/numpyversionincompatibility:https://github.com/tensorflow/tensorflow/issues/30427#issuecomment-527891497 importwarnings warnings.filterwarnings(‘ignore’,category=FutureWarning) importtensorflow #SuppresswarningfromTF:to_int32(fromtensorflow.python.ops.math_ops)isdeprecated:https://github.com/aamini/introtodeeplearning/issues/25#issuecomment-578404772 importlogging logging.getLogger(‘tensorflow’).setLevel(logging.ERROR) importkeras fromnumpyimportloadtxt fromkeras.modelsimportSequential fromkeras.layersimportDense Reply JasonBrownlee April17,2020at6:21am # IrecommendusingKeras2.3andTensorFlow2.1. Reply MattGurney April17,2020at12:18pm # Yes,upgradingtotensorFlow2.1fixedit,IhavenowremovedmywarningssuppressionandIdon’tseethewarningsintheoutput IupgradedTFlikethis: pipinstall–upgradetensorflow Ididfollowyourinstallationinstructionsfromhttps://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/andendedupwithTFversion1.13.1.ThecommandIranwas: condainstall-cconda-forgetensorflow IamonMac,IseepossiblerelevantdiscussionhereonTF2.1notonconda:https://github.com/tensorflow/tensorflow/issues/35754 Reply JasonBrownlee April17,2020at1:31pm # Welldone! Iusemacportsmyself: https://machinelearningmastery.com/install-python-3-environment-mac-os-x-machine-learning-deep-learning/ Reply meryem April17,2020at1:25am # ThankyouJasonforthetutoriel.IappliedyourexampletominebyaddingdropoutandstandarisationofX X=dataset[:,0:7] y=dataset[:,7] scaler=MinMaxScaler(feature_range=(0,1)) X=scaler.fit_transform(X) #definethekerasmodel model=Sequential() model.add(Dense(6,input_dim=7,activation=’relu’)) model.add(Dropout(rate=0.3)) model.add(Dense(6,activation=’relu’)) model.add(Dropout(rate=0.3)) model.add(Dense(1,activation=’sigmoid’)) model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’] history=model.fit(X,y,epochs=30,batch_size=30,validation_split=0.1) _,accuracy=model.evaluate(X,y) print(‘Accuracy:%.2f’%(accuracy*100)) showsmeanaccuracyof100whichisnotnormal.toadjustmymodel,whatshouldIdo? Reply JasonBrownlee April17,2020at6:22am # Welldone! Perhapsevaluateyourmodelusingk-foldcrossvalidation. Reply meryem April17,2020at7:29am # yesifollowedyourexampleusingk-flodcrossvalidationitgivesmealways100% ifimovestandarisationhegives83%,canyouguidemeplease seed=4 numpy.random.seed(seed) dataset=loadtxt(‘data.csv’,delimiter=’,’) X=dataset[:,0:7] Y=dataset[:,7] fromsklearn.preprocessingimportStandardScaler sc=StandardScaler() X=sc.fit_transform(X) kfold=StratifiedKFold(n_splits=5,shuffle=True,random_state=seed) cvscores=[] fortrain,testinkfold.split(X,Y): model=Sequential() model.add(Dense(12,input_dim=7,activation=”relu”)) model.add(Dropout(rate=0.2)) model.add(Dense(6,activation=”relu”)) model.add(Dropout(rate=0.2)) model.add(Dense(1,activation=”sigmoid”)) model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) model.fit(X[train],Y[train],epochs=20,batch_size=10,verbose=1) scores=model.evaluate(X[test],Y[test],verbose=0) print(“%s:%.2f%%”%(model.metrics_names[1],scores[1]*100)) cvscores.append(scores[1]*100) print(“%.2f%%(+/-%.2f%%)”%(numpy.mean(cvscores),numpy.std(cvscores))) Reply JasonBrownlee April17,2020at7:48am # Nicework!Perhapsyourpredictiontaskistrivial? Reply meryem April17,2020at8:08am # youareveryhelpful. orbecauseIdon’thaveenoughdata.SothereisnothingelseIcanuse? Reply JasonBrownlee April17,2020at1:28pm # Perhaps. Reply FarjadHaider April17,2020at11:00pm # SirJasonyouareawesome!Suchaniceandeasytocomprehendthetutorial.GreatWork! Reply JasonBrownlee April18,2020at5:57am # Thanks! Reply JoanEstrada April19,2020at3:51am # “Note,themostconfusingthinghereisthattheshapeoftheinputtothemodelisdefinedasanargumentonthefirsthiddenlayer.ThismeansthatthelineofcodethataddsthefirstDenselayerisdoing2things,definingtheinputorvisiblelayerandthefirsthiddenlayer.” Couldyoubetterexplainthis?Thanks,nicework! Reply JasonBrownlee April19,2020at6:02am # Yes,seethis: https://machinelearningmastery.com/faq/single-faq/how-do-you-define-the-input-layer-in-keras Reply Hany April19,2020at9:57am # Actually,IcannotthankyouenoughDr.Brownlee. GodBlessyou. Reply JasonBrownlee April19,2020at1:14pm # Thanks.You’reverywelcome! Reply Rahim April22,2020at5:56am # DearJason Thanksforthisinterestingcode.Itestedthiscodeonpima-indians-diabetesinmycomputerwithkeras2.3.1butstrangelyIgottheaccuracyof52%.Iwonderwhythereisthismuchdifferencebetweenyouraccuracy(76%)andmine(52%). Reply JasonBrownlee April22,2020at6:10am # You’rewelcome. Perhapstryrunningtheexampleafewtimes? Reply Sarmad April24,2020at8:04pm # wanttoask:inthefirstlayer(ahiddenlayer)aswedefinedinput_dim=8w.r.tfeatureswehaveright.andwespecifyneurons=12.butconcernedisthatathingistudiedisthatwespecifyneuronsw.r.ttoinputs(features).Meansifwehave8inputssoneuronswillalsobe8.butyouspecifiedas12.Why? 2)Inanyofproblemwehavetospecifiedaneuralnetworkright.itcanbeanyeg:convolutional,recurrentetc.sowhichneuralnetworkwehavechoosehere.andwhere? 3)wehavetoassignweights.sowherewehaveassigned? pleaseletmeknow.Thankssir. Reply JasonBrownlee April25,2020at6:44am # Thefirstlineofthemodeldefines2things,theinputorvisiblelayer(8)andthefirsthiddenlayer(12).Morehere: https://machinelearningmastery.com/faq/single-faq/how-do-you-define-the-input-layer-in-keras Thesetwothingscanhavedifferentvalues,theyarenotdirectlyrelated. Yes,thiswillhelpyouchoosemodels: https://machinelearningmastery.com/when-to-use-mlp-cnn-and-rnn-neural-networks/ Weightsareassignedsmallrandomnumbersautomaticallywhenyoucallcompile(): https://machinelearningmastery.com/why-initialize-a-neural-network-with-random-weights/ Reply Sarmad April26,2020at7:44pm # sirstillconfusethatasinMLalgorithmwespecifywhichalgorithmtoimplementwrttoscenariolikeforregressionwecanchooselinearregression,logisticregressionetc. nowatthistimewhatneuralnetwehavechosen?convoltiona,rntnetc? Reply JasonBrownlee April27,2020at5:33am # Linearregressionisforregression,logisticregressionisforclassification. Herearesomeregressionalgorithmstotryonaregressiontask: https://machinelearningmastery.com/spot-check-regression-machine-learning-algorithms-python-scikit-learn/ Reply Sarmad April24,2020at8:31pm # wherearetheweights,biasandinputvalues? Reply JasonBrownlee April25,2020at6:46am # Weightsareinitializedtosmallrandomvalueswhenwecallcompile(). Reply mouna April26,2020at8:51pm # HelloJason, Congratulationsfroallthegoodjob,iwanttoaskyou: Howwecanknowofallepochstheaverageoftrainingtimeandvalidationtimeforamodel? Reply JasonBrownlee April27,2020at5:34am # Youcouldextrapolatethetimeofoneepochtothenumberofepochsyouwanttotrain. Reply JasonChia April28,2020at2:41pm # HiJason, Iamverynewtodeeplearning.Iunderstandthatyoudomodel.fittofitthedataandmodel.predicttopredictthevaluesoftheclassvariabley.However,isitalsopossibletoextracttheparameterestimateandderivef(X)=y(similartoregression)? Reply JasonBrownlee April29,2020at6:15am # Perhapsforsmallmodels,butitwouldbeamesswiththousandsofcoefficients.Themodeliscomplexcircuit. Reply Dina April28,2020at4:34pm # HiJAson,doyouhaveanideaonhowtopredictpriceorrangeofvalue? Reply Dina April28,2020at4:39pm # IfIusekerasmodeltopredictprice/rangeofvalue,itispossibleformetofindtheaccuracyofkerasmodel?becauseinyourarticleonlytopredictthebinaryoutput Reply JasonBrownlee April29,2020at6:19am # Youaredescribingaregressionproblem,Irecommendstartinghere: https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/ Reply JasonBrownlee April29,2020at6:17am # Apredictionrangeiscalledapredictioninterval,learnmorehere: https://machinelearningmastery.com/prediction-intervals-for-machine-learning/ Reply Hume May5,2020at10:54am # thankyouforyourexplanation,iamabeginnerformachinelearningaswellaspython.woluldyoupleasehelpmeingettingtheexactCSVdatafileforpredictingtheHepatitisBvirus. Reply JasonBrownlee May5,2020at1:37pm # Thiswillhelpyoulocateadataset: https://machinelearningmastery.com/faq/single-faq/where-can-i-get-a-dataset-on-___ Reply AbabouNabil May12,2020at2:01pm # 768/768[==============================]–2s3ms/step Accuracy:76.56 Reply JasonBrownlee May13,2020at6:21am # Welldone! Reply MAHESHMADHUSHAN May24,2020at11:29am # Whydidn’tyounormalizedata?Isnotthatnecessary?Ihaveseenonsometutorials,theynormalizedataforcommonscaleusingas–>fromsklearn.preprocessingimportStandardScaler.Whatisthedifferencethatmethodandyourmethod? Reply JasonBrownlee May25,2020at5:43am # Itcanhelpforsomealgorithmstonormalizeorstandardizethedatainputdata.Perhapstryitandsee. Reply HenryLevkine May26,2020at7:49am # Jason, Youarethebest! Mynameforyourprogramhereis“helloDL.py” Iamsureyourfuturebook“HelloDeepLearning”willbethemostpopularonthemarket. Peopleneedinprograms helloClassification.py helloRegression.py helloHelloPrediction.py helloDogsCats.py helloFaces.py andsoon! Thankyouforyourhardwork! Reply JasonBrownlee May26,2020at1:19pm # Thanks. Youcanfindalloftheseontheblog,usethesearch. Reply Thijs June12,2020at12:46am # Hello, isthereapossibilitytoaccesstheaccuracyofthelastepoch?Ifyes,howcaniaccessthisandsaveit? Kindregards Reply JasonBrownlee June12,2020at6:14am # Yes,thehistoryobjectcontainsthescorescalculatedoneachepoch: https://machinelearningmastery.com/display-deep-learning-model-training-history-in-keras/ Reply Krishan June16,2020at11:08am # Accuracy:82.42 epochs=1500 batch_size=1 Idon’tknowifwhatIdidwasappropriate.Anyadviseisappreciated. Reply JasonBrownlee June16,2020at1:39pm # Welldone! Reply Saad June19,2020at9:20pm # HiJason, Thanksalotforthiswonderfullearningplatform. Whywere12neuronsusedinthefirsthiddenlayer,whatisthecriteriabehindit?Isitrandomorthereisanunderlyingreason/calculation? (Ipresumedthatthenumberofneuronsinahiddenlayerwouldalwaysbebetweenthenumberofinputsandthenumberofoutputs) Reply JasonBrownlee June20,2020at6:12am # Ichosetheconfigurationafteralittletrialanderror. Thereisnogoodtheoryforconfiguringneuralnets: https://machinelearningmastery.com/faq/single-faq/how-many-layers-and-nodes-do-i-need-in-my-neural-network Reply ParasMemon July30,2020at9:05am # HelloJason, Ihavethisshapeoftrainingandtestingdatasets: xTrain_CN.shape,yTrain_CN.shape,xTest_CN.shape ((320,56,6251),(320,),(80,56,6251)) Iamgettingthiserror:ValueError:Errorwhencheckinginput:expecteddense_20_inputtohave2dimensions,butgotarraywithshape(320,56,6251) Belowisthecode: defnn_keras(xTrain_CN,yTrain_CN,xTest_CN): model=Sequential() model.add(Dense(12,input_dim=6251,activation=’relu’)) model.add(Dense(8,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’)) #compilethekerasmodel model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) #fitthekerasmodelonthedataset model.fit(xTrain_CN,yTrain_CN,epochs=150,batch_size=10) #evaluatethekerasmodel _,accuracy=model.evaluate(xTrain_CN,yTrain_CN) print(‘TrainingAccuracy:%.2f’%(accuracy*100)) _,accuracy=model.evaluate(xTrain_CN,yTrain_CN) print(‘TestingAccuracy:%.2f’%(accuracy*100)) nn_keras(xTrain_CN,yTrain_CN,xTest_CN) Reply JasonBrownlee July30,2020at1:44pm # AMLPmusttake2ddataasinput(rowsandcolumns)and1ddataasoutputduringtraining. Reply Joanne August12,2020at1:25am # HiJason, Thisisagreattutorial,veryeasytounderstand!!Isthereatutorialforhowtoaddweightandbiasintoourmodel? Reply JasonBrownlee August12,2020at6:11am # Thanks! Reply LuisCordero August20,2020at12:05pm # Hello,ifIhaveapredictionproblem,itisabsolutelynecessarytoscaletheinputvariablestousethesigmoidorreluactivationfunctionsortheoneyoudecidetouse? Reply JasonBrownlee August20,2020at1:37pm # No,buttryitandcompareresults. Reply LuisCordero August20,2020at1:15pm # howIcancreateaconfigurationthathasmorethanoneoutput,i.e.theoutputlayerhas2ormorevalues Reply JasonBrownlee August20,2020at1:39pm # Yes,justspecifythenumberoftargetsintheoutputlayerandprepareyourtrainingdataaccordingly. Ihaveatutorialonexactlythiswrittenandscheduled–fornextweekIthink. Reply LuisCordero September1,2020at4:29pm # whatwillbeennameoftutorialtofindit Reply JasonBrownlee September2,2020at6:24am # Righthere: https://machinelearningmastery.com/deep-learning-models-for-multi-output-regression/ Reply SimonSuarez August30,2020at8:27am # HiJason. Ithankyouforthegreatqualityofthisarticle.IamexperiencedwithMachineLearningusingScikit-Learn,andreadingthispost(andsomeofyourpreviousonthetopic)helpedmealottogetintomakingMultilayerPerceptrons. ItestedtheknowledgeIlearnedherewiththeWisconsinDiagnosticBreastCancer(WDBC)dataset.Igotaround92.965%Accuracyfortrainand96.491%fortest,onlyusing3features(radius,texture,smoothness)andthefollowingtopology: • Epochs=250 • Batch_size=60 • Funcióndeactivación=ReLu • Optimizador=‘Nadam’ Layer;Numberofneurons;Activationfunction Input;3;None Hidden1;4;ReLu Hidden2;4;ReLu Hidden3;2;ReLu Output;1;Sigmoid Trainandtestweresplittedusing:train_test_split(X,y,test_size=0.33,random_state=42) Thanks! Reply JasonBrownlee August31,2020at5:58am # Thanks. WelldoneonyourresultsSimon! Reply BernsBuenaobra September7,2020at7:32am # 0s833us/step–loss:0.4607–accuracy:0.7773 Reply JasonBrownlee September7,2020at8:36am # Welldone! Reply BernsBuenaobra September7,2020at7:37am # SeconditerationwithlaptopGPUgives: 0s958us/step–loss:0.4119–accuracy:0.8216 Accuracy:82.16 Reply AhmedNuru September8,2020at5:01pm # Hijansonhowcanpredictimageforgeryandgenuineusingpretraineddeep-learningmodel Reply JasonBrownlee September9,2020at6:44am # Perhapsprepareadatasetofrealandfakeimagesandtrainabinaryclassificationmodeltodifferentiatethetwo. Perhapsthistutorialwillhelpyoutogetstarted: https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-to-classify-photos-of-dogs-and-cats/ Reply FatmaZohra September11,2020at2:30am # HelloJason, CanyoupleaseguidemehowtomakeaqueryandadocumentasaninputinourNN(knowingthattheybotharerepresentedbyfrequencyvectors)? Reply JasonBrownlee September11,2020at6:01am # Perhapsstarthere: https://machinelearningmastery.com/start-here/#nlp Reply fatmazohra September13,2020at2:41am # HiDrJason, Thanksalotforthereply,thelinkwasusefulforme, yeti’amstilllostabitsincei’amnewdealingwithNN,actualyiwanttocalculatethesimilaritybetweenthequeryandthedocusingtheNN,theinputsare(theTFvectorofthedocandTFvectorofthequery,andtheoutputisthesimilarity(0ifno,1ifyes),ihavetheideaofmyNNbutidon’tknowfromwheretostart… iwouldbegratfulifyoucouldhelpme(asimilarcodethaticantakeasexemplemaybe), Waitingforyourreply..thanksinadvance Reply JasonBrownlee September13,2020at6:10am # Ithinkyou’reaskingaboutcalculatingtextsimilarity.Ifso,sorryIdon’thavetutorialsonthattopic. Reply fatmazohra September13,2020at6:38am # yeah,thisiswhatiwasaskingfor,anywaysthanksalotforyourtutorialstheyareveryclearandfruitful.. Reply JasonBrownlee September13,2020at8:28am # You’rewelcome. Reply yibrahfisseha September22,2020at11:41pm # Iwouldliketothankyoualotforyourtutorials.canyoupleaseguidemeonhowtoevaluatethemodelusingconfusionmatrixparameterssuchasrecall,precision,f1score? Reply JasonBrownlee September23,2020at6:40am # Yes,hereareexamples: https://machinelearningmastery.com/how-to-calculate-precision-recall-f1-and-more-for-deep-learning-models/ Reply derya September23,2020at5:03am # greattutorialhelpedalot! Reply JasonBrownlee September23,2020at6:44am # Thanks! Reply SeanH.Kelley September23,2020at6:38am # HiJason,thankyouverymuchforthis. Iappreciatetheextraindepthexplanationsinthelinkstootherpages. Iamwonderinghowtokeepthestateofmind.Likeyoutrainitwhileitrunsandgetalevelofaccuracy.Ifyoufinallygetthelevelofaccuracyfromtrainingacertainconfiguration,howdoyoukeepthatconfiguration/stateofmind/levelofaccuracyoftheartificialneuralnetwithouthavingtotrainitalloveragain? Canyoustoreasnapshotofthat“stateofmind”somewheresothatwhenyouhaveagoodworkingmodel,youjustusethattorunnewdataagainstoramIstillmissingsomekeyelementsinmyattemptingtograspthis? Thankyou! Reply JasonBrownlee September23,2020at6:46am # Youcansaveyourmodelandloaditlatertomakepredictions,seethistutorial: https://machinelearningmastery.com/save-load-machine-learning-models-python-scikit-learn/ Reply SeanH.Kelley September24,2020at12:53am # Thankyouverymuch! Reply JasonBrownlee September24,2020at6:16am # You’rewelcome. Reply MuhammadAsadArshed October10,2020at12:34am # Awesomeblogandtechnicalskillwouldyouliketorefermetosomeotherblogs. Reply JasonBrownlee October10,2020at7:06am # Thanks! Reply Brijesh October10,2020at5:57pm # Hi CanweuseonlyCSVfileformat? Reply JasonBrownlee October11,2020at6:44am # No,deeplearningcanuseimages,textdata,audiodata,almostanythingthatcanberepresentedwithnumbers. Reply imene October18,2020at4:49am # withepoch=10000andbatch-size=20agotaccuracy=84%andloss=loss:0.3434 Reply JasonBrownlee October18,2020at6:12am # Welldone! Reply YAŞARSAİDDERDİMAN December27,2020at4:12pm # thisisgoodbutprobably,yourmodel’sgeneralizationerrorishigher.Becausemoreepochmeansmoreoverfitting,Thereforeyoushoulduselessepochforanydeeplearningtraining. Reply JasonBrownlee December28,2020at5:58am # Goodadvice. Reply imene October18,2020at4:59am # firstthanksforyourgoodexplanation, howcanisavethetrainedmodeltobeusedfortestbecausthetrainnigrepeateachtimeitrytoexecutetheprogram tanks. Reply JasonBrownlee October18,2020at6:12am # Goodquestion,thiswillshowyouhow: https://machinelearningmastery.com/save-load-keras-deep-learning-models/ Reply Fatima October24,2020at5:18am # HiJason,IappliedtheDeepNeuralNetworkalgorithm(DNN)todotheprediction,Itworksanditisperfect,IhaveaprobleminevaluatingthepredictedresultsIused(metrics.confusion_matrix),Itgavemethiserror: ValueError:Classificationmetricscan’thandleamixofbinaryandcontinuoustargets anysuggestionstosolvetheerror? note:myclasslabel(outcomevariable)isbinary(0,1) Thanksinadvanced Reply JasonBrownlee October24,2020at7:12am # Seethistutorial: https://machinelearningmastery.com/how-to-calculate-precision-recall-f1-and-more-for-deep-learning-models/ Reply KAl October27,2020at2:53am # Firstofall,pleaseallowmetothankyouforthisgreattutorialandforyourvaluabletime. Iwonder:youtrainedandevaluatedthenetworkonthesamedataset.Whydidnotitgeneratea100%accuracythen? Thanks Reply JasonBrownlee October27,2020at6:46am # Allmodelshaveerror. Ifwegetperfectskill/100%accuracythentheproblemislikelytoosimpleandmachinelearningisnotrequired: https://machinelearningmastery.com/faq/single-faq/what-does-it-mean-if-i-have-0-error-or-100-accuracy Reply Zuzana November1,2020at11:15pm # Hi,greattutorial,everythingworks,exceptwhentryingtoaddpredictions,Igetthefollowingerrormessage.Couldyouplease,help?Thanksalot. WARNING:tensorflow:FromC:/Users/ZuzanaŠútová/Desktop/RTPnew/3_training_deep_learning/data_PDS/keras_first_network_including_predictions.py:27:Sequential.predict_classes(fromtensorflow.python.keras.engine.sequential)isdeprecatedandwillberemovedafter2021-01-01. Instructionsforupdating: Pleaseuseinstead:*np.argmax(model.predict(x),axis=-1),ifyourmodeldoesmulti-classclassification(e.g.ifitusesasoftmaxlast-layeractivation).*(model.predict(x)>0.5).astype("int32"),ifyourmodeldoesbinaryclassification(e.g.ifitusesasigmoidlast-layeractivation). Warning(fromwarningsmodule): File“C:\Users\ZuzanaŠútová\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\keras\engine\sequential.py”,line457 return(proba>0.5).astype(‘int32’) RuntimeWarning:invalidvalueencounteredingreater Traceback(mostrecentcalllast): File“C:\Users\ZuzanaŠútová\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\indexes\base.py”,line2895,inget_loc returnself._engine.get_loc(casted_key) File“pandas\_libs\index.pyx”,line70,inpandas._libs.index.IndexEngine.get_loc File“pandas\_libs\index.pyx”,line101,inpandas._libs.index.IndexEngine.get_loc File“pandas\_libs\hashtable_class_helper.pxi”,line1032,inpandas._libs.hashtable.Int64HashTable.get_item File“pandas\_libs\hashtable_class_helper.pxi”,line1039,inpandas._libs.hashtable.Int64HashTable.get_item KeyError:0 Theaboveexceptionwasthedirectcauseofthefollowingexception: Traceback(mostrecentcalllast): File“C:/Users/ZuzanaŠútová/Desktop/RTPnew/3_training_deep_learning/data_PDS/keras_first_network_including_predictions.py”,line30,in print(‘%s=>%d(expected%d)’%(X[i].tolist(),predictions[i],y[i])) File“C:\Users\ZuzanaŠútová\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\frame.py”,line2902,in__getitem__ indexer=self.columns.get_loc(key) File“C:\Users\ZuzanaŠútová\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\indexes\base.py”,line2897,inget_loc raiseKeyError(key)fromerr KeyError:0 Reply JasonBrownlee November2,2020at6:40am # Sorrytohearthat,thismayhelp: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply Zuzana November2,2020at6:50am # Iamsorrybutnoneofthathelped:/ Reply JulianAEpps November3,2020at7:58am # WherecanIfinddocumentationonthesekerasfunctionsthatyouareusing.Idon’tknowhowanyofthesefunctionswork. Reply JasonBrownlee November3,2020at10:08am # Goodquestion,here: https://keras.io/api/ Reply UmairRasool November8,2020at4:42am # HelloSir,iamnotactuallyfamiliarwithMLsosomeonedoingmytaskforpredictionusingrasterdatasetwithpython.HejustgivingfinalresultsandCSVfileratherthanfinalpredictionmapasraster,CouldyoupleaseguidemeMLworkslikethisorheismissingsomethingtogeneratefinalmap.PleaseResponse.Thanks Reply UmairRasool November8,2020at4:44am # sorryihavealittlemistake“finalresultasCSVfile” Reply JasonBrownlee November8,2020at6:42am # Perhapsthisframeworkwillhelp: https://machinelearningmastery.com/start-here/#process Reply Halil November27,2020at6:09am # Thankyouforthisbrilliantlyexplainedtutorial!Actually,Iamboredofwatchingvideoswhichhavelotsofboringtalksandsuperficialexplanations.Idiscoveredmymainresourcenow Bytheway,Iguessthereisanerrorhere.No? rounded=[round(x[0])forxinpredictions]—>shouldbe“round(X…..” Reply JasonBrownlee November27,2020at6:44am # You’rewelcome. Therearemanywaystoroundanarray. Reply Halil November30,2020at5:45am # Imean,that“x”shouldbe“X”.No? Reply RAJSHREESRIVASTAVA November28,2020at4:05am # Hijason, Hopeyouaredoingwell.IamworkingonANNforimageclassificationingooglecolab.Iamgettingthiserror,canyouhelpmetofindsolutionforthis? InvalidArgumentError:Incompatibleshapes:[100,240,240,1]vs.[100,1] [[nodegradient_tape/mean_squared_error/BroadcastGradientArgs(definedat:14)]][Op:__inference_train_function_11972] Functioncallstack: train_function Waittingforyourreply. Reply JasonBrownlee November28,2020at6:41am # Sorry,Idon’tknowaboutcolab: https://machinelearningmastery.com/faq/single-faq/do-code-examples-run-on-google-colab Reply RAJSHREESRIVASTAVA November28,2020at8:14pm # Hijasonthanksforyourreply. okinpythonIamworkingonANNforimageclassification.Iamgettingthiserror,canyouhelpmetofindsolutionforthis? InvalidArgumentError:Incompatibleshapes:[100,240,240,1]vs.[100,1] [[nodegradient_tape/mean_squared_error/BroadcastGradientArgs(definedat:14)]][Op:__inference_train_function_11972] Functioncallstack: train_function Reply JasonBrownlee November29,2020at8:12am # Sorry,thecauseoftheerrorisnotclear,youmayneedtodebugyourmodel. Herearesomesuggestions: https://machinelearningmastery.com/faq/single-faq/can-you-read-review-or-debug-my-code Reply Hanem December17,2020at11:07am # Thanksamillion,ithelpedmealot.Actually,allofyourarticlesareinformativeandgoogguideforme. Reply JasonBrownlee December17,2020at12:59pm # You’rewelcome,I’mhappytohearthat! Reply JohnSmith December28,2020at7:58am # ThiswasabrillianttutorialIthinkwhatcouldbedonetoimprovethisisaddinganexampleofactualpredictions. ThepredictionbitisquitebriefIdon’tquitehaveanunderstandinghowtousethatarrayof“predictions”toactuallypredictsomething. LikeifIwantedtofeeditsometestdataandgetapredictionhowcouldIdothat? Iwillconsultsomeofyourotherhelpfulguidesbutwouldbegreattohaveitallinthis1tutorial. Reply JohnSmith December28,2020at8:07am # IdidnothavemycoffeewhenIwrotethis. Iseenowwearepassingtheoriginalvariablesbackintothemodelandpredictingandprintingoutthepredicationvsactual. 🙂 Thanks–youmadeagreattutorial! Haveagoodchristmasandnewyear. Reply JasonBrownlee December28,2020at8:19am # Noproblematall! I’mhappyithelpedyoukickstartyourjourneywithdeeplearning. Reply Joe January3,2021at5:00am # HiJason, Happynewyear! Youarepredictingonthesamedataset,X,thatyouusedtotrainthemodel. Iwouldhavethoughtthatthemodelwould’veproducedcloseto100%accuracyinthiscasesincethemodelissowelltrainedspecificallywithrespecttoX(maybeevenoverfitted). Whyareweonlygetting76.9%accuracy,notcloseto100%? Thanks Joe Reply JasonBrownlee January3,2021at6:00am # Yes,Ithattokeeptheexamplesimple,Iexplainmorehere: https://machinelearningmastery.com/faq/single-faq/why-do-you-use-the-test-dataset-as-the-validation-dataset Nomodelisperfect,theyarealltryingtogeneralizefromthetrainingdata. Reply RobertoAguirreMaturana January7,2021at12:19pm # Excelenttutorial,wellexplainedandveryeasytofollow.Itseemsyouhavetoupdateonelinethatwasdeprecatedin2021: #insteadof #predictions=model.predict(X) #nowyouhavetouse predictions=(model.predict(X)>0.5).astype(“int32”) Reply JasonBrownlee January7,2021at2:04pm # Thanks. Idon’tthinkso: https://keras.io/api/models/model_training_apis/#predict-method And: https://www.tensorflow.org/api_docs/python/tf/keras/Sequential#predict Ifyouwantlabelsyoucanusemodel.predict_classes(),thiswillhelp: https://machinelearningmastery.com/how-to-make-classification-and-regression-predictions-for-deep-learning-models-in-keras/ Reply GirishAhire January8,2021at8:27pm # Igot65% Reply JasonBrownlee January9,2021at6:41am # Welldone! Reply TomRauch January15,2021at6:37am # Hi,IhavetheseinstalledinmyVirtualEnv(alongwithotherlibraries) Keras==2.4.3 Keras-Preprocessing==1.1.2 ButwhenIrunthis: #firstneuralnetworkwithkerastutorial fromnumpyimportloadtxt fromkeras.modelsimportSequential fromkeras.layersimportDense Igeta‘DeadKernel’errormessageinjupyter;thefirstlinerunsfinebutthe‘deadkernel’messageappearswhenitgetstokeras. Anyideaonhowtofix? Thanks! Reply JasonBrownlee January15,2021at8:46am # Irecommendnotusinganotebookastheycauseproblemsforalmosteveryone: https://machinelearningmastery.com/faq/single-faq/why-dont-use-or-recommend-notebooks Insteadsavethecodeusingasimpletexteditorlikesublimeoratomandrunthescriptfromthecommandline: https://machinelearningmastery.com/faq/single-faq/how-do-i-run-a-script-from-the-command-line Reply TomRauch January15,2021at9:32am # ThankyouJason!Iwillgivethecommandlineatry. Tom Reply JasonBrownlee January15,2021at11:32am # You’rewelcome. Reply TomRauch January15,2021at12:22pm # HiJason,IfollowedyourinstructionsbutstillrunningintoissueswithKeras,maybeIdidnotinstallitcorrectly? (rec_engine)[email protected]:~/code$pythonkeras.py Traceback(mostrecentcalllast): File“keras.py”,line3,in fromkeras.modelsimportSequential File“/home/tom/code/keras.py”,line3,in fromkeras.modelsimportSequential ModuleNotFoundError:Nomodulenamed‘keras.models’;‘keras’isnotapackage butwhenIrunthis,Idoseeitinstalled (rec_engine)[email protected]:~/code$piplist|grepKeras Keras2.4.3 Keras-Preprocessing1.1.2 Ifollowedthepipinstallfoundinthisguide: https://www.liquidweb.com/kb/how-to-install-keras/ IthinkmynextstepmaybetocreateanewVirtualEnvforjustKerasandTensorFlow. Thanks,Tom Reply JasonBrownlee January15,2021at1:26pm # Ithinktheremaybeanissuewithyourenvironment,perhapsthistutorialwillhelp: https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/ Reply GovindKelkar January15,2021at10:58pm # HiDr.Jason, IexecutedyourcodeingooglecolabandgotitexecutingonlychangeIfoundiswhilepredictingthenewdata youhadlistedthesequenceas10101andIgotitas01010 Alsodidthefewchangestothecode. NonethelessIgotthecodeworkingatleast.NowIwilltryandplaywithittogetmoreaccuracies. Reply JasonBrownlee January16,2021at6:55am # Welldone! Reply TomRauch January16,2021at9:18am # HiJason,IcreatedanewvirtualenvandloadedKeras,TensorFlowetcandcreateda.pywithallofyourcode,thenranitatthecommandlineinthedirectorythatcontainsboththecsvandpy. But,Igotthiserror: (ML)[email protected]:~/code$pythonmykerasloader.py Illegalinstruction(coredumped) IstherealoggerIshouldbeusingtoseemoredetail? Thanks,Tom Reply JasonBrownlee January16,2021at1:20pm # Thatdoesnotlookgood,Isuspectthereissomethingupwithyourenvironment. Perhapsyoucantryposting/searchingonstackoverflow.com Reply FranciscoSantiago January17,2021at9:51am # Creatingneuralnetwork 24/24[==============================]–0s756us/step–loss:0.3391–accuracy:0.8503 Accuracy:85.03 Wohooo!! Reply JasonBrownlee January17,2021at1:27pm # Welldone! Reply Jeremy January17,2021at4:38pm # Dr.Brownlee, Goodmorning,sir!Curiousforyourthoughtsonsomething:istherevalueinrunningthealgorithm,say,fiftytimesandaveragingtheaccuracy?I’veusedthattechniquebeforetogoodeffect,butsincethisisrelativelynewtome,havinganexperiencedteacherofmachinessetmestraightwouldbehelpful. Ifthisissomethingyouthinkisuseful,Ihaveonemorequestionthatcomesfrommystilllimitedunderstandingofthings:wherewouldIstartthe‘for’loop?Myfirstthoughtwasstartingitbefore‘model=Sequential()’,butthatwouldmeanredefiningtheNNstructureeachtime,whichdoesn’tmakemuchsense.Secondthoughtwasstartingitbefore‘model.fit()’,inwhichcasethemodelstaysthesame,andloss/optimizationfunctionsstaythesame. Thankyouverymuchforyourtime! V/r, Jeremy Reply JasonBrownlee January18,2021at6:05am # Yes,itreducesthevarianceinthemethodandcanbeusedforbothevaluatingmodelperformanceandmakingpredictions. Moredetailsarehere: https://machinelearningmastery.com/faq/single-faq/why-do-i-get-different-results-each-time-i-run-the-code Theloopisaroundthedefinition,trainingandevaluationofthemodel. Reply TomRauch January18,2021at6:33am # HiJason,anytutsonusingyourcodeinthispostinginGooglecolabs?Notsurehowtopointtothecsvusingcolabs. Thanks,Tom Reply JasonBrownlee January18,2021at8:58am # ThisisacommonquestionthatIanswerhere: https://machinelearningmastery.com/faq/single-faq/do-code-examples-run-on-google-colab Reply Anna January21,2021at8:53am # HelloJasonIhaveaquestion. Iwanttocreateamodeltopredicttheurbandevelopment.Istartedwithyourmodelabove. Iusetheinformationabouttheurbanandthenon-urbanpointsfor4years(2000,2006,2012,2018).Ialsouseinformationabouttheslopeandsomedistancesforeverypoint. Ihavecreateadatasetwitchcontainsinformationinthecolumnslikethis. 2000-2006 2006-2012 AfterthetrainIhaveaccuracy94% ButwhenIgivetothemodeltheyear2006itdoesn’tpredictthe2012verywell.Theremanyproblems. Ithoughtthatwiththisaccuracythemodelwouldhavepredictthe2012verywell. Idon’twhereitmightbetheproblem…Atthetrainsection,atthepredictorsomewhereelse?? PleasetellyouropinionbecauseIamstuckinthisforweeksandIhavetofindthesolutionquickly!!!! Reply JasonBrownlee January22,2021at7:13am # Itsoundslikeyourworkingwithatimeseriesdataset. Ifso,itwouldnotbevalidtotrainthemodelonthefutureandpredictthepast. Irecommendstartinghere: https://machinelearningmastery.com/start-here/#deep_learning_time_series Reply JamesParker January22,2021at8:43pm # ThankyouforthisgreatarticlebutIhaveaquestionwhatdoes_,beforeaccuracystandsfor Isearcheditontheinternetbutcouldn’tfindit Reply JasonBrownlee January23,2021at7:04am # Weuseunderscore(_)inpythontoeatupreturnvaluesorvariableswedon’tcareabout.Inthiscasetheloss,asweonlycareaboutaccuracy. Reply FOGANGFOKOA January24,2021at12:43pm # Hello, Inputanarrayof(50385,)whereeachisanarrayof(x,127)intoMLP) Iwanttoinputanumpy2darrayintoMLPbutIhaveanarrayof50395rowsthatcontainsmany2darrayofshape(x,129).xbecausesomematriceshavedifferentrownumbers.Hereisanexample: train[‘spec’].shape >>(50395,) train[‘spec’][0].shape >>(41,129) train[‘spec’][5].shape >>(71,129) Hereansnippetofmycode: X_train=train[‘spec’].values;X_valid=valid[‘spec’].values y_train=train[‘label’].values;y_valid=valid[‘label’].values model.add(Dense(12,input_shape=(50395,),activation=’relu’)); model.fit(X_train,y_train,validation_data=(X_valid,y_valid),epochs=500,batch_size=1); Igetthiserroronlastline(model.fit): ValueError:Errorwhencheckinginput:expecteddense_54_inputtohaveshape(50395,)butgotarraywithshape(1,) Howtofixthisproblemsothatthenetworkcantakeasinputall50395matricesofshape(x,129)? Reply JasonBrownlee January24,2021at12:52pm # Perhapstheyare“timesteps”andifsothismayhelp: https://machinelearningmastery.com/faq/single-faq/what-is-the-difference-between-samples-timesteps-and-features-for-lstm-input Andthenpadallsequencestothesamelength: https://machinelearningmastery.com/data-preparation-variable-length-input-sequences-sequence-prediction/ Reply FOGANGFOKOA January24,2021at1:40pm # InfactIabsolutelymustuseanMLP.Ihadsoundsof1soffrequency16000hz.Asaresult,allofmyaudiogavemeanarrayof16000.Afterremovingthesilenceinthoseaudios,Iendedupwitharraysofdifferentsizes. ThenItransformedtheseaudiointoanumpymatrixofnumbersusingthespectrogramealgorithmtoinputthemtotheneuralnetwork. Iendedupwithmatricesof2dimensionsofthesamecolumnsbutofdifferentrows. isitpossibletopasstheminknowingthatthematrixhavedifferentsizes? Reply JasonBrownlee January25,2021at5:47am # Asafirststep,perhapstrypaddingallinputstothesamesizeanduseamaskinginputlayerfollowedbydense/mlparchitecture. Reply FOGANGFOKOA January28,2021at12:56am # Ididasyouadvisedme.AndIpassedthisdifficultythere!Nowmycodelookslikethis model=Sequential(); model.add(Dense(units=8,input_shape=(71,129),activation=’relu’)); model.add(Dense(units=8,activation=’relu’)); model.add(Dense(units=11,activation=’sigmoid’)); #Compilemodel model.compile(loss=’categorical_crossentropy’,optimizer=’sgd’,metrics=[‘accuracy’]); #model=mpl_model(); X_train=list(train_df[‘spec’]);X_valid=list(valid_df[‘spec’]); y_train=train_df[‘label’];y_valid=valid_df[‘label’]; #labels=[‘yes’,‘no’,‘up’,‘down’,‘left’,’right’,‘on’,‘off’,‘stop’,‘go’]; encoder=LabelEncoder(); encoder.fit(y_train); encoded_y_train=encoder.transform(y_train); dummy_y_train=to_categorical(encoded_y_train); #Fitmodel,validation_data=(np.array(X_valid),y_valid) model.fit(np.array(X_train),np.array(list(dummy_y_train)),epochs=50,batch_size=50); andIgetthiserror: ValueError:Atargetarraywithshape(50395,11)waspassedforanoutputofshape(None,71,11)whileusingaslosscategorical_crossentropy.Thislossexpectstargetstohavethesameshapeastheoutput. Reply JasonBrownlee January28,2021at6:01am # Ouch,looksliketheshapeofthedatadoesnotmatchtheexpectationsofthemodel. Perhapsfocusontheprepareddataandinspectitaftereachchange–getthatright,thenfocusonthemodelingpart. Reply FOGANGFOKOA January29,2021at7:28am # Okay.It’sdoneandItworkswell..thankyou Reply JasonBrownlee January29,2021at7:40am # Nicework! Reply KinsonVERNET January29,2021at1:06am # Hello,thankyouforthistutorial. For100timesIgotscore=76.82fortheaccuracy. Reply JasonBrownlee January29,2021at6:06am # Welldone! Reply Kamal January30,2021at12:14pm # It’sasuperbtutorialtoimplementyourfirstdeepneuralnetworkinPython.Thankyou,dearJasonBrownlee. Reply JasonBrownlee January30,2021at12:35pm # Thanks,welldoneonyourprogress! Reply Rob February18,2021at1:59pm # Hithere, I’mcurrentlystuckonfittingthemodel.OnlythingIhavedonedifferentlyisuseread_csvsoIdidn’thavetoputanythinglocally.ButI’vevalidatedtheX/youtputstobethesame. Myerroris: ValueError:logitsandlabelsmusthavethesameshape((None,11)vs(None,1)) Reply JasonBrownlee February19,2021at5:53am # Itsuggestsyourdatawasnotloadedcorrectly,perhapsthiswillhelp: http://machinelearningmastery.com/load-machine-learning-data-python/ Reply Rob March1,2021at11:59pm # Ahthanks,itturnsoutitwasanissuewiththewrongnumberofnodesonthesigmoidlayer. Reply JasonBrownlee March2,2021at5:45am # Happytohearyousolvedyourproblem! Reply Sofia February24,2021at3:53am # Anothergreattutorial!! WhenIruntheprogramitcrasheswithanerrorasseenbelow: 2021-02-2318:50:50.497125:Wtensorflow/stream_executor/platform/default/dso_loader.cc:59]Couldnotloaddynamiclibrary‘cudart64_101.dll’;dlerror:cudart64_101.dllnotfound 2021-02-2318:50:50.498601:Itensorflow/stream_executor/cuda/cudart_stub.cc:29]IgnoreabovecudartdlerrorifyoudonothaveaGPUsetuponyourmachine. Traceback(mostrecentcalllast): File“C:/Users/USER/PycharmProjects/Sofia/main.py”,line26,in X=dataset[:,0:8] File“C:\Users\USER\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\core\frame.py”,line3024,in__getitem__ indexer=self.columns.get_loc(key) File“C:\Users\USER\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\core\indexes\base.py”,line3080,inget_loc returnself._engine.get_loc(casted_key) File“pandas\_libs\index.pyx”,line70,inpandas._libs.index.IndexEngine.get_loc File“pandas\_libs\index.pyx”,line75,inpandas._libs.index.IndexEngine.get_loc TypeError:‘(slice(None,None,None),slice(0,8,None))’isaninvalidkey HowwouldIgoaboutfixingthiserror?Thankyouinadvance! Reply JasonBrownlee February24,2021at5:38am # Thanks! Sorrytohearthat,perhapsthesetipswillhelp: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply Slava February27,2021at3:46am # Itlookslikethemodel.predict_classes()wasdeprecatedon2021-01-01. Cheers, Slava Reply JasonBrownlee February27,2021at6:09am # Thanks. Reply AtsushiIsobe March3,2021at11:26pm # Whatisthenewmethodtouse?Icannotrunthepredictmethodafterfinishingthetraining. Reply JasonBrownlee March4,2021at5:50am # Thiswillhelp: https://machinelearningmastery.com/how-to-make-classification-and-regression-predictions-for-deep-learning-models-in-keras/ Reply Mitchell March11,2021at8:16am # Jason,Ihaveacoupleofquestionsregardingthelayersandhowtheychoosefilters. model=Sequential() model.add(Dense(12,input_dim=8,activation=’relu’)) model.add(Dense(8,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’) 1)Whatisthefiltersizeforeachlayerabove?3×3or7×7. 2)Arethereanypre-defined3×3filters,7×7filers,? 3)Inhiddenlayers,filtersareusedtoproducenextlayerusually.Howdoesthemodelchoosefilters?Forexample,ifalayerhas16nodes,andhowwouldIchoose32filterssothatthenextlayerwillhave32nodes(neurons)? Whenyoucreateamodel,doyouneedtospecifyfiltersforeachlayerneeded?likesizeofafilterandhowmanyfilters.. Thanks! Reply JasonBrownlee March11,2021at1:25pm # TherearenofiltersinaDenselayer,filtersissomethingtodowithconvolutionallayers: https://machinelearningmastery.com/convolutional-layers-for-deep-learning-neural-networks/ Reply marineboy March12,2021at8:22pm # helloJason ihaveaproblem!canuhaveme: whenIpredict_classes(Z)#Z=[100,100,100,100,100,100,100,100]asyouseethisdatasodifferencebutoutputstill0or1.iwantoutput=don’tknowlabel:(((((howcanimakeitplshaveme thanksyousomuch,sir Reply JasonBrownlee March13,2021at5:29am # Sorry,Idon’tunderstand. Perhapsyoucanrephrasetheproblemyou’rehaving? Reply Franklin March17,2021at3:00pm # It’sanawesomeblog.Keepthegoodwork. Reply JasonBrownlee March18,2021at5:15am # Thanks! Reply Hamza March19,2021at12:38am # 79.53accuracy Reply JasonBrownlee March19,2021at6:23am # Welldone! Reply OriyomiRaheem March20,2021at6:06am # Iamtryingtotrainapermeabilitydatainlasfileandpredictthemafterwards.Pleasehelp Reply JasonBrownlee March21,2021at6:00am # Perhapsthisprocesswillhelpyoutoworkthroughyourproject: https://machinelearningmastery.com/start-here/#process Reply Bangash李忠勇 March31,2021at6:41pm # accuracy:0.7865 Accuracy:78.65 Reply JasonBrownlee April1,2021at8:08am # Welldone! Reply Pankaj April23,2021at7:19am # Withcategoricalfeatures,howwouldIpreventaKerasmodelfrommakingapredictionontestsamplesthatithasnotseeninthetrainingset,andinsteadeitheruseanothermodelorthrowanexception? Reply JasonBrownlee April24,2021at5:13am # Sorry,Idon’tunderstand.Perhapsyoucanelaborate? Reply Luca April26,2021at8:31pm # Allthecontentyoucreateandofferisabsolutelyamazing. Veryinformative,veryup-to-dateandcristal-clear. THANKYOU! Reply JasonBrownlee April27,2021at5:16am # You’rewelcome. Reply RonaldSsebadduka May5,2021at4:53pm # File“/Users/ronaldssebadduka/PycharmProjects/pythonProject1/venv/lib/python3.9/site-packages/numpy/lib/npyio.py”,line1067,inread_data items=[conv(val)for(conv,val)inzip(converters,vals)] File“/Users/ronaldssebadduka/PycharmProjects/pythonProject1/venv/lib/python3.9/site-packages/numpy/lib/npyio.py”,line1067,in items=[conv(val)for(conv,val)inzip(converters,vals)] File“/Users/ronaldssebadduka/PycharmProjects/pythonProject1/venv/lib/python3.9/site-packages/numpy/lib/npyio.py”,line763,infloatconv returnfloat(x) ValueError:couldnotconvertstringtofloat:‘\ufeff”6’ Iˆgetthiserrorwhenirunyourcode! HowcanIfixit? Reply JasonBrownlee May6,2021at5:42am # Sorrytohearthat,perhapssomeofthesetipswillhelp: https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-me Reply Shilpa May28,2021at4:43am # Contentsareexplainedinasimplewayandaresoclear.ThanxJason Reply JasonBrownlee May28,2021at6:49am # You’rewelcome. Reply ToniNehme May28,2021at7:56pm # PleasepleasehelpmetobuildaMultilayerPerceptrontouseitforregressionproblem.Thankyou Reply JasonBrownlee May29,2021at6:50am # Sure,seethis: https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/ Reply JamesMayr May29,2021at11:02pm # Thankyousooomuchforyourtutorial!IstruggledaroundwiththeinputlayerandtheKerashelpwasnothelpful.Butyourexplanationgavemetheinsightandthethingsbecametotalclear!Thatwasverygreat,Thankyou! Reply JasonBrownlee May30,2021at5:50am # You’rewelcome! Reply Meenakshi June3,2021at8:28pm # GreatworkSir.Simple,detailedexplanationofcomplexthings. IwouldliketolearnmodellingforDDoSattacksdetectioninNeuralnetworks.Pleasesuggesttheway. Tanksinadvance. Reply JasonBrownlee June4,2021at6:48am # Perhapsthetutorialsherewillhelpifyouaremodelingyourproblemasatimeseries: https://machinelearningmastery.com/start-here/#deep_learning_time_series Reply Meenakshi June5,2021at11:34pm # Thankyouverymuch.IwillgothroughitSir. Reply JasonBrownlee June6,2021at5:51am # You’rewelcome. Reply JC June24,2021at4:13am # Thefollowingaretheoutcomeofthefirst10consecutiveexecutionsonmy8GBRAM64bitWindows10platform: Accuracy:65.49 Accuracy:70.70 Accuracy:75.91 Accuracy:76.04 Accuracy:78.26 Accuracy:76.04 Accuracy:77.86 Accuracy:79.17 Accuracy:78.52 Accuracy:78.91 ThecomputerdoesnothaveGPU.Thescriptgivessomewarningmessages.Oneofthemis:“NoneoftheMLIROptimizationPassesareenabled(registered2)” Reply JasonBrownlee June24,2021at6:06am # Welldone! Reply Sneha July2,2021at8:31am # Hi, Ihaveaquestionregardingtheinputamount.Iamattemptingtofitaneuralnetworkforaclassificationmodel.However,thefeaturesinmymodelarecategoricalsoIneedtoone-hotencodethem.Forinstance,ifacategoricalvariablehas3valuesandIone-hotencodeit,wouldthatmake‘input_dim’1or3? Reply JasonBrownlee July3,2021at6:05am # Yes,categoricalvariableswillneedtobeencoded. 3categorieswillbecome3binaryinputvariableswhenusingaonehotencoding. Reply Rohan July3,2021at10:15am # Myresults: Accuracy:75.78 Accuracy:78.26 Accuracy:76.30 Accuracy:77.47 Accuracy:77.47 Reply JasonBrownlee July4,2021at5:58am # Welldone! Reply Patrick July10,2021at8:32pm # HiJason, Thankyouforallofyourcontent.AllveryinsightfulforsomeonenewtoKerasandmachinelearning.Ifyoucouldofferanyguidance/insightintothebelowproblemI’mtryingtotackle,thenitwouldbemuchappreciated. IamtryingtoreplicateasimilarBallPredictionModelasdiscussedhere: https://towardsdatascience.com/predicting-t20-cricket-matches-with-a-ball-simulation-model-1e9cae5dea22 Thisisamulticlassifcationproblem(thankyouforyourarticleonthis).Thereare8outputsthatIamtryingtopredict(0,1,2,3,4,6,Wide,Wicket)columnHinmydataset(https://i.stack.imgur.com/DmTNb.png). Thisdatasetisball-by-ball(match)dataofmanycricketmatches.ColumnsA-Garetheinputvariablesthatshouldbeusedtopredicttheprobabilityofeachoutcome(innings,over,batsman,bowleretc.) Model: X=my_data[:,0:7] y=my_data[:,7] model=Sequential() model.add(Dense(12,input_dim=7,activation=’relu’)) model.add(Dense(8,activation=’relu’)) model.add(Dense(1,activation=’sigmoid’)) model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) model.fit(X,y,epochs=150,batch_size=10,verbose=0) _,accuracy=model.evaluate(X,y,verbose=0) print(‘Accuracy:%.2f’%(accuracy*100)) Runningtheabovemodelontheball-by-balldatasetgivesanaccuracyof30%.Asthearticlesuggests,Iwanttoincludemoredatai.e.thehistoricalprobabilityofeachindividualbatsmanandbowlerachievingeachofthe8outcomes. ThismeansIhave3datasetswhichshouldbeusedtoinfluencetheprobabilityofeachoutcome. HowandwhenshouldIbetryingtointroducethese3linkeddatasets?Ipresumablywantthemodeltoconsiderallthisinformationatthesametimeandnotinisolation. Isitacaseoftryingtoincorporatethebatsman/bowlerdatasetsintothematch-by-matchdata?TheonlyissueIhavewiththisisthattherearec.200,000rowsofmatchdata,whereasaplayerdatabasewillhavec.500rows. MaybeIamwrong,andIshouldberunningthemultipledatasetsthroughthemodelindividuallyandthensomehowpoolingtheoutcomes–isthisevenpossible?AlthoughIdoubtthatthisisevenrecommended/worthwhile Ifyouhaveanysuggestionsonhowtoimprovetheabove,orachievethedesiredoutcome,thenitwouldbemostwelcomed. Thanksagainforallyourhardworkinmaintainingagreatdatasciencesite. Reply JasonBrownlee July11,2021at5:39am # Definingthedata/problemforamodelistherealworkinappliedmachinelearning. Thereisnogood/bestway,Irecommendreadingpapersonorrelatedtothetopictogetideas,prototype,experiment,etc. Also,thismayalsohelpondefiningtheproblem: https://machinelearningmastery.com/how-to-define-your-machine-learning-problem/ Also,moregenerally,thesetutorialsexplainhowtogetbetterperformancefromneuralnets: https://machinelearningmastery.com/start-here/#better Reply JoleneWang July23,2021at5:08am # HiJason! Thankyouforprovidingallofthiscontent.IamtryingtoreplicatethismodelbyusingmyowncsvfilehoweveritcontainsmanyNaNandthuscannotbeloadedthroughtheloadtxt()function.As0isaveryimportantnumberinmydataset,IcannotchangemyNAsto0.WhatcanIdo? Thankyouagainforallofyourhelp. Reply JasonBrownlee July23,2021at6:04am # Youmustimputethemissingvaluesfirst,therearemanymethods: https://machinelearningmastery.com/?s=missing&post_type=post&submit=Search Reply JoleneWang July23,2021at5:13am # IforgottomentionbutisthereawayformetokeeptheNaNinthedatasetandhavethemodelreaditasjustamissingvalue?ItwouldbedifficultformetoassigntheNaNsaspecificvalueasitcouldmessupthedataset. Reply JasonBrownlee July23,2021at6:04am # No.NaNwillcauseallcomputationtofailinamlmodel,includinganeuralnet. Reply IsiyakuSaleh July31,2021at10:09am # ThankverymuchDr,Jasonthetutorialhasreallyservedbewell. Reply JasonBrownlee August1,2021at4:49am # You’rewelcome! Reply TimPapa August3,2021at8:02pm # Thistutorialbuildsaneuralnetwork,butwhatspecificallythisneuralnetworkis?IsitanANNorCNNorRNN? Reply JasonBrownlee August4,2021at5:13am # Itisamulti-layerperceptron(MLP)whichisatypeoffeed-forwardneuralnetwork.ItisnotaCNNorRNN. Reply EdwinBrown August13,2021at7:26am # Firstandforemost,thankyouJasonBrownleeforgettingmestartedwithmyfirstdeeplearningproject.Ifollowedstep-by-stepandfoundmyselfstuckforawhile;however,aftercountlesshoursofresearchingIfoundmycodebelowtoworkforPython3.8.10,Tensorflow2.5.0,IPython7.26.0,andKeras2.6.0respectedenvironments.IapologizeifIovercommented,IwastakingnotesasIwasreadingthroughJason’ssourcecodesandnotes.IusedAnaconda-SpyderandIwantedtoseetheresultsaswellinJupyterNotebook.Ihopethishelps: importsys importtensorflowastf fromtensorflowimportkeras fromnumpyimportloadtxt fromtensorflow.keras.modelsimportSequential fromtensorflow.keras.layersimportDense #LoadthedataandsplittheX(input)&y(output)variables #Besureyourdataisintherespectedfileastheproject dataset=loadtxt(r’pima-indians-diabetes.csv’,delimiter=’,’) X=dataset[:,0:8] y=dataset[:,8] #Createoursequentialmodel #input_dimsetsnumberofarguementsforthenumberofinputvariables #Thisstructureasthreelayers #Fullyconnectedlayersaredefinedbythedenseclass #formoreondenseclassviewonKerashomepage #ReLUonthefirsttolayersandSigmoidfunctionontheoutputlayer(thirdlayer) #Defaultthresholdof0.5andbetterperformancefromReLU #ReLUmeasuresoutputbetween0and1asseeninprobability #Themodelexpectsrowsofdatawith8variables(theinput_dim=8argument) #Thefirsthiddenlayerhas12nodesandusesthereluactivationfunction. #Thesecondhiddenlayerhas8nodesandusesthereluactivationfunction. #Theoutputlayerhasonenodeandusesthesigmoidactivationfunction. model=Sequential() model.add(Dense(12,input_dim=8,kernel_initializer=’normal’,activation=’relu’)) model.add(Dense(8,kernel_initializer=’normal’,activation=’relu’)) model.add(Dense(1,kernel_initializer=’normal’,activation=’sigmoid’)) #Compilethemodel model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) #Fitthemodelontothedataset #Epoch:Onepassthroughalloftherowsinthetrainingdataset. #Batch:Oneormoresamplesconsideredbythemodelwithinanepochbeforeweightsareupdated. #TheCPUorGPUhandlesitfromhere,usually,largerdatasetsneedtheGPU model.fit(X,y,epochs=150,batch_size=10,verbose=0) #Evaluatethedata _,accuracy=model.evaluate(X,y,verbose=0) print(‘Accuracy:%.2f’%(accuracy*100)) #makeprobabilitypredictionswiththemodel predictions=model.predict(X) #roundpredictions rounded=[round(x[0])forxinpredictions] Reply AdrianTam August14,2021at2:33am # Goodwork! Reply Bonjour20 August15,2021at9:43pm # IuseWindowssystemonmylaptop,andIdonotknowifIshouldhaveaLinuxdestro>IamconfusedaboutwhereshouldIdownloadtheDataset>Hementioned:”onthesameplacewhereptyhonisinstalled”,whatisthisriddle? Itisariddleforabeginnerlikemecomingfromnontechnologicalbackground. Reply AdrianTam August17,2021at7:30am # Usuallythatmeans,youjustneedtoplacethedatafilesandthepythoncodefiletogetheratthesamefolder. Reply samasamaan August30,2021at6:19am # Hello Thanksforthisgreattutorial🙂 Questionno.1:canweapplydeeplearninginApacheSpark? Questionno.2:Ihavethefollowingdatasethttps://www.kaggle.com/leandroecomp/sdn-traffic Itriedthemulti-classclassificationcodebutitstopworking.Whatcouldbethereasonbehindthatfault? Thanks Reply AdrianTam September1,2021at7:39am # (1)yes(2)whatspecificallystoppedworking? Reply MALAVIKA September23,2021at11:17pm # Firstofall,Iamoverwhelmedbythenumberofcommentsandpromptrepliesbytheauthor.Youarereallyalifesavertomany,Jason. Now,Ihaveadoubt.Ihavebeensearchingforasimplefeed-forward-back-propagationANNcodeinpython,andIcouldseeonlyfeed-forwardneuralnetworkseverywhere.Inyourexample,isbackpropagationhappening?Doesn’tANNmeanboththeprocessesbydefault? Shouldn’tweapplybackpropagationinANN,normally? Reply AdrianTam September24,2021at4:41am # Feed-forwardhappenswhenyougiveinputtotheANN.Backpropagationhappenswhenyoucalculatethegradientandupdatetheweightsineachneuron. Reply MALAVIKA September24,2021at5:06pm # So,Isupposeit’s(back-propagation)nothappeningintheabovetutorial.Canyoushowushowtocodetheback-propagationinpython,ordirectmetoanypoststhatshowthesame? ThankYou. Reply AdrianTam September25,2021at4:36am # Whenyoucallfit()function,backpropagationisusedtoupdatethemodelparameters.That’spartofthetrainingprocess.Wedon’tnormallydothisexplicitly.Ifyouareinterested,seeatoyexamplehere:https://machinelearningmastery.com/implement-backpropagation-algorithm-scratch-python/ Reply Elham October8,2021at1:11am # Hi,Thanksalotforthisawesometutorial.I’musingtensorflowversion2.6andinmakingclasspredictionswiththemodelwiththeselinesofcode, predict_x=model.predict(X) classes_x=np.argmax(predict_x,axis=1) foriinrange(5): print(‘%s=>%d(expected%d)’%(X[i].tolist(),classes_x[i],y[i])) theoutpoutis: [6.0,148.0,72.0,35.0,0.0,33.6,0.627,50.0]=>0(expected1) [1.0,85.0,66.0,29.0,0.0,26.6,0.351,31.0]=>0(expected0) [8.0,183.0,64.0,0.0,0.0,23.3,0.672,32.0]=>0(expected1) [1.0,89.0,66.0,23.0,94.0,28.1,0.167,21.0]=>0(expected0) [0.0,137.0,40.0,35.0,168.0,43.1,2.288,33.0]=>0(expected1) Whyareallclasses_xzero? Reply AdrianTam October13,2021at5:09am # Becausethepredictionhereisabinaryone,hencepredict_xisNx1matrixwhichargmaxwillonlyreport0.Yoursyntaxiscorrectformulti-class,whichtheneuralnetworkhasoutputlayerasDense(n)withn>1 I’veupdatedthesamplecodeheretoreflectwhatyoushoulddo.Thanksforalertingme. Reply christoper October17,2021at6:41am # hellothisishelpful.Iamstudyingneuralnetworksandimjustabeginner.Yousaidthisismlptypeofneuralnetworkright?Ijustwanttoask,howaboutthis?Whatkindofneuralnetworkarchitectureusedhere?isitrnn?ann?orltstm?linkbelow: https://towardsdatascience.com/how-to-create-a-chatbot-with-python-deep-learning-in-less-than-an-hour-56a063bdfc44 Reply AdrianTam October20,2021at8:52am # MLP=MultilayerPerceptron,whichusuallymeansaneuralnetworkwith3ormorelayers.ThelinkyouprovideduseDense(),whichisfully-connectedlayer.HenceitisalsoMLP. Reply Flo October25,2021at11:29pm # HiJasonandAdrian,Icameacrossyourverynicetutorial,becauseIhaveaquitesimilarproblem. Ihaveacoupleofnumericalprocessparametersofanengineeringproblem(similartoyourinputparametershere),whichIwanttochecktoanoutcomevalue(whichisdifferenttoyourtutorialagainanumericalvalue,notaclassification).Canyoutellme(ordoyouevennowaaccordinglyhandytutoriallikethisone),howIneedtomodifythecode? Thanksalot! Reply AdrianTam October27,2021at2:23am # Itsoundstomethatitisaregressionprobleminsteadofclassificationproblem.Inthiscase,twothingsyoumayconsidertochange 1.ThelastDense()layer,youmaywantadifferentactivation(e.g.,linear?)becausesigmoidalisboundedbetween0and1 2.model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’])shouldhavealossandmetricchanged.Forexample,youmayconsidertouseMSEbecausecrossentropyandaccuracyaremeasuresspecifictoclassification Reply DrShaziaSaqib October28,2021at3:14am # awesome,greatservice,veryhelpful,amsharingwithmystudents,LordBlessyouameen Reply veejay November5,2021at11:23pm # Awesometutorial,verywell-detailed.Ihaveaquestionthough, HowtoimproveValidationLossandValidationAccuracy?IamverynewtoNeuralNetwork.Ionlyscratchedthesurfaceofit.Weights,biases,activationfunction,lossfunction,architecturesandhowtobuildlayersonkerasandotherfundamentalterminologies(thanksfromyouanddeeplizardtutorialsfromyoutube.)IamstudyingandpracticingitandIwanttotryandreplicatesomeprojectandIcameacrossthistutorialfromDataflairwherehe’screatingachatbotandItriedtoimitateit.LINK:https://data-flair.training/blogs/python-chatbot-project/. SofromwhatIhaveobservedandbasedonmylearnings,themodelthathecreatedisanANN-MLP.Myproblemis,whenItrainedthemodelandsetthevalidation_split=0.3,thetraininglossandaccuracyaregoodbutthevalidationlossandaccuracydotheopposite.Iknowthatitmaybeanoverfittingproblemso… Here’swhatIdid: -addedregularizationwithL2 –SlowedthelearningrateandIalsotriedtospeeditup -Dropouts(0.2-0.5) -BatchSize -Removinglayers -Addinglayers -Experimenteddifferentactivationandlossfunctions(sigmoid,softplus,binary_crossentropy) -Ieventriedtoadddataonmydatasets(from320to796inputs) Itriedallofthisbutval_lossandval_accstillhighandlowrespectively. (BestthatIdidisloss:0.1/accuracy98percentvalloss:1.9/val_accuracy:52percent. whiletheworstisval_loss:over3.0andval_accuracy35-40percent) Thedatasetthati’musingisfromdataflairbutIexpandedit.here’smyvisualizedmodel:https://i.stack.imgur.com/HE1jU.png Reply AdrianTam November7,2021at10:35am # Can’treallytellwhatwentwronghere.Didyouverifythevalidationlossasyoutrainedit?Atfirst,thetraininglossandvalidationlossshouldbeequallybad.Howdidtheyprogressedineachtrainingepoch?Thismaygiveyouclues. Reply Veejay November9,2021at6:54am # YesIbothtrainedandvalidatethem.Theyareequallybadatfirstandastheyprogressed,thelossimprovedbymilesbutval_lossandval_accuracyimprovedaninch.T_T Reply AdrianTam November14,2021at1:36pm # That’sexpected.Youmodelwaslookingatthetraininglossandtrytoimproveitself,butitwasnotabletoseethevalidationdatasoitisharderandslowertoimprove. Reply Mak November17,2021at6:24pm # YourbookshelpedmeunderstandLSTMsgreatly,Iamhavingtroublewithdevelopinganattentionlayer,pleasecanyoudoatutorialonusingAttention/MultiheadAttention Thankyou. Reply AdrianTam November18,2021at5:38am # Pleaseseetheseries:https://machinelearningmastery.com/category/attention/ Reply NikhilGupta November25,2021at5:47pm # TheaccuracyfromANNforthisdatasetisbetween70-78%.UsingLogisticsRegression,wearegetting78%accuracyforthesamedataset.So,what’stheadvantageofusingANN? Reply AdrianTam November26,2021at2:09am # ANNismoreflexible.Occam’srazor–youusethesimplestmodelforthejob.Iflogisticregressionfitswell,youhavenoreasontouseANN.Itusemorememoryandrunsslower. Reply Flo December3,2021at8:38pm # ThanksfortheTutorial Itriedyourapproachanditworkednicelyonmydata.ForafirstshotIjustuseddata,whichismeasuredaftertheprocess(e.g.processtime,temperaturedifferenceduringtheprocess,etc.).Forafurther,deeperinvestigation,Iwouldliketousemeasureddatacurves,forexamplethedevelopmentoftheprocesstemperaturebytimeduringtheprocessitself.Byuseofthesecurves,Iexpectahigherdegreeofinformation. Couldyouprovideahint,howtoworkwiththisdata?ForthefirstshotIsimplygeneratedatablewithmyprocessparametersinthefirst6columnsandmyoutputvalueincolumn7,whichcouldbeeasilyfeededintothemodell. Thanksalot! Reply AdrianTam December8,2021at6:59am # Everythingsoundsstraightforwardtome.Didyoutriedimplementedthis?Anyerror? Reply Flo December10,2021at6:32pm # Tobehonest,Ihavenocluehowtoprovidethedata.Inthefirstcase,Ihadatablewith7columns:6Inputprocessparameterandonecolumnwithoutputvalues. NowIwouldliketoreplace(areadd)someinputcolumnswithtime-recordeddatacurves,whicharesomehowtables(firstcolumnthetimestamp,secondcolumnthetime-specificprocessparameter)itself.HowdoIworkwiththis? Reply AdrianTam December15,2021at5:38am # UsuallyIwouldhavepandastoprocessdataandconvertittonumpyarraybeforefeedingtoKerasmodel.Pandasallowsyoutomanipulatetableseasier Reply Rick December28,2021at7:46am # MayneedtoadjusttheimportsettingsforcompatibilitywithnewerTensorflowversions. Insteadof: … fromkeras.modelsimportSequential fromkeras.layersimportDense Use: … fromtensorflow.keras.modelsimportSequential fromtensorflow.keras.layersimportDense SolvedmyissueswithConda. Thanksfortheexcellenttutorialsandarticles!! Reply JamesCarmichael December29,2021at11:44am # ThankyouforthefeedbackRick!IalsooftentrytoruncodeinbothAnacondaandGoogleColabtoidentifyandcorrectcompatibilityissues. Reply Preeti February10,2022at4:18pm # MyAccuracy:76.95 Thankyouforthecodeanddetailedexplanation Reply JamesCarmichael February11,2022at8:35am # Youareverywelcome,Preeti!Keepupthegreatwok! Reply Alan March9,2022at8:46pm # HiJames Greatwork Nevermindneuralnetworks,thisiscausingmealotofdeepthinking. Iamrunningyourtutorialonapi400with64bitOSonThonny. Worksreasonablywellonthismachine. Howevercameacrossanerrorinoneofyourexamples…Kerasneuralnetworkusing‘pima-indians-diabetes.csv’ ”fromtensorflow.python.eager.contextimportget_config ImportError:cannotimportname‘get_config’from‘tensorflow.python.eager.context’(/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/context.py)” SodiscoveredthatthefaultlaywithKeras.modelsandlayersandhaverejiggedthesketchasfollows:- #firstneuralnetworkwithkerastutorial fromnumpyimportloadtxt fromtensorflow.kerasimportmodels,layers#******************** #fromkeras.modelsimportSequential#****************** #fromkeras.layersimportDense#******************** #loadthedataset dataset=loadtxt(‘/home/pi/Documents/pima-indians-diabetes.csv’,delimiter=’,’) #splitintoinput(X)andoutput(y)variables X=dataset[:,0:8] y=dataset[:,8] #definethekerasmodel model=models.Sequential()#******************** model.add(layers.Dense(12,input_dim=8,activation=’relu’))#******************** model.add(layers.Dense(8,activation=’relu’))#******************** model.add(layers.Dense(1,activation=’sigmoid’))#******************** #compilethekerasmodel model.compile(loss=’binary_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) #fitthekerasmodelonthedataset model.fit(X,y,epochs=150,batch_size=10) #evaluatethekerasmodel _,accuracy=model.evaluate(X,y) print(‘Accuracy:%.2f’%(accuracy*100)) Nowthatproduces Accuracy:74.35 Reply JamesCarmichael March10,2022at10:37am # HiAlan…Thankyouforthefeedbackandsupport!InterestingapplicationtotheRaspberryPi!Keepinmindthatourimplementationsmaynotbefullycompatiblewiththelibrariesthataredevelopedforthatplatform.Keepupthegreatwork! Reply Nishanth March14,2022at3:38am # Hi, Amazingtutorial!Simpleandeasy.Itriedthesamethingonmydatasetbutthelastforloopdoesnotseemtowork.Couldplshelpmewithit? Hereistheforloop: foriinrange(5): print(‘%s=>%d(expected%d)’%(X[i].tolist(),predictions[i],y[i])) Thanks Reply JamesCarmichael March14,2022at11:48am # HiNishanth…areyoucopyingandpastingthecodeortypingitin?Becarefulregardingcopyingandpastingcodeandhowitmayaffectthecodelayoutaserrorsmaybeverydifficulttospotvisually. Reply Nishanth March14,2022at3:40am # Hihereinthecommenttheprintstatementlooksun-indentedbutinmycode,Iindentitandstilldoesnotwork. Reply JamesCarmichael March14,2022at11:52am # HiNishanth…pleaseseepreviousreplies. Reply Nishanth March14,2022at3:45am # Hi, AmazingTutorial!SimpleandEasytofollow.Itrieditonmydatasetbutthelastforloopthatprintsfirst5examplesdoesnotwork.ItgivesmeKeyError:0 Couldyouhelpmewithit? Thanks Reply JamesCarmichael March14,2022at11:47am # HiNishanth…pleasesharethefullerrormessagesowecanbetterassistyou. Reply Nishanth March14,2022at11:41pm # Foundawayout.Thingisthatherethedatasetisnumpyarrayandminewasapandas.DataFrame.Thanksforthehelp. Reply NVRaman April2,2022at1:51am # HelloJason, Reallywonderfultutorial WhenIranthecodeeverythingworkedexceptwhileprintingthepredictionsIgetakeyerror. Reply JamesCarmichael April2,2022at12:18pm # HiNV…Canyouprovidetheexacterrormessagesothatwecanbetterassistyou? Reply Susia April9,2022at1:30am # Hi,I’velearnedthesametutorialtodevelopthefirstneuralnetinKerasinoneofyourmini_courses.Todevelopmyownmodelonmyowndataset,I’vetriedtoadaptthistutorial.TheproblemismytargetYiscountdata(numberoftrafficflowforexample).Inmycase,howtodefinetheactivationfunctionfortheoutputlayer.Isitrelu?Howtochoosethelossfunction?I’vetriedMeanSquaredError,thelossvalueisquitelarge,orcategorical_crossentropy,thelossvalueisnan.IamconsideringtoorderthecompletebookofDeepLearningWithPython.What’sthedifferenceofthetutorialsinsidethebookandthemini_course? Reply JamesCarmichael April9,2022at8:34am # HiSusia…Thefollowingresourcemayaddclarityinhowtochooseanactivationfunction: https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/ Reply Nasrin April23,2022at4:32am # Isir,thanksamillionforyourawesomepost couldyoupleaseexplainhowwecandivideXandyintothetrainandtestsampleindeeplearning? thiscodeiscorrecthere? fromsklearn.model_selectionimporttrain_test_split X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.33,random_state=42) Reply JamesCarmichael April24,2022at3:25am # HiNasrin…thesamplecodeyouprovidedlooksaccurate.Feelfreetoimplementitandletusknowifyouencounteranyissues. Reply ShivaManhar April23,2022at3:25pm # 24/24[==============================]–0s489us/step–loss:0.4517–accuracy:0.7956 Accuracy:79.56 Reply JackSparrow June3,2022at5:38am # DeepLearningwithkerasmnistdataset: fromcgiimporttest frompyexpatimportmodel importnumpyasnp fromkeras.modelsimportSequential fromkerasimportlayers #fromkeras.layersimportConvolution2D,MaxPooling2D#trainonimagedata fromkeras.utilsimportnp_utils#veridönüşümüiçingerekli fromkeras.datasetsimportmnist#imagedata (X_train,y_train),(X_test,y_test)=mnist.load_data() print(“Reshapeöncesi”,X_train.shape) X_train=X_train.reshape(-1,28,28,1) X_test=X_test.reshape(-1,28,28,1) print(“Reshapesonrası”,X_train.shape) X_train=X_train.astype(‘float32’) X_test=X_test.astype(‘float32′) X_train/=255 X_test/=255 Y_train=np_utils.to_categorical(y_train) Y_test=np_utils.to_categorical(y_test) model=Sequential() model.add(layers.Convolution2D(32,3,3,activation=’relu’,input_shape=(28,28,1))) model.add(layers.Convolution2D(32,3,3,activation=’relu’)) model.add(layers.MaxPooling2D(pool_size=(2,2))) model.add(layers.Dropout(0.25)) model.add(layers.Flatten()) model.add(layers.Dense(128,activation=’relu’)) model.add(layers.Dropout(0.5)) model.add(layers.Dense(10,activation=’softmax’)) model.compile(loss=’categorical_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’]) model.fit(X_train,Y_train, batch_size=32,epochs=10,verbose=1) test_loss,test_acc=model.evaluate(X_test,Y_test,verbose=0) print(“TestLoss”,test_loss) print(“TestAccuracy”,test_acc) DeepLearningwithdata_diagnosisdataset: importimp frompickletoolsimportoptimize fromrandomimportrandom fromstatisticsimportmode fromtabnannyimportverbose fromwarningsimportfilters frommatplotlib.pyplotimportaxis importpandasaspd importnumpyasnp dataSet=pd.read_csv(“.\data_diagnosis.csv”) dataSet.drop([“id”,”Unnamed:32″],axis=1,inplace=True) dataSet.diagnosis=[1ifeach==“M”else0foreachindataSet.diagnosis] y=dataSet.diagnosis.values x_data=dataSet.drop([“diagnosis”],axis=1) x_data.astype(“uint8”) fromsklearn.preprocessingimportStandardScaler scaler=StandardScaler() x=scaler.fit_transform(x_data) fromkeras.utilsimportto_categorical Y=to_categorical(y) fromsklearn.model_selectionimporttrain_test_split trainX,testX,trainy,testy=train_test_split(x,Y,test_size=0.2,random_state=42) trainX=trainX.reshape(trainX.shape[0],testX.shape[1],1) testX=testX.reshape(testX.shape[0],testX.shape[1],1) fromkerasimportlayers fromkerasimportSequential verbose,epochs,batch_size=0,10,8 n_features,n_outputs=trainX.shape[1],trainy.shape[1] model=Sequential() input_shape=(trainX.shape[1],1) model.add(layers.Conv1D(filters=8,kernel_size=5,activation=’relu’,input_shape=input_shape)) model.add(layers.BatchNormalization()) model.add(layers.MaxPooling1D(pool_size=3)) model.add(layers.Conv1D(filters=16,kernel_size=5,activation=’relu’)) model.add(layers.BatchNormalization()) model.add(layers.MaxPooling1D(pool_size=2)) model.add(layers.Flatten()) model.add(layers.Dense(200,activation=’relu’)) model.add(layers.Dense(n_outputs,activation=’softmax’)) model.summary() print(‘başladı’) importkeras importtensorflow #model.compile(loss=’categorical_crossentropy’,optimizer=’adam’,metrics=[‘accuracy’]) model.compile(loss=’binary_crossentropy’, optimizer=tensorflow.keras.optimizers.Adam(), metrics=[‘accuracy’])#编译 dataSet.info() model.fit(trainX,trainy,epochs=epochs,verbose=1) _,accuracy=model.evaluate(testX,testy,verbose=0) print(accuracy) Reply JamesCarmichael June3,2022at9:12am # ThankyouforthefeedbackJack!Keepupthegreatwork! Reply Jack June17,2022at5:25am # 24/24[==============================]–0s1ms/step [6.0,148.0,72.0,35.0,0.0,33.6,0.627,50.0]=>1(expected1) [1.0,85.0,66.0,29.0,0.0,26.6,0.351,31.0]=>0(expected0) [8.0,183.0,64.0,0.0,0.0,23.3,0.672,32.0]=>1(expected1) [1.0,89.0,66.0,23.0,94.0,28.1,0.167,21.0]=>0(expected0) [0.0,137.0,40.0,35.0,168.0,43.1,2.288,33.0]=>1(expected1) myaccuracyis77.99butthisshowsit100isthisright? Reply JamesCarmichael June17,2022at9:28am # ThankyouforthefeedbackJack! Reply NicolaMenga June22,2022at5:53pm # Hi. Thankyouforthistutorial.Itisveryuseful. Ihaveaquestion.Thisisatutorialforabinaryclassificationpurpose. However,IwanttobuildaFeedForwardNeuralNetworkwhichpredictsmorethanonevariable(morethanoneneuronintheoutputlayer),whichhaveavaluebetween0and1(forexample0.956,0.878,0.897andsoon),unlikethecaseofthistutorial,inwhichthevariabletobepredictedtakesonlythevalues0or1. Itriedtoapplythenetworkdevelopedinthistutorialforthispurpose,butresultsarebad. Mytestdatasethave257observations.IfIapplythisnetwork,thepredictionarrayisconstitutedby257values(oneforeachobservation),butthesevaluesareallthesame(forexample1:0.985;2:0.985;3:0.985;…;256:0.985;257:0.985).IhopeIexplained. Thereisakerasmodel/functionadequateformyproblem(i.e.thepredictionofavariablewhichisnot0or1)? Thankyouforyourhelp. NicolaMenga. Reply JamesCarmichael June23,2022at10:59am # HiNicola…Pleaseclarifyand/orelaborateonyourquestionsothatwemaybetterassistyou. Reply Sadegh July7,2022at3:49am # Hithere, IalwaysgetwarningwhenI’musingNNmodelthatismadewithkerasinanaconda’sspyderconsul. Thewarningisasfollow: WARNING:AutoGraphcouldnottransformandwillrunitas-is. Cause:Unabletolocatethesourcecodeof.Notethatfunctionsdefinedincertainenvironments,liketheinteractivePythonshell,donotexposetheirsourcecode.Ifthatisthecase,youshoulddefinethemina.pysourcefile.Ifyouarecertainthecodeisgraph-compatible,[email protected]_not_convert.Originalerror:linenoisoutofbounds Tosilencethiswarning,[email protected]_not_convert Ireallyappreciateifyoucanhelpmeoutofthis. Reply LeaveaReplyClickheretocancelreply.Comment*Name(required) Email(willnotbepublished)(required) Δ Welcome! I'mJasonBrownleePhD andIhelpdevelopersgetresultswithmachinelearning. Readmore Nevermissatutorial: Pickedforyou: YourFirstDeepLearningProjectinPythonwithKerasStep-By-StepHowtoGridSearchHyperparametersforDeepLearningModelsinPythonWithKerasRegressionTutorialwiththeKerasDeepLearningLibraryinPythonMulti-ClassClassificationTutorialwiththeKerasDeepLearningLibraryHowtoSaveandLoadYourKerasDeepLearningModel LovingtheTutorials? TheDeepLearningwithPythonEBookiswhereyou'llfindtheReallyGoodstuff. >>SeeWhat'sInside

請為這篇文章評分？

延伸文章資訊

[讀書筆記]Deep learning 深度學習必讀：Keras 大神帶你用 ...

Keras提供的循環層有SimpleRNN、LSTM和GRU。SimpleRNN理論上應該能夠 ... [讀書筆記]Deep learning 深度學習必讀：Keras 大神帶你用Python ...

About Keras

Keras is a deep learning API written in Python, running on top of the machine learning platform T...

Keras - 維基百科，自由的百科全書

Keras是一個用Python編寫的開源神經網路庫，能夠在TensorFlow、Microsoft Cognitive ... Chollet, François; Allaire, J. J....

Your First Deep Learning Project in Python with Keras Step-By ...

Your First Deep Learning Project in Python with Keras Step-By-Step · 1. Load Data · 2. Define Ker...

Day 02：撰寫第一支Neural Network 程式-- 阿拉伯數字辨識

Machine Learning 框架(Framework)GitHub評比，資料來源:【AI關鍵技術】三大熱門 ... 完成一個Neural Network 模型，非常適合入門學習，因此我們就...

Your First Deep Learning Project in Python with Keras Step-By ...

文章推薦指數： 80 %

請為這篇文章評分？

延伸文章資訊

最新文章

相關網站資訊

時代國際英日語中心托福

時代國際費用

時代師資

一天見效瘦腿

玩遊戲賺錢app

玩遊戲賺錢2020