Initialization of Weights in Neural Networks
Priyesh Patel
, Meet Nandu , Purva Raut
Neural Networks, Variable Initialization
When training and building a neural network, a number of subtle but important decisions needs to be taken. Zeroing down on the loss function to be used, the number of layers, kernel size, and the stride for each convolution layer, best-suited optimization algorithm for the network, etc. Compared to all these things, the choice of initialization of weights may seem trivial pre-training detail. But weight initialization contributes as a significant factor on the final quality of a network as well as its convergence rate. This paper discusses different approaches to weight initialization and compares their results on few datasets to find out the best technique that can be employed to achieve higher accuracy in relatively lower duration.
"Initialization of Weights in Neural Networks", IJSDR - International Journal of Scientific Development and Research (www.IJSDR.org), ISSN:2455-2631, Vol.3, Issue 11, page no.73 - 79, November-2018, Available :https://ijsdr.org/papers/IJSDR1811013.pdf
Volume 3
Issue 11,
November-2018
Pages : 73 - 79
Paper Reg. ID: IJSDR_180734
Published Paper Id: IJSDR1811013
Downloads: 000347219
Research Area: Engineering
Country: MUMBAI, MAHARASHTRA, India
ISSN: 2455-2631 | IMPACT FACTOR: 9.15 Calculated By Google Scholar | ESTD YEAR: 2016
An International Scholarly Open Access Journal, Peer-Reviewed, Refereed Journal Impact Factor 9.15 Calculate by Google Scholar and Semantic Scholar | AI-Powered Research Tool, Multidisciplinary, Monthly, Multilanguage Journal Indexing in All Major Database & Metadata, Citation Generator
Publisher: IJSDR(IJ Publication) Janvi Wave
