My favourite papers and talks at Nips 2013

I just came back from NIPS 2013, an illustrious world conference on machine learning in South Lake Tahoe, California.

Here’s the best papers and talks from the conference:

• “Dropout training as adaptive regularization” by Stefan Wager, Sida Wang, Percy Liang (Stanford). Interesting research in which authors attribute the dropout algorithm training to an adaptive regularizer, and find common aspects with AdaGrad, an online learning method based on the adaptive gradient descent.
• “Adaptive dropout for training deep neural networks” by Jimmy Ba, Brendan Frey (University of Toronto). Again on the dropout algorithm, authors investigate some alternatives to picking 0.5 as unity dropout probability, during the dropout training.
• “Understanding dropout” by Pierre Baldi, Peter J. Sadowski (University of California Irvine). Authors investigate important mathematical issues of the dropout algorithm.
• “Training and Analysing Deep Recurrent Neural Networks”  by Michiel Hermans, Benjamin Schrauwen (Universiteit Gent). Authors apply deep recurrent neural networks to sequence time series prediction, and show some interesting applications to the character sequence prediction of English Wikipedia text.
• “Deep supervised and convolutional generative stochastic network for protein secondary structure prediction” by Jian Zhou and Olga Troyanskaya (Princeton), from the Deep Learning workshop. An interesting application of generative stochastic network (GSN) to the important issue of the protein secondary structure prediction.
• “Tissue-dependent alternative splicing prediction using deep neural network” by Michaek K. K. Leung, Hui Yuan Xiong, Leo J. Lee and Brendan J. Frey (University of Toronto), from the Machine Learning in Computational Biology workshop. Authors apply the dropout algorithm in a deep neural network to predict new regions of tissue alternative splicing.

A really top-level conference with intriguing workshops… thanks a lot to all the organizers!

[EDIT: check out these blog posts by Paul Mineiro, hundalhh, Yisong Yue, Sebastien Bubeck, Memming]