recentpopularlog in

pozorvlak : machinelearning   142

« earlier  
[1806.07366] Neural Ordinary Differential Equations
We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.
maths  deeplearning  machinelearning  computers  programming  ai 
5 weeks ago by pozorvlak
Unprovability comes to machine learning
Ben-David and colleagues then prove that the ability to carry out a weak form of monotone compression is related to the size of certain infinite sets. The set that the authors ultimately use in their work is the unit interval, which is the set of real numbers between 0 and 1. Their results imply that the finite subsets of the unit interval have monotone-compression schemes, and therefore are learnable in EMX, if and only if the continuum hypothesis is true, which is known to be unprovable.
machinelearning  maths  logic 
5 weeks ago by pozorvlak
Dark knowledge
Use "soft targets" (temperature-smoothed average predictions from ensemble models) to train smaller summary models - they carry much of the information from the teacher models.

Random dropout can achieve a similar effect more cheaply.

Ensembles-of-specialists can perform well, but must be combined with care.
computers  machinelearning  deeplearning  ai  google 
10 weeks ago by pozorvlak
Out of shape? Why deep learning works differently than we thought
Current deep learning techniques for object recognition primarily rely on textures, not on object shapes.
computers  programming  ai  machinelearning  deeplearning  science  computervision 
10 weeks ago by pozorvlak
[1611.03530] Understanding deep learning requires rethinking generalization
Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization, and occurs even if we replace the true images by completely unstructured random noise. We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice. 
computers  machinelearning  deeplearning  badscience 
october 2018 by pozorvlak
Togelius: Empiricism and the limits of gradient descent
Evolutionary algorithms might be able to learn things that gradient descent can't; a tortured analogy claiming (gradient descent:evolutionary algorithms)::(strict empiricism:Popperian hypothetico-deductivism).
machinelearning  deeplearning  ai  evolution  algorithms  philosophy 
june 2018 by pozorvlak
Generative Adversarial Networks (GANs) in 50 lines of code (PyTorch)
In 2014, Ian Goodfellow and his colleagues at the University of Montreal published a stunning paper introducing the world to GANs, or generative adversarial networks. Through an innovative…
ai  machinelearning  deeplearning  computers  python  programming 
march 2018 by pozorvlak
Copista: Training models for TensorFlow Mobile – Andrew G – Medium
For those who missed the first part Copista: Developing Neural Style Transfer application with TensorFlow Mobile, this blog is a software engineer take on Machine Learning. In this part, you can find…
android  computers  programming  machinelearning  deeplearning 
november 2017 by pozorvlak
Copista: Developing Neural Style Transfer application with TensorFlow Mobile
I should confess that I am not a Data Scientist :). It’s my take on Machine Learning as a Software Engineer. Its all started when I came across Pete Warden’s blog TensorFlow for Mobile Poets in the…
machinelearning  android  deeplearning  computers  programming 
november 2017 by pozorvlak
Zayd's Blog – Why is machine learning 'hard'?
In conclusion, fast and effective debugging is the skill that is most required for implementing modern day machine learning pipelines.
computers  programming  debugging  machinelearning  datascience 
march 2017 by pozorvlak
« earlier      
per page:    204080120160

Copy this bookmark:

to read