recentpopularlog in

pozorvlak : deeplearning   70

[1806.07366] Neural Ordinary Differential Equations
We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.
maths  deeplearning  machinelearning  computers  programming  ai 
4 weeks ago by pozorvlak
Dark knowledge
Use "soft targets" (temperature-smoothed average predictions from ensemble models) to train smaller summary models - they carry much of the information from the teacher models.

Random dropout can achieve a similar effect more cheaply.

Ensembles-of-specialists can perform well, but must be combined with care.
computers  machinelearning  deeplearning  ai  google 
9 weeks ago by pozorvlak
Out of shape? Why deep learning works differently than we thought
Current deep learning techniques for object recognition primarily rely on textures, not on object shapes.
computers  programming  ai  machinelearning  deeplearning  science  computervision 
10 weeks ago by pozorvlak
The deepest problem with deep learning – Gary Marcus – Medium
It fails for anything outside perceptual classification, and fails badly for out-of-dataset examples even there. The author suggests that it should be supplemented with symbolic manipulation.
computers  programming  ai  deeplearning 
10 weeks ago by pozorvlak
[1611.03530] Understanding deep learning requires rethinking generalization
Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization, and occurs even if we replace the true images by completely unstructured random noise. We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice. 
computers  machinelearning  deeplearning  badscience 
october 2018 by pozorvlak
Fnord — AI Image Echo Chamber
What happens when you feed the output of an image-recognition AI into an image-generating AI?
ai  deeplearning 
august 2018 by pozorvlak
OpenAI Five
AI that defeated a semi-pro team at the video game DotA 2.
computers  games  ai  deeplearning  cloudcomputing 
july 2018 by pozorvlak
Togelius: Empiricism and the limits of gradient descent
Evolutionary algorithms might be able to learn things that gradient descent can't; a tortured analogy claiming (gradient descent:evolutionary algorithms)::(strict empiricism:Popperian hypothetico-deductivism).
machinelearning  deeplearning  ai  evolution  algorithms  philosophy 
june 2018 by pozorvlak
Generative Adversarial Networks (GANs) in 50 lines of code (PyTorch)
In 2014, Ian Goodfellow and his colleagues at the University of Montreal published a stunning paper introducing the world to GANs, or generative adversarial networks. Through an innovative…
ai  machinelearning  deeplearning  computers  python  programming 
march 2018 by pozorvlak
Copista: Training models for TensorFlow Mobile – Andrew G – Medium
For those who missed the first part Copista: Developing Neural Style Transfer application with TensorFlow Mobile, this blog is a software engineer take on Machine Learning. In this part, you can find…
android  computers  programming  machinelearning  deeplearning 
november 2017 by pozorvlak
Copista: Developing Neural Style Transfer application with TensorFlow Mobile
I should confess that I am not a Data Scientist :). It’s my take on Machine Learning as a Software Engineer. Its all started when I came across Pete Warden’s blog TensorFlow for Mobile Poets in the…
machinelearning  android  deeplearning  computers  programming 
november 2017 by pozorvlak

Copy this bookmark:

to read