Three Architecture Ilities found in Deep Learning Systems

Three Architecture Ilities found in Deep Learning Systems

People in software development are familiar with the phrase “-ilities”. It is actually not a word, but you can google it: Informally these are sometimes called the “ilities”, from attributes like stability and portability. Qualities — that is non-functional requirements — can be divided into two main categories: Execution qualities, such as security and usability, which are observable at run time. Non-functional requirement — Wikipedia You may think because it is not a real world, that it is some informal convention, some kind of loose jargon….

Read More Read More

Five Capability Levels of Deep Learning Systems

Five Capability Levels of Deep Learning Systems

Arend Hintze has a good short article on “Understanding the four types of AI, from reactive robots to self-aware beings” where he Reactive Machine — The most basic type that is unable to form memories and use past experiences to inform decisions. They can’t function outside the the specific tasks that they were designed for. Limited Memory — Are able to look into the past to inform current decisions. The memory however is transient and aren’t used for future experiences. Theory of Mind — These systems…

Read More Read More

Rethinking Generalization in Deep Learning

Rethinking Generalization in Deep Learning

The ICLR 2017 submission “Understanding Deep Learning required Rethinking Generalization“ is certainly going to disrupt our understanding of Deep Learning . Here is a summary of what the had discovered through experiments: 1. The effective capacity of neural networks is large enough for a brute-force memorization of the entire data set. 2. Even optimization on random labels remains easy. In fact, training time increases only by a small constant factor compared with training on the true labels. 3. Randomizing labels is…

Read More Read More

11 Biases Why Experts will Miss the Deep Learning Revolution

11 Biases Why Experts will Miss the Deep Learning Revolution

I spend most of my waking time ( and likely my subconscious works overtime while I sleep ) studying Deep Learning.  Peter Thiel  has a phrase, “The Last Company Advantage”.  Basically you don’t necessarily need to have the “First Mover Advantage” however you absolutely want to be the last company standing in your kind of business.  So Google may be the last Search company, Amazon may be the last E-Commerce company and Facebook hopefully will not be the last Social…

Read More Read More

Meta-Learning in Deep Learning is now Reality

Meta-Learning in Deep Learning is now Reality

Note:  This is a short version of “Deep Learning – The Unreasonable Effectiveness of Randomness”. The paper submissions for ICLR 2017 in Toulon France deadline has arrived and instead of a trickle of new knowledge about Deep Learning we get a massive deluge. This is a gold mine of research that’s hot off the presses. Many papers are incremental improvements of algorithms of the state of the art. I had hoped to find more fundamental theoretical and experimental results of…

Read More Read More

Deep Learning – The Unreasonable Effectiveness of Randomness

Deep Learning – The Unreasonable Effectiveness of Randomness

The paper submissions for ICLR 2017 in Toulon France deadline has arrived and instead of a trickle of new knowledge about Deep Learning we get a massive deluge. This is a gold mine of research that’s hot off the presses. Many papers are incremental improvements of algorithms of the state of the art. I had hoped to find more fundamental theoretical and experimental results of the nature of Deep Learning, unfortunately there were just a few. There was however 2…

Read More Read More

Why Deep Learning is Radically Different from Machine Learning

Why Deep Learning is Radically Different from Machine Learning

There is a lot of confusion these days about Artificial Intelligence (AI), Machine Learning (ML) and Deep Learning (DL).   There certainly is a massive uptick of articles about AI being a competitive game changer and that enterprises should begin to seriously explore the opportunities.   The distinction between AI, ML and DL are very clear to practitioners in these fields.  AI is the all encompassing umbrella that covers everything from Good Old Fashion AI (GOFAI) all the way to…

Read More Read More

The Unreasonable Simplicity of Universal Machines

The Unreasonable Simplicity of Universal Machines

Rule 110 cellular automata, or more specifically the one dimensional cellular automata (you can explore those here http://atlas.wolfram.com/01/01/ ) that has the following rule: current pattern 111 110 101 100 011 010 001 000 new state for center cell 0 1 1 0 1 1 1 0 is all the complexity that one needs to create a machine that has all the computational capability of a Turing Machine, hence any computer system. NAND gates (or alternatively NOR gates): INPUT OUTPUT…

Read More Read More

10 Lessons Learned from Building Deep Learning Systems

10 Lessons Learned from Building Deep Learning Systems

Deep Learning is a sub-field of Machine Learning that has its own peculiar ways of doing things.  Here are 10 lessons that we’ve uncovered while building Deep Learning systems.  These lessons are a bit general, although they do focus on applying Deep Learning in a area that involves structured and unstructured data. The More Experts the Better The one tried and true way to improve accuracy is to have more networks perform the inferencing and combining the results.  In fact,…

Read More Read More

A Development Methodology for Deep Learning

A Development Methodology for Deep Learning

The practice of software development has created development methodologies such agile development and lean methodology to tackle the complexity of development with the objective of improving the quality and efficiency of software creation. Although Deep Learning is built from software it is a different kind of software and therefore a different kind of methodology is needed. Deep Learning differs most from traditional software development in that a substantial portion of the process involves the machine learning how to achieve objectives….

Read More Read More