What is shallow architecture in machine learning?

by mattosmat   Last Updated September 11, 2019 17:19 PM

What is a precise definition of shallow architecture in machine learning?

Answers 4

It means an artificial neural networks with few hidden layers. To be contrasted with deep neural networks. The precise boundary is debatable.

June 10, 2014 00:23 AM

Shallow and deep refer to two different (but related) ways to go about modelling a problem. Shallow architectures rely on the paper by Cybenko: Approximation by Superposition of a Sigmoidal Function, where the shows that arbitrary decision regions can be arbitrarily well approximated by MLPs.

Basically it means that you would just add more neurons to your layer, thus making your network wider, as the complexity of your problem at hand increases (in very rough terms). My point is thus, that there is no precise definition to that term. Is more about how you approach the complexity of a problem.

June 14, 2014 10:36 AM

I believe usually "shallow" means only one hidden layer. For example:

1. MLP with one hidden layer: data --> hidden --> softmax (class label)
2. SVM: data --> feature (can be considered as hidden) --> class label

Anything with more than 2 hidden layers (inclusive) can be called deep.

July 20, 2014 00:49 AM

according to one book, shallow-structured architectures, typically contain at most one or two layers of non-linear feature transformations. Examples of shallow architectures are Gaussian mixture models (GMMs), linear or non-linear dynamical systems, conditional random fields (CRFs), maximum entropy (MaxEnt) models, support vector machines (SVMs), logistic regression, kernel regression, multilayer perceptrons (MLPs) with a single hidden layer including extreme learning machines (ELMs). For instance, SVMs use a shallow linear pattern separation model with one or zero feature transformation layer when the kernel trick is used or otherwise.

September 11, 2019 16:23 PM

Related Questions

Updated July 17, 2017 06:19 AM

Updated March 03, 2017 23:19 PM

Updated March 09, 2017 03:19 AM

Updated April 15, 2018 17:19 PM

Updated September 06, 2018 07:19 AM