How does a usual fully-connected feed-forward neural network work?
Deep LearningLevel: IntermediateNeural networks
How does a usual fully-connected feed-forward neural network work?
Hints
- Trace data and gradients through the model, noting shapes.
- Separate training from inference and identify a necessary assumption.
Answer
In a usual fully-connected feed-forward network, each neuron receives input from every element of the previous layer and thus the receptive field of a neuron is the entire previous layer. They are usually used to represent feature vectors for input data in classification problems but can be expensive to train because of the number of computations involved.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0