Organic
An organic trainer attempts to train a model of progressive complexity by growing layers in width and depth, as well as adding functional modules where linearities allow growth without disrupting learned features.
The process runs as such:
Bootstrap baby network with foundation data.
Use network to evaluate training data and determine order of familiarity.
Use familiarity to select next training data.
During training, grow the network to accommodate plateaus or premature stalls below the acceptable loss threshold.
After training, prune the network to remove unused or redundant layers and connections, and prevent overfitting.
Parameters
pruning
The number of iterations after which the network will be pruned to remove unused or redundant layers and connections.