How Deep Should It Be? On the Optimality of Hierarchical Architectures