If you look at the configuration of Fe in Fe(CO)5 it is 1s2 2s2 2p6 3s2 3p6 3d8 . So the outermost shell corresponds to n=3. Which means all electrons in n=3 shall be considered valence electrons. Thus there are 2+6+8 = 16 valence electrons. Hybridsation is dsp3 (all 8 electrons are paired in the 3d orbital )which is trigonal bipyramidal.