The growth and evolution of Ag nanolayers on differently-passivated Si(001) substrates at ambient condition have been studied. Initial compactness and smoothness of Ag nanolayer on the H-passivated Si(001) surface are found better compared to those on the Br-passivated Si(001) surface, which can be understood considering surface free energy and surface mobility of the passivated surfaces. As the time passes, the growth of dewetted three-dimensional (3D) islandlike structures (Volmer-Weber-type mode) from comparatively wetted Ag nanolayer (Stranski-Krastanov-type mode) is evident at ambient conditions. Such evolution of growth is through dewetting (related to the change in the interfacial energy due to the oxide growth), migration, and coalesce of Ag, which can even produce large epitaxial [Ag(001)/Si(001)] 3D islands on H-passivated Si(001) surface. The growth rate, size, number density, and epitaxy/nonepitaxy of 3D islands are different for different passivated surfaces. These differences can be realized considering the growth time of oxide (i.e., instability of passivated surface), in-plane inhomogeneity of interfacial energy (i.e., inhomogeneous nature of passivation), and in-plane diffusion of Ag on the passivated surfaces.