A new scheme for decomposing the total dipole moment and polarizability of a system into site-specific contributions is presented. The scheme is based on partitioning the system volume into cells associated with its atoms or groups of atoms. The site-specific dipole moments and polarizabilities are computed from the charge densities within the individual cells and the responses of these densities to an external electric field. These dipole moments and polarizabilities are further partitioned into local/dipole and charge-transfer components. The utility of the scheme is illustrated through analysis of the structure-/shape- and size-specific aspects of the dipole moments and polarizabilities of silicon clusters. It is shown that the polarizabilities associated with the individual constituent Si atoms vary considerably with the structure/shape of the cluster and the location of the atom or site within a given structure. Surface atoms, and especially those at edges, have larger polarizabilities than interior atoms. The overall contribution of the charge-transfer components to the total cluster polarizability increases with the cluster size. Finally, the anisotropy of the total polarizability correlates with the anisotropy of the cluster shape, and the charge-transfer component is the part dominantly responsible for the polarizability anisotropy.