Sleep disruption is associated with cognitive decline and dementia in older adults; however, the underlying mechanisms are unclear. In rodents, sleep disruption causes microglial activation, inhibition of which improves cognition. However, data from humans are lacking. We studied participants in two cohort studies of older persons—the Rush Memory and Aging Project and the Religious Orders Study. We assessed sleep fragmentation by actigraphy and related this to cognitive function, to neocortical microglial marker gene expression measured by RNA sequencing, and to the neocortical density of microglia assessed by immunohistochemistry. Greater sleep fragmentation was associated with higher neocortical expression of genes characteristic of aged microglia, and a higher proportion of morphologically activated microglia, independent of chronological age- and dementia-related neuropathologies. Furthermore, these were, in turn, associated with worse cognition. This suggests that sleep fragmentation is accompanied by accelerated microglial aging and activation, which may partially underlie its association with cognitive impairment.