Assistive technologies that incorporate affective computing methods may be useful for increasing inclusion across the autism spectrum. The involvement of this target population during development and design is key to ensuring that the technology is beneficial and has the potential to be adopted. This article reports on a participatory design process used for the development of an affective technology intended for users on the autism spectrum. We first briefly recapitulate concepts and theories about emotions to illuminate emotion-specific challenges faced in the design process. We then introduce biomusic, an affective technology that communicates emotional states by translating physiological signals into auditory output. The necessity of stakeholder involvement during the development of biomusic is discussed, with a focus specifically on including individuals with autism spectrum condition (ASC). Finally, we present a concrete example of a participatory sound design workshop focusing on the development of biomusic's auditory display in collaboration with adolescents with ASC. From this experience, challenges of using participatory design for the development of affective technologies with people with ASC are presented and discussed.