The effectiveness and reliability of automation aids are critical topics in the area of human-robot interaction (HRI). As more tasks are subsumed by robots and autonomous systems, it is important to examine the relationships between these entities and their human operators. Research to date has covered various manipulations of autonomy, but this broad body of research is in need of focus and consistency. The current study presents a qualitative overview of research regarding levels and reliability of autonomy/control and the effects they have on important HRI-relevant outcome variables. Results indicate that autonomy and automation aids operate uniquely for different tasks, and that there are many complex factors that can affect not only performance but also usability, confidence, and safety. Unresolved issues in the field and challenges and opportunities for future research are also presented.