Performing actions in a specific order is an essential part of many daily activities. Working memory is known to support memory for temporal order in digit and spatial sequences, but less is known about the ability to learn temporal order in regard to action sequences, which are an essential part of many daily activities. Research with children has shown that memory for order is sensitive to deviant input and is the least prioritized in a hierarchy of processing steps when learning action sequences, following object identification and sub-action identification. Similar research had not been conducted with adult participants until now. The present study investigated the effects of deviant order on temporal memory for actions with adult participants, with varied ease of object categorization used to assess whether adults' memory for ordered sequences is supported by the processing hierarchy. Results showed that the ease of object categorization had a significant effect on memory for target actions, as participants recalled more actions in the easy condition. Additionally, the presence of a deviant order had a significant effect on memory for order in the difficult-to-categorize object condition, but the effect was not significant in the easy-to-categorize object condition. This suggests that the presence of a deviant order hinders adults' ability to learn order sequences, but does so to a lesser extent when the actions in the order sequence are performed with easily categorized objects. These results support the theorized processing hierarchy, with the presence of a deviant order having a larger effect on memory for novel action sequences when the objects are difficult to categorize. These results have implications for theories of working memory and learning in real world contexts.