Action understanding requires a many-to-one mapping of perceived input onto abstract representations that generalize across concrete features. It is debated whether such abstract action concepts are encoded in ventral premotor cortex (PMv; motor hypothesis) or, alternatively, are represented in lateral occipitotemporal cortex (LOTC; cognitive hypothesis). We used fMRI-based multivoxel pattern analysis to decode observed actions at concrete and abstract, object-independent levels of representation. Participants observed videos of 2 actions involving 2 different objects, using either an explicit or implicit task with respect to conceptual action processing. We decoded concrete action representations by training and testing a classifier to discriminate between actions within each object category. To identify abstract action representations, we trained the classifier to discriminate actions in one object and tested the classifier on actions performed on the other object, and vice versa. Region-of-interest and searchlight analyses revealed decoding in LOTC at both concrete and abstract levels during both tasks, whereas decoding in PMv was restricted to the concrete level during the explicit task. In right inferior parietal cortex, decoding was significant for the abstract level during the explicit task. Our findings are incompatible with the motor hypothesis, but support the cognitive hypothesis of action understanding.