We prove for each integer ℓ⩾1 an unconditional upper bound for the size of the ℓ-torsion subgroup ClK[ℓ] of the class group of K, which holds for all but a zero density set of number fields K of degree d∈{4,5} (with the additional restriction in the case d=4 that the field be non-D4). For sufficiently large ℓ this improves recent results of Ellenberg, Matchett Wood and Pierce, and is also stronger than the best currently known pointwise bounds under GRH. Conditional on GRH and on a weak conjecture on the distribution of number fields our bounds also hold for arbitrary degrees d.