Definitions are a bit blurry here. PE and gradualism are not in direct conflict (although Gould sometimes put it like that). In fact, I sometimes think that gradualism was basically a strawman put forth by Gould (and Edredge, I think), to underline their point.
If we talk about speciation we are necessarily talking about populations. So we are not talking about individual changes but rather about the change of allele frequency in the population.
As such the gradual element is for instance the change in allele frequency (e.g. mutation rates), the probability of fixation of new alleles and so on. Given a sufficient large population large scale changes (like split off into new species) are very unlikely. So Gould argues that factors necessary for speciation are for instance geographic isolation coupled with genetic drift. Something which was not a precisely new concept (peripatric speciation, a special case of allopatric speciation).
The evolutionary rate does not change as AlphaWolf pointed out, as such it is not a saltatory evolution. So "punctuated" does not refer to the evolutionary rate, but rather to the point on which speciation can occur, usually coupled with a constraint in gene flow by, as already mentioned, for example geographic isolation.
Or in other words, one has to be careful not to mix up evolution with speciation...