I'm not sure why the copy constructor is being evoked instead of the move copy constructor, but the problem is being caused by the reallocation of the vector. For this small sample you can probably increase the speed by altering the amount of memory reserved for the vector. With my compiler the capacity of the vector increases like:
I find the fastest way to increase the speed of vector operations is to try to avoid as many reallocations as possible.
If you reserve a larger amount then your reallocation will happen much less often.