@yommie There are two things here, in my opinion.
Our body have a setting point, a certain amount of weight/body fat they are happy with. The body will always try to come back there, and it can take years to change the setting point.
The less fat you have, the harder it is to lose the weight...