Adjust the speed of an animation?

Using the Follow Path constraint I have a circle going in a loop on a very simple path(an oval)

I'm looking for a way to smoothly increase and decrease the speed of the circle on hover on/off

I've tried blending and transitioning to the same timeline with higher speed but I always get a stutter depending on when the action happens

Am I trying to do something not yet possible?

A gif of the desired outcome bellow
https://i.giphy.com/media/v1.Y2lkPTc5MGI3NjExbzBveXl2eXBzdjY2MTBuN2kxdnNvMnhjODdoaWQzemJ4aG9remo2byZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/ZFA6TDmqYWEBOCciEq/giphy.gif