Should I increase the speed value and then somehow remember to reduce
it next turn or should I implement some sort of object which holds
modifiers that is checked every turn?
This is the heart of your question and you're really asking, "should I use an object oriented approach?" In general, you're likely to find more advocates than critics of the object-oriented approach, although critics exist.
My suspicion is that if you go with option one (the non-OOP approach), you're going to find yourself with an increasingly convoluted and complicated class, whereas checking status effect modifier objects each turn will lead to much cleaner, more extensible code.
Your players should hold a collection of StatusEffects. StatusEffect can be an object you extend or an interface. (I'm calling them StatusEffect rather than Power-Up, because you mentioned you'd use this for poison too, so it feels like a better name.) In your StatusEffect class you're going to want to put anything that would apply to all of them. totalDuration (number of turns it lasts), currentDuration (how many turns it has been in effect), etc.
Then you'll implement particular StatusEffects, speed-up, poison, etc. You might consider having each statuseffect implement comparable() so you can sort the collection held by the player (i.e., does poison take effect before a healing effect?). Give each effect a applyEffect() method of some sort, where the modifications to the player are applied. Also give them an isExpired() method to see if the currentDuration has reached the totalDuration. When you do a turn, you'll iterate through the effects, apply them, and remove them if necessary.