It seems to me that your best bet that will combine portability and modernity is an mp3 player. Despite the rise of the smartphone and its effects on dedicated mp3 players — the iPod peaked around 2005/6 and steadily declined till 2014 when sales numbers were last available — similar devices continue to exist under a plethora of brand names.
Many are marketed with your use case in mind. For example, this one (picked after a very cursory read of a couple reviews) is lighter than most phones and claims to be safe for sports through shock absorption. It also resolves the cable issue through Bluetooth connectivity for earbuds.
A further step in that direction is the clip form factor for mp3 players, e.g. this one. This has no base unit — the earbuds with their wraparound band are themselves an mp3 player. What a time to be alive.
I can't think of a better way to get portable music without bringing a serious device. That said, I also like Willeke's suggestion. As someone who plays and writes for piano, I do find that my attention can sometimes be fully occupied by mentally playing or coming up with music. Can't hurt to try before buying a device.
mp3 player clip bluetooth
and the first one that popped up is 40,6 x 16 x 66 mm, with bluetooth (so you can use the wireless earplugs) and water resistant. At just over 100 grams it should not weigh you down too much. (I am sure there are others available as well.) – Willeke Apr 28 '19 at 16:45