Tend to agree - sounds like a bad fob. I have a spare physical key ($37) and I tape the E-key inside my helmet cheek pad.
Despite having rid myself of 2 x rideless-key bikes some time ago, I must say that I too thought of this idea, for almost any bike/ key. Then I surmised that a nasty helmet-crunching fall might end up penetrating brain with key - ouch! I'd warn against it.