May be it is derived from “bugbear”? As per Merriam-Webster:
1) an imaginary goblin or specter used to excite fear
2) a source of dread, a continuing source of irritation: a problem.
First known use 1552.