Do you mean... it only turns on (when pressing the button) if a game is inserted? If so, this is how it is meant to be.
Or... do you mean you insert a game and it turns itself on without you pressing the on button? Like, it just powers itself up when you put a game in? If so, guessing the on button might be making permanent contact or something, so maybe open the Lynx up and gently clean around that area of the board.