The optimal solution is for there to be a separate button visible that you can press that will retain priority, which has the shortcut of CTRL+pass priority hotkey.