I think I found a solution! It's not about the audio output - it's about what display is your 'main display'. You need to go to System preferences > Display > Arrangement. Here, drag that little white title bar on the main display screen, which currently will be on the HDMi display, to your mac display. Apple will always keep the audio output corresponding to the main display as the main audio source. Once you change the main display to your inbuilt display, the HDMi audio output is never used by default! This solved it for me.