Although there are ways to do this with Firefox using multi-account containers and the AWS SSO plug-in this native capability is a long time coming. I am in and out of various accounts all day long and while this is a bit easier with SSO it was a pain if you wanted to compare settings in two accounts, etc.
Try it today by signing in to the AWS Console, selecting the account menu, and selecting “Turn on multi-session”. You can opt out at any time from the account menu.