There is detailed discussion here about autoplay policy for OIS:
Make sure you are not having autoplay related issues, as you said you joined with audio/video on so GUM will let you autoplay, so you didn’t face this issue initially but if you turn off your audio and video during the call then the audio received after that will have same autoplay issue causing these new audio streams to not play. Make sure you are not in this scenario if you have this scenario then keep audio captured from microphone to overcome it.
If this is not the cause of the issue then you are probably making some mistake in consuming etc, so you need to debug it via remote debugging from your safari browser to mac and see what is causing the issue.