Here's a map that might help you find where the house stood. Who knows, you might even be staying in the neighbourhood.
For those interested in where Yemmerrawanyea lived in London
Below is 125 Mount Street, Mayfair, the address where Yemmerrewanyea and Bennelong stayed in London. The likelihood that this is the same building is virtually non-existent. And the likelihood the Loewe shop was there is, of course, completely non-existent. However, if you're interested, or in the vicinity, it would be good to at least pause on the footpath, irritating the heck out of the hoity toities who live in this area, look around, and although all the buildings will have changed, just try to capture the mixture of wonderment, fear and strength Yemmerrawanyea and Bennelong experienced here.