Yeah, there's quite a few in a short walking distance in
Soho.
Sister Ray and
Reckless cover most styles, both new and used.
Sounds of the Universe specialises a bit more in Soul, Funk, Jazz, both new and used.
Phonica is more of an EDM specialist, mainly new if I remember rightly and
IF Music is a quirky one that you have to buzz for entry to and also specialises more towards the jazzy end of thing, pricey but has lots of rare pieces. I haven't been but I think there's still a
Third Man shop there now also.
Camden has a few rock/punk specialists I think but not really my scene so I couldn't vouch for them.
If you're more in the City/Shoreditch area,
Rough Trade East is pretty varied and I think there are a few other little ones towards Hackney.
I stumbled on
this by pure chance last week on Discogs that lists loads I wasn't even aware of too and splits them up by parts of the city you're in. Good luck and have fun in the best city in the world.