The surprising answer is that it comes from the Malay word menteri ('minister'). Originally, mandarin was used to refer to high officials in China, and then it came to refer to the language that they used. See the Wikipedia article (here).
I don't usually rely too much on Wikipedia, as so much of it is flawed. But Webster's Dictionary confirms that Mandarin does indeed come from menteri (via Portuguese), so maybe this time Wikipedia is right:
I am only aware of a few words that have been borrowed into standard English from Malay: - amok
- orangutan (lit. 'forest person')
- durian (lit. 'spiky thing')
- rambutan (lit. 'hairy thing')
- parang (though perhaps the word machete is more common in English)
- compound (as in police compound, from the Malay kampung, 'village')
