NEW DELHI—A deadly clash with China high in the Himalayas has posed a tricky domestic challenge for Indian Prime Minister Narendra Modi, who has cultivated an image as a strong nationalist leader and struggled to deliver on promises to transform India’s economy.

The showdown has stoked the sort of nationalist feelings that Mr. Modi has proved adept at turning into political support, but unlike past confrontations with Pakistan, India can’t afford to turn China into an archenemy, analysts say.

…