I did it and I'm not sure it was worthwhile. But I don't have a PlayStation VR. Diablo III in 4K looks good, and MLB The Show in "high performance" mode has a high framerate at 1080p, but it wasn't cheap.
Well, my 1080p tv got fried a few months ago and I ended up buying a 4k tv that was on sale. Went from an Xbox One to a One X but couldn't really see any difference, even with Dark Souls remastered.