You can force your PlayStation Vita to turn off in most situations. To do this press and hold the power button for two seconds to turn off the system. You'll see an option to "Power Off", please tap this. If the system does not turn off or you can't see the "Power Off" option, press and hold the power button for more than 30 seconds until the screen turns off. After a moment, press and hold the power button for five seconds to turn on your system.
With PlayStation Vita System Update v1.60 and later, you'll see a "Safe Mode" menu when your PlayStation Vita restarts after being turned off in this way. It's completely normal for this screen to appear and you can restart your PlayStation Vita normally by choosing option one.