A hacked PS Vita can play PSP ad hoc games online through PSP Relay servers, using Kethen's aemu plugin inside Adrenaline or ARK-4 standalone. Relay is aemu's default mode, so you don't need port forwarding, UPnP or a VPN. You can play with PPSSPP and real PSP players on the same server.

What you need

A hacked PS Vita or PS TV with Adrenaline (6.61) or ARK-4 standalone. For PSN eboots, aemu's notes say to use ARK-4 standalone, Adrenaline with ARK pentazemin, or the newest isage Adrenaline builds.
The latest aemu release from GitHub.

Set up the Vita

In the Vita's Settings > Network > Wi-Fi Settings, connect to your Wi-Fi, then run Internet Connection Test to make sure it works.
In the Vita's Settings > Power Save Settings, turn off Use Wi-Fi in Power Save Mode.
Adrenaline only: in the PSP XMB, go to Settings > Power Save Settings and set WLAN Power Save to Off.
Change the custom firmware settings:
Adrenaline: press Select in the XMB to open the recovery menu, and set CPU Speed in UMD/ISO = 333/166 and Advanced > Force High Memory Layout = Disabled.
ARK-4 standalone: press Triangle in the Custom Launcher, open CFW Settings, and set CPU Clock in Game = OverClock, Use Extra Memory = Auto, Memory Stick Speedup = Off, Inferno Cache = Off.
Unzip aemu and copy the kd and seplugins folders from dist_660_release to your PSP memory stick folder, which is ux0:/pspemu unless you changed it in Adrenaline.
Turn on the plugin by adding this line, if it isn't there already:
Adrenaline: ms0:/seplugins/atpro.prx 1 in ux0:/pspemu/seplugins/game.txt
ARK-4 standalone: game, ms0:/seplugins/atpro.prx, on in ux0:/pspemu/seplugins/plugins.txt
Open ux0:/pspemu/seplugins/server.txt and type the relay server address. The default is socom.cc; change it to the server your group plays on, for example psp.mgn.pub. See the full PSP Relay server list.
Leave ux0:/pspemu/seplugins/port_offset.txt at 10000, which matches PPSSPP's default.
Delete ux0:/pspemu/kd/pspnet_miniupnc.prx if it exists.
On ARK-4 standalone, if a game doesn't accept the nickname taken from your PSN account, create ux0:/pspemu/seplugins/nickname.txt with a simple name (plain letters and numbers).

The Vita uses its own Wi-Fi connection, so you don't need to fill in hotspot.txt.

Play

Start the game and open the aemu overlay to check that it loaded: L + R + Select on ARK-4 standalone, or the Home button on Adrenaline. If it doesn't appear, go back over the steps.
Go into the game's ad hoc multiplayer mode and create or join a room. The overlay also has a chat while you're in a group.

Optional: faster networking

Kethen's pspemu_inet_multithread can make some games run faster during ad hoc play. It is experimental, needs firmware 3.60/3.65 with Enso, and doesn't seem to work with PSN games yet.

Put pspemu_inet_multithread.suprx in ur0:/tai.
Add these lines at the end of ur0:/tai/config.txt:

# pspemu_inet_multithread
*ALL
ur0:/tai/pspemu_inet_multithread.suprx

Copy pspnet_inet_kermit_redirect.prx to ux0:/pspemu/seplugins.
Add ms0:/seplugins/pspnet_inet_kermit_redirect.prx 1 at the end of ux0:/pspemu/seplugins/game.txt. On ARK-4 standalone, add it to plugins.txt in the ARK format instead, as described in the ARK-4 plugin wiki.
Restart the Vita. You can switch it on or off by enabling or disabling the PSP plugin in your custom firmware's settings.

Tips

Everyone needs the same relay server, port offset 10000, and the same game region and version.
For lag or random disconnects, pick the server closest to you and use a quiet 2.4 GHz Wi-Fi channel. A few Vitas can usually share one channel.
Some users report that Adrenaline Bubble Manager doesn't work with the plugin, so start games from the PSP XMB.
In Dissidia 012 on Vita, if you can't find other players after a match, go back to the main menu and then back to the lobby.
Still stuck? Copy kd and seplugins from dist_660_debug instead and check ux0:/pspemu/atpro.log. More game notes are in aemu's known issues and tips.

Video guides

How to play Ad-hoc multiplayer between a real PSP / Vita and PPSSPP emulator by PSP Mav (December 2023). This one is older: it uses the original Pro Online plugin on a single Wi-Fi network, but installing the plugin in Adrenaline works much the same way. Use the current aemu files and the relay settings from this guide. A viewer wrote: "this worked wonderfully between Vita and PPSSPP on Steamdeck."

PSP Relay on PS Vita (Adrenaline)