For macOS we just follow the example steps from the Readme coming with MBS Plugins examples in URL Scheme folder. As you can read there, you change the info.plist file of the app. Than you use the URLScheme functions in MBS Plugin to set the default scheme handler. Instead of a FileMaker Pro (which usually is not installed on the clients machines), the runtime will now handle the FMP URLs.
For Windows, the installer for the runtime, registers in the registry the FMP URL scheme and points to the runtime exe file. This requires admin permissions.
In both cases, FileMaker Pro (if installed), won't be the FMP URL handler anymore. So please keep it an option only for clients who need it and have no FileMaker Pro installed!
This can be reverted on macOS by deleting the runtime (or some terminal commands for lsregister). On Windows you can revert it with registry changes or using the FileMaker Pro installer to install over the existing installation.