obs-outputs: Surface RTMP publish error to user

[nathanduchesne]

Description

When an RTMP server denies a publish, OBS showed a generic error even when the server provided a human-readable reason in the onStatus description. Three things were in the way: NetStream.Publish.Failed was not among the recognized failure codes (it fell through to the "Unhandled" branch), the description was only logged and never stored for the output, and the OBS_OUTPUT_INVALID_STREAM failure message did not include the output's last error.

This captures the server's onStatus description on a failed publish (adding NetStream.Publish.Failed), reports it via obs_output_set_last_error on the publish-denied path, and lets the invalid-stream failure message include it. The reason is then shown to the user instead of only a generic message.

Motivation and Context

Servers frequently reject a publish for actionable reasons (quota/credit limits, expired or disabled keys, etc.). OBS currently discards that message, leaving the user with no explanation. Surfacing it addresses https://obsproject.com/forum/threads/feature-request-alert-with-netstream-publish-failed-commands-description.66266/

How Has This Been Tested?

Built OBS locally and streamed to an RTMP server that rejects the publish with an onStatus description (an ingest that denies the stream with a reason).

• Before: only the generic "Could not access the specified channel or stream key…".
• After: that message followed by the server's reason (e.g. "You have surpassed your monthly streaming minutes.").

Verified on the 32.1.2 tag; master currently requires a newer macOS SDK than the test machine has, and the changed code is identical on both.

Types of changes

• Tweak (non-breaking change to improve existing functionality)

Checklist:

• I have read the contributing document.
• My code has been run through clang-format.
• My code follows the project's style guidelines.
• My code is not on the master branch.
• My code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[WizardCM]

Please fill in the PR template.

[nathanduchesne]

Please fill in the PR template.

Sorry Matt, my bad! Should be good now 😁

[notr1ch]

It's a security risk to allow the server to set a message that's displayed directly in the UI like this. A malicious server could send a "Your OBS Studio needs to be updated, please visit https://definitely-not-malware.invalid/update.exe" error and create a convincing malware / phishing prompt for the user, even more so if Qt renders HTML entities (I think the autodetection should stop at the newline though - but that makes the format string a security barrier which is not obvious at all).

I think we need to be very clear that this is a message from server.example.com, its contents are untrusted and its informational only. A QMessageBox might be insufficient to convey this.

Perhaps only servers in services.json should have access to this feature (disabled on custom RTMP)?

See also spesmilo/electrum#4968 for similar discussions on this topic.

[nathanduchesne]

@notr1ch Great point Richard, thanks for raising it! I'll get to it whenever I find the time 🤝 🙏🏻