[BUG] Undefined behavior in nvdsvideotemplate custom library instantiation function

[CristiFati]

Given the fact that:

• There's a .claude dir in the repo
• A web search didn't reveal much information on this topic

I have a strong feeling that I'm missing something crucial (soc, or smth), or these sources are not the ones used to build existing libraries, because people should have been complaining about this (and the other 2 issues that I opened).

Describe the bug
NVIDIA/DeepStream - customlib_impl.cpp:150 declares:

extern "C" IDSCustomLibrary *CreateCustomAlgoCtx(DSCustom_CreateParams *params);

but

• NVIDIA/DeepStream -gstnvdsvideotemplate.cpp:663 calls it: nvdsvideotemplate->algo_ctx = nvdsvideotemplate->algo_factory->CreateCustomAlgoCtx(nvdsvideotemplate->custom_lib_name, G_OBJECT(btrans)); where btrans is a pointer to itself (as aBseTransform)
• NVIDIA/DeepStream -nvdscustomlib_factory.hpp:72 calls it return createAlgoCtx ? createAlgoCtx(object) : nullptr; where object is btrans (from previous step)

It works now since the example function ignores the argument, but if someone would try to (dereference and) use it as DSCustom_CreateParams* that would be Undefined Behavior (and a great chance to crash). Not to mention that in (ancient) DS 6.2, videotemplate called this function with no argument at all, and simply trying to dereference it would have ended up in this situation.

Steps/Code to reproduce bug

Create a dummy custom library with:

extern "C" IDSCustomLibrary *CreateCustomAlgoCtx(DSCustom_CreateParams *params)
{
  if (params) {
    params->m_gpuId = 0;
  }
  return new SampleAlgorithm();
  // return new SampleAlgorithm(params);  // and do the access there
}

Expected behavior
Everything should work as expected, no segfault should be encountered when running e.g.:
NVDS_CUSTOMLIB="path/to/libcustom.s0" gst-inspect-1.0 nvdsvideotemplate

Environment details (please complete the following information):
N/A. I used an Ubuntu DS container

Additional context
Add any other context about the problem here.

---

By submitting this issue, you agree to follow our code of conduct and our contribution guidelines.