Fix #1641: Handle cleanup when exceptions are thrown#1669
Fix #1641: Handle cleanup when exceptions are thrown#1669mdboom wants to merge 2 commits intoNVIDIA:mainfrom
Conversation
|
Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
/ok to test |
|
|
/ok to test |
| cdef _HelperInputVoidPtrStruct cycallbackHelper | ||
| cdef void* cycallback = _helper_input_void_ptr(callback, &cycallbackHelper) | ||
| cdef _HelperInputVoidPtrStruct cypayloadHelper | ||
| cdef void* cypayload = _helper_input_void_ptr(payload, &cypayloadHelper) | ||
| with nogil: | ||
| err = cynvrtc.nvrtcSetFlowCallback(cyprog, cycallback, cypayload) | ||
| _helper_input_void_ptr_free(&cycallbackHelper) | ||
| _helper_input_void_ptr_free(&cypayloadHelper) |
There was a problem hiding this comment.
Q: Maybe we should encapsulate _helper_input_void_ptr and _helper_input_void_ptr_free as a C++ class (not Cython cdef class, which was the previous approach IIRC)?
There was a problem hiding this comment.
Yeah, that's not a bad idea. We'd get RAII for free.
There was a problem hiding this comment.
Is it worth considering specialized guards to simplify a few scenarios like malloc/free, buffer release, and the generic _HelperInputVoidPtr?
For example,
struct FreeGuard {
void* ptr;
FreeGuard(void* p) : ptr(p) {}
~FreeGuard() { std::free(ptr); }
void* release() { void* p = ptr; ptr = nullptr; return p; }
};
The usage pattern is:
cdef void* cbData = malloc(sizeof(CallbackData))
if cbData == NULL:
return ERROR
cdef FreeGuard guard = FreeGuard(cbData)
# ... precall that might throw ...
# ... CUDA call ...
m_global._allocated[key] = guard.release() # transfer ownership
This would avoid the issue I pointed out in another comment. It's also straightforward to build this on top of std::unique_ptr. Unfortunately, Cython does not support C++ lambdas, so each case needs its own class, but there don't seem to be many cases.
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
| finally: | ||
| if err != cydriver.CUDA_SUCCESS: | ||
| free(cbData) | ||
| else: | ||
| m_global._allocated[int(callback)] = cbData | ||
| _helper_input_void_ptr_free(&cyuserDataHelper) |
There was a problem hiding this comment.
This clean up block looks questionable on an early return from line 31021 (malloc failed). What is the value of err if control never reached line 31027? Will a NULL pointer be stored in m_global._allocated?
| cdef _HelperInputVoidPtrStruct cycallbackHelper | ||
| cdef void* cycallback = _helper_input_void_ptr(callback, &cycallbackHelper) | ||
| cdef _HelperInputVoidPtrStruct cypayloadHelper | ||
| cdef void* cypayload = _helper_input_void_ptr(payload, &cypayloadHelper) | ||
| with nogil: | ||
| err = cynvrtc.nvrtcSetFlowCallback(cyprog, cycallback, cypayload) | ||
| _helper_input_void_ptr_free(&cycallbackHelper) | ||
| _helper_input_void_ptr_free(&cypayloadHelper) |
There was a problem hiding this comment.
Is it worth considering specialized guards to simplify a few scenarios like malloc/free, buffer release, and the generic _HelperInputVoidPtr?
For example,
struct FreeGuard {
void* ptr;
FreeGuard(void* p) : ptr(p) {}
~FreeGuard() { std::free(ptr); }
void* release() { void* p = ptr; ptr = nullptr; return p; }
};
The usage pattern is:
cdef void* cbData = malloc(sizeof(CallbackData))
if cbData == NULL:
return ERROR
cdef FreeGuard guard = FreeGuard(cbData)
# ... precall that might throw ...
# ... CUDA call ...
m_global._allocated[key] = guard.release() # transfer ownership
This would avoid the issue I pointed out in another comment. It's also straightforward to build this on top of std::unique_ptr. Unfortunately, Cython does not support C++ lambdas, so each case needs its own class, but there don't seem to be many cases.
3 participants
In driver, runtime and nvrtc, the generated code follows the following pattern:
For certain argument types, $precall and $postcall are required to be pairs, such as
malloc/free. If an exception is thrown when parsing one of the other arguments in$precall,$postcallwill not get called, leaking memory or other resources.This reorganizes the code (whenever
$postcallis non-empty) to:This diff is larger than it otherwise might need to be since
Cythononly allowscdefdeclarations at the top level. So all thecdef's that used to be part of$precallneed to be moved to$init.Performance impact
try/finallyis implemented by Cython withgotos. It generates two copies of thefinallyclause: one for success and one for failure, so there is a cost in overall code size. (This is similar to CPython's bytecode implementation oftry/finally). However, the runtime performance penalty is below the noise threshold in the #659 benchmark. There is a small performance penalty of around 200ns for the error case, but that's to be expected -- the old implementation leaked memory.Alternatives considered
We could use C++ RAII to automatically free memory. In fact, some of our code in these files already does that in the use of
std::vector. However, this is not generally applicable as a solution, since it's not possible to implement a custom RAII struct Cython that is not also a PyObject. This would make theHelperVoidPtrStructoptimization impossible.We could require that $init does all validation (but not any resource allocation), and the $precall would never raise, so $postcall would always be guaranteed to run. It seems like this may have been part of the original design, but then has not been strictly enforced everywhere over time. Even if we could get to that, it has two problems. (1) Some exceptions in $precall are unavoidable, even if the inputs are valid, such as
mallocfailing. (2) Doing a separate validation and conversion pass on all the arguments is never going to be as performant as a single pass.