build: split a CUDA-free cuopt_client library out of libcuopt - #1798
Closed
ramakrishnap-nv wants to merge 1 commit into
Closed
build: split a CUDA-free cuopt_client library out of libcuopt#1798ramakrishnap-nv wants to merge 1 commit into
ramakrishnap-nv wants to merge 1 commit into
Conversation
Calling a remote cuopt_grpc_server currently requires the full GPU stack: `pip install cuopt` pulls cudf, cupy, rmm, pylibraft, numba-cuda and libcuopt (which itself pulls cuda-toolkit) onto a machine that only serializes protobuf over a socket. The coupling turned out to be largely accidental. The gRPC client sources are already GPU-free, and LP's data model is host-only in both languages. What tied them to CUDA was that every Cython extension links cuopt::cuopt, and the host-side implementations they need were compiled into .cu translation units. This adds a CPU-only `cuopt_client` library holding the host-side problem representation (parsers, data_model_view, mps_data_model, writers), the gRPC wire protocol (generated protos + mappers) and the gRPC clients. libcuopt and cuopt_grpc_server both link it, so there is exactly one mapper implementation rather than a client-side fork. Getting there needed the host halves of several CUDA translation units split out. Each split follows one rule: host code moves to a .cpp, device members stay in the .cu, and the moved members are instantiated explicitly per-member rather than via `template class` -- the latter would drag device ctors/dtors back into the client. math_optimization/solver_settings.cu -> .cpp + _gpu.cu mip_heuristics/solver_settings.cu -> .cpp + .cu pdlp/solution_conversion.cu -> + solution_conversion_cpu.cpp pdlp/solver_settings.cu -> + solver_settings_accessors.cpp pdlp/cpu_optimization_problem.cpp -> + cpu_optimization_problem_to_gpu.cpp Two changes were structural rather than mechanical: * populate_from_data_model_view() inlined both the GPU and CPU warm-start paths, so every TU including the header instantiated the device conversions. It is now split into apply_warmstart_gpu_target() (declared in the header, defined in libcuopt) and apply_warmstart_cpu_target(), with a kHostOnly template parameter dispatched via `if constexpr` so a host-only caller never instantiates the GPU branch. * to_optimization_problem() was a virtual member, so it occupied a slot in cpu_optimization_problem_t's vtable. Vtable relocations resolve eagerly at load, which would make the client library unloadable without libcuopt.so. It is now a free function in libcuopt that dispatches on the concrete type. pdlp_solver_settings_t held pdlp_warm_start_data_t by value; since that type owns rmm::device_uvector and its default ctor lives in a CUDA TU, merely constructing settings required CUDA. It is now held by shared_ptr, chosen over unique_ptr because shared_ptr type-erases its deleter into the control block, so a host-only TU can destroy it without the complete type. Result: libcuopt_client.so links with no CUDA, rmm or raft in NEEDED, and `ldd -r` across both libraries resolves cleanly. Note: cuopt_client deliberately uses default visibility, unlike cuopt_objs. libcuopt depends on ~214 of its symbols -- essentially the whole host-side API -- because this library was carved out of the internals rather than designed as a curated public surface. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Ramakrishna Prabhu <ramakrishnap@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Collaborator
Author
|
Superseded by a stacked series — same change, split for reviewability:
The stack reconciles exactly to this branch (zero diff), and is rebased onto current |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Calling a remote
cuopt_grpc_servercurrently requires the full GPU stack.pip install cuoptpullscudf,cupy-cuda13x[ctk],rmm,pylibraft,numba-cuda,scipy,pandasandlibcuopt(which itself pullscuda-toolkit) onto a machine that only serializes protobuf over a socket.The coupling turned out to be largely accidental:
cpp/src/grpc/**forthrust|rmm|raft|device_found two hits, both insolve_remote.cpp(the local-vs-remote dispatcher, which stays GPU-side)..cpp.cuopt::cuopt, and the host-side implementations they need were compiled into.cutranslation units.What this does
Adds a CPU-only
cuopt_clientlibrary holding the host-side problem representation (parsers,data_model_view,mps_data_model, writers), the gRPC wire protocol (generated protos + mappers), and the gRPC clients.libcuoptandcuopt_grpc_serverboth link it, so there is exactly one mapper implementation — not a client-side fork.Both gRPC arms go in: the routing mappers added in #1597 reference no
raft/rmm/thrusteither.The splits
Each follows one rule: host code moves to a
.cpp, device members stay in the.cu, and moved members are instantiated explicitly per-member rather than viatemplate class— the latter would drag device ctors/dtors back into the client.math_optimization/solver_settings.cu.cpp+_gpu.cumip_heuristics/solver_settings.cu.cpp+.cupdlp/solution_conversion.cusolution_conversion_cpu.cpppdlp/solver_settings.cusolver_settings_accessors.cpppdlp/cpu_optimization_problem.cppcpu_optimization_problem_to_gpu.cppTwo structural changes
populate_from_data_model_view()inlined both the GPU and CPU warm-start paths, so every TU including the header instantiated the device conversions. Now split intoapply_warmstart_gpu_target()(declared in the header, defined in libcuopt) andapply_warmstart_cpu_target(), with akHostOnlytemplate parameter dispatched viaif constexpr— so a host-only caller never instantiates the GPU branch, rather than merely not executing it.to_optimization_problem()wasvirtual, so it occupied a slot incpu_optimization_problem_t's vtable. Vtable relocations resolve eagerly at load, which would make the client library unloadable withoutlibcuopt.so. It is now a free function in libcuopt dispatching on the concrete type (6 call sites updated).pdlp_solver_settings_theldpdlp_warm_start_data_tby value. That type owns ninermm::device_uvectorand its default ctor lives in a CUDA TU, so merely constructing settings required CUDA. Now held byshared_ptr— chosen overunique_ptrbecauseshared_ptrtype-erases its deleter into the control block, letting a host-only TU destroy it without the complete type.Result
Undefined
cuopt::symbols in the client went 94 -> 0 for LP/MIP.Known gaps
cuopt::routing::symbols remain undefined in the client — the host-only accessors ofrouting::solver_settings_t/routing::assignment_t, still living inrouting/solver_settings.cuandassignment.cu. Same split pattern as above; left for a follow-up to keep this reviewable.solver_wrapper.pyxcome next.cuopt_clientdeliberately uses default visibility, unlikecuopt_objs. libcuopt depends on ~214 of its symbols — essentially the whole host-side API — because this library was carved out of the internals rather than designed as a curatedCUOPT_EXPORTsurface. Hiding them makeslibcuopt.sofail to load.Testing
Built against a fresh
.cuopt_env;libcuopt.so,libcuopt_client.so,cuopt_grpc_server,cuopt_cliand all 126 test binaries compile with 0 errors.The C++ suite caught two real bugs during development, both fixed here:
cuopt_staticneeded the client objects (internal tests reach parser internals the shared library doesn't export), and the visibility issue above.Important
Verifying the full suite locally required #1795 applied on top — current
rapidsai-nightlylibrmm26.10 deletesdevice_scalar's rvalue constructor, which breaks pristinemainwith 158 errors. That is independent of this PR (zero file overlap). With #1795 applied, 111/125 tests pass; the remaining 14 fail on a locally wedgednvidia_uvm(cudaGetDeviceCount->unknown erroreven for a standalone CUDA program), not on code. The GPU-dependent tests still need a clean CI run to confirm.🤖 Generated with Claude Code