Skip to content

build: split a CUDA-free cuopt_client library out of libcuopt - #1798

Closed
ramakrishnap-nv wants to merge 1 commit into
mainfrom
detach-grpc-client-from-cuda
Closed

build: split a CUDA-free cuopt_client library out of libcuopt#1798
ramakrishnap-nv wants to merge 1 commit into
mainfrom
detach-grpc-client-from-cuda

Conversation

@ramakrishnap-nv

Copy link
Copy Markdown
Collaborator

Why

Calling a remote cuopt_grpc_server currently requires the full GPU stack. pip install cuopt pulls cudf, cupy-cuda13x[ctk], rmm, pylibraft, numba-cuda, scipy, pandas and libcuopt (which itself pulls cuda-toolkit) onto a machine that only serializes protobuf over a socket.

The coupling turned out to be largely accidental:

  • The C++ gRPC client sources are already GPU-free — grepping cpp/src/grpc/** for thrust|rmm|raft|device_ found two hits, both in solve_remote.cpp (the local-vs-remote dispatcher, which stays GPU-side).
  • LP's data model is host-only in both languages; its C++ types are plain .cpp.
  • What actually tied them to CUDA is that every Cython extension links cuopt::cuopt, and the host-side implementations they need were compiled into .cu translation units.

What this does

Adds a CPU-only cuopt_client library holding the host-side problem representation (parsers, data_model_view, mps_data_model, writers), the gRPC wire protocol (generated protos + mappers), and the gRPC clients. libcuopt and cuopt_grpc_server both link it, so there is exactly one mapper implementation — not a client-side fork.

Both gRPC arms go in: the routing mappers added in #1597 reference no raft/rmm/thrust either.

The splits

Each follows one rule: host code moves to a .cpp, device members stay in the .cu, and moved members are instantiated explicitly per-member rather than via template class — the latter would drag device ctors/dtors back into the client.

File Split into
math_optimization/solver_settings.cu .cpp + _gpu.cu
mip_heuristics/solver_settings.cu .cpp + .cu
pdlp/solution_conversion.cu + solution_conversion_cpu.cpp
pdlp/solver_settings.cu + solver_settings_accessors.cpp
pdlp/cpu_optimization_problem.cpp + cpu_optimization_problem_to_gpu.cpp

Two structural changes

populate_from_data_model_view() inlined both the GPU and CPU warm-start paths, so every TU including the header instantiated the device conversions. Now split into apply_warmstart_gpu_target() (declared in the header, defined in libcuopt) and apply_warmstart_cpu_target(), with a kHostOnly template parameter dispatched via if constexpr — so a host-only caller never instantiates the GPU branch, rather than merely not executing it.

to_optimization_problem() was virtual, so it occupied a slot in cpu_optimization_problem_t's vtable. Vtable relocations resolve eagerly at load, which would make the client library unloadable without libcuopt.so. It is now a free function in libcuopt dispatching on the concrete type (6 call sites updated).

pdlp_solver_settings_t held pdlp_warm_start_data_t by value. That type owns nine rmm::device_uvector and its default ctor lives in a CUDA TU, so merely constructing settings required CUDA. Now held by shared_ptr — chosen over unique_ptr because shared_ptr type-erases its deleter into the control block, letting a host-only TU destroy it without the complete type.

Result

libcuopt_client.so
  NEEDED : libgrpc++, libprotobuf, libabsl_*, librapids_logger, libc, libstdc++, libgomp, libdl
           -> no libcudart, no rmm, no raft, no cudss
  ldd -r across libcuopt.so + libcuopt_client.so : 0 unresolved symbols

Undefined cuopt:: symbols in the client went 94 -> 0 for LP/MIP.

Known gaps

  • 14 cuopt::routing:: symbols remain undefined in the client — the host-only accessors of routing::solver_settings_t / routing::assignment_t, still living in routing/solver_settings.cu and assignment.cu. Same split pattern as above; left for a follow-up to keep this reviewable.
  • No Python packaging change yet. This is the C++/linkage prerequisite; relinking the Cython extensions and splitting the CPU half of solver_wrapper.pyx come next.
  • cuopt_client deliberately uses default visibility, unlike cuopt_objs. libcuopt depends on ~214 of its symbols — essentially the whole host-side API — because this library was carved out of the internals rather than designed as a curated CUOPT_EXPORT surface. Hiding them makes libcuopt.so fail to load.

Testing

Built against a fresh .cuopt_env; libcuopt.so, libcuopt_client.so, cuopt_grpc_server, cuopt_cli and all 126 test binaries compile with 0 errors.

The C++ suite caught two real bugs during development, both fixed here: cuopt_static needed the client objects (internal tests reach parser internals the shared library doesn't export), and the visibility issue above.

Important

Verifying the full suite locally required #1795 applied on top — current rapidsai-nightly librmm 26.10 deletes device_scalar's rvalue constructor, which breaks pristine main with 158 errors. That is independent of this PR (zero file overlap). With #1795 applied, 111/125 tests pass; the remaining 14 fail on a locally wedged nvidia_uvm (cudaGetDeviceCount -> unknown error even for a standalone CUDA program), not on code. The GPU-dependent tests still need a clean CI run to confirm.

🤖 Generated with Claude Code

Calling a remote cuopt_grpc_server currently requires the full GPU stack:
`pip install cuopt` pulls cudf, cupy, rmm, pylibraft, numba-cuda and libcuopt
(which itself pulls cuda-toolkit) onto a machine that only serializes protobuf
over a socket.

The coupling turned out to be largely accidental. The gRPC client sources are
already GPU-free, and LP's data model is host-only in both languages. What tied
them to CUDA was that every Cython extension links cuopt::cuopt, and the
host-side implementations they need were compiled into .cu translation units.

This adds a CPU-only `cuopt_client` library holding the host-side problem
representation (parsers, data_model_view, mps_data_model, writers), the gRPC
wire protocol (generated protos + mappers) and the gRPC clients. libcuopt and
cuopt_grpc_server both link it, so there is exactly one mapper implementation
rather than a client-side fork.

Getting there needed the host halves of several CUDA translation units split
out. Each split follows one rule: host code moves to a .cpp, device members
stay in the .cu, and the moved members are instantiated explicitly per-member
rather than via `template class` -- the latter would drag device ctors/dtors
back into the client.

  math_optimization/solver_settings.cu -> .cpp + _gpu.cu
  mip_heuristics/solver_settings.cu    -> .cpp + .cu
  pdlp/solution_conversion.cu          -> + solution_conversion_cpu.cpp
  pdlp/solver_settings.cu              -> + solver_settings_accessors.cpp
  pdlp/cpu_optimization_problem.cpp    -> + cpu_optimization_problem_to_gpu.cpp

Two changes were structural rather than mechanical:

* populate_from_data_model_view() inlined both the GPU and CPU warm-start paths,
  so every TU including the header instantiated the device conversions. It is
  now split into apply_warmstart_gpu_target() (declared in the header, defined
  in libcuopt) and apply_warmstart_cpu_target(), with a kHostOnly template
  parameter dispatched via `if constexpr` so a host-only caller never
  instantiates the GPU branch.

* to_optimization_problem() was a virtual member, so it occupied a slot in
  cpu_optimization_problem_t's vtable. Vtable relocations resolve eagerly at
  load, which would make the client library unloadable without libcuopt.so. It
  is now a free function in libcuopt that dispatches on the concrete type.

pdlp_solver_settings_t held pdlp_warm_start_data_t by value; since that type
owns rmm::device_uvector and its default ctor lives in a CUDA TU, merely
constructing settings required CUDA. It is now held by shared_ptr, chosen over
unique_ptr because shared_ptr type-erases its deleter into the control block,
so a host-only TU can destroy it without the complete type.

Result: libcuopt_client.so links with no CUDA, rmm or raft in NEEDED, and
`ldd -r` across both libraries resolves cleanly.

Note: cuopt_client deliberately uses default visibility, unlike cuopt_objs.
libcuopt depends on ~214 of its symbols -- essentially the whole host-side
API -- because this library was carved out of the internals rather than
designed as a curated public surface.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Ramakrishna Prabhu <ramakrishnap@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 25, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@ramakrishnap-nv

Copy link
Copy Markdown
Collaborator Author

Superseded by a stacked series — same change, split for reviewability:

The stack reconciles exactly to this branch (zero diff), and is rebased onto current main now that #1795 has landed. Closing in favour of those.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant