Skip to content

[FEA]: Add support for CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW in LaunchConfig #2626

Description

@lijinf2

Is this a duplicate?

Area

cuda.core

Is your feature request related to a problem? Please describe.

LaunchConfig does not expose CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW, which sets an L2 access-policy window for a launch (Ampere+).

Parent tracking: #496.

Describe the solution you'd like

  • Users can set CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW via LaunchConfig.
  • Mapping to the native launch attribute is tested.

Describe alternatives you've considered

Use cuda.bindings.driver (CUlaunchAttribute / CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW) and cuLaunchKernelEx directly.

Downside: leaves cuda.core users without a first-class LaunchConfig API; they must drop to low-level bindings, manage the attribute array themselves, and lose consistency with other LaunchConfig launch attributes (is_cooperative, programmatic_stream_serialization, etc.).

Additional context

Incremental LaunchConfig attribute coverage under #496 (same approach as #1334 for programmatic_stream_serialization).

Example (C API) — agent-generated; reviewed but not executed:

// 1. Reserve L2 cache space
cudaDeviceProp prop;
cudaGetDeviceProperties(&prop, 0);
size_t l2_setaside = min((size_t)(prop.l2CacheSize * 0.75),
                         (size_t)prop.persistingL2CacheMaxSize);
cudaDeviceSetLimit(cudaLimitPersistingL2CacheSize, l2_setaside);

// 2. Configure access policy window
size_t window_size = min((size_t)prop.accessPolicyMaxWindowSize,
                         num_bytes);

CUlaunchAttribute attr;
attr.id = CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW;
attr.value.accessPolicyWindow.base_ptr = (void*)data_persistent;
attr.value.accessPolicyWindow.num_bytes = window_size;
attr.value.accessPolicyWindow.hitRatio  = 0.6;
attr.value.accessPolicyWindow.hitProp   = CU_ACCESS_PROPERTY_PERSISTING;
attr.value.accessPolicyWindow.missProp  = CU_ACCESS_PROPERTY_STREAMING;

// 3. Launch with cuLaunchKernelEx
CUlaunchConfig config = {
    .gridDimX = grid, .gridDimY = 1, .gridDimZ = 1,
    .blockDimX = 256, .blockDimY = 1, .blockDimZ = 1,
    .sharedMemBytes = 0,
    .hStream = stream,
    .attrs = &attr,
    .numAttrs = 1
};

for (int i = 0; i < 10; i++) {
    cuLaunchKernelEx(&config, kernelA, NULL, NULL);
}

// 4. Disable window (release L2 for other kernels)
attr.value.accessPolicyWindow.num_bytes = 0;
cuLaunchKernelEx(&config, kernelB, NULL, NULL);

// 5. Clear persisting L2 lines
cuCtxResetPersistingL2Cache();

Metadata

Metadata

Assignees

Labels

cuda.coreEverything related to the cuda.core modulefeatureNew feature or request

Type

No type

Projects

Status
Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions