Skip to content

Conversation

@zhaixuejun1993
Copy link

@zhaixuejun1993 zhaixuejun1993 commented Feb 9, 2026

Enabling the OpenVINO backend fallback to Llama.cpp CPU backend mechanisms.

Below is a summary of the main process:

  1. Dynamic Dimension Computation
    Function: compute_cgraph_dynamic_dims()
    Purpose: Determines the dynamic dimensions for each node in the computation graph. This is essential for handling nodes with variable shapes during runtime.
    Process:
    Traverses the computation graph.
    Assigns dynamic dimension indices to nodes based on their operation type and dependencies.
    Handles specific operations like [GGML_OP_VIEW], [GGML_OP_RESHAPE], and others to propagate dynamic dimensions.

  2. Adding Extra Model Outputs
    Function: add_extra_model_outputs_for_fallback()
    Purpose: Ensures that all relevant nodes in the computation graph are included as model outputs for fallback scenarios.
    Process:
    Maps tensor data addresses to their corresponding nodes, excluding [GGML_OP_VIEW] nodes.
    Adds nodes to the [m_model_outputs] map if they are not already present.

  3. Adding Extra Model Inputs
    Function: add_extra_model_inputs_for_fallback()
    Purpose: Ensures that all necessary input nodes are included as model inputs for fallback scenarios.
    Process:
    Iterates through the source nodes of each computation graph node.
    Skips nodes already in [m_model_weights] or [m_model_inputs].
    Excludes intermediate nodes from [m_node_info_list].
    Creates OpenVINO parameter nodes for eligible source nodes and updates the [m_inputs] and [m_model_inputs] maps.

@github-actions github-actions bot added the ggml label Feb 9, 2026
@wine99 wine99 force-pushed the dev_backend_openvino branch from 6d71ded to 900dd76 Compare February 11, 2026 02:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants