Select configurable sub-tables for generic table entries

[HS24]

• Added config to specify additional tables to fetch for each query result
• Added output formats for sub-table result: XML and TSV (tab-separated values)
• Added filter options for sub-table results to only select specific values
• Added example to query the short description of activity groups
• Added example to query single roles that are part of a composite role
• Added example to query SAP flags for activity groups (these include a flag for the composite/single role distinction)

[HS24]

@martin-lizner Can you have a look at this extension of the connector? This enables the connector to load more details for ActivityGroups and other object types.

I also created a PR for the public documentation: Evolveum/docs#58

[HS24]

@semancik Can you have a look at this connector extension?

[martin-lizner]

Thank you for the PR! I may not have time to full test it on SAP backend, but these are my remarks based on the code review (and i apologize if i missed smth):

• SapBasicFilterTranslator no longer fully uses AbstractFilterTranslator? Seems like only UID filter gets to the SAP and other filters are in-memory only?
• Connector now skips rows whose concatenated key doesn’t exactly match the searched key..? SapConnector.java (lines 641-713). Could it impact existing deployments which uses tableParameterNames connector param?
• Why is this workaround removed? "// SAP returned multiple entries on UID search, we need to pick the best entry to pass to upper layer"

My basic concern is disruption for any existing deployments. New features are generally welcomed.

[HS24]

Thank you for your review! I will try to answer your questions.

FilterTranslator vs. AbstractFilterTranslator:

• The AbstractFilterTranslator only adds convenience methods to implement individual filters (like equals, and...) and tries to simplify the query, so that complex queries might be simpler.
  Since the SAP BAPI only accepts a single search value (which is used as this SAP-style startsWith key), nearly no filter can be applied inside the BAPI call. They will be applied in-memory on the search results.
• The BAPI search key is still extracted in the same way as before: The query value from the equals filter for the attributes __NAME__ or __UID__ will be passed to the BAPI.
• The behavior for equals filter for __NAME__ or __UID__ changes a little, but this should not be noticeable (more on that in the next answer).
• The behavior of queries changes, that use different filters:
  1. This includes the GUI panel for resource objects, because basic filter values are queried as contains. Previously these filters didn't work and all objects were returned without filtering. With this version, the filters can actually be used.
  2. Reconcile queries for specific object classes will be affected as well. These filters didn't work on-resource, so enableFilteredResultsHandler had to be activated to enable in-memory filtering inside midPoint. The setting is not needed any more, but it doesn't hurt if it is still used.
     But: The setting might still be needed for account queries.

Skip rows that don't match the concatenated key:

• Only if the connector executes an equals query for __NAME__ or __UID__, the connector will now skip entries, that don't match the exact query key (stored in basicByNameEquals).
  If an in-memory filter is specified, the connector will also skip the entries, which don't match this filter.
  This is results in a big performance optimization, especially if the new sub-tables are queried.
• The workaround in the finally block did the same thing previously.
• It only would have an impact if an equals query was used improperly instead of a startsWith (but I doubt there is a real use-case to do this).

Removed workaround:

• Previously, the query results were all converted to connector objects and filtered afterwards. The conversion will be more time-consuming if the new sub-queries are executed. Because of the negative performance impact, the check is now moved into the loop (see the basicByNameEquals check in line 677) to stop the execution after all relevant objects were handled.

Overall this change shouldn't be disruptive to existing configurations.

[HS24]

Hi @martin-lizner, did you have time to look at this PR again?