Skip to content

Fix FlowAppendView support for 'once' flag by forcing batch read#22

Open
liamperritt wants to merge 1 commit intodatabricks-solutions:mainfrom
liamperritt:fix/append-view-once
Open

Fix FlowAppendView support for 'once' flag by forcing batch read#22
liamperritt wants to merge 1 commit intodatabricks-solutions:mainfrom
liamperritt:fix/append-view-once

Conversation

@liamperritt
Copy link

Implementing a fix for appending views using the 'once' flag by ensuring the returned value is a batch DataFrame, not a streaming DataFrame.

As outlined in the append_flow documentation:

Using once=True changes the flow: the return value must be a batch DataFrame in this case, not a streaming DataFrame.

Currently, configuring the 'once' flag with a batch data source fails with:
View is not a streaming view and must be referenced using read.

I've tested this fix manually, and it resolves the issue: it successfully appends a one-time batch flow into the target streaming table, as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant