Support custom transformation rules in Clojure - #1763
Open
markwort wants to merge 2 commits into
Open
Conversation
…e CAST section of .load files, as is already promised in the introductory comments in clojure/src/pgloader/transforms.clj
Author
|
@dimitri is this something you would consider merging? Or is there any other plan to bring back the custom transformation rules? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I need to migrate some data from a MySQL database into PostgreSQL, where in the original schema some columns use
longblobdatatype, but when the application is connected to PostgreSQL, it expects to read plain text, something that it can process as VARCHAR in its SQL library.The application in question is Znuny.
The documentation brought me onto the idea of using custom transformation rules, which worked fine with pgloader 3.6 and the lisp files.
In the clojure rewrite,
clojure/src/pgloader/transforms.clj, it is stated that custom transform functions can be used, but my AI helper and I could not get it to work.It seems like pgloaders grammar currently doesn't handle this.
I've had my AI helper produce a patch to support this, which I am sending in this PR.
To be perfectly honest, I don't know anything about Clojure and I cannot attest that there are no side effects or other weirdnesses introduced by this, but it seems to work for my usecase.
Other points to eventually fix would be to update the Documentation (which still talks about Lisp transformers), and possibly to think about using files to hand over clojure funcions, as defining them in the
CASTsection of a .load file (and repeating them for each of the 25 columns that require this same treatment) is a bit unsightly.Disclosure: This patch was written by AI. This description of the PR was written by me.