Optimization: Can we use chunking for String ( CHAR,VARCAR,NVARCHAR) data type to remove extra memory allocation

[subrata-ms]

Describe the bug

Currently we allocate fixed size memory ( coulumnsize+1) to handle string data type. This was causing problem for CP1252 character set. We have fix this issue under reported SQLAlchemy bug ( #435 ).
However, we do see an opportunity improve the memory allocation for the string data type. One of the consideration/approach could be chunking.
This will help to reduce unnecessary memory allocation considerably during multithreaded execution and for large dataset.

Exception message:
Stack trace:

To reproduce

This is considered as code optimization. Below is the related bug -
#435

Expected behavior

All string data type should handle CP1252 character set.

[github-actions]

Hi @subrata-ms, thank you for opening this issue!

Our team will review it shortly. We aim to triage all new issues within 24-48 hours and get back to you.

If you have additional information to share, please feel free to update the issue.

Thank you for your patience!