u/Last_Jeweler8838

I am loading data from parquet files to LakeHouse tables using a Spark notebook (and subsequently warehouse views on LH tables). I noticed one thing: all my string fields which have more than 8,000 characters values ( JSON payload) are getting trimmed to 8000 characters, only when I'm querying the data through the SQL endpoint. When I query the same data in Spark notebook, it returns me the values without any loss.

My problem with this is: how do I fetch all the correct results without any trimming of data, as it will cause an error in JSON parsing? Does anyone have experience with this, or any configuration change that needs to be done? Please guide.