DEV Community

Igor Alexandrov for JetRockets

Posted on • Originally published at jetrockets.pro

5 1

How to parse CSV with double quote (") character in Crystal

We use microservice written in Crystal to parse large CSV files (about 1.5Gb). Some rows in these files may contain no closed " characters:

,Y,FEDERAL NATIONAL MORTGAGE ASSOCIATION "F,,
Enter fullscreen mode Exit fullscreen mode

With Crystal default CSV parse settings this row and everything after it won't be parsed correctly because DEFAULT_QUOTE_CHAR constant is equal to ". Of course you can override quote_char param in CSV constructor with something that cannot be found in your document.

From my point of view the best is to use zero byte which is '\u0000' in Crystal.

csv = CSV.new(file, headers: true, strip: true, quote_char: '\u0000')
while csv.next
  # ... 
end
Enter fullscreen mode Exit fullscreen mode

Hack!

Top comments (0)

👋 Kindness is contagious

Engage with a wealth of insights in this thoughtful article, valued within the supportive DEV Community. Coders of every background are welcome to join in and add to our collective wisdom.

A sincere "thank you" often brightens someone’s day. Share your gratitude in the comments below!

On DEV, the act of sharing knowledge eases our journey and fortifies our community ties. Found value in this? A quick thank you to the author can make a significant impact.

Okay