How can I convert CSV to JSON with data type in python or node.js?

Question

I wanna convert CSV to JSON correct data type

csv file 2nd row is data type.
data has over 300 properties
example data:

Name	DMG	HP	Human
string	number	number	boolean
knight	100	500	true
archer	50	200	true
dog	–	–	–

if string empty return null

if number empty return 0

if boolean empty return false

my node.js code:

const fs = require('fs')
const papa = require("papaparse")

const results = [];

const options = { header: true, dynamicTyping: true };

fs.createReadStream("characters.csv")
    .pipe(papa.parse(papa.NODE_STREAM_INPUT, options))
    .on("data", (data) => {
        results.push(data);
    }).on("end", () => {
        console.log(results)
    })

output I expecting:

[
    {
        "Name": "knight",
        "DMG": 100,
        "HP": 500,
        "Human": true,
    },
    {
        "Name": "archer",
        "DMG": 50,
        "HP": 200,
        "Human": true,
    },
    {
        "Name": "dog",
        "DMG": 0,
        "HP": 0,
        "Human": false,
    },
]

Asked By: JameyPlay

||

Source

Answer 1

Try overriding the properties in the on.data event like so

let i = 0; //define this outside, its only to skip that first row with headers.
if(i > 0){ //skipping first row
    data.DMG = parseInt(data.DMG) || 0; //an int or 0
    data.HP = parseInt(data.HP) || 0; //an int or 0
    data.Human = data.Human === !0; //!0 = not false, will match true string or bool
    results.push(data);
}
i++;//iterate

const fs = require('fs')
const papa = require("papaparse")

const results = [];

const options = { header: true, dynamicTyping: true };
let i = 0;
fs.createReadStream("characters.csv")
    .pipe(papa.parse(papa.NODE_STREAM_INPUT, options))
    .on("data", (data) => {
        //add this here
        if(i > 0){
            data.DMG = parseInt(data.DMG) || 0; //an int or 0
            data.HP = parseInt(data.HP) || 0; //an int or 0
            data.Human = data.Human === !0; //!0 = not false, will match true string or bool
            results.push(data);
        }
        i++;
        //end add this here
    }).on("end", () => {
    console.log(results)
})

Answered By: joeycrash135

Answer 2

Options options..

in this approach I cached the headerTypes and made a small helper function to return the intended type

define the vars let i = 0, headerTypes = {};

replace your on.data code with this

.on("data", (data) => {
    if(i > 0){
       for(let prop in data){
            if(data.hasOwnProperty(prop)){
                const value = data[prop];
                data[prop] = typecast(headerTypes[prop], value);
            }
        }
    }else{
        //save types from row 0
        headerTypes = data;
    }
    i++;
})

add this helper function

function typecast(type = '', value){
    switch(type){
      case "number":
        return +value || 0; //unary for int or float
      case "boolean":
        return value === !0; //typecast to Boolean will let any filled string like '-' be true, do this instead.
      case "string":
      default:
        return String(value);
    }
}

Answered By: joeycrash135

How can I convert CSV to JSON with data type in python or node.js?

Question:

Answers: