$show=/label

Python - TypeError: Object of type 'int64' is not JSON serializable (Works for all data types - Integer, Float, Array, etc)

SHARE:

1. Overview In this tutorial, You will be learning how to solve the problem in python " TypeError: Object of type 'int64' i...

1. Overview


In this tutorial, You will be learning how to solve the problem in python "TypeError: Object of type 'int64' is not JSON serializable" and "TypeError: (Integer) is not JSON serializable". These two errors come into existence when we working with JSON Serialization. Solutions shown in this article will work for apart from the int or int64 type such as float, array and etc.

Below is the scenario where my friend faced the problem.

Python TypeError Object of type  int64 is not JSON serializable


2. Problem 1:  Statement (Scenario): TypeError: Object of type 'int64' is not JSON serializable


I have a Dataframe that stores Store name and daily sales count. I am trying to insert this to Salesforce using the below Python script. I, however, get an error

    TypeError: Object of type 'int64' is not JSON serializable

Given below is the view of the Dataframe

Storename,Count
Store A,10
Store B, 12
Store C, 5


I use the below code to insert it to Salesforce

update_list = []
for i in range((len(store))):
    update_data = {
               'name' :    store['entity_name'].iloc[i],
                'count__c': store['count'].iloc[i] }

    update_list.append(update_data)

sf_data_cursor = sf_datapull.salesforce_login()
sf_data_cursor.bulk.Account.update(update_list)

Get the error the last line above gets executed.

Error:

TypeError: Object of type 'int64' is not JSON serializable

You can try this problem solving without seeing the answer first.

3. Solution 1 for TypeError: Object of type 'int64' is not JSON serializable


JSON does not recognize NumPy data types. Convert the number to a Python int before serializing the object. Just add int typecasting to the store(..) value.

'count__c': int(store['count'].iloc[i])

4. Solution 2 to TypeError: Object of type 'int64' is not JSON serializable


import numpy package and create your own decoder for solving the problem of any data type.


import json
import numpy as np

class NpEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.integer):
            return int(obj)
        elif isinstance(obj, np.floating):
            return float(obj)
        elif isinstance(obj, np.ndarray):
            return obj.tolist()
        else:
            return super(NpEncoder, self).default(obj)

# Your codes .... 
json.dumps(data, cls=NpEncoder)


5. Solution 3 with typecasting to String all values in CSV file


Another option is that when you create the data frame that treats all values as string from CSV file. For that, we need to use "dtype=str" which defines the type for each field as String. So that String is serialized by default.

For example, if you loaded the store from a CSV file:

import pandas as pd
store = pd.read_csv('store.csv', dtype=str)


Then everything has a type of str that can be serialized to JSON.

6. Problem 2: preserve int64 values when parsing JSON in Go


The problem faced in the below program.

I am processing a JSON POST in Go that contains an array of objects containing 64bit integers. When using JSON. Unmarshal these values seem to be converted to a float64 which isn't very helpful.

body := []byte(`{"tags":[{"id":4418489049307132905},{"id":4418489049307132906}]}`)

var dat map[string]interface{}
if err := json.Unmarshal(body, &dat); err != nil {
    panic(err)
}

tags := dat["tags"].([]interface{})

for i, tag := range tags {

    fmt.Println("tag: ", i, " id: ", tag.(map[string]interface{})["id"].(int64))

}

Is there any way to preserve the original int64 in the output of JSON.Unmarshal?

7. Solution 1: To preserve the int64 type in unmarshalling


You can use a Decoder and UseNumber to decode your numbers without loss.

The Number type is defined like this :

// A Number represents a JSON number literal.
type Number string

which means you can easily convert it :

package main

import (
    "encoding/json"
    "fmt"
    "bytes"
    "strconv"
)

func main() {
    body := []byte("{\"tags\":[{\"id\":4418489049307132905},{\"id\":4418489049307132906}]}")
    dat := make(map[string]interface{})
    d := json.NewDecoder(bytes.NewBuffer(body))
    d.UseNumber()
    if err := d.Decode(&dat); err != nil {
        panic(err)
    }
    tags := dat["tags"].([]interface{})
    n := tags[0].(map[string]interface{})["id"].(json.Number)
    i64, _ := strconv.ParseUint(string(n), 10, 64)
    fmt.Println(i64) // prints 4418489049307132905
}


8. Solution 2: To preserve the int64 type in unmarshalling


You can also decode into a specific structure tailored as per your needs.

package main

import (
    "encoding/json"
    "fmt"
)

type A struct {
    Tags []map[string]uint64 // "tags"
}

func main() {
    body := []byte("{\"tags\":[{\"id\":4418489049307132905},{\"id\":4418489049307132906}]}")
    var a A
    if err := json.Unmarshal(body, &a); err != nil {
        panic(err)
    }
    fmt.Println(a.Tags[0]["id"]) // logs 4418489049307132905
}

Personally, I generally prefer this solution which feels more structured and easier to maintain.

Note:

A small note if you use JSON because your application is partly in JavaScript: JavaScript has no 64 bits integers but only one number type, which is the IEEE754 double-precision float. So you wouldn't be able to parse this JSON in JavaScript without loss using the standard parsing function.

9. Conclusion


In this article, we have seen the solutions for the problems that we face in data types int, int64, float and array serialization. We have shown the problems here for int type. But the same solutions will work for float and remaining types as well.

Just give a try. Keep posting your questions in the comment section for answers.

COMMENTS

BLOGGER

About Us

Author: Venkatesh - I love to learn and share the technical stuff.
Name

accumulo,1,ActiveMQ,2,Adsense,1,API,37,ArrayList,16,Arrays,16,Bean Creation,3,Bean Scopes,1,BiConsumer,1,Blogger Tips,1,Books,1,C Programming,1,Collection,5,Collections,23,Collector,1,Command Line,1,Compile Errors,1,Configurations,7,Constants,1,Control Statements,8,Conversions,6,Core Java,84,Corona India,1,Create,2,CSS,1,Date,3,Date Time API,13,Dictionary,1,Difference,1,Download,1,Eclipse,2,Efficiently,1,Error,1,Errors,1,Exception,1,Exceptions,3,Fast,1,Files,10,Float,1,Font,1,Form,1,Freshers,1,Function,3,Functional Interface,2,Garbage Collector,1,Generics,4,Git,4,Grant,1,Grep,1,HashMap,1,HomeBrew,2,HTML,2,HttpClient,2,Immutable,1,Installation,1,Interview Questions,5,Iterate,2,Jackson API,3,Java,30,Java 10,1,Java 11,5,Java 12,5,Java 13,2,Java 14,2,Java 8,78,Java 8 Difference,2,Java 8 Stream Conversions,2,java 8 Stream Examples,3,Java 9,1,Java Conversions,11,Java Design Patterns,1,Java Files,1,Java Program,3,Java Programs,93,java.lang,5,java.util. function,1,jQuery,1,Kotlin,10,Kotlin Conversions,3,Kotlin Programs,6,Lambda,1,lang,29,Leap Year,1,live updates,1,Logging,1,Mac OS,2,Math,1,Matrix,5,Maven,1,Method References,1,Mockito,1,MongoDB,3,New Features,1,Operations,1,Optional,6,Oracle,5,Oracle 18C,1,Partition,1,Patterns,1,Programs,1,Property,1,Python,2,Quarkus,1,Read,1,Real Time,1,Recursion,2,Remove,2,Rest API,1,Schedules,1,Serialization,1,Servlet,2,Sort,1,Sorting Techniques,8,Spring,2,Spring Boot,23,Spring Email,1,Spring MVC,1,Streams,23,String,58,String Programs,12,String Revese,1,Swing,1,System,1,Tags,1,Threads,10,Tomcat,1,Tomcat 8,1,Troubleshoot,16,Unix,2,Updates,3,util,5,While Loop,1,
ltr
item
JavaProgramTo.com: Python - TypeError: Object of type 'int64' is not JSON serializable (Works for all data types - Integer, Float, Array, etc)
Python - TypeError: Object of type 'int64' is not JSON serializable (Works for all data types - Integer, Float, Array, etc)
https://1.bp.blogspot.com/-d6fxboRvyDU/XdgTkxtnScI/AAAAAAAAB8g/SpCxK6TrCnEgPbU8o6UAKOE2UfBeYm1FgCLcBGAsYHQ/s400/Python%2BTypeError%2BObject%2Bof%2Btype%2B%2Bint64%2Bis%2Bnot%2BJSON%2Bserializable.png
https://1.bp.blogspot.com/-d6fxboRvyDU/XdgTkxtnScI/AAAAAAAAB8g/SpCxK6TrCnEgPbU8o6UAKOE2UfBeYm1FgCLcBGAsYHQ/s72-c/Python%2BTypeError%2BObject%2Bof%2Btype%2B%2Bint64%2Bis%2Bnot%2BJSON%2Bserializable.png
JavaProgramTo.com
https://www.javaprogramto.com/2019/11/python-typeerror-integer-json-not-serializable.html
https://www.javaprogramto.com/
https://www.javaprogramto.com/
https://www.javaprogramto.com/2019/11/python-typeerror-integer-json-not-serializable.html
true
3124782013468838591
UTF-8
Loaded All Posts Not found any posts VIEW ALL Readmore Reply Cancel reply Delete By Home PAGES POSTS View All RECOMMENDED FOR YOU LABEL ARCHIVE SEARCH ALL POSTS Not found any post match with your request Back Home Sunday Monday Tuesday Wednesday Thursday Friday Saturday Sun Mon Tue Wed Thu Fri Sat January February March April May June July August September October November December Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec just now 1 minute ago $$1$$ minutes ago 1 hour ago $$1$$ hours ago Yesterday $$1$$ days ago $$1$$ weeks ago more than 5 weeks ago Followers Follow THIS PREMIUM CONTENT IS LOCKED STEP 1: Share to a social network STEP 2: Click the link on your social network Copy All Code Select All Code All codes were copied to your clipboard Can not copy the codes / texts, please press [CTRL]+[C] (or CMD+C with Mac) to copy Table of Content