Reading binary file using numpy in Streamlit

Question

I have the following code snippet that works perfectly well in python, however, I am trying to use streamlit to upload the binary files, but I can’t seem to make it work. Here is the working code in python:

def read_bin():
   dt = np.dtype([('col1','d'),('col2','d'),('col3','d'),('col4','d')])  
   data = np.fromfile('Files/Bin Files/myfile.bin',dtype=dt,sep='')
   df= pd.DataFrame(data) 
   return df

Now, I want to have the user to upload the binary file and perform the operation using streamlit interface. Here is the code which doesn’t work for me:

if options == ls:
    st.sidebar.title('Upload Binary File')
    bin_file = st.sidebar.file_uploader('Upload File', key = 'ls')
    if bin_file:
        st.sidebar.success('The file was uploaded successfully!', icon="✅")     
 
def read_bin():
     dt = np.dtype([('col1','d'),('col2','d'),
                 ('col3','d'),('col4','d')])                 
     data = np.fromfile(bin_file,dtype=dt,sep='')
     df = pd.DataFrame(data)
     return df

if options == ls:
    if bin_file:
        displayBin = st.checkbox('Display File')
        if displayBin:
            df = read_bin()
            st.write(df)

So, basically, instead of showing the path to my folder where the bin file is located, I am showing the path to the uploaded file. But it doesn’t seem to work for me.
Upon some suggestions I tried the following but they didn’t work either:

def read_bin(file):
     dt = np.dtype([('col1','d'),('col2','d'),
                 ('col3','d'),('col4','d')])               

     with NamedTemporaryFile(dir='.',suffix='.bin') as f:
        f.write(file.getbuffer())      
        data = np.fromfile(f.name, dtype=dt, sep='')
        df = pd.DataFrame(data)
        return df

if options == ls:
    if bin_file:
        displayBin = st.checkbox('Display File')
        if displayBin:
            df = read_bin(bin_file)
            st.write(df)

I also tried to use shutil library as suggested here post and it returned an empty df as well: Please see my code here below where I attempted to utilise shutil.

def read_bin(fl):  

    dt = np.dtype([('col1','d'),('col2','d'),
                 ('col3','d'),('col4','d')])   
    
    with open('par.bin', 'wb') as buffer:
        shutil.copyfileobj(fl, buffer)
        data = np.fromfile('par.bin', dtype=dt, sep='')
        df = pd.DataFrame(data)
        return df

if options == ls:
    if bin_file:
        displayBin = st.checkbox('Display File')
        if displayBin:
            df = read_bin(bin_file)
            st.write(df)

Is there any way to read bin files from the uploaded_files directly just like we can read csv or pdf or other file types without the need for creating a temporary file in buffer?

Asked By: serdar_bay

||

Source

Answer 1

get_user_upload() will crate a new directory called user_uploads which saves the uploaded file to local disc and returns the file path. Now you pass that file path to read_bin() as an argument

Example:

import os


def read_bin(bin_data):
    dt = np.dtype([('col1','d'), ('col2','d'), ('col3','d'),('col4','d')])                 
    data = np.fromfile(bin_data, dtype=dt, sep='')
    df = pd.DataFrame(data)
    return df


def get_user_upload(file):
    file_path = os.path.join("user_uploads/", file.name)
    with open(file_path, "wb") as user_file:
        user_file.write(file.getbuffer())
    return file_path


# Where are the options?
... 
if options == "ls":
    st.sidebar.title('Upload Binary File')
    bin_file = st.sidebar.file_uploader('Upload File', key = 'ls')
    if bin_file:
        st.sidebar.success('The file was uploaded successfully!', icon="✅") 
        
        file_path = get_user_upload(bin_file)

        displayBin = st.checkbox('Display File')
        if displayBin:
            df = read_bin(file_path)
            st.write(df)

Answered By: Jamiu Shaibu

Reading binary file using numpy in Streamlit

Question:

Answers: