hexadecimal string to byte array in python
Question:
I have a long Hex string that represents a series of values of different types. I wish to convert this Hex String into a byte array so that I can shift each value out and convert it into its proper data type.
Answers:
A good one liner is:
byte_list = map(ord, hex_string)
This will iterate over each char in the string and run it through the ord() function. Only tested on python 2.6, not too sure about 3.0+.
-Josh
provided I understood correctly, you should look for binascii.unhexlify
import binascii
a='45222e'
s=binascii.unhexlify(a)
b=[ord(x) for x in s]
You should be able to build a string holding the binary data using something like:
data = "fef0babe"
bits = ""
for x in xrange(0, len(data), 2)
bits += chr(int(data[x:x+2], 16))
This is probably not the fastest way (many string appends), but quite simple using only core Python.
Suppose your hex string is something like
>>> hex_string = "deadbeef"
Convert it to a bytearray (Python 3 and 2.7):
>>> bytearray.fromhex(hex_string)
bytearray(b'xdexadxbexef')
Convert it to a bytes object (Python 3):
>>> bytes.fromhex(hex_string)
b'xdexadxbexef'
Note that bytes is an immutable version of bytearray.
Convert it to a string (Python ≤ 2.7):
>>> hex_data = hex_string.decode("hex")
>>> hex_data
"xdexadxbexef"
Assuming you have a byte string like so
“x12x45x00xAB”
and you know the amount of bytes and their type you can also use this approach
import struct
bytes = 'x12x45x00xAB'
val = struct.unpack('<BBH', bytes)
#val = (18, 69, 43776)
As I specified little endian (using the ‘<‘ char) at the start of the format string the function returned the decimal equivalent.
0x12 = 18
0x45 = 69
0xAB00 = 43776
B is equal to one byte (8 bit) unsigned
H is equal to two bytes (16 bit) unsigned
More available characters and byte sizes can be found here
The advantages are..
You can specify more than one byte and the endian of the values
Disadvantages..
You really need to know the type and length of data your dealing with
There is a built-in function in bytearray that does what you intend.
bytearray.fromhex("de ad be ef 00")
It returns a bytearray and it reads hex strings with or without space separator.
def hex2bin(s):
hex_table = ['0000', '0001', '0010', '0011',
'0100', '0101', '0110', '0111',
'1000', '1001', '1010', '1011',
'1100', '1101', '1110', '1111']
bits = ''
for i in range(len(s)):
bits += hex_table[int(s[i], base=16)]
return bits
You can use the Codecs module in the Python Standard Library, i.e.
import codecs
codecs.decode(hexstring, 'hex_codec')
Sometimes byte-array conversion doesn’t work so in this case you can do this:
hex_string = "deadbeef"
data = ["0x" + hex_string[i:i+2] for i in range(0,len(hex_string), 2)]
I have a long Hex string that represents a series of values of different types. I wish to convert this Hex String into a byte array so that I can shift each value out and convert it into its proper data type.
A good one liner is:
byte_list = map(ord, hex_string)
This will iterate over each char in the string and run it through the ord() function. Only tested on python 2.6, not too sure about 3.0+.
-Josh
provided I understood correctly, you should look for binascii.unhexlify
import binascii
a='45222e'
s=binascii.unhexlify(a)
b=[ord(x) for x in s]
You should be able to build a string holding the binary data using something like:
data = "fef0babe"
bits = ""
for x in xrange(0, len(data), 2)
bits += chr(int(data[x:x+2], 16))
This is probably not the fastest way (many string appends), but quite simple using only core Python.
Suppose your hex string is something like
>>> hex_string = "deadbeef"
Convert it to a bytearray (Python 3 and 2.7):
>>> bytearray.fromhex(hex_string)
bytearray(b'xdexadxbexef')
Convert it to a bytes object (Python 3):
>>> bytes.fromhex(hex_string)
b'xdexadxbexef'
Note that bytes is an immutable version of bytearray.
Convert it to a string (Python ≤ 2.7):
>>> hex_data = hex_string.decode("hex")
>>> hex_data
"xdexadxbexef"
Assuming you have a byte string like so
“x12x45x00xAB”
and you know the amount of bytes and their type you can also use this approach
import struct
bytes = 'x12x45x00xAB'
val = struct.unpack('<BBH', bytes)
#val = (18, 69, 43776)
As I specified little endian (using the ‘<‘ char) at the start of the format string the function returned the decimal equivalent.
0x12 = 18
0x45 = 69
0xAB00 = 43776
B is equal to one byte (8 bit) unsigned
H is equal to two bytes (16 bit) unsigned
More available characters and byte sizes can be found here
The advantages are..
You can specify more than one byte and the endian of the values
Disadvantages..
You really need to know the type and length of data your dealing with
There is a built-in function in bytearray that does what you intend.
bytearray.fromhex("de ad be ef 00")
It returns a bytearray and it reads hex strings with or without space separator.
def hex2bin(s):
hex_table = ['0000', '0001', '0010', '0011',
'0100', '0101', '0110', '0111',
'1000', '1001', '1010', '1011',
'1100', '1101', '1110', '1111']
bits = ''
for i in range(len(s)):
bits += hex_table[int(s[i], base=16)]
return bits
You can use the Codecs module in the Python Standard Library, i.e.
import codecs
codecs.decode(hexstring, 'hex_codec')
Sometimes byte-array conversion doesn’t work so in this case you can do this:
hex_string = "deadbeef"
data = ["0x" + hex_string[i:i+2] for i in range(0,len(hex_string), 2)]